/_iE;UdZddlmZddlZddlZddlZddlmZmZm Z m Z m Z m Z ddl mZddlmZddlmZmZmZer ddlZddlmZeeZd ed <Gd d e Zdd ZGddZGddZy)z@Provides global MediaFileManager object as `media_file_manager`.) annotationsN) TYPE_CHECKINGBinaryIOFinalTextIO TypedDictcast) get_logger)$convert_data_to_bytes_and_infer_mime) MediaFileKindMediaFileStorageMediaFileStorageError)Callabler_LOGGERc:eZdZUdZded<ded<ded<ded<y ) DeferredCallableEntryz/Typed metadata for deferred download callables.>r c\eZdZdZej fddZeddZed dZ d dZ y) MediaFileMetadatazBMetadata that the MediaFileManager needs for each file it manages.c ||_d|_yNF)_kind_is_marked_for_delete)selfkinds r!__init__zMediaFileMetadata.__init__Bs %*"r c|jSN)r-r/s r!r0zMediaFileMetadata.kindFs zzr c|jSr3r.r4s r!is_marked_for_deletez&MediaFileMetadata.is_marked_for_deleteJs)))r cd|_y)NTr6r4s r!mark_for_deletez!MediaFileMetadata.mark_for_deleteNs %)"r N)r0r returnNone)r:r )r:boolr:r;) rrrrr MEDIAr1propertyr0r7r9rr r!r*r*?s@L-:-@-@+***r r*ceZdZdZd dZd dZddZddZddZdddZ d dd Z d dd Z dd Z y)MediaFileManageraIn-memory file manager for MediaFile objects. This keeps track of: - Which files exist, and what their IDs are. This is important so we can serve files by ID -- that's the whole point of this class! - Which files are being used by which AppSession (by ID). This is important so we can remove files from memory when no more sessions need them. - The exact location in the app where each file is being used (i.e. the file's "coordinates"). This is is important so we can mark a file as "not being used by a certain session" if it gets replaced by another file at the same coordinates. For example, when doing an animation where the same image is constantly replace with new frames. (This doesn't solve the case where the file's coordinates keep changing for some reason, though! e.g. if new elements keep being prepended to the app. Unlikely to happen, but we should address it at some point.) c||_i|_tjt|_i|_tj|_ yr3) _storage_file_metadata collections defaultdictdict_files_by_session_and_coord_deferred_callables threadingLock_lock)r/storages r!r1zMediaFileManager.__init__esG =?  # #D ) ( FH ^^% r ct|jj}|jj D]!}|j |j #|S)zCompute the set of files that are stored in the manager, but are not referenced by any active session. These are files that can be safely deleted. Thread safety: callers must hold `self._lock`. )setrDkeysrHvaluesdifference_update)r/file_idssession_file_ids_by_coords r!_get_inactive_file_idsz'MediaFileManager._get_inactive_file_idsys_t**//12*.)I)I)P)P)R K %  & &'@'G'G'I J Kr ctjd|j5|jD]}|j|}|j t jk(r|j|A|j t jk(s_|jr|j|}|j|jdddy#1swYyxYw)zuRemove all files that are no longer referenced by any active session. Safe to call from any thread. zRemoving orphaned files...N) rdebugrLrUrDr0r r> _delete_file DOWNLOADABLEr7r9#_remove_orphaned_deferred_callables)r/file_idfiles r!remove_orphaned_filesz&MediaFileManager.remove_orphaned_filess  23 ZZ 7668 /**7399 3 33%%g.YY-"<"<<00))'2,,. /  4 4 6 7 7 7sA/CACC$cdtjdtt}|jj D]!}|j |j #|jDcgc]}||vr| }}|D]%}tjd||j|='ycc}w)zRemove deferred callables that are not referenced by any active session. Thread safety: callers must hold `self._lock`. z'Removing orphaned deferred callables...zRemoving deferred callable: %sN)rrWrOrrHrQupdaterI)r/active_file_idsrTr[deferred_ids_to_removes r!rZz4MediaFileManager._remove_orphaned_deferred_callabless  ?@c(*)-)I)I)P)P)R G %  " "#<#C#C#E F G  33" o- " " . 2G MM:G D((1 2 " s3 B-ctjd||jj||j|=y)zDelete the given file from storage, and remove its metadata from self._files_by_id. Thread safety: callers must hold `self._lock`. zDeleting File: %sN)rrWrC delete_filerD)r/r[s r!rXzMediaFileManager._delete_files3  )73 !!'*    (r Nc| t}tjd||j5||jvr |j|=dddtjd|jj tjdt |jt |jy#1swYuxYw)a#Remove the given session's file references. (This does not remove any files from the manager - you must call `remove_orphaned_files` for that.) Should be called whenever ScriptRunner completes and when a session ends. Safe to call from any thread. Nz*Disconnecting files for session with ID %szSessions still active: %rz"Files: %s; Sessions with files: %s)r(rrWrLrHrPlenrD)r/r&s r!clear_session_refsz#MediaFileManager.clear_session_refss  (*J BJO ZZ AT===44Z@ A  ')I)I)N)N)P   0 ## $ 00 1  A As B??Ccjt}|j5|rtjntj}|j j ||||}t|} | |j|<||j||<|j j|cdddS#1swYyxYw)aAdd a new MediaFile with the given parameters and return its URL. If an identical file already exists, return the existing URL and registers the current session as a user. Safe to call from any thread. Parameters ---------- path_or_data : bytes or str If bytes: the media file's raw data. If str: the name of a file to load from disk. mimetype : str The mime type for the file. E.g. "audio/mpeg". This string will be used in the "Content-Type" header when the file is served over HTTP. coordinates : str Unique string identifying an element's location. Prevents memory leak of "forgotten" file IDs when element media is being replaced-in-place (e.g. an st.image stream). coordinates should be of the form: "1.(3.-14).5" file_name : str or None Optional file_name. Used to set the filename in the response header. is_for_static_download: bool Indicate that data stored for downloading as a file, not as a media for rendering at page. [default: False] Returns ------- str The url that the frontend can use to fetch the media. Raises ------ If a filename is passed, any Exception raised when trying to read the file will be re-raised. r0N) r(rLr rYr>rCload_and_get_idr*rDrHget_url) r/ path_or_datarr file_nameis_for_static_downloadr&r0r[metadatas r!addzMediaFileManager.adds\%& ZZ 2***"((  mm33hiG)d3H+3D   (HOD , ,Z 8 E==((1 2 2 2s BB))B2c t}|j5tjj}t d||||d|j |<||j||<|cdddS#1swYyxYw)aRegister a callable for deferred execution. Returns placeholder file_id. The callable will be executed later when execute_deferred() is called, typically when the user clicks a download button. Safe to call from any thread. Parameters ---------- data_callable : Callable[[], bytes | str | BinaryIO | TextIO | io.RawIOBase] A callable that returns the file data when invoked. mimetype : str or None The mime type for the file. E.g. "text/csv". If None, the mimetype will be inferred from the data type when execute_deferred() is called. coordinates : str Unique string identifying an element's location. file_name : str or None Optional file_name. Used to set the filename in the response header. Returns ------- str A placeholder file_id that can be used to execute the callable later. r)rrrrN)r(rLuuiduuid4hexr rIrH)r/ data_callablerrrlr&r[s r! add_deferredzMediaFileManager.add_deferreds@%& ZZ jjl&&G15' - ( )#. 1D $ $W -IPD , ,Z 8 E'   s AA11A:c x|j5||jvrtd|d|j|}ddd d}t |tdt |\}}|dxs|}|j5|j j||tj|d }ttj } | |j|<|j j|cdddS#1swYxYw#t$r}td||d}~wwxYw#1swYyxYw) aExecute a deferred callable and return the URL to the generated file. This method retrieves the callable registered with add_deferred(), executes it, stores the result, and returns a URL to access it. Safe to call from any thread. Parameters ---------- file_id : str The placeholder file_id returned by add_deferred(). Returns ------- str The URL that can be used to download the generated file. Raises ------ MediaFileStorageError If the file_id is not found or if the callable execution fails. zDeferred file z not foundNrzCallable execution failed: z$Callable returned unsupported type: )unsupported_errorrrrh) rLrIr Exceptionr typerCrir rYr*rDrj) r/r[deferreddatae data_as_bytesinferred_mime_type mime_typeactual_file_idrns r!execute_deferredz!MediaFileManager.execute_deferredVsR0ZZ 9d666+nWIZ,PQQ//8H  9 R'8J')D -Q 36tDzlC- ) )"*-C1C ZZ 9!]]::**$ N)m.H.HIH2:D   /==((8' 9 91 9 9 R'*EaS(IJPQ Q R 9 9s0-D DA3D0D  D-D((D-0D9)rMr r:r;)r:zset[str]r=)r[rr:r;r3)r&rr:r;r,) rkz bytes | strrrrrrlrrmr<r:r) rtrrrrrrlrr:r)r[rr:r) rrrrr1rUr]rZrXrfrorurrr r!rArARs$&( 7*2,) L!%', >2!>2>2 >2  >2 !% >2 >2J!% 5S55 5  5 5nC9r rA)r:r)r __future__rrErJrqtypingrrrrrr streamlit.loggerr $streamlit.runtime.download_data_utilr $streamlit.runtime.media_file_storager r riocollections.abcrrrrrr(r*rArr r!rsnG" JJ'U  (H%%I **&G9G9r