o lh @sdZddlZddlZddlZddlZddlmZddlmZmZddl m Z ddl m Z er7ej Z ejZeZnej Z ejZzddlZWn eyNdZYnwzddlZWn ey`dZYnwdZd d Zd d Zd dZddZddZejdddZdddZdZdddZ dS)zFUtilities for fast persistence of big data, with optional compression.N)closing) PY3_OR_LATERPY27) _ZFILE_PREFIX) _COMPRESSORSicCs&trt|d|}t|tjSt|tS)z=Check if fileobj is a raw file object, e.g created with open.raw)rgetattr isinstanceioFileIOfile)fileobjrK/var/www/html/env/lib/python3.10/site-packages/joblib/numpy_pickle_utils.py _is_raw_file+s   rcCs(ddtD}|ttg7}t|S)NcSsg|]}t|jqSr)lenprefix).0 compressorrrr 6sz)_get_prefixes_max_len..)rvaluesrrmax)prefixesrrr_get_prefixes_max_len4srcCsht}t|dr||}n ||}|d|trdStD]\}}||j r1|Sq#dS)zReturn the compressor matching fileobj. Parameters ---------- fileobj: file object Returns ------- str in {'zlib', 'gzip', 'bz2', 'lzma', 'xz', 'compat', 'not-compressed'} peekrcompatnot-compressed) rhasattrrreadseek startswithrritemsr)rmax_prefix_len first_bytesnamerrrr_detect_compressor=s      r&cCs*trtdurt|tjr|Stj|tdS)z0Return a buffered version of a read file object.N buffer_size)rbz2r BZ2Filer BufferedReader_IO_BUFFER_SIZEfobjrrr_buffered_read_file]sr/cCs.trtdurt|tjrt|Stj|tdS)z1Return a buffered version of a write file object.Nr')rr)r r*rr BufferedWriterr,r-rrr_buffered_write_filegsr1ccst|}|dkrtjd|tdd|VdS|tvr)t|}||}t|}|dur[t|tj r>tjdt ddn|dkrMtjdt ddnt |s[tjd t dd|VdS) a-Utility function opening the right fileobject from a filename. The magic number is used to choose between the type of file object to open: * regular file object (default) * zlib file object * gzip file object * bz2 file object * lzma file object (for xz and lzma compressor) Parameters ---------- fileobj: file object compressor: str in {'zlib', 'gzip', 'bz2', 'lzma', 'xz', 'compat', 'not-compressed'} filename: str filename path corresponding to the fileobj parameter. mmap_mode: str memory map mode that should be used to open the pickle file. This parameter is useful to verify that the user is not trying to one with compression. Default: None. Returns ------- a file like object rzjThe file '%s' has been generated with a joblib version less than 0.10. Please regenerate this pickle file.) stacklevelNzuIn memory persistence is not compatible with mmap_mode "%(mmap_mode)s" flag passed. mmap_mode option will be ignored.rztmmap_mode "%(mmap_mode)s" is not compatible with compressed file %(filename)s. "%(mmap_mode)s" flag will be ignored.zP"%(fileobj)r" is not a raw file, mmap_mode "%(mmap_mode)s" flag will be ignored.) r&warningswarnDeprecationWarningrdecompressor_filer/r r BytesIOlocalsr)rfilename mmap_modercompressor_wrapperinstrrr_read_fileobjectssB    r>zlibcCsP|d}|d}|tvrt|j||d}t|Stdj||d}t|S)z6Return the right compressor file object in write mode.rr) compresslevelr@)rkeyscompressor_filer1)r:compresscompressmethodrB file_instancerrr_write_fileobjects rHiran out of datacCst} z||t|}||7}t|dkst||kr Wn Wn tjy+Ynwqt||kr@d}t|||t|f|S)aRead from file-like object until size bytes are read. Raises ValueError if not EOF is encountered before size bytes are read. Non-blocking objects only supported if they derive from io objects. Required as e.g. ZipExtFile in python 2.6 can return less data than requested. This function was taken from numpy/lib/format.py in version 1.10.2. Parameters ---------- fp: file-like object size: int error_template: str Returns ------- a bytes object The data read in bytes. Trz)EOF: reading %s, expected %d bytes got %d)bytesrrr BlockingIOError ValueError)fpsizeerror_templatedatarmsgrrr _read_bytess  rS)N)r?)rI)!__doc__pickler r4 contextlibr_compatrrrrr _Unpickler Unpickler_PicklerPicklerrangexrangenumpynp ImportErrorr)r,rrr&r/r1contextmanagerr>rH BUFFER_SIZErSrrrrsH             E