o 4Z@s ddlZddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Tddl TddlTGdddeZGdddeZGdddeZGd d d eZGd d d eZGd ddeZGdddeZGdddeZGdddeZGdddeZGdddeZGdddeZGdddeZGdddeZGdd d eZGd!d"d"eZGd#d$d$eZ Gd%d&d&eZ!Gd'd(d(eZ"Gd)d*d*eZ#Gd+d,d,eZ$Gd-d.d.eZ%Gd/d0d0eZ&Gd1d2d2eZ'Gd3d4d4eZ(Gd5d6d6eZ)Gd7d8d8eZ*d9d:Z+d;d<Z,d=d>Z-dqd?d@Z.drdAdBZ/dCdDZ0dEdFZ1dGdHZ2GdIdJdJZ3GdKdLdLZ4dMdNZ5dOdPZ6dQdRZ7dSdTZ8dUdVZ9dWdXZ:GdYdZdZe;ZGd_d`d`e>Z?Gdadbdbe?Z@Gdcdddde=ZAGdedfdfe<ZBGdgdhdhe<ZCe+Gdidjdje<ZDdkdlZEdmdnZFGdodpdpe<ZGGdqdrdre<ZHGdsdtdte<ZIe+dudvZJe+dwdxZKe+dydzZLe+d{d|ZMe+d}d~ZNe+ddZOe+ddZPe+ddZQe+ddZRe+ddZSe+ddZTe+ddZUe+ddZVe+ddZWe+ddZXe+ddZYe+ddZZe+ddZ[e+ddZ\e+ddZ]e+ddZ^e+ddZ_e+ddZ`e+ddZae+ddZbe+ddZce+ddZdeMZeeNZfeOZgePZhe+ddZie+ddZje+ddZke+ddZle+ddZme+ddZneiZoelZpejqdkZre+ddZse+ddZte+ddZue+ddZve+ddZwe+ddÄZxe+Gddńde<ZyezdddddddddddddddɍZ{dd˄Z|Gdd̈́de>Z}ddτZ~ddфZddӄZddՄZe+Gddׄde<ZGddلdeZGddۄdeZGdd݄de>ZGdd߄deZGddde>ZGddde>ZGddde<ZGddde<ZGddde=ZGddde=ZGddde=ZGddde=ZGddde=ZGddde=ZGddde<Ze+Gddde<ZGddde=ZGddde=ZGddde<Ze+Gddde<ZGddde<Ze+Gddde<Ze+Gddde<ZGddde>Zdd ZGd d d e>ZGd d d e>ZGddde<ZGddde<ZddZddZGddde<ZGddde<ZddZGddde<Zdsdd ZGd!d"d"e=ZGd#d$d$e=Zd%d&Zd'd(ZGd)d*d*e=ZGd+d,d,e=ZGd-d.d.e<Ze+Gd/d0d0e<Ze+Gd1d2d2e<Ze+Gd3d4d4e<ZGd5d6d6e=Zd7d8Zd9d:ZGd;d<dZGd?d@d@e=ZGdAdBdBe=ZGdCdDdDe=ZGdEdFdFe=ZGdGdHdHe=ZGdIdJdJe=ZGdKdLdLe=ZGdMdNdNe=ZGdOdPdPe<ZGdQdRdReAZGdSdTdTe=ZGdUdVdVe=ZGdWdXdXezZGdYdZdZe<ZGd[d\d\eƃZGd]d^d^e=ZGd_d`d`e<ZGdadbdbe>ZGdcddddeʃZGdedfdfe@Z̐dgdhZ͐didjZΐdkdlZGdmdndne>ZGdodpdpe>ZdS(tN)*c@ eZdZdS)ConstructErrorN__name__ __module__ __qualname__r r @/var/www/html/env/lib/python3.10/site-packages/construct/core.pyr rc@r) SizeofErrorNrr r r r r r r c@r)AdaptationErrorNrr r r r r r r c@r)ValidationErrorNrr r r r rr rc@r) StreamErrorNrr r r r rr rc@r)FormatFieldErrorNrr r r r rr rc@r) IntegerErrorNrr r r r rr rc@r) StringErrorNrr r r r rr rc@r) MappingErrorNrr r r r rr rc@r) RangeErrorNrr r r r rr rc@r) RepeatErrorNrr r r r r!r rc@r) ConstErrorNrr r r r r#r rc@r)IndexFieldErrorNrr r r r r%r rc@r) CheckErrorNrr r r r r'r rc@r) ExplicitErrorNrr r r r r)r rc@r)NamedTupleErrorNrr r r r r+r rc@r)TimestampErrorNrr r r r r-r rc@r) UnionErrorNrr r r r r/r rc@r) SelectErrorNrr r r r r1r rc@r) SwitchErrorNrr r r r r3r rc@r)StopFieldErrorNrr r r r r5r rc@r) PaddingErrorNrr r r r r 7r r c@r)TerminatedErrorNrr r r r r!9r r!c@r) RawCopyErrorNrr r r r r";r r"c@r) RotationErrorNrr r r r r#=r r#c@r) ChecksumErrorNrr r r r r$?r r$c@r) CancelParsingNrr r r r r%Ar r%cs}fdd|_|S)NcsjSN)rr argr r Jszsingleton..) __reduce__)r(xr r'r singletonHsr,cCsd|dkr td|z||}Wntytd|fwt||kr0td|t|f|S)Nr%length must be non-negative, found %sz(stream.read() failed, requested %s bytesz=stream read less then specified amount, expected %d, found %d)rread Exceptionlen)streamlengthdatar r r stream_readNs   r4cC"z|WStytdw)Nz+stream.read() failed when reading until EOF)r.r/rr1r r r stream_read_entireZ   r7cCst|ts td|f|durt|}|dkrtd|t||kr.td|t|fz||}WntyCtd|fw||krPtd||fdS)N*given non-bytes value, perhaps unicode? %rrr-z3bytes object of wrong length, expected %d, found %dzstream.write() failed, given %rz;stream written less then specified, expected %d, written %d) isinstancebytestringtyperr0rwriter/)r1r3r2writtenr r r stream_writeas    r>cCs.z|||WStytd||fw)Nz*stream.seek() failed, offset %s, whence %s)seekr/r)r1offsetwhencer r r stream_seekrs  rBcCr5)Nzstream.tell() failed)tellr/rr6r r r stream_tellyr8rDcCs"|}|dd}|||SNr)rCr?)r1fallbackendr r r stream_sizes  rIcCs"|}|d}||| SN)rCr.r?)r1rGr3r r r stream_iseofs  rLc@s,eZdZddZddZddZddZd S) CodeGencCs"g|_d|_i|_i|_i|_dSNr)blocksnextid parsercachelinkedinstances linkedparsersselfr r r __init__  zCodeGen.__init__cC|jd7_|jSrJrPrTr r r allocateIdzCodeGen.allocateIdcsddd|D}|d}t|t|dfdd|D}||jvr0|j|dSdS)NcSsg|]}|r|qSr )strip.0sr r r z"CodeGen.append..r c3s|] }|dVqdSr&r r]trimr r z!CodeGen.append..) splitlinesr0lstripjoinrOappend)rUblock firstliner rcr rjs zCodeGen.appendcCsd|jdgS)Nrb)rirOrTr r r toStringzCodeGen.toStringN)rrrrVrZrjrnr r r r rMs  rMc@seZdZddZddZdS)KsyGencCsi|_i|_i|_d|_dSrN) instancesenumstypesrPrTr r r rVs zKsyGen.__init__cCrXrJrYrTr r r rZr[zKsyGen.allocateIdN)rrrrVrZr r r r rps rpcCsdd|DS)NcSs$i|]\}}|ddd|qS)_-)replacerstripr^kvr r r $z!hyphenatedict..)items)dr r r hyphenatedictrorcCsdd|DS)NcSsg|]}t|qSr )r)r^r~r r r r`z!hyphenatelist..r )lr r r hyphenatelistrcs>fddg}|D]}|jr||q ||q |S)Ncs:t|ttfr |jSt|tttttfr|j St d)NzFEmbedding only works with: Struct Sequence FocusedSeq Union LazyStruct) r:RenamedEmbeddedsubconStructSequence FocusedSeqUnion LazyStructsubconsrscselectr r rs  zmergefields..select) flagembeddedextendrj)rresultrr rr mergefieldss  rcCs*t|tr t|jSt|trtd|S)NzEmbedded itself is not parsable)r:r extractfieldrrrrr r r rs   rcCst|r||S|Sr&)callable)paramcontextr r r evaluatercCs |rdSdS)Nz# zpass; r ) conditionr r r disableif rc@sHeZdZdZddZddZddZdd Zd d Zd d Z ddZ ddZ ddZ ddZ ddZddZddZddZddZd d!Zd"d#ZdPd%d&Zd'd(Zd)d*Zd+d,Zd-d.Zd/d0ZdPd1d2ZdQd4d5ZdRd8d9ZdRd:d;ZdRdd?Z d@dAZ!dBdCZ"dDdEZ#e#Z$dFdGZ%dHdIZ&dJdKZ'dLdMZ(dNdOZ)d$S)S Constructa? The mother of all constructs. This object is generally not directly instantiated, and it does not directly implement parsing and building, so it is largely only of interest to subclass implementors. There are also other abstract classes sitting on top of this one. The external user API: * `parse` * `parse_stream` * `parse_file` * `build` * `build_stream` * `build_file` * `sizeof` * `compile` * `benchmark` Subclass authors should not override the external methods. Instead, another API is available: * `_parse` * `_build` * `_sizeof` * `_actualsize` * `_emitparse` * `_emitbuild` * `__getstate__` * `__setstate__` Attributes and Inheritance: All constructs have a name and flags. The name is used for naming struct members and context dictionaries. Note that the name can be a string, or None by default. A single underscore "_" is a reserved name, used as up-level in nested containers. The name should be descriptive, short, and valid as a Python identifier, although these rules are not enforced. The flags specify additional behavioral information about this construct. Flags are used by enclosing constructs to determine a proper course of action. Flags are often inherited from inner subconstructs but that depends on each class. cCs"d|_d|_d|_d|_d|_dS)NrmF)namedocs flagbuildnonerparsedrTr r r rVrWzConstruct.__init__cCsHd|jj|jr d|jnd|jrdnd|jrdnd|jr dfSdfS)Nz <%s%s%s%s%s> rm +nonbuild +embedded +docs) __class__rrrrrrTr r r __repr__sHzConstruct.__repr__cCsvi}t|dr ||jg}|j}|dur(t|dr!||j|j}|dus|D]}t||r8t||||<q*|S)N__dict__ __slots__)hasattrupdaterrrr__base__getattr)rUattrsslotscrr r r __getstate__ s     zConstruct.__getstate__cCs"|D] \}}t|||qdSr&)r}setattr)rUrrvaluer r r __setstate__szConstruct.__setstate__cCs t|j}||||Sr&)object__new__rrr)rUself2r r r __copy__s zConstruct.__copy__cKs|jt|fi|S)a Parse an in-memory buffer (often bytes object). Strings, buffers, memoryviews, and other complete buffers can be parsed with this method. Whenever data cannot be read, ConstructError or its derivative is raised. This method is NOT ALLOWED to raise any other exceptions although (1) user-defined lambdas can raise arbitrary exceptions which are propagated (2) external libraries like numpy can raise arbitrary exceptions which are propagated (3) some list and dict lookups can raise IndexError and KeyError which are propagated. Context entries are passed only as keyword parameters \*\*contextkw. :param \*\*contextkw: context entries, usually empty :returns: some value, usually based on bytes read from the stream but sometimes it is computed from nothing or from the context dictionary, sometimes its non-deterministic :raises ConstructError: raised for any reason ) parse_streamioBytesIO)rUr3 contextkwr r r parse"szConstruct.parsecKsLtdi|}d|_d|_d|_||_z|||dWSty%YdSw)z Parse a stream. Files, pipes, sockets, and other streaming sources of data are handled by this method. See parse(). TFz (parsing)Nr ) Container_parsing _building_sizing_params _parsereportr%)rUr1rrr r r r2s zConstruct.parse_streamcKs@t|d}|j|fi|WdS1swYdS)z: Parse a closed binary file. See parse(). rbN)openr)rUfilenamerfr r r parse_file@s $zConstruct.parse_filecCs(||||}|jdur||||Sr&)_parserrUr1rpathobjr r r rGs  zConstruct._parsereportcCtzOverride in your subclass.NotImplementedErrorrUr1rrr r r rMzConstruct._parsecKs$t}|j||fi||S)a Build an object in memory (a bytes object). Whenever data cannot be written, ConstructError or its derivative is raised. This method is NOT ALLOWED to raise any other exceptions although (1) user-defined lambdas can raise arbitrary exceptions which are propagated (2) external libraries like numpy can raise arbitrary exceptions which are propagated (3) some list and dict lookups can raise IndexError and KeyError which are propagated. Context entries are passed only as keyword parameters \*\*contextkw. :param \*\*contextkw: context entries, usually empty :returns: bytes :raises ConstructError: raised for any reason )rr build_streamgetvalue)rUrrr1r r r buildQszConstruct.buildcKs:tdi|}d|_d|_d|_||_||||ddS)zF Build an object directly into a stream. See build(). FTz (building)Nr )rrrrr_build)rUrr1rrr r r rcs zConstruct.build_streamcKsDt|d}|j||fi|WddS1swYdS)zI Build an object into a closed binary file. See build(). wbN)rr)rUrrrrr r r build_filens "zConstruct.build_filecCrrrrUrr1rrr r r rurzConstruct._buildcKs2tdi|}d|_d|_d|_||_||dS)aP Calculate the size of this object, optionally using a context. Some constructs have fixed size (like FormatField), some have variable-size and can determine their size given a context entry (like Bytes(this.otherfield1)), and some cannot determine their size (like VarInt). Whenever size cannot be determined, SizeofError is raised. This method is NOT ALLOWED to raise any other exception, even if eg. context dictionary is missing a key, or subcon propagates ConstructError-derivative exception. Context entries are passed only as keyword parameters \*\*contextkw. :param \*\*contextkw: context entries, usually empty :returns: integer if computable, SizeofError otherwise :raises SizeofError: size could not be determined in actual context, or is impossible to be determined FTz(sizeof)Nr )rrrrr_sizeof)rUrrr r r sizeofys  zConstruct.sizeofcCrrr rUrrr r r rrzConstruct._sizeofcCs |||Sr&rrr r r _actualsizerzConstruct._actualsizeNc Cst}|d|d||f|}|r3t|d }||Wdn1s.wYtt|   }t |}t|dd}t||j|j|_|j|_|j}||_||_||_||_|S)a Transforms a construct into another construct that does same thing (has same parsing and building semantics) but is much faster when parsing. Already compiled instances just compile into itself. Optionally, partial source code can be saved to a text file. This is meant only to inspect the generated code, not to import it from external scripts. :returns: Compiled instance aM # generated by Construct, this source is for inspection only! do not import! from construct import * from construct.lib import * from io import BytesIO import struct import collections import itertools def read_bytes(io, count): if not count >= 0: raise StreamError data = io.read(count) if not len(data) == count: raise StreamError return data def restream(data, func): return func(BytesIO(data)) def reuse(obj, func): return func(obj) linkedinstances = {} linkedparsers = {} len_ = len sum_ = sum min_ = min max_ = max abs_ = abs z} def parseall(io, this): return %s compiled = Compiled(None, None, parseall) wtNrmexec)rMrj _compileparsernrr<hexlifyhashlibsha1encodedigestdecodeimp new_modulecompilerrrRrScompiledsourcemodule modulename defersubcon) rUrcoderrrrrrr r r rs,       zConstruct.compilecCsdt||jvr|jt|S|dt||ft|}||jt|<|j|jt|<dt|S)Used internally.z1 # linkedinstances[%s] is %r zlinkedinstances[%s])idrRrjrrrS)rUrfieldr r r _compileinstances  zConstruct._compileinstancecCsfzt||jvr|jt|WS||}||jt|<|WSty2||dt|YSw)rz$linkedparsers[%s](io, this, '(???)'))rrQ _emitparserr)rUremittedr r r rs   zConstruct._compileparsecCr)rrrUrr r r _compilebuildrzConstruct._compilebuildcCrrrrr r r rrzConstruct._emitparsecCrrrrr r r _emitbuildrzConstruct._emitbuildc sddlm}zd}d}d}|fdddd}tdtdtd |}|dkr:|fd d|d|}d |}|fd ddd}tdtdtd |}|dkrl|fd d|d|}d |}}|ksJ|fdddd}tdtdtd |}|dkr|fdd|d|}d |}Wn tyYnwgd} d | |||} |rt |d} | | Wd| S1swY| S)a Measures performance of your construct (its parsing and building runtime), both for the original instance and the compiled instance. Uses timeit module, over at min 1 sample, and at max over 1 second time. Optionally, results are saved to a text file for later inspection. Otherwise you can print the result string to terminal. Also this method checks correctness, by comparing parsing/building results from both instances. :param sampledata: bytes, a valid blob parsable by this construct :param filename: optional, string, source is saved to that file :returns: string containing measurements r)timeitfailedc Sr&rr  sampledatarUr r r) z%Construct.benchmark..rK)numberig?crr&rr rr r r)rz{:.10f} sec/callcrr&rr  sampleobjrUr r r)rcrr&rr rr r r)rc Sr&rr rrr r r)#rcrr&rr rr r r)&r)zTimeit measurements:zparsing: {}zparsing compiled: {}zbuilding: {}rmrbrN) rrminmaxintformatrrr/rirr<) rUrrr parsetime buildtime parsetime2runsrlinesresultsrr )rrrrUr benchmarksJ           zConstruct.benchmarkunnamed_schemac Csddlm}|}d|_t}t}tt|d|||j|j |j d}| ||| }|rLt |d} | |Wd|S1sGwY|S)Nr)YAMLFr)metaseqrqrrrsr) ruamel.yamlrdefault_flow_stylerStringIOrpdict _compileseqrqrrrsdumprrr<) rU schemanamerryamloutputgenmainrrr r r export_ksy:s $    zConstruct.export_ksyFrc CsX|dkrtdz t|||WSty+tdddi||||dgYSw)N'construct does not implement KSY exportrr+rKr )rr_emitseqrr_compilefulltyperUksybitwise recursionr r r rIs &zConstruct._compileseqc Cs`|dkrtdz|||WSty/d|}t||||dd|j|<|YSw)Nr%r&ztype_%srK)r)r_emitprimitivetyperrZrrrs)rUr*r+r,rr r r _compileprimitivetypeQs  zConstruct._compileprimitivetypec CsN|dkrtdz t|||WSty&t||||ddYSw)Nr%r&rK)type)rr _emitfulltyperrr.r)r r r r([s zConstruct._compilefulltypecCrrrrUr*r+r r r r'crzConstruct._emitseqcCrrrr1r r r r-grzConstruct._emitprimitivetypecCrrrr1r r r r0krzConstruct._emitfulltypecCs t||dS)zc Used for renaming subcons, usually part of a Struct, like Struct("index" / Byte). )newname)rrUrr r r __rtruediv__os zConstruct.__rtruediv__cC2t|tr t||dSt|rt||dStdzz Used for adding docstrings and parsed hooks to subcons, like "field" / Byte * "docstring" * processfunc. )newdocs) newparsedz1operator * can only be used with string or lambdar: stringtypesrrrrUotherr r r __mul__w   zConstruct.__mul__cCr5r6r9r;r r r __rmul__r>zConstruct.__rmul__cC8t|tr|jn|g}t|tr|jn|g}t||S)zM Used for making Struct like ("index"/Byte + "prefix"/Byte). )r:rrrUr<lhsrhsr r r __add__ zConstruct.__add__cCr@)zA Used for making Sequences like (Byte >> Short). )r:rrrAr r r __rshift__rEzConstruct.__rshift__cCs6t|tr tdt|tst|rt||Std)zK Used for making Arrays like Byte[5] and Byte[this.count]. zNsubcon[N] syntax can only be used for Arrays, use GreedyRange(subcon) instead?z2subcon[N] syntax expects integer or context lambda)r:slicerr rArray)rUcountr r r __getitem__s  zConstruct.__getitem__r&)rN)Fr)*rrr__doc__rVrrrrrrrrrrrrrrrrrrrrrrrr$rr.r(r'r-r0r4__rdiv__r=r?rDrFrJr r r r rsP!  @   =      rc@eZdZdZfddZddZddZdd Zd d ZZ S) Subconstructz Abstract subconstruct (wraps an inner construct, inheriting its name and flags). Parsing and building is by default deferred to subcon, same as sizeof. :param subcon: Construct instance cs:t|ts tdtt|||_|j|_|j|_dS)Nz"subcon should be a Construct field) r:r TypeErrorsuperrNrVrrrrUrrr r rVs  zSubconstruct.__init__cCsLd|jj|jr d|jnd|jrdnd|jrdnd|jrdndt|jfS)Nz<%s%s%s%s%s %s>rrmrrr)rrrrrrreprrrTr r r rsLzSubconstruct.__repr__cCs|j|||Sr&)rrrr r r rzSubconstruct._parsecC|j||||Sr&rrrr r r rrozSubconstruct._buildcC|j||Sr&rrrr r r rrzSubconstruct._sizeof) rrrrKrVrrrr __classcell__r r rRr rNs rNc@0eZdZdZddZddZddZdd Zd S) Adapterz Abstract adapter class. Needs to implement `_decode()` for parsing and `_encode()` for building. :param subcon: Construct instance cCs|j|||}||||Sr&)rr_decoderr r r rzAdapter._parsecCs$||||}|j||||}|Sr&)_encoderr)rUrr1rrobj2buildretr r r rszAdapter._buildcCrr&rrUrrrr r r r\zAdapter._decodecCrr&rrar r r r^rbzAdapter._encodeN)rrrrKrrr\r^r r r r r[s  r[c@eZdZdZddZdS)SymmetricAdapterz Abstract adapter class. Needs to implement `_decode()` only, for both parsing and building. :param subcon: Construct instance cCs||||Sr&)r\rar r r r^rzSymmetricAdapter._encodeN)rrrrKr^r r r r rds rdc@ eZdZdZddZddZdS) Validatorz Abstract class that validates a condition on the encoded/decoded object. Needs to implement `_validate()` that returns a bool (or a truthy value) :param subcon: Construct instance cCs ||||std|f|S)Nzobject failed validation: %s) _validaterrar r r r\szValidator._decodecCrr&rrar r r rgrbzValidator._validateN)rrrrKr\rgr r r r rfs rfc@8eZdZdZddZddZddZdd Zd d Zd S) Tunnelz Abstract class that allows other constructs to read part of the stream as if they were reading the entrie stream. See Prefixed for example. Needs to implement `_decode()` for parsing and `_encode()` for building. cCs*t|}||||}|jj|fi|Sr&)r7r\rrrUr1rrr3r r r rsz Tunnel._parsecCs>t}|j||||}|}||||}t|||Sr&)rrrrrr^r>rUrr1rrstream2r`r3r r r rs  z Tunnel._buildcCrr&rrr r r r rbzTunnel._sizeofcCrr&rrUr3rrr r r r\ rbzTunnel._decodecCrr&rrmr r r r^rbzTunnel._encodeN) rrrrKrrrr\r^r r r r ris ricsLeZdZdZfddZddZddZdd Zdd d Zdd dZ Z S)Compiledrc$tt|||_||_||_dSr&)rPrnrVrr parsefunc)rUrrrprRr r rV zCompiled.__init__cCs |||Sr&)rprr r r rrzCompiled._parsecCrUr&)rrrr r r rrozCompiled._buildcCrWr&)rrrr r r r"rzCompiled._sizeofNcC|Sr&r )rUrr r r r%rbzCompiled.compilecCrWr&)rr)rUrrr r r r(rzCompiled.benchmarkr&) rrrrKrVrrrrrrYr r rRr rns  rncHeZdZdZfddZddZddZdd Zd d Zd d Z Z S)Bytesa Field consisting of a specified number of bytes. Parses into a bytes (of given length). Builds into the stream directly (but checks that given object matches specified length). Can also build from an integer for convenience (although BytesInteger should be used instead). Size is the specified length. :param length: integer or context lambda :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises StringError: building from non-bytes value, perhaps unicode Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> d = Bytes(4) >>> d.parse(b'beef') b'beef' >>> d.build(b'beef') b'beef' >>> d.build(0) b'\x00\x00\x00\x00' >>> d.sizeof() 4 >>> d = Struct( ... "length" / Int8ub, ... "data" / Bytes(this.length), ... ) >>> d.parse(b"\x04beef") Container(length=4)(data=b'beef') >>> d.sizeof() construct.core.SizeofError: cannot calculate size, key not found in context ctt|||_dSr&)rPrtrVr2)rUr2rRr r rVR zBytes.__init__cCs$t|jr ||n|j}t||Sr&)rr2r4)rUr1rrr2r r r rVs z Bytes._parsecCsBt|jr ||n|j}t|trt||n|}t||||Sr&)rr2r:r  integer2bytesr>rUrr1rrr2r3r r r rZs z Bytes._buildc Cs:zt|jr ||WS|jWSttfytdwN/cannot calculate size, key not found in context)rr2KeyErrorAttributeErrorr rr r r r`s z Bytes._sizeofcC d|jfS)Nzread_bytes(io, %s)r2rr r r rfrzBytes._emitparsecCs t|jdSNsize)rr2r1r r r r0irzBytes._emitfulltype rrrrKrVrrrrr0rYr r rRr rt/s "rtc@rZ) GreedyBytesa Field consisting of unknown number of bytes. Parses the stream to the end. Builds into the stream directly (without checks). Size is undefined. :raises StreamError: stream failed when reading until EOF :raises StringError: building from non-bytes value, perhaps unicode Example:: >>> GreedyBytes.parse(b"asislight") b'asislight' >>> GreedyBytes.build(b"asislight") b'asislight' cCt|Sr&)r7rr r r rzGreedyBytes._parsecCs|||Sr&)r<rr r r rs zGreedyBytes._buildcCdS)Nz io.read()r rr r r rrbzGreedyBytes._emitparsecC tddS)NT)size_eosrr1r r r r0 zGreedyBytes._emitfulltypeN)rrrrKrrrr0r r r r rms  rc sz}tt|dt|d}Wnty&ttdtddd}Ynwfdd}fdd}fd d }||_||_||_|S) aO Converts the stream from bytes to bits, and passes the bitstream to underlying subcon. Bitstream is a stream that contains 8 times as many bytes, and each byte is either \\x00 or \\x01 (in documentation those bytes are called bits). Parsing building and size are deferred to subcon, although size gets divided by 8. :param subcon: Construct instance, any field that works with bits (like BitsInteger) or is bit-byte agnostic (like Struct or Flag) See :class:`~construct.core.Transformed` and :class:`~construct.core.Restreamed` for raisable exceptions. Example:: >>> d = Bitwise(Struct( ... 'a' / Nibble, ... 'b' / Bytewise(Float32b), ... 'c' / Padding(4), ... )) >>> d.parse(bytes(5)) Container(a=0)(b=0.0)(c=None) >>> d.sizeof() 5 rKcSs|dSNrr nr r r r)zBitwise..cj|ddSNTr+rr*r+rr r r'rzBitwise.._emitseqcrrr.rrr r r-rz#Bitwise.._emitprimitivetypecrrr(rrr r r0rzBitwise.._emitfulltype) r Transformed bytes2bits bits2bytesr Restreamedr'r-r0rrmacror'r-r0r rr Bitwise    rc sz}tt|dt|d}Wnty&ttdtddd}Ynwfdd}fdd}fd d }||_||_||_|S) a Converts the bitstream back to normal byte stream. Must be used within :class:`~construct.core.Bitwise`. Parsing building and size are deferred to subcon, although size gets multiplied by 8. :param subcon: Construct instance, any field that works with bytes or is bit-byte agnostic See :class:`~construct.core.Transformed` and :class:`~construct.core.Restreamed` for raisable exceptions. Example:: >>> d = Bitwise(Struct( ... 'a' / Nibble, ... 'b' / Bytewise(Float32b), ... 'c' / Padding(4), ... )) >>> d.parse(bytes(5)) Container(a=0)(b=0.0)(c=None) >>> d.sizeof() 5 rrKcSs|dSrr rr r r r)rzBytewise..crNFrrrrr r r'rzBytewise.._emitseqcrrrrrr r r-rz$Bytewise.._emitprimitivetypecrrrrrr r r0rzBytewise.._emitfulltype) rrrrr rr'r-r0rr rr Bytewiserrcrs) FormatFieldae Field that uses `struct` module to pack and unpack CPU-sized integers and floats. This is used to implement most Int* Float* fields, but for example cannot pack 24-bit integers, which is left to :class:`~construct.core.BytesInteger` class. See `struct module `_ documentation for instructions on crafting format strings. Parses into an integer. Builds from an integer into specified byte count and endianness. Size is determined by `struct` module according to specified format string. :param endianity: string, character like: < > = :param format: string, character like: f d B H L Q b h l q :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises FormatFieldError: wrong format string, or struct.(un)pack complained about the value Example:: >>> d = FormatField(">", "H") or Int16ub >>> d.parse(b"\x01\x00") 256 >>> d.build(256) b"\x01\x00" >>> d.sizeof() 2 csh|tdvr td||tdvrtd|tt||||_t|||_t |||_ dS)Nz=<>zendianity must be like: = < > fdBHLQbhlqz(format must be like: f d B H L Q b h l q) listrrPrrVfmtstrstructcalcsizer2rpacker)rU endianityr rRr r rVs     zFormatField.__init__cCs<t||j}z |j|dWStytd|jw)Nrzstruct %r error during parsing)r4r2runpackr/rrrjr r r rs  zFormatField._parsecCsDz|j|}Wntytd|j|fwt|||j|S)Nz/struct %r error during building, given value %r)rpackr/rrr>r2)rUrr1rrr3r r r r s zFormatField._buildcC|jSr&r~rr r r rzFormatField._sizeofcCs.d|}|d||jfd||jfS)Nzformatfield_%sz%s = struct.Struct(%r)z %s.unpack(read_bytes(io, %s))[0])rZrjrr2rUrfnamer r r rs zFormatField._emitparsecCs|j\}}|}|dkp|dkotjdk}|dvr>|r,|r J|r$Jdd|jfSd|r1dnd |j|r:d fSd fS|d vrS|rFJd |j|rOd fSd fSdS)N<=littlebhlqBHLQb%sr%s%s%sr_ulebefdzf%s%s)rislowersys byteorderr2)rUr*r+rr signedswappedr r r r-s $zFormatField._emitprimitivetype rrrrKrVrrrrr-rYr r rRr rs  rcJeZdZdZdfdd ZddZddZd d Zd d Zd dZ Z S) BytesIntegera Field that packs arbitrarily large integers. Some Int24* fields use this class. Parses into an integer. Builds from an integer into specified byte count and endianness. Size is specified in ctor. Analog to :class:`~construct.core.BitsInteger` that operates on bits. In fact, ``BytesInteger(n)`` is equivalent to ``Bitwise(BitsInteger(8*n))`` and ``BitsInteger(n)`` is equivalent to ``Bytewise(BytesInteger(n//8)))`` . :param length: integer or context lambda, number of bytes in the field :param signed: bool, whether the value is signed (two's complement), default is False (unsigned) :param swapped: bool, whether to swap byte order (little endian), default is False (big endian) :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises IntegerError: lenght is negative, given a negative value when field is not signed, or not an integer Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> d = BytesInteger(4) or Int32ub >>> d.parse(b"abcd") 1633837924 >>> d.build(1) b'\x00\x00\x00\x01' >>> d.sizeof() 4 Fcror&)rPrrVr2rrrUr2rrrRr r rVFrqzBytesInteger.__init__cCsP|j}t|r ||}|dkrtdt||}|jr"|ddd}t||jS)Nrlength must be non-negative)r2rrr4r bytes2integerrrUr1rrr2r3r r r rLs  zBytesInteger._parsecCst|ts td|f|dkr|jstd|f|j}t|r%||}|dkr-tdt||}|jr<|ddd}t||||S)Nvalue %r is not an integerr-value %r is negative, but field is not signedrr) r: integertypesrrr2rrwrr>rxr r r rWs   zBytesInteger._buildc C8z|j}t|r ||}|WSttfytdwryr2rr{r|r rUrrr2r r r rgzBytesInteger._sizeofcCd|j|jrdnd|jfS)Nz'bytes2integer(read_bytes(io, %s)%s, %s)[::-1]rmr2rrrr r r rpzBytesInteger._emitparsecCsP|r|jrJ|jr Jdd|jfSd|jrdnd|j|jr$dfSdfS)Nrrrr_rrrrrr2r1r r r r-ss   (zBytesInteger._emitprimitivetypeFFrr r rRr r*s  rcr) BitsIntegera Field that packs arbitrarily large (or small) integers. Some fields (Bit Nibble Octet) use this class. Must be enclosed in :class:`~construct.core.Bitwise` context. Parses into an integer. Builds from an integer into specified bit count and endianness. Size (in bits) is specified in ctor. Note that little-endianness is only defined for multiples of 8 bits. Analog to :class:`~construct.core.BytesInteger` that operates on bytes. In fact, ``BytesInteger(n)`` is equivalent to ``Bitwise(BitsInteger(8*n))`` and ``BitsInteger(n)`` is equivalent to ``Bytewise(BytesInteger(n//8)))`` . :param length: integer or context lambda, number of bits in the field :param signed: bool, whether the value is signed (two's complement), default is False (unsigned) :param swapped: bool, whether to swap byte order (little endian), default is False (big endian) :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises IntegerError: lenght is negative, given a negative value when field is not signed, or not an integer Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> d = Bitwise(BitsInteger(8)) or Bitwise(Octet) >>> d.parse(b"\x10") 16 >>> d.build(255) b'\xff' >>> d.sizeof() 1 Fcror&)rPrrVr2rrrrRr r rVrqzBitsInteger.__init__cCsZ|j}t|r ||}|dkrtdt||}|jr'|d@r#tdt|}t||jS)Nrr9little-endianness is only defined for multiples of 8 bits)r2rrr4r swapbytes bits2integerrrr r r rs  zBitsInteger._parsecCst|ts td|f|dkr|jstd|f|j}t|r%||}|dkr-tdt||}|jrA|d@r=tdt|}t ||||S)Nrrrrrr) r:rrrr2r integer2bitsrrr>rxr r r rs   zBitsInteger._buildc Crryrrr r r rrzBitsInteger._sizeofcCr)Nz&bits2integer(read_bytes(io, %s)%s, %s)rrmrrr r r rrzBitsInteger._emitparsecCs |jrJ|jr Jd|jfSNrrr1r r r r-s   zBitsInteger._emitprimitivetyperrr r rRr r|s  rcCtdS)z>A 1-bit integer, must be enclosed in a Bitwise (eg. BitStruct)rKrr r r r BitrcCr)z>A 4-bit integer, must be enclosed in a Bitwise (eg. BitStruct)rr r r r NibblerrcCr)z>A 8-bit integer, must be enclosed in a Bitwise (eg. BitStruct)rrr r r r OctetrrcC tddS)z"Unsigned, big endian 8-bit integer>Brr r r r Int8ub rcCr)z#Unsigned, big endian 16-bit integerrHrr r r r Int16ubrrcCr)z#Unsigned, big endian 32-bit integerrLrr r r r Int32ubrrcCr)z#Unsigned, big endian 64-bit integerrQrr r r r Int64ubrrcCr)z Signed, big endian 8-bit integerrbrr r r r Int8sbrrcCr)z!Signed, big endian 16-bit integerrhrr r r r Int16sbrrcCr)z!Signed, big endian 32-bit integerrrrr r r r Int32sbrrcCr)z!Signed, big endian 64-bit integerrqrr r r r Int64sbrrcCr)z%Unsigned, little endian 8-bit integerrrrr r r r Int8ulrrcCr)z&Unsigned, little endian 16-bit integerrrrr r r r Int16ulrrcCr)z&Unsigned, little endian 32-bit integerrrrr r r r Int32ulrrcCr)z&Unsigned, little endian 64-bit integerrrrr r r r Int64ul rrcCr)z#Signed, little endian 8-bit integerrrrr r r r Int8slrrcCr)z$Signed, little endian 16-bit integerrrrr r r r Int16slrrcCr)z$Signed, little endian 32-bit integerrrrr r r r Int32slrrcCr)z$Signed, little endian 64-bit integerrrrr r r r Int64slrrcCr)z(Unsigned, native endianity 8-bit integerrrrr r r r Int8un"rrcCr)z)Unsigned, native endianity 16-bit integerrrrr r r r Int16un&rrcCr)z)Unsigned, native endianity 32-bit integerrrrr r r r Int32un*rrcCr)z)Unsigned, native endianity 64-bit integerrrrr r r r Int64un.rrcCr)z&Signed, native endianity 8-bit integerrrrr r r r Int8sn3rrcCr)z'Signed, native endianity 16-bit integerrrrr r r r Int16sn7rrcCr)z'Signed, native endianity 32-bit integerrrrr r r r Int32sn;rrcCr)z'Signed, native endianity 64-bit integerrrrr r r r Int64sn?rrcCr)z-Big endian, 32-bit IEEE floating point numberrrrr r r r Float32bIrrcCr)z0Little endian, 32-bit IEEE floating point numberrrrr r r r Float32lMrrcCr)z3Native endianity, 32-bit IEEE floating point numberrrrr r r r Float32nQrrcCr)z-Big endian, 64-bit IEEE floating point numberrr~rr r r r Float64bVrrcCr)z0Little endian, 64-bit IEEE floating point numberrr~rr r r r Float64lZrr cCr)z3Native endianity, 64-bit IEEE floating point numberrr~rr r r r Float64n^rr rcCtddddS)zFA 3-byte big-endian unsigned integer, as used in ancient file formats.r%Frrrr r r r Int24ubhrcCtddddS)zIA 3-byte little-endian unsigned integer, as used in ancient file formats.r%FTr r r r r r Int24ullrrcCtddtdS)zIA 3-byte native-endian unsigned integer, as used in ancient file formats.r%Fr rnativer r r r Int24unprrcCr)zDA 3-byte big-endian signed integer, as used in ancient file formats.r%TFr r r r r r Int24sbtrrcCr )zGA 3-byte little-endian signed integer, as used in ancient file formats.r%Tr r r r r r Int24slxrrcCr)zGA 3-byte native-endian signed integer, as used in ancient file formats.r%Tr rr r r r Int24sn|rrc@s(eZdZdZddZddZddZdS) VarInta# VarInt encoded integer. Each 7 bits of the number are encoded in one byte of the stream, where leftmost bit (MSB) is unset when byte is terminal. Scheme is defined at Google site related to `Protocol Buffers `_. Can only encode non-negative numbers. Parses into an integer. Builds from an integer. Size is undefined. :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises IntegerError: given a negative value, or not an integer Example:: >>> VarInt.build(1) b'\x01' >>> VarInt.build(2**100) b'\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x04' cCsPg} tt|d}||d@|d@snqd}t|D]}|d>|B}q|S)NTrKrr)byte2intr4rjreversed)rUr1rraccrnumr r r rs z VarInt._parsecCspt|ts td|dkrtd|f|}|dkr.t|td|d@Bd|dL}|dkst|t|d|S)Nzvalue is not an integerrz,varint cannot build from negative number: %rrrrKr)r:rrr>int2byte)rUrr1rrr+r r r rs z VarInt._buildcCr)Nvlq_base128_ler r1r r r r-rbzVarInt._emitprimitivetypeN)rrrrKrrr-r r r r rs  rrKrFr)asciiutf8utf_8u8utf16utf_16u16 utf_16_be utf_16_leutf32utf_32u32 utf_32_be utf_32_lecCs4|dd}|tvrtd|tftt|S)rrurtzencoding %r not found among %r)rvlowerpossiblestringencodingsrbytesencodingr r r encodingunits r5c8eZdZdZfddZddZddZdd ZZS) StringEncodedrcs&tt|||std||_dS)Nz)String* classes require explicit encoding)rPr7rVrr4)rUrr4rRr r rVs zStringEncoded.__init__cCs ||jSr&)rr4rar r r r\rzStringEncoded._decodecCs*t|ts td|dkrdS||jS)Nz/string encoding failed, expected unicode stringrm)r:unicodestringtyperrr4rar r r r^s  zStringEncoded._encodecCd|j||jfS)Nz(%s).decode(%r))rrr4rr r r rzStringEncoded._emitparse rrrrKrVr\r^rrYr r rRr r7s  r7cs4tttttd}fdd}||_|S)u+ Configurable, fixed-length or variable-length string field. When parsing, the byte string is stripped of null bytes (per encoding unit), then decoded. Length is an integer or context lambda. When building, the string is encoded and then padded to specified length. If encoded string is larger then the specified length, it fails with PaddingError. Size is same as length parameter. .. warning:: PaddedString and CString only support encodings explicitly listed in :class:`~construct.core.possiblestringencodings` . :param length: integer or context lambda, length in bytes (not unicode characters) :param encoding: string like: utf8 utf16 utf32 ascii :raises StringError: building a non-unicode string :raises StringError: selected encoding is not on supported list Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> d = PaddedString(10, "utf8") >>> d.build(u"Афон") b'\xd0\x90\xd1\x84\xd0\xbe\xd0\xbd\x00\x00' >>> d.parse(_) u'Афон' padcstddS)Nstrz)rr/r4rrr4r2r r r0rz#PaddedString.._emitfulltype)r7 FixedSized NullStrippedrr5r0)r2r4rr0r r@r PaddedStringsrCcs(ttt}fdd}||_|S)u Length-prefixed string. The length field can be variable length (such as VarInt) or fixed length (such as Int64ub). :class:`~construct.core.VarInt` is recommended when designing new protocols. Stored length is in bytes, not characters. Size is not defined. :param lengthfield: Construct instance, field used to parse and build the length (like VarInt Int64ub) :param encoding: string like: utf8 utf16 utf32 ascii :raises StringError: building a non-unicode string Example:: >>> d = PascalString(VarInt, "utf8") >>> d.build(u"Афон") b'\x08\xd0\x90\xd1\x84\xd0\xbe\xd0\xbd' >>> d.parse(_) u'Афон' cs$td||dtddddgS)N lengthfieldrr/r3str)rrr/r4rr.rr4rDr r r'szPascalString.._emitseq)r7Prefixedrr')rDr4rr'r rHr PascalStringsrJcs,ttttd}fdd}||_|S)u@ String ending in a terminating null byte (or null bytes in case of UTF16 UTF32). .. warning:: String and CString only support encodings explicitly listed in :class:`~construct.core.possiblestringencodings` . :param encoding: string like: utf8 utf16 utf32 ascii :raises StringError: building a non-unicode string :raises StringError: selected encoding is not on supported list Example:: >>> d = CString("utf8") >>> d.build(u"Афон") b'\xd0\x90\xd1\x84\xd0\xbe\xd0\xbd\x00' >>> d.parse(_) u'Афон' )termcs tddS)Nr?)r/r4rrr3r r r0-rzCString.._emitfulltype)r7NullTerminatedrr5r0r4rr0r r3r CStrings rNcs tt}fdd}||_|S)u  String that reads entire stream until EOF, and writes a given string as-is. Analog to :class:`~construct.core.GreedyBytes` but also applies unicode-to-bytes encoding. :param encoding: string like: utf8 utf16 utf32 ascii :raises StringError: building a non-unicode string :raises StreamError: stream failed when reading until EOF Example:: >>> d = GreedyString("utf8") >>> d.build(u"Афон") b'\xd0\x90\xd1\x84\xd0\xbe\xd0\xbd' >>> d.parse(_) u'Афон' cstdddS)NTrF)rr/r4rrr3r r r0Erz#GreedyString.._emitfulltype)r7rr0rMr r3r GreedyString3s  rOc@rh) Flaga One byte (or one bit) field that maps to True or False. Other non-zero bytes are also considered True. Size is defined as 1. :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes Example:: >>> Flag.parse(b"\x01") True >>> Flag.build(True) b'\x01' cCst|ddkS)NrK)r4rr r r r]rz Flag._parsecCst||rdndd|S)NrQrK)r>rr r r r`sz Flag._buildcCrrJr rr r r rdrbz Flag._sizeofcCr)Nz(read_bytes(io, 1) != b'\x00')r rr r r rgrbzFlag._emitparsecCst|rdddSdddS)Nb1u1rP)r/_construct_renderrr1r r r r0jzFlag._emitfulltypeN) rrrrKrrrrr0r r r r rPNs  rPc@seZdZdZdS) EnumIntegerrN)rrrrKr r r r rWnsrWc@s,eZdZdZddZddZeddZdS) EnumIntegerStringrcCsd|jt|fS)NzEnumIntegerString.new(%s, %s))intvaluerFrrTr r r rvrzEnumIntegerString.__repr__cCrr&)rYrTr r r __int__yrzEnumIntegerString.__int__cCst|}||_|Sr&)rXrY)rY stringvalueretr r r new|szEnumIntegerString.newN)rrrrKrrZ staticmethodr]r r r r rXss rXcrs)Enuma Translates unicode label names to subcon values, and vice versa. Parses integer subcon, then uses that value to lookup mapping dictionary. Returns an integer-convertible string (if mapping found) or an integer (otherwise). Building is a reversed process. Can build from an integer flag or string label. Size is same as subcon, unless it raises SizeofError. There is no default parameter, because if no mapping is found, it parses into an integer without error. This class supports enum34 module. See examples. This class supports exposing member labels as attributes, as integer-convertible strings. See examples. :param subcon: Construct instance, subcon to map to/from :param \*merge: optional, list of enum.IntEnum and enum.IntFlag instances, to merge labels and values from :param \*\*mapping: dict, mapping string names to values :raises MappingError: building from string but no mapping found Example:: >>> d = Enum(Byte, one=1, two=2, four=4, eight=8) >>> d.parse(b"\x01") 'one' >>> int(d.parse(b"\x01")) 1 >>> d.parse(b"\xff") 255 >>> int(d.parse(b"\xff")) 255 >>> d.build(d.one or "one" or 1) b'\x01' >>> d.one 'one' import enum class E(enum.IntEnum or enum.IntFlag): one = 1 two = 2 Enum(Byte, E) <--> Enum(Byte, one=1, two=2) FlagsEnum(Byte, E) <--> FlagsEnum(Byte, one=1, two=2) csptt|||D] }|D]}|j||j<qq dd|D|_dd|D|_dd|D|_dS)NcSsi|] \}}t|||qSr rXr]rxr r r r{z!Enum.__init__..cSsi|] \}}|t||qSr r`rxr r r r{racSi|]\}}||qSr r rxr r r r{) rPr_rVrrr} encmapping decmapping ksymapping)rUrmergemappingenum enumentryrRr r rVsz Enum.__init__cCs||jvr |j|j|Str&)rdrer|r3r r r __getattr__s zEnum.__getattr__cCs(z|j|WStyt|YSwr&)rer{rWrar r r r\s    z Enum._decodecCs:zt|tr |WS|j|WStytd|fw)Nz"building failed, no mapping for %r)r:rrdr{rrar r r r^s   z Enum._encodecCs4d|}|d||jfd|j||fS)N factory_%s%s = %rz.reuse(%s, lambda x: %s.get(x, EnumInteger(x)))rZrjrerrrr r r r zEnum._emitparsecCsd|}|j|j|<|S)Nzenum_%s)rZrfrr)rUr*r+rr r r r-s  zEnum._emitprimitivetype) rrrrKrVrkr\r^rr-rYr r rRr r_s + r_c@rc)BitwisableStringrcCstd||S)Nz{}|{})rpr r;r r r __or__rTzBitwisableString.__or__N)rrrrKrqr r r r rps rpcrs) FlagsEnuma Translates unicode label names to subcon integer (sub)values, and vice versa. Parses integer subcon, then creates a Container, where flags define each key. Builds from a container by bitwise-oring of each flag if it matches a set key. Can build from an integer flag or string label directly, as well as | concatenations thereof (see examples). Size is same as subcon, unless it raises SizeofError. This class supports enum34 module. See examples. This class supports exposing member labels as attributes, as bitwisable strings. See examples. :param subcon: Construct instance, must operate on integers :param \*merge: optional, list of enum.IntEnum and enum.IntFlag instances, to merge labels and values from :param \*\*flags: dict, mapping string names to integer values :raises MappingError: building from object not like: integer string dict :raises MappingError: building from string but no mapping found Can raise arbitrary exceptions when computing | and & and value is non-integer. Example:: >>> d = FlagsEnum(Byte, one=1, two=2, four=4, eight=8) >>> d.parse(b"\x03") Container(one=True, two=True, four=False, eight=False) >>> d.build(dict(one=True,two=True)) b'\x03' >>> d.build(d.one|d.two or "one|two" or 1|2) b'\x03' import enum class E(enum.IntEnum or enum.IntFlag): one = 1 two = 2 Enum(Byte, E) <--> Enum(Byte, one=1, two=2) FlagsEnum(Byte, E) <--> FlagsEnum(Byte, one=1, two=2) csNtt|||D] }|D]}|j||j<qq ||_dd|D|_dS)NcSrbr r rxr r r r{ rcz&FlagsEnum.__init__..)rPrrrVrrflagsr} reverseflags)rUrrgrsrirjrRr r rVszFlagsEnum.__init__cCs||jvr t|Str&)rsrpr|r3r r r rks zFlagsEnum.__getattr__cCs8t}d|_|jD]\}}||@|k|t|<q |SNT)r _flagsenumrsr}rp)rUrrrr_rrr r r r\s zFlagsEnum._decodecCszOt|tr |WSt|tr(d}|dD]}|}|r$||j|O}q|WSt|trId}|D]\}}|dsE|rE||j|O}q3|WSt d|ft y]t d|fw)Nr|rtz#building failed, unknown object: %rz"building failed, unknown label: %r) r:rr:splitr\rsrr} startswithrr{)rUrrrrsrrr r r r^s.     zFlagsEnum._encodecCs*d|j|ddd|jDfS)Nz"reuse(%s, lambda x: Container(%s)), css |] \}}d||fVqdS)z%s=bool(x & %s)Nr rxr r r re2z'FlagsEnum._emitparse..)rrrirsr}rr r r r1*zFlagsEnum._emitparsec CsX|jd}g}t|D]}d|>}|j|d|}|t|dt|ddq |S)NrrKz unknown_%srSrP)rr/docrU)rrrangertgetrjrhex)rUr*r+ bitstotalrirrr r r r'4s zFlagsEnum._emitseq) rrrrKrVrkr\r^rr'rYr r rRr rrs &rrcr6) MappingaY Adapter that maps objects to other objects. Translates objects after parsing and before building. Can for example, be used to translate between enum34 objects and strings, but Enum class supports enum34 already and is recommended. :param subcon: Construct instance :param mapping: dict, for encoding (building) mapping, the reversed is used for parsing mapping :raises MappingError: parsing or building but no mapping found Example:: >>> x = object >>> d = Mapping(Byte, {x:0}) >>> d.parse(b"\x00") x >>> d.build(x) b'\x00' cs.tt||dd|D|_||_dS)NcSrbr r rxr r r r{Srcz$Mapping.__init__..)rPrrVr}rerd)rUrrhrRr r rVQs zMapping.__init__c C.z|j|WSttfytd|fw)Nz*parsing failed, no decoding mapping for %r)rer{rOrrar r r r\V  zMapping._decodec Cr)Nz+building failed, no encoding mapping for %r)rdr{rOrrar r r r^\rzMapping._encodecCs4d|}|d||jfd||j|fS)Nrlrmz%s[%s]rnrr r r rbrozMapping._emitparser<r r rRr r>s  rcPeZdZdZfddZddZddZdd Zd d Zd d Z ddZ Z S)ra  Sequence of usually named constructs, similar to structs in C. The members are parsed and build in the order they are defined. If a member is anonymous (its name is None) then it gets parsed and the value discarded, or it gets build from nothing (from None). Some fields do not need to be named, since they are built without value anyway. See: Const Padding Check Error Pass Terminated Seek Tell for examples of such fields. :class:`~construct.core.Embedded` fields do not need to (and should not) be named. Operator + can also be used to make Structs (although not recommended). Parses into a Container (dict with attribute and key access) where keys match subcon names. Builds from a dict (not necessarily a Container) where each member gets a value from the dict matching the subcon name. If field has build-from-none flag, it gets build even when there is no matching entry in the dict. Size is the sum of all subcon sizes, unless any subcon raises SizeofError. This class does context nesting, meaning its members are given access to a new dictionary where the "_" entry points to the outer context. When parsing, each member gets parsed and subcon parse return value is inserted into context under matching key only if the member was named. When building, the matching entry gets inserted into context before subcon gets build, and if subcon build returns a new value (not None) that gets replaced in the context. This class supports embedding. :class:`~construct.core.Embedded` semantics dictate, that during instance creation (in ctor), each field is checked for embedded flag, and its subcon members are merged. This changes behavior of some code examples. Only few classes are supported: Struct Sequence FocusedSeq Union LazyStruct, although those can be used interchangably (a Struct can embed a Sequence, or rather its members). This class exposes subcons as attributes. You can refer to subcons that were inlined (and therefore do not exist as variable in the namespace) by accessing the struct attributes, under same name. Also note that compiler does not support this feature. See examples. This class exposes subcons in the context. You can refer to subcons that were inlined (and therefore do not exist as variable in the namespace) within other inlined fields using the context. Note that you need to use a lambda (`this` expression is not supported). Also note that compiler does not support this feature. See examples. This class supports stopping. If :class:`~construct.core.StopIf` field is a member, and it evaluates its lambda as positive, this class ends parsing or building as successful without processing further fields. :param \*subcons: Construct instances, list of members, some can be anonymous :param \*\*subconskw: Construct instances, list of members (requires Python 3.6) :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises KeyError: building a subcon but found no corresponding key in dictionary Example:: >>> d = Struct("num"/Int8ub, "data"/Bytes(this.num)) >>> d.parse(b"\x04DATA") Container(num=4)(data=b"DATA") >>> d.build(dict(num=4, data=b"DATA")) b"\x04DATA" >>> d = Struct(Const(b"MZ"), Padding(2), Pass, Terminated) >>> d.build({}) b'MZ\x00\x00' >>> d.parse(_) Container() >>> d.sizeof() 4 >>> d = Struct( ... "animal" / Enum(Byte, giraffe=1), ... ) >>> d.animal.giraffe 'giraffe' >>> d = Struct( ... "count" / Byte, ... "data" / Bytes(lambda this: this.count - this._subcons.count.sizeof()), ... ) >>> d.build(dict(count=3, data=b"12")) b'\x0312' Alternative syntax (not recommended): >>> ("a"/Byte + "b"/Byte + "c"/Byte + "d"/Byte) Alternative syntax, but requires Python 3.6 or any PyPy: >>> Struct(a=Byte, b=Byte, c=Byte, d=Byte) cftt|t|tdd|D}t||_tdd|jD|_t dd|jD|_ dS)Ncs|] \}}||VqdSr&r rxr r r rerfz"Struct.__init__..cs |] }|jr|j|fVqdSr&rr^rr r r rer{cs|]}|jVqdSr&rrr r r re) rPrrVrr}rrr_subconsallrrUr subconskwrRr r rV  zStruct.__init__cC||jvr |j|Str&rr|r3r r r rk  zStruct.__getattr__c Cst}||_t||jd|j|j|j|j||ddd }|jd||_ |j D]#}z| |||}|j r>|||j <|||j <Wq't yJY|Sw|SN_index rtr_rootrrrr_iorr)rrrrrrrrrtrrrrr)rUr1rrrrsubobjr r r rs ,    z Struct._parsec C|durt}t||jd|j|j|j|j||ddd }|jd||_| ||j D]7}z*|j r;||j d}n||j }|j rH|||j <| ||||}|j rX|||j <Wq-tydY|Sw|SrrrrrrrrrtrrrrrrrrUrr1rrrrr`r r r rs*,      z Struct._buildc ttjdjjj|jdddd jd_zt fdd|j DWSt t fy9t dw)Nrrrc3|] }|VqdSr&rrrrr r rerfz!Struct._sizeof..rzrrrrrrrrtrsumrr{r|r rr rr r,zStruct._sizeofcCsjd|}d|f}|jD]}|d|jrd|j|jfnd||f7}q|d7}||d|fS)Nzparse_struct_%sae def %s(io, this): result = Container() this = Container(_ = this, _params = this['_params'], _root = None, _parsing = True, _building = False, _sizing = False, _subcons = None, _io = io, _index = this.get('_index', None)) this['_root'] = this['_'].get('_root', this) try: z& %s%s zresult[%r] = this[%r] = rm pass except StopFieldError: pass return result %s(io, this))rZrrrrjrUrrrkrr r r rs  "  zStruct._emitparsecfdd|jDS)Ncg|]}|qSr rrr+r*r r r`raz#Struct._emitseq..rr1r rr r'r;zStruct._emitseq rrrrKrVrkrrrrr'rYr r rRr rks <rcr)ra Sequence of usually un-named constructs. The members are parsed and build in the order they are defined. If a member is named, its parsed value gets inserted into the context. This allows using members that refer to previous members. :class:`~construct.core.Embedded` fields do not need to (and should not) be named. Operator >> can also be used to make Sequences (although not recommended). Parses into a ListContainer (list with pretty-printing) where values are in same order as subcons. Builds from a list (not necessarily a ListContainer) where each subcon is given the element at respective position. Size is the sum of all subcon sizes, unless any subcon raises SizeofError. This class does context nesting, meaning its members are given access to a new dictionary where the "_" entry points to the outer context. When parsing, each member gets parsed and subcon parse return value is inserted into context under matching key only if the member was named. When building, the matching entry gets inserted into context before subcon gets build, and if subcon build returns a new value (not None) that gets replaced in the context. This class supports embedding. :class:`~construct.core.Embedded` semantics dictate, that during instance creation (in ctor), each field is checked for embedded flag, and its subcon members are merged. This changes behavior of some code examples. Only few classes are supported: Struct Sequence FocusedSeq Union LazyStruct, although those can be used interchangably (a Struct can embed a Sequence, or rather its members). This class exposes subcons as attributes. You can refer to subcons that were inlined (and therefore do not exist as variable in the namespace) by accessing the struct attributes, under same name. Also note that compiler does not support this feature. See examples. This class exposes subcons in the context. You can refer to subcons that were inlined (and therefore do not exist as variable in the namespace) within other inlined fields using the context. Note that you need to use a lambda (`this` expression is not supported). Also note that compiler does not support this feature. See examples. This class supports stopping. If :class:`~construct.core.StopIf` field is a member, and it evaluates its lambda as positive, this class ends parsing or building as successful without processing further fields. :param \*subcons: Construct instances, list of members, some can be named :param \*\*subconskw: Construct instances, list of members (requires Python 3.6) :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises KeyError: building a subcon but found no corresponding key in dictionary Example:: >>> d = Sequence(Byte, Float32b) >>> d.build([0, 1.23]) b'\x00?\x9dp\xa4' >>> d.parse(_) [0, 1.2300000190734863] # a ListContainer >>> d = Sequence( ... "animal" / Enum(Byte, giraffe=1), ... ) >>> d.animal.giraffe 'giraffe' >>> d = Sequence( ... "count" / Byte, ... "data" / Bytes(lambda this: this.count - this._subcons.count.sizeof()), ... ) >>> d.build([3, b"12"]) b'\x0312' Alternative syntax (not recommended): >>> (Byte >> "Byte >> "c"/Byte >> "d"/Byte) Alternative syntax, but requires Python 3.6 or any PyPy: >>> Sequence(a=Byte, b=Byte, c=Byte, d=Byte) cr)Ncsrr&r rxr r r re1rfz$Sequence.__init__..csrr&rrr r r re3r{csrr&rrr r r re4r) rPrrVrr}rrrrrrrrRr r rV/rzSequence.__init__cCrr&rr3r r r rk6rzSequence.__getattr__c Cst}t||jd|j|j|j|j||ddd }|jd||_ t |j D]%\}}z| |||}| ||jr?|||j<Wq&tyKY|Sw|Sr) ListContainerrrrrrrrrtr enumeraterrrjrr)rUr1rrrrrrr r r r;s,   zSequence._parsec Cs|durtdd|jD}t||jd|j|j|j|j||ddd }|j d||_ t}t t |j|D]0\}\}}z|j rF|||j <|||||} || |j r[| ||j <Wq7tygY|Sw|S)NcSsg|]}dqSr&r rr r r r`Ksz#Sequence._build..rrr)rrrrrrrrrrtrrziprrrjr) rUrr1rrretlistrrrr`r r r rIs&,    zSequence._buildc r)Nrrrc3rr&rrrr r re`rfz#Sequence._sizeof..rzrrr rr r\rzSequence._sizeofcCsfd|}d|f}|jD]}|d||7}|jr$|d|jf7}q|d7}||d|fS)Nzparse_sequence_%sai def %s(io, this): result = ListContainer() this = Container(_ = this, _params = this['_params'], _root = None, _parsing = True, _building = False, _sizing = False, _subcons = None, _io = io, _index = this.get('_index', None)) this['_root'] = this['_'].get('_root', this) try: z3 result.append(%s) z; this[%r] = result[-1] rr)rZrrrrjrr r r rds    zSequence._emitparsecr)Ncrr rrrr r r`raz%Sequence._emitseq..rr1r rr r'~r;zSequence._emitseqrr r rRr rs 2rcr)rHa Homogenous array of elements, similar to C# generic T[]. Parses into a ListContainer (a list). Parsing and building processes an exact amount of elements. If given list has more or less than count elements, raises RangeError. Size is defined as count multiplied by subcon size, but only if subcon is fixed size. Operator [] can be used to make Array instances (recommended syntax). :param count: integer or context lambda, strict amount of elements :param subcon: Construct instance, subcon to process individual elements :param discard: optional, bool, if set then parsing returns empty list :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises RangeError: specified count is not valid :raises RangeError: given object has different length than specified count Can propagate any exception from the lambdas, possibly non-ConstructError. Example:: >>> d = Array(5, Byte) or Byte[5] >>> d.build(range(5)) b'\x00\x01\x02\x03\x04' >>> d.parse(_) [0, 1, 2, 3, 4] Fc tt||||_||_dSr&)rPrHrVrIdiscard)rUrIrrrRr r rV zArray.__init__cCsj|j}t|r ||}d|kstd|ft}t|D]}||_|j|||}|js2| |q|S)Nrinvalid count %s) rIrrrr~rrrrrj)rUr1rrrIrrer r r rs  z Array._parsec C|j}t|r ||}d|kstd|ft||ks&td|t|ft}t|D]\}}||_|j||||} | | q-|SNrrzexpected %d elements, found %d rIrrr0rrrrrrj rUrr1rrrIrrrr`r r r rs  z Array._buildc CHz |j}t|r ||}Wn ttfytdw||j||SryrIrr{r|r rrrUrrrIr r r rsz Array._sizeofcCr:)NzHListContainer((this.__setitem__('_index',i),(%s))[1] for i in range(%s)))rrrIrr r r rr;zArray._emitparsecCst|j||d|jdS)Nexpr)r/repeat repeat_expr)rrr.rIr1r r r r0rVzArray._emitfulltypeFrr r rRr rHs rHcBeZdZdZd fdd ZddZddZd d Zd d ZZ S) GreedyRangeag Homogenous array of elements, similar to C# generic IEnumerable, but works with unknown count of elements by parsing until end of stream. Parses into a ListContainer (a list). Parsing stops when an exception occured when parsing the subcon, either due to EOF or subcon format not being able to parse the data. Either way, when GreedyRange encounters either failure it seeks the stream back to a position after last successful subcon parsing. Builds from enumerable, each element as-is. Size is undefined. This class supports stopping. If :class:`~construct.core.StopIf` field is a member, and it evaluates its lambda as positive, this class ends parsing or building as successful without processing further fields. :param subcon: Construct instance, subcon to process individual elements :param discard: optional, bool, if set then parsing returns empty list :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises StreamError: stream is not seekable and tellable Can propagate any exception from the lambdas, possibly non-ConstructError. Example:: >>> d = GreedyRange(Byte) >>> d.build(range(8)) b'\x00\x01\x02\x03\x04\x05\x06\x07' >>> d.parse(_) [0, 1, 2, 3, 4, 5, 6, 7] Fctt||||_dSr&)rPrrVr)rUrrrRr r rV zGreedyRange.__init__cCst}z!tD]}||_t|}|j|||}|js!||qW|St y.Y|St y5t yCt ||Y|Swr&) r itertoolsrIrrDrrrrjrrr/rB)rUr1rrrrrGrr r r rs(      zGreedyRange._parsec CsVz t}t|D]\}}||_|j||||}||q|WSty*YdSwr&)rrrrrrjr) rUrr1rrrrrr`r r r r s  zGreedyRange._buildcCrr&rrr r r r rbzGreedyRange._sizeofcCst|j||ddS)Neos)r/r)rrr.r1r r r r0 r;zGreedyRange._emitfulltyper rrrrKrVrrrr0rYr r rRr rs rcr) RepeatUntila Homogenous array of elements, similar to C# generic IEnumerable, that repeats until the predicate indicates it to stop. Note that the last element (that predicate indicated as True) is included in the return list. Parse iterates indefinately until last element passed the predicate. Build iterates indefinately over given list, until an element passed the precicate (or raises RepeatError if no element passed it). Size is undefined. :param predicate: lambda that takes (obj, list, context) and returns True to break or False to continue (or a truthy value) :param subcon: Construct instance, subcon used to parse and build each element :param discard: optional, bool, if set then parsing returns empty list :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises RepeatError: consumed all elements in the stream but neither passed the predicate Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> d = RepeatUntil(lambda x,lst,ctx: x > 7, Byte) >>> d.build(range(20)) b'\x00\x01\x02\x03\x04\x05\x06\x07\x08' >>> d.parse(b"\x01\xff\x02") [1, 255] >>> d = RepeatUntil(lambda x,lst,ctx: lst[-2:] == [0,0], Byte) >>> d.parse(b"\x01\x00\x00\xff") [1, 0, 0] Fcrr&)rPrrV predicater)rUrrrrRr r rV. rzRepeatUntil.__init__csl|jts fddt}tD]}||_|j|||}|js)| ||||r3|SqdS)NcSr&r _1_2_3rr r r)6 z$RepeatUntil._parse..) rrrrrIrrrrrj)rUr1rrrrrr rr r3 s    zRepeatUntil._parsec s~|jts fddt}t}t|D]#\}}||_|j||||} || || |||r:|Sqtd)Ncrr&r rrr r r)C rz$RepeatUntil._build..z3expected any item to match predicate, when building) rrrrrrrrjr) rUrr1rr partiallistrrrr`r rr r@ s    zRepeatUntil._buildcCtd)Nz4cannot calculate size, amount depends on actual datarrr r r rQ rzRepeatUntil._sizeofcCs8d|}d||j||jf}||d|fS)Nzparse_repeatuntil_%sa) def %s(io, this): list_ = ListContainer() for i in itertools.count(): this['_index'] = i obj_ = %s list_.append(obj_) if (%s): return list_ r)rZrrrrj)rUrrrkr r r rT s   zRepeatUntil._emitparsecCs&t|j||dt|jdddS)Nuntilobj_rt)r/r repeat_until)rrr.rSrrvr1r r r r0c s&zRepeatUntil._emitfulltyperrr r rRr r s rc eZdZdZfddZZS)rap Special wrapper that allows outer multiple-subcons construct to merge fields from another multiple-subcons construct. Embedded does not change a field, only wraps it like a candy with a flag. .. warning:: Can only be used between Struct Sequence FocusedSeq Union LazyStruct, although they can be used interchangably, for example Struct can embed fields from a Sequence. There is also :class:`~construct.core.EmbeddedSwitch` macro that pseudo-embeds a Switch. Its not possible to embed IfThenElse. Parsing building and sizeof are deferred to subcon. :param subcon: Construct instance, its fields to embed inside a struct or sequence Example:: >>> outer = Struct( ... Embedded(Struct( ... "data" / Bytes(4), ... )), ... ) >>> outer.parse(b"1234") Container(data=b'1234') ctt||d|_dSru)rPrrVrrQrRr r rV rzEmbedded.__init__rrrrKrVrYr r rRr rj srcsbeZdZdZdfdd ZddZddZd d Zd d Zd dZ ddZ ddZ ddZ Z S)ra' Special wrapper that allows a Struct (or other similar class) to see a field as having a name (or a different name) or having a parsed hook. Library classes do not have names (its None). Renamed does not change a field, only wraps it like a candy with a label. Used internally by / and * operators. Also this wrapper is responsible for building a path info (a chain of names) that gets attached to error message when parsing, building, or sizeof fails. Fields that are not named do not appear in the path string. Parsing building and size are deferred to subcon. :param subcon: Construct instance :param newname: optional, string :param newdocs: optional, string :param newparsed: optional, lambda Example:: >>> "number" / Int32ub NcsJtt|||r |n|j|_|r|n|j|_|r||_dS|j|_dSr&)rPrrVrrr)rUrr2r7r8rRr r rV szRenamed.__init__cCs t|j|Sr&)rrr3r r r rk rzRenamed.__getattr__cCs |d|jf7}|j|||SNz -> %s)rrrrr r r r szRenamed._parsecCs"|d|jf7}|j||||Sr)rrrrr r r r zRenamed._buildcCs|d|jf7}|j||Sr)rrrrr r r r r]zRenamed._sizeofcC |j|Sr&rrrr r r r rzRenamed._emitparsecCrWr&rrr1r r r r' rzRenamed._emitseqcCrWr&rr.r1r r r r- rzRenamed._emitprimitivetypecCsFt}|jr |j|jd||j|||jr!|j|jd|S)Nr)r})rrrrr(r)rUr*r+rr r r r0 szRenamed._emitfulltype)NNN)rrrrKrVrkrrrrr'r-r0rYr r rRr r srcr)Consta Field enforcing a constant. It is used for file signatures, to validate that the given pattern exists. Data in the stream must strictly match the specified value. Note that a variable sized subcon may still provide positive verification. Const does not consume a precomputed amount of bytes, but depends on the subcon to read the appropriate amount (eg. VarInt is acceptable). Whatever subcon parses into, gets compared against the specified value. Parses using subcon and return its value (after checking). Builds using subcon from nothing (or given object, if not None). Size is the same as subcon, unless it raises SizeofError. :param value: expected value, usually a bytes literal :param subcon: optional, Construct instance, subcon used to build value from, assumed to be Bytes if value parameter was a bytes literal :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises ConstError: parsed data does not match specified value, or building from wrong value :raises StringError: building from non-bytes value, perhaps unicode Example:: >>> d = Const(b"IHDR") >>> d.build(None) b'IHDR' >>> d.parse(b"JPEG") construct.core.ConstError: expected b'IHDR' but parsed b'JPEG' >>> d = Const(255, Int32ul) >>> d.build(None) b'\xff\x00\x00\x00' NcsL|durt|tstd|ftt|}tt||||_d|_ dS)Nr9T) r:r;rrtr0rPrrVrr)rUrrrRr r rV s   zConst.__init__cCs0|j|||}||jkrtd|j|f|S)Nz!parsing expected %r but parsed %r)rrrrrr r r r s z Const._parsecCs4|d|jfvrtd|j|f|j|j|||S)Nz'building expected None or %r but got %r)rrrrrr r r r sz Const._buildcCrWr&rXrr r r r rz Const._sizeofcCs |dd|j||jfS)Nz def parse_const(value, expected): if not value == expected: raise ConstError return value zparse_const(%s, %r))rjrrrrr r r r s zConst._emitparsecCs |j|j}ttt|dS)N)contents)rrrrr iterateints)rUr*r+r3r r r r0 szConst._emitfulltyper&rr r rRr r s rcrM) Computeda Field computing a value from the context dictionary or some outer source like os.urandom or random module. Underlying byte stream is unaffected. The source can be non-deterministic. Parsing and Building return the value returned by the context lambda (although a constant value can also be used). Size is defined as 0 because parsing and building does not consume or produce bytes into the stream. :param func: context lambda or constant value Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> d = Struct( ... "width" / Byte, ... "height" / Byte, ... "total" / Computed(this.width * this.height), ... ) >>> d.build(dict(width=4,height=5)) b'\x04\x05' >>> d.parse(b"12") Container(width=49, height=50, total=2450) >>> d = Computed(7) >>> d.parse(b"") 7 >>> d = Computed(lambda ctx: 7) >>> d.parse(b"") 7 >>> import os >>> d = Computed(lambda ctx: os.urandom(10)) >>> d.parse(b"") b'\x98\xc2\xec\x10\x07\xf5\x8e\x98\xc2\xec' ctt|||_d|_dSru)rPrrVfuncrrUrrRr r rV&  zComputed.__init__cCst|jr ||S|jSr&rrrr r r r+ rVzComputed._parsecCst|jr ||S|jSr&rrr r r r. rVzComputed._buildcCrrNr rr r r r1 rbzComputed._sizeofcCr})Nz%r)rrr r r r4 rzComputed._emitparse rrrrKrVrrrrrYr r rRr r s !rcr6) Indexa Indexes a field inside outer :class:`~construct.core.Array` :class:`~construct.core.GreedyRange` :class:`~construct.core.RepeatUntil` context. Note that you can use this class, or use `this._index` expression instead, depending on how its used. See the examples. Parsing and building pulls _index key from the context. Size is 0 because stream is unaffected. :raises IndexFieldError: did not find either key in context Example:: >>> d = Array(3, Index) >>> d.parse(b"") [0, 1, 2] >>> d = Array(3, Struct("i" / Index)) >>> d.parse(b"") [Container(i=0), Container(i=1), Container(i=2)] >>> d = Array(3, Computed(this._index+1)) >>> d.parse(b"") [1, 2, 3] >>> d = Array(3, Struct("i" / Computed(this._._index+1))) >>> d.parse(b"") [Container(i=1), Container(i=2), Container(i=3)] ct|j|d|_dSrurPrrVrrTrRr r rVT rzIndex.__init__cCs |ddSNrrrrr r r rX rz Index._parsecCs |ddSrrrr r r r\ rz Index._buildcCrrNr rr r r r` rbz Index._sizeof rrrrKrVrrrrYr r rRr r8 s  rcrs)Rebuilda Field where building does not require a value, because the value gets recomputed when needed. Comes handy when building a Struct from a dict with missing keys. Useful for length and count fields when :class:`~construct.core.Prefixed` and :class:`~construct.core.PrefixedArray` cannot be used. Parsing defers to subcon. Building is defered to subcon, but it builds from a value provided by the context lambda (or constant). Size is the same as subcon, unless it raises SizeofError. Difference between Default and Rebuild, is that in first the build value is optional and in second the build value is ignored. :param subcon: Construct instance :param func: context lambda or constant value :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> d = Struct( ... "count" / Rebuild(Byte, len_(this.items)), ... "items" / Byte[this.count], ... ) >>> d.build(dict(items=[1,2,3])) b'\x03\x01\x02\x03' c tt||||_d|_dSru)rPrrVrr)rUrrrRr r rV} rzRebuild.__init__cCs,t|jr ||n|j}|j||||Sr&)rrrrrr r r r szRebuild._buildcCrr&rrr r r r rzRebuild._emitparsecCrWr&rr1r r r r' rzRebuild._emitseqcCrWr&rr1r r r r- rzRebuild._emitprimitivetypecCrWr&rr(r1r r r r0 rzRebuild._emitfulltype rrrrKrVrrr'r-r0rYr r rRr rd s rcrs)Defaulta3 Field where building does not require a value, because the value gets taken from default. Comes handy when building a Struct from a dict with missing keys. Parsing defers to subcon. Building is defered to subcon, but it builds from a default (if given object is None) or from given object. Building does not require a value, but can accept one. Size is the same as subcon, unless it raises SizeofError. Difference between Default and Rebuild, is that in first the build value is optional and in second the build value is ignored. :param subcon: Construct instance :param value: context lambda or constant value :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> d = Struct( ... "a" / Default(Byte, 0), ... ) >>> d.build(dict(a=1)) b'\x01' >>> d.build(dict()) b'\x00' crru)rPrrVrr)rUrrrRr r rV rzDefault.__init__cCs8|durt|jr||n|jn|}|j||||Sr&)rrrrrr r r r s&zDefault._buildcCrr&rrr r r r rzDefault._emitparsecCrWr&rr1r r r r' rzDefault._emitseqcCrWr&rr1r r r r- rzDefault._emitprimitivetypecCrWr&rr1r r r r0 rzDefault._emitfulltyperr r rRr r s rcrM) Checka Checks for a condition, and raises CheckError if the check fails. Parsing and building return nothing (but check the condition). Size is 0 because stream is unaffected. :param func: bool or context lambda, that gets run on parsing and building :raises CheckError: lambda returned false Can propagate any exception from the lambda, possibly non-ConstructError. Example:: Check(lambda ctx: len(ctx.payload.data) == ctx.payload_len) Check(len_(this.payload.data) == this.payload_len) crru)rPrrVrrrrRr r rV rzCheck.__init__cCs*t|jr ||n|j}|stddS)Nzcheck failed during parsingrrr)rUr1rrpassedr r r r z Check._parsecCs*t|jr ||n|j}|stddS)Nzcheck failed during buildingr)rUrr1rrrr r r r rz Check._buildcCrrNr rr r r r rbz Check._sizeofcC|dd|jfS)Nzd def parse_check(condition): if not condition: raise CheckError zparse_check(%s))rjrrr r r r s  zCheck._emitparserr r rRr r s rcrM) Errorag Raises ExplicitError, unconditionally. Parsing and building always raise ExplicitError. Size is undefined. :raises ExplicitError: unconditionally, on parsing and building Example:: >>> d = Struct("num"/Byte, Error) >>> d.parse(b"data...") construct.core.ExplicitError: Error field was activated during parsing crrurrTrRr r rV rzError.__init__cCr)Nz(Error field was activated during parsingrrr r r r rz Error._parsecCr)Nz)Error field was activated during buildingrrr r r r rz Error._buildcCr)NzDError does not have size, because it interrupts parsing and buildingrrr r r r rz Error._sizeofcCs|ddS)NzL def parse_error(): raise ExplicitError z parse_error())rjrr r r r s zError._emitparserr r rRr r s rcr)ra Allows constructing more elaborate "adapters" than Adapter class. Parse does parse all subcons in sequence, but returns only the element that was selected (discards other values). Build does build all subcons in sequence, where each gets build from nothing (except the selected subcon which is given the object). Size is the sum of all subcon sizes, unless any subcon raises SizeofError. This class does context nesting, meaning its members are given access to a new dictionary where the "_" entry points to the outer context. When parsing, each member gets parsed and subcon parse return value is inserted into context under matching key only if the member was named. When building, the matching entry gets inserted into context before subcon gets build, and if subcon build returns a new value (not None) that gets replaced in the context. This class supports embedding. :class:`~construct.core.Embedded` semantics dictate, that during instance creation (in ctor), each field is checked for embedded flag, and its subcon members are merged. This changes behavior of some code examples. Only few classes are supported: Struct Sequence FocusedSeq Union LazyStruct, although those can be used interchangably (a Struct can embed a Sequence, or rather its members). This class exposes subcons as attributes. You can refer to subcons that were inlined (and therefore do not exist as variable in the namespace) by accessing the struct attributes, under same name. Also note that compiler does not support this feature. See examples. This class exposes subcons in the context. You can refer to subcons that were inlined (and therefore do not exist as variable in the namespace) within other inlined fields using the context. Note that you need to use a lambda (`this` expression is not supported). Also note that compiler does not support this feature. See examples. This class is used internally to implement :class:`~construct.core.PrefixedArray`. :param parsebuildfrom: string name or context lambda, selects a subcon :param \*subcons: Construct instances, list of members, some can be named :param \*\*subconskw: Construct instances, list of members (requires Python 3.6) :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises UnboundLocalError: selector does not match any subcon Can propagate any exception from the lambda, possibly non-ConstructError. Excample:: >>> d = FocusedSeq("num", Const(b"SIG"), "num"/Byte, Terminated) >>> d.parse(b"SIG\xff") 255 >>> d.build(255) b'SIG\xff' >>> d = FocusedSeq("animal", ... "animal" / Enum(Byte, giraffe=1), ... ) >>> d.animal.giraffe 'giraffe' >>> d = FocusedSeq("count", ... "count" / Byte, ... "data" / Padding(lambda this: this.count - this._subcons.count.sizeof()), ... ) >>> d.build(4) b'\x04\x00\x00\x00' PrefixedArray <--> FocusedSeq("items", "count" / Rebuild(lengthfield, len_(this.items)), "items" / subcon[this.count], ) csVtt|||_t|tdd|D}t||_tdd|jD|_ dS)Ncsrr&r rxr r r reJ rfz&FocusedSeq.__init__..csrr&rrr r r reL r{) rPrrVparsebuildfromrr}rrrr)rUrrrrRr r rVG s  zFocusedSeq.__init__cCrr&rr3r r r rkN rzFocusedSeq.__getattr__c Cst||jd|j|j|j|j||ddd }|jd||_t |j |}t |j D]\}}| |||}|jr<|||j<|j|krC|}q)|Sr)rrrrrrrrtrrrrrrr) rUr1rrrrrparseretfinalretr r r rS s,   zFocusedSeq._parsec Cst||jd|j|j|j|j||ddd }|jd||_t |j |}|||<t |j D]"\}}| |j|kr:|nd|||}|jrH|||j<|j|krO|} q-| Sr)rrrrrrrrtrrrrrrr) rUrr1rrrrrr`rr r r r_ s,   zFocusedSeq._buildc r)Nrrrc3rr&rrrr r rep rfz%FocusedSeq._sizeof..rzrrr rr rl rzFocusedSeq._sizeofcCspd|}d|f}|jD]}|d||f7}|jr%|d|jf7}q|d|jf7}||d|fS)Nzparse_focusedseq_%saG def %s(io, this): result = [] this = Container(_ = this, _params = this['_params'], _root = None, _parsing = True, _building = False, _sizing = False, _subcons = None, _io = io, _index = this.get('_index', None)) this['_root'] = this['_'].get('_root', this) z/ result.append(%s) z7 this[%r] = result[-1] z) return this[%r] r)rZrrrrrjrr r r rt s$     zFocusedSeq._emitparsecr)Ncrr rrrr r r` raz'FocusedSeq._emitseq..rr1r rr r' r;zFocusedSeq._emitseqrr r rRr r s 2  rc@re)Pickleda Preserves arbitrary Python objects. Parses using `pickle.load() `_ and builds using `pickle.dump() `_ functions, using default Pickle binary protocol. Size is undefined. :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes Can propagate pickle.load() and pickle.dump() exceptions. Example:: >>> x = [1, 2.3, {}] >>> Pickled.build(x) b'\x80\x03]q\x00(K\x01G@\x02ffffff}q\x01e.' >>> Pickled.parse(_) [1, 2.3, {}] cCs t|Sr&)pickleloadrr r r r rzPickled._parsecCst|||Sr&)rrrr r r r s zPickled._buildNrrrrKrrr r r r r s rc@re)Numpya Preserves numpy arrays (both shape, dtype and values). Parses using `numpy.load() `_ and builds using `numpy.save() `_ functions, using Numpy binary protocol. Size is undefined. :raises ImportError: numpy could not be imported during parsing or building :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes Can propagate numpy.load() and numpy.save() exceptions. Example:: >>> import numpy >>> a = numpy.asarray([1,2,3]) >>> Numpy.build(a) b"\x93NUMPY\x01\x00F\x00{'descr': '>> Numpy.parse(_) array([1, 2, 3]) cCsddl}||SrN)numpyr)rUr1rrrr r r r s z Numpy._parsecCsddl}||||SrN)rsave)rUrr1rrrr r r r s z Numpy._buildNrr r r r r s rcr) NamedTuplea Both arrays, structs, and sequences can be mapped to a namedtuple from `collections module `_. To create a named tuple, you need to provide a name and a sequence of fields, either a string with space-separated names or a list of string names, like the standard namedtuple. Parses into a collections.namedtuple instance, and builds from such instance (although it also builds from lists and dicts). Size is undefined. :param tuplename: string :param tuplefields: string or list of strings :param subcon: Construct instance, either Struct Sequence Array GreedyRange :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises NamedTupleError: subcon is neither Struct Sequence Array GreedyRange Can propagate collections exceptions. Example:: >>> d = NamedTuple("coord", "x y z", Byte[3]) >>> d = NamedTuple("coord", "x y z", Byte >> Byte >> Byte) >>> d = NamedTuple("coord", "x y z", "x"/Byte + "y"/Byte + "z"/Byte) >>> d.parse(b"123") coord(x=49, y=50, z=51) csHt|ttttfs tdtt||||_ ||_ t |||_ dS)N3subcon is neither Struct Sequence Array GreedyRange)r:rrrHrrrPrrV tuplename tuplefields collections namedtuplefactory)rUr r rrRr r rV s zNamedTuple.__init__cCsFt|jtr|d=|jdi|St|jtttfr|j|Std)Nrz>subcon is neither Struct Sequence Array GreedyRangeGreedyRanger )r:rrr rrHrrrar r r r\ s  zNamedTuple._decodecsHt|jtrtfdd|jjDSt|jtttfr tSt d)Ncs"i|] }|jr|jt|jqSr )rrrrr r r{ s"z&NamedTuple._encode..r) r:rrrrrrHrrrrar rr r^ s zNamedTuple._encodecCsrd|}|d||j|jft|jtr"d||j|fSt|jtt t fr5d||j|fSt d)Nrlz9 %s = collections.namedtuple(%r, %r) z %s(**(%s))z %s(*(%s))r) rZrjr r r:rrrrrHrrrr r r r s   zNamedTuple._emitparsecCrWr&rr1r r r r' rzNamedTuple._emitseqcCrWr&rr1r r r r- rzNamedTuple._emitprimitivetypecCrWr&rr1r r r r0 rzNamedTuple._emitfulltype) rrrrKrVr\r^rr'r-r0rYr r rRr r s  rc sddltts tdttttfstdttjtfs&tddks.dkr^tdt dd t d d t d d t d dt ddt d }Gfdddt }||}nttrjddGfdddt }|}fdd}fdd}||_ ||_ |S)a Datetime, represented as `Arrow `_ object. Note that accuracy is not guaranteed, because building rounds the value to integer (even when Float subcon is used), due to floating-point errors in general, and because MSDOS scheme has only 5-bit (32 values) seconds field (seconds are rounded to multiple of 2). Unit is a fraction of a second. 1 is second resolution, 10**-3 is milliseconds resolution, 10**-6 is microseconds resolution, etc. Usually its 1 on Unix and MacOSX, 10**-7 on Windows. Epoch is a year (if integer) or a specific day (if Arrow object). Usually its 1970 on Unix, 1904 on MacOSX, 1600 on Windows. MSDOS format doesnt support custom unit or epoch, it uses 2-seconds resolution and 1980 epoch. :param subcon: Construct instance like Int* Float*, or Int32ub with msdos format :param unit: integer or float, or msdos string :param epoch: integer, or Arrow instance, or msdos string :raises ImportError: arrow could not be imported during ctor :raises TimestampError: subcon is not a Construct instance :raises TimestampError: unit or epoch is a wrong type Example:: >>> d = Timestamp(Int64ub, 1., 1970) >>> d.parse(b'\x00\x00\x00\x00ZIz\x00') >>> d = Timestamp(Int32ub, "msdos", "msdos") >>> d.parse(b'H9\x8c"') rNzPsubcon should be Int*, experimentally Float*, or Int32ub when using msdos formatz%unit must be one of: int float stringz&epoch must be one of: int Arrow stringmsdosyearrmonthrdayhourminutesecondcs eZdZfddZddZdS)z(Timestamp..MsdosTimestampAdaptercs8dddj|j|jd|jd|j|j|jddS)NrKrF)yearsmonthsdayshoursminutesseconds)Arrowshiftrrrrrrraarrowr r r\: s8z0Timestamp..MsdosTimestampAdapter._decodecSs0|}t|jd|j|j|j|j|jddS)NrrF)rrrrrr) timetuplertm_yeartm_montm_mdaytm_hourtm_mintm_sec)rUrrrtr r r r^< s(z0Timestamp..MsdosTimestampAdapter._encodeNrrrr\r^r r!r r MsdosTimestampAdapter9 s  r,rKcs(eZdZfddZfddZdS)z#Timestamp..TimestampAdaptercsj|dS)N)r)r raepochunitr r r\E rTz+Timestamp..TimestampAdapter._decodecst|Sr&)r  total_secondsrar-r r r^G rz+Timestamp..TimestampAdapter._encodeNr+r r-r r TimestampAdapterD sr1c ||Sr&rrrr r r0K rz Timestamp.._emitfulltypecr2r&rrrr r r-M rz%Timestamp.._emitprimitivetype) r"r:rrrfloatr:r BitStructrr[r0r-) rr/r.str,rr1r0r-r )r"r.rr/r Timestamp s6           r6c@@eZdZdZddZddZddZdd Zd d Zd d Z dS)Hexa Adapter for displaying hexadecimal/hexlified representation of integers/bytes/RawCopy dictionaries. Parsing results in int-alike bytes-alike or dict-alike object, whose only difference from original is pretty-printing. If you look at the result, you will be presented with its `repr` which remains as-is. If you print it, then you will see its `str` whic is a hexlified representation. Building and sizeof defer to subcon. To obtain a hexlified string (like before Hex HexDump changed semantics) use binascii.(un)hexlify on parsed results. Example:: >>> d = Hex(Int32ub) >>> obj = d.parse(b"\x00\x00\x01\x02") >>> obj 258 >>> print(obj) 0x00000102 >>> d = Hex(GreedyBytes) >>> obj = d.parse(b"\x00\x00\x01\x02") >>> obj b'\x00\x00\x01\x02' >>> print(obj) unhexlify('00000102') >>> d = Hex(RawCopy(Int32ub)) >>> obj = d.parse(b"\x00\x00\x01\x02") >>> obj {'data': b'\x00\x00\x01\x02', 'length': 4, 'offset1': 0, 'offset2': 4, 'value': 258} >>> print(obj) unhexlify('00000102') c CsPt|trt|dd|j||St|trt|St|tr&t |S|S)Nz0%sXrF) r:rHexDisplayedIntegerr]rrr;HexDisplayedBytesrHexDisplayedDictrar r r r\w s   z Hex._decodecC|Sr&r rar r r r^ rbz Hex._encodecCrr&rrr r r r rzHex._emitparsecCrWr&rr1r r r r' rz Hex._emitseqcCrWr&rr1r r r r- rzHex._emitprimitivetypecCrWr&rr1r r r r0 rzHex._emitfulltypeN rrrrKr\r^rr'r-r0r r r r r8T s"  r8c@r7)HexDumpa Adapter for displaying hexlified representation of bytes/RawCopy dictionaries. Parsing results in bytes-alike or dict-alike object, whose only difference from original is pretty-printing. If you look at the result, you will be presented with its `repr` which remains as-is. If you print it, then you will see its `str` whic is a hexlified representation. Building and sizeof defer to subcon. To obtain a hexlified string (like before Hex HexDump changed semantics) use construct.lib.hexdump on parsed results. Example:: >>> d = HexDump(GreedyBytes) >>> obj = d.parse(b"\x00\x00\x01\x02") >>> obj b'\x00\x00\x01\x02' >>> print(obj) hexundump(''' 0000 00 00 01 02 .... ''') >>> d = HexDump(RawCopy(Int32ub)) >>> obj = d.parse(b"\x00\x00\x01\x02") >>> obj {'data': b'\x00\x00\x01\x02', 'length': 4, 'offset1': 0, 'offset2': 4, 'value': 258} >>> print(obj) hexundump(''' 0000 00 00 01 02 .... ''') cCs(t|tr t|St|trt|S|Sr&)r:r;HexDumpDisplayedBytesrHexDumpDisplayedDictrar r r r\ s  zHexDump._decodecCr<r&r rar r r r^ rbzHexDump._encodecCrr&rrr r r r rzHexDump._emitparsecCrWr&rr1r r r r' rzHexDump._emitseqcCrWr&rr1r r r r- rzHexDump._emitprimitivetypecCrWr&rr1r r r r0 rzHexDump._emitfulltypeNr=r r r r r> s r>crs)ra" Treats the same data as multiple constructs (similar to C union) so you can look at the data in multiple views. Fields are usually named (so parsed values are inserted into dictionary under same name). :class:`~construct.core.Embedded` fields do not need to (and should not) be named. Parses subcons in sequence, and reverts the stream back to original position after each subcon. Afterwards, advances the stream by selected subcon. Builds from first subcon that has a matching key in given dict. Size is undefined (because parsefrom is not used for building). This class does context nesting, meaning its members are given access to a new dictionary where the "_" entry points to the outer context. When parsing, each member gets parsed and subcon parse return value is inserted into context under matching key only if the member was named. When building, the matching entry gets inserted into context before subcon gets build, and if subcon build returns a new value (not None) that gets replaced in the context. This class supports embedding. :class:`~construct.core.Embedded` semantics dictate, that during instance creation (in ctor), each field is checked for embedded flag, and its subcon members are merged. This changes behavior of some code examples. Only few classes are supported: Struct Sequence FocusedSeq Union LazyStruct, although those can be used interchangably (a Struct can embed a Sequence, or rather its members). This class exposes subcons as attributes. You can refer to subcons that were inlined (and therefore do not exist as variable in the namespace) by accessing the struct attributes, under same name. Also note that compiler does not support this feature. See examples. This class exposes subcons in the context. You can refer to subcons that were inlined (and therefore do not exist as variable in the namespace) within other inlined fields using the context. Note that you need to use a lambda (`this` expression is not supported). Also note that compiler does not support this feature. See examples. .. warning:: If you skip `parsefrom` parameter then stream will be left back at starting offset, not seeked to any common denominator. :param parsefrom: how to leave stream after parsing, can be integer index or string name selecting a subcon, or None (leaves stream at initial offset, the default), or context lambda :param \*subcons: Construct instances, list of members, some can be anonymous :param \*\*subconskw: Construct instances, list of members (requires Python 3.6) :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises StreamError: stream is not seekable and tellable :raises UnionError: selector does not match any subcon, or dict given to build does not contain any keys matching any subcon :raises IndexError: selector does not match any subcon :raises KeyError: selector does not match any subcon Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> d = Union(0, ... "raw" / Bytes(8), ... "ints" / Int32ub[2], ... "shorts" / Int16ub[4], ... "chars" / Byte[8], ... ) >>> d.parse(b"12345678") Container(raw=b'12345678', ints=[825373492, 892745528], shorts=[12594, 13108, 13622, 14136], chars=[49, 50, 51, 52, 53, 54, 55, 56]) >>> d.build(dict(chars=range(8))) b'\x00\x01\x02\x03\x04\x05\x06\x07' >>> d = Union(None, ... "animal" / Enum(Byte, giraffe=1), ... ) >>> d.animal.giraffe 'giraffe' >>> d = Union(None, ... "chars" / Byte[4], ... "data" / Bytes(lambda this: this._subcons.chars.sizeof()), ... ) >>> d.parse(b"\x01\x02\x03\x04") Container(chars=[1, 2, 3, 4])(data=b'\x01\x02\x03\x04') Alternative syntax, but requires Python 3.6 or any PyPy: >>> Union(0, raw=Bytes(8), ints=Int32ub[2], shorts=Int16ub[4], chars=Byte[8]) csht|tr tdtt|||_t|tdd|D}t ||_ t dd|j D|_ dS)Nz9parsefrom should be either: None int str context-functioncsrr&r rxr r r re rfz!Union.__init__..csrr&rrr r r re r{) r:rrrPrrV parsefromrr}rrrr)rUrArrrRr r rV s  zUnion.__init__cCrr&rr3r r r rk rzUnion.__getattr__c Cst}t||jd|j|j|j|j||ddd }|jd||_t |}i}t |j D]-\}}| |||} |j rD| ||j <| ||j <t |||<|j rTt |||j <t||q,|j} t| re| |} | durpt||| |Sr)rrrrrrrrtrrDrrrrrBrAr) rUr1rrrrGforwardsrrrrAr r r r s(,    z Union._parsec Cst||jd|j|j|j|j||ddd }|jd||_| ||j D]9}|j r3||j d}n |j |vr>||j }nq&|j rG|||j <| ||||}|j rW|||j <t|j |iStd|f)Nrrrz=cannot build, none of subcons were found in the dictionary %r)rrrrrrrrtrrrrrrrrr r r r' s ,      z Union._buildcCr)Nz=Union builds depending on actual object dict, size is unknownrrr r r r= rz Union._sizeofc Csbt|jr tdd|}d|f}t|jtdr"d}d}d}t|jtr@|j}|j|d}|j||jdk}t|jt rcddt |jD|j}d}|j||jdk}t |jD].\}}|d |j rvd |j nd | |f7}||kr|s|d 7}|t |jd kr|d7}qh|s|d7}|s|d7}|d7}||d|fS)Nz-Union does not compile non-constant parsefromzparse_union_%saP def %s(io, this): this = Container(_ = this, _params = this['_params'], _root = None, _parsing = True, _building = False, _sizing = False, _subcons = None, _io = io, _index = this.get('_index', None)) this['_root'] = this['_'].get('_root', this) fallback = io.tell() rFTcSsi|] \}}|jr|j|qSr rr^rrr r r r{T raz$Union._emitparse..z" %s%s z this[%r] = rmz5 forward = io.tell() rKz3 io.seek(fallback) z/ io.seek(fallback) z. io.seek(forward) zf del this['_'] del this['_index'] return this r)rrArrZr:r/r rrrFrrrr0rj) rUrrrkindex skipfallback skipforwardrrr r r r@ sF        zUnion._emitparse) rrrrKrVrkrrrrrYr r rRr r s 8 rc0eZdZdZfddZddZddZZS)Selectu? Selects the first matching subconstruct. Parses and builds by literally trying each subcon in sequence until one of them parses or builds without exception. Stream gets reverted back to original position after each failed attempt, but not if parsing succeeds. Size is not defined. :param \*subcons: Construct instances, list of members, some can be anonymous :param \*\*subconskw: Construct instances, list of members (requires Python 3.6) :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises StreamError: stream is not seekable and tellable :raises SelectError: neither subcon succeded when parsing or building Example:: >>> d = Select(Int32ub, CString("utf8")) >>> d.build(1) b'\x00\x00\x00\x01' >>> d.build(u"Афон") b'\xd0\x90\xd1\x84\xd0\xbe\xd0\xbd\x00' Alternative syntax, but requires Python 3.6 or any PyPy: >>> Select(num=Int32ub, text=CString("utf8")) cs^tt|t|tdd|D|_tdd|jD|_tdd|jD|_dS)Ncsrr&r rxr r r re rfz"Select.__init__..csrr&rrr r r re rcsrr&)rrr r r re r) rPrHrVrr}rrrrrrRr r rV s zSelect.__init__c Cs^|jD]'}t|}z ||||}W|Styty*t||Yqwtd)Nzno subconstruct matched)rrDrrrrBr)rUr1rrrrGrr r r r s   z Select._parsec Csb|jD]&}z |j|fi|}Wntyty Yqwt|||Std|f)Nzno subconstruct matched: %s)rrrr/r>r)rUrr1rrrr3r r r r s    z Select._buildrrrrKrVrrrYr r rRr rHu s   rHcCs t|tS)a Makes an optional field. Parsing attempts to parse subcon. If sub-parsing fails, returns None and reports success. Building attempts to build subcon. If sub-building fails, writes nothing and reports success. Size is undefined, because whether bytes would be consumed or produced depends on actual data and actual context. :param subcon: Construct instance Example:: Optional <--> Select(subcon, Pass) >>> d = Optional(Int64ul) >>> d.parse(b"12345678") 4050765991979987505 >>> d.parse(b"") None >>> d.build(1) b'\x01\x00\x00\x00\x00\x00\x00\x00' >>> d.build(None) b'' )rHPassrr r r Optional s rKcs$tt}fdd}||_|S)aq If-then conditional construct. Parsing evaluates condition, if True then subcon is parsed, otherwise just returns None. Building also evaluates condition, if True then subcon gets build from, otherwise does nothing. Size is either same as subcon or 0, depending how condfunc evaluates. :param condfunc: bool or context lambda (or a truthy value) :param subcon: Construct instance, used if condition indicates True :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes Can propagate any exception from the lambda, possibly non-ConstructError. Example:: If <--> IfThenElse(condfunc, subcon, Pass) >>> d = If(this.x > 0, Byte) >>> d.build(255, x=1) b'\xff' >>> d.build(255, x=0) b'' cs t||tdddS)Nthis.rm)r/if_)rr.rSrvrcondfuncrr r r0  zIf.._emitfulltype) IfThenElserJr0)rOrrr0r rNr If s rRcrs)rQaR If-then-else conditional construct, similar to ternary operator. Parsing and building evaluates condition, and defers to either subcon depending on the value. Size is computed the same way. :param condfunc: bool or context lambda (or a truthy value) :param thensubcon: Construct instance, used if condition indicates True :param elsesubcon: Construct instance, used if condition indicates False :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> d = IfThenElse(this.x > 0, VarInt, Byte) >>> d.build(255, dict(x=1)) b'\xff\x01' >>> d.build(255, dict(x=0)) b'\xff' cs2tt|||_||_||_|jo|j|_dSr&)rPrQrVrO thensubcon elsesubconr)rUrOrSrTrRr r rV s zIfThenElse.__init__cCs4|j}t|r ||}|r|jn|j}||||Sr&)rOrrSrTr)rUr1rrrOrr r r r zIfThenElse._parsecCs6|j}t|r ||}|r|jn|j}|||||Sr&)rOrrSrTr)rUrr1rrrOrr r r r  zIfThenElse._buildcCs2|j}t|r ||}|r|jn|j}|||Sr&)rOrrSrTr)rUrrrOrr r r rs  zIfThenElse._sizeofcCs d|j||j|j|fS)Nz((%s) if (%s) else (%s)))rSrrOrTrr r r rrPzIfThenElse._emitparsecCsNtd|j||t|jdddtd|j||t|jdddgS)N thenvaluerLrm)rr/rMrT)rrSr.rSrOrvrTr1r r r r's$&zIfThenElse._emitseq) rrrrKrVrrrrr'rYr r rRr rQ s rQcr)Switcha+ A conditional branch. Parsing and building evaluate keyfunc and select a subcon based on the value and dictionary entries. Dictionary (cases) maps values into subcons. If no case matches then `default` is used (that is Pass by default). Note that `default` is a Construct instance, not a dictionary key. Size is evaluated in same way as parsing and building, by evaluating keyfunc and selecting a field accordingly. :param keyfunc: context lambda or constant, that matches some key in cases :param cases: dict mapping keys to Construct instances :param default: optional, Construct instance, used when keyfunc is not found in cases, Pass is default value for this parameter, Error is a possible value for this parameter :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> d = Switch(this.n, { 1:Int8ub, 2:Int16ub, 4:Int32ub }) >>> d.build(5, n=1) b'\x05' >>> d.build(5, n=4) b'\x00\x00\x00\x05' >>> d = Switch(this.n, {}, default=Byte) >>> d.parse(b"\x01", n=255) 1 >>> d.build(1, n=255) b"\x01" NcsV|durt}tt|||_||_||_t||g}t dd|D|_ dS)Ncsrr&rrr r r reHrz"Switch.__init__..) rJrPrXrVkeyfunccasesdefaultrvaluesrr)rUrYrZr[allcasesrRr r rV@szSwitch.__init__cCs4|j}t|r ||}|j||j}||||Sr&)rYrrZrr[r)rUr1rrrYrr r r rJrUz Switch._parsecCs6|j}t|r ||}|j||j}|||||Sr&)rYrrZrr[r)rUrr1rrrYrr r r rQrVz Switch._buildc CsPz|j}t|r ||}|j||j}|||WSttfy'tdwry) rYrrZrr[rr{r|r )rUrrrYrr r r rXszSwitch._sizeofc snd}d|dfdd|jDfd}d||jfd||j|fS) Nrlz %s = {%s}rzc3s&|]\}}d||fVqdS)z%r : lambda io,this: %sNr)r^keyrrr r ree$z$Switch._emitparse..z compiled_%sz%s = lambda io,this: %sz%s.get(%s, %s)(io, this))rZrjrirZr}r[rrY)rUrr defaultfnamer r`r rcs , zSwitch._emitparser&rr r rRr rX#s  rXcCsJt|j}|D]\}}|jD]}||jt||k|qq t|S)aI Macro that simulates embedding Switch, which under new embedding semantics is not possible. This macro does NOT produce a Switch. It generates classes that behave the same way as you would expect from embedded Switch, only that. Instance created by this macro CAN be embedded. Both `merged` and all values in `mapping` must be Struct instances. Macro re-creates a single struct that contains all fields, where each field is wrapped in `If(selector == key, ...)`. Note that resulting dictionary contains None values for fields that would not be chosen by switch. Note also that if selector does not match any cases, it passes successfully (default Switch behavior). All fields should have unique names. Otherwise fields that were not selected during parsing may return None and override other fields context entries that have same name. This is because `If` field returns None value if condition is not met, but the Struct inserts that None value into the context entry regardless. :param merged: Struct instance :param selector: this expression, that references one of `merged` fields :param mapping: dict with values being Struct instances Example:: d = EmbeddedSwitch( Struct( "type" / Byte, ), this.type, { 0: Struct("name" / PascalString(Byte, "utf8")), 1: Struct("value" / Byte), } ) # generates essentially following d = Struct( "type" / Byte, "name" / If(this.type == 0, PascalString(Byte, "utf8")), "value" / If(this.type == 1, Byte), ) # both parse like following >>> d.parse(b"\x00\x00") Container(type=0, name=u'', value=None) >>> d.parse(b"\x01\x00") Container(type=1, name=None, value=0) )rrr}rjrrRr)mergedselectorrhmerged2r_r5rr r r EmbeddedSwitchls ' rfcrM) StopIfa Checks for a condition, and stops certain classes (:class:`~construct.core.Struct` :class:`~construct.core.Sequence` :class:`~construct.core.GreedyRange`) from parsing or building further. Parsing and building check the condition, and raise StopFieldError if indicated. Size is undefined. :param condfunc: bool or context lambda (or truthy value) :raises StopFieldError: used internally Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> Struct('x'/Byte, StopIf(this.x == 0), 'y'/Byte) >>> Sequence('x'/Byte, StopIf(this.x == 0), 'y'/Byte) >>> GreedyRange(FocusedSeq(0, 'x'/Byte, StopIf(this.x == 0))) crru)rPrgrVrOr)rUrOrRr r rVrzStopIf.__init__cCs"|j}t|r ||}|rtdSr&rOrr)rUr1rrrOr r r r z StopIf._parsecCs"|j}t|r ||}|rtdSr&rh)rUrr1rrrOr r r rriz StopIf._buildcCr)NzxStopIf cannot determine size because it depends on actual context which then depends on actual data and outer constructsrrr r r rrzStopIf._sizeofcCr)Nzy def parse_stopif(condition): if condition: raise StopFieldError zparse_stopif(%s))rjrOrr r r rs  zStopIf._emitparserr r rRr rgs rgrQcs6tt|d}fdd}fdd}||_||_|S)ap Appends null bytes. Parsing consumes specified amount of bytes and discards it. Building writes specified pattern byte multiplied into specified length. Size is same as specified. :param length: integer or context lambda, length of the padding :param pattern: b-character, padding pattern, default is \\x00 :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises PaddingError: length was negative :raises PaddingError: pattern was not bytes (b-character) Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> d = Padding(4) or Padded(4, Pass) >>> d.build(None) b'\x00\x00\x00\x00' >>> d.parse(b"****") None >>> d.sizeof() 4 )patterncs|stdfSrrrr~r r r- z#Padding.._emitprimitivetypecs|rttdSr)rrrr~r r r0rkzPadding.._emitfulltype)PaddedrJr-r0)r2rjrr-r0r r~r Paddings   rmcr)rla Appends additional null bytes to achieve a length. Parsing first parses the subcon, then uses stream.tell() to measure how many bytes were read and consumes additional bytes accordingly. Building first builds the subcon, then uses stream.tell() to measure how many bytes were written and produces additional bytes accordingly. Size is same as `length`, but negative amount results in error. Note that subcon can actually be variable size, it is the eventual amount of bytes that is read or written during parsing or building that determines actual padding. :param length: integer or context lambda, length of the padding :param subcon: Construct instance :param pattern: optional, b-character, padding pattern, default is \\x00 :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises PaddingError: length is negative :raises PaddingError: subcon read or written more than the length (would cause negative pad) :raises PaddingError: pattern is not bytes of length 1 Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> d = Padded(4, Byte) >>> d.build(255) b'\xff\x00\x00\x00' >>> d.parse(_) 255 >>> d.sizeof() 4 >>> d = Padded(4, VarInt) >>> d.build(1) b'\x01\x00\x00\x00' >>> d.build(70000) b'\xf0\xa2\x04\x00' rQc>t|tr t|dkrtdtt||||_||_dS)NrKz(pattern expected to be bytes of length 1) r:r;r0r rPrlrVr2rj)rUr2rrjrRr r rV  zPadded.__init__c Csrt|j|}|dkrtdt|}|j|||}t|}|||}|dkr2td|||ft|||S)Nrlength cannot be negativez.subcon parsed %d bytes but was allowed only %d)rr2r rDrrr4) rUr1rrr2 position1r position2r>r r r rs   z Padded._parsec Cs|t|j|}|dkrtdt|}|j||||}t|}|||} | dkr3td|||ft||j| | |SNrrpz-subcon build %d bytes but was allowed only %d)rr2r rDrrr>rj) rUrr1rrr2rqr`rrr>r r r r,s  z Padded._buildc Cs>zt|j|}|dkrtd|WSttfytdw)Nrrprz)rr2r r{r|r rr r r r9s zPadded._sizeofcCsd|j||j|jfS)Nz#(%s, read_bytes(io, (%s)-(%s) ))[0])rrr2rrr r r rBzPadded._emitparsecCst|j|j||dS)N)rr/)rr2rr.r1r r r r0EzPadded._emitfulltyperQrr r rRr rls!   rlcr)Aligneda Appends additional null bytes to achieve a length that is shortest multiple of a modulus. Note that subcon can actually be variable size, it is the eventual amount of bytes that is read or written during parsing or building that determines actual padding. Parsing first parses subcon, then consumes an amount of bytes to sum up to specified length, and discards it. Building first builds subcon, then writes specified pattern byte to sum up to specified length. Size is subcon size plus modulo remainder, unless SizeofError was raised. :param modulus: integer or context lambda, modulus to final length :param subcon: Construct instance :param pattern: optional, b-character, padding pattern, default is \\x00 :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises PaddingError: modulus was less than 2 :raises PaddingError: pattern was not bytes (b-character) Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> d = Aligned(4, Int16ub) >>> d.parse(b'\x00\x01\x00\x00') 1 >>> d.sizeof() 4 rQcrn)NrKz&pattern expected to be bytes character) r:r;r0r rPrwrVmodulusrj)rUrxrrjrRr r rVdrozAligned.__init__c Csft|jr ||n|j}|dkrtdt|}|j|||}t|}|| |}t|||SNrFexpected modulo 2 or greater)rrxr rDrrr4) rUr1rrrxrqrrrr>r r r rks zAligned._parsec Cspt|jr ||n|j}|dkrtdt|}|j||||}t|}|| |} t||j| | |Sry)rrxr rDrrr>rj) rUrr1rrrxrqr`rrr>r r r rvszAligned._buildc Csdz$t|jr ||n|j}|dkrtd|j||}|| |WSttfy1tdw)NrFrzrz)rrxr rrr{r|r )rUrrrx subconlenr r r rszAligned._sizeofcCsd|j||j|jfS)Nz'(%s, read_bytes(io, -(%s) %% (%s) ))[0])rrrrxrr r r rrtzAligned._emitparservrr r rRr rwIs   rwcs4t|tdd|D}tfdd|DS)a Makes a structure where each field is aligned to the same modulus (it is a struct of aligned fields, NOT an aligned struct). See :class:`~construct.core.Aligned` and :class:`~construct.core.Struct` for semantics and raisable exceptions. :param modulus: integer or context lambda, passed to each member :param \*subcons: Construct instances, list of members, some can be anonymous :param \*\*subconskw: Construct instances, list of members (requires Python 3.6) Example:: >>> d = AlignedStruct(4, "a"/Int8ub, "b"/Int16ub) >>> d.build(dict(a=0xFF,b=0xFFFF)) b'\xff\x00\x00\x00\xff\xff\x00\x00' csrr&r rxr r r rerfz AlignedStruct..csg|] }|jt|qSr )rrwrrxr r r`sz!AlignedStruct..)rr}r)rxrrr r|r AlignedStructsr}cOstt|i|S)a Makes a structure inside a Bitwise. See :class:`~construct.core.Bitwise` and :class:`~construct.core.Struct` for semantics and raisable exceptions. :param \*subcons: Construct instances, list of members, some can be anonymous :param \*\*subconskw: Construct instances, list of members (requires Python 3.6) Example:: BitStruct <--> Bitwise(Struct(...)) >>> d = BitStruct( ... "a" / Flag, ... "b" / Nibble, ... "c" / BitsInteger(10), ... "d" / Padding(1), ... ) >>> d.parse(b"\xbe\xef") Container(a=True)(b=7)(c=887)(d=None) >>> d.sizeof() 2 )rr)rrr r r r4sr4cr)PointeraC Jumps in the stream forth and back for one field. Parsing and building seeks the stream to new location, processes subcon, and seeks back to original location. Size is defined as 0 but that does not mean no bytes are written into the stream. Offset can be positive, indicating a position from stream beginning forward, or negative, indicating a position from EOF backwards. :param offset: integer or context lambda, positive or negative :param subcon: Construct instance :param stream: None to use original stream (default), or context lambda to provide a different stream :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises StreamError: stream is not seekable and tellable Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> d = Pointer(8, Bytes(1)) >>> d.parse(b"abcdefghijkl") b'i' >>> d.build(b"Z") b'\x00\x00\x00\x00\x00\x00\x00\x00Z' Ncrr&)rPr~rVr@r1)rUr@rr1rRr r rVrzPointer.__init__cCsZt|j|}t|j|p |}t|}t|||dkrdnd|j|||}t|||SrE)rr@r1rDrBrr)rUr1rrr@rGrr r r rs  zPointer._parsecCs\t|j|}t|j|p |}t|}t|||dkrdnd|j||||}t|||SrE)rr@r1rDrBrr)rUrr1rrr@rGr`r r r rs  zPointer._buildcCrrNr rr r r rrbzPointer._sizeofcCs |dd|j|j|fS)Nz def parse_pointer(io, offset, func): fallback = io.tell() io.seek(offset, 2 if offset < 0 else 0) obj = func() io.seek(fallback) return obj z!parse_pointer(io, %s, lambda: %s))rjr@rrrr r r rs zPointer._emitparsecCsLt|jr |jn|j}d|}tdd|i|j|||j|<|S)Nz instance_%sposr )rr@ __getfield__rZrrr(rq)rUr*r+r@rr r r r-s "zPointer._emitprimitivetyper&rr r rRr r~s   r~crM) Peeka Peeks at the stream. Parsing sub-parses (and returns None if failed), then reverts stream to original position. Building does nothing (its NOT deferred). Size is defined as 0 because there is no building. This class is used in :class:`~construct.core.Union` class to parse each member. :param subcon: Construct instance :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises StreamError: stream is not seekable and tellable Example:: >>> d = Sequence(Peek(Int8ub), Peek(Int16ub)) >>> d.parse(b"\x01\x02") [1, 258] >>> d.sizeof() 0 crru)rPrrVrrQrRr r rVrz Peek.__init__c Csft|}z(z|j|||WWt||Styty$YnwWt||dSt||wr&)rDrrrBrr)rUr1rrrGr r r r!s   z Peek._parsecCr<r&r rr r r r,rbz Peek._buildcCrrNr rr r r r/rbz Peek._sizeofcCs|dd|j|fS)NaJ def parse_peek(io, func): fallback = io.tell() try: return func() except ExplicitError: raise except ConstructError: pass finally: io.seek(fallback) zparse_peek(io, lambda: %s))rjrrrr r r r2s  zPeek._emitparserr r rRr rs  rcr)SeekaZ Seeks the stream. Parsing and building seek the stream to given location (and whence), and return stream.seek() return value. Size is not defined. .. seealso:: Analog :class:`~construct.core.Pointer` wrapper that has same side effect but also processes a subcon, and also seeks back. :param at: integer or context lambda, where to jump to :param whence: optional, integer or context lambda, is the offset from beginning (0) or from current position (1) or from EOF (2), default is 0 :raises StreamError: stream is not seekable Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> d = (Seek(5) >> Byte) >>> d.parse(b"01234x") [5, 120] >>> d = (Bytes(10) >> Seek(5) >> Byte) >>> d.build([b"0123456789", None, 255]) b'01234\xff6789' rcs$tt|||_||_d|_dSru)rPrrVatrAr)rUrrArRr r rV\rqz Seek.__init__cCs@t|jr ||n|j}t|jr||n|j}t|||Sr&rrrArB)rUr1rrrrAr r r rb z Seek._parsecCs@t|jr ||n|j}t|jr||n|j}t|||Sr&r)rUrr1rrrrAr r r rgrz Seek._buildcCr)Nz2Seek only moves the stream, size is not meaningfulrrr r r rlrz Seek._sizeofcCsd|j|jfS)Nzio.seek(%s, %s))rrArr r r rorTzSeek._emitparserrr r rRr rBsrcrM) Tella1 Tells the stream. Parsing and building return current stream offset using using stream.tell(). Size is defined as 0 because parsing and building does not consume or add into the stream. Tell is useful for adjusting relative offsets to absolute positions, or to measure sizes of Constructs. To get an absolute pointer, use a Tell plus a relative offset. To get a size, place two Tells and measure their difference using a Compute field. However, its recommended to use :class:`~construct.core.RawCopy` instead of manually extracting two positions and computing difference. :raises StreamError: stream is not tellable Example:: >>> d = Struct("num"/VarInt, "offset"/Tell) >>> d.parse(b"X") Container(num=88)(offset=1) >>> d.build(dict(num=88)) b'X' crrurrTrRr r rVrz Tell.__init__cCrr&rDrr r r rrz Tell._parsecCst|Sr&rrr r r rrz Tell._buildcCrrNr rr r r rrbz Tell._sizeofcCr)Nz io.tell()r rr r r rrbzTell._emitparserr r rRr rss rcrs)rJa No-op construct, useful as default cases for Switch and Enum. Parsing returns None. Building does nothing. Size is 0 by definition. Example:: >>> Pass.parse(b"") None >>> Pass.build(None) b'' >>> Pass.sizeof() 0 crrurrTrRr r rVrz Pass.__init__cCsdSr&r rr r r rrbz Pass._parsecCr<r&r rr r r rrbz Pass._buildcCrrNr rr r r rrbz Pass._sizeofcCr)NNoner rr r r rrbzPass._emitparsecCr)Nrrrr1r r r r0rzPass._emitfulltyperr r rRr rJs rJcr6) Terminateda5 Asserts end of stream (EOF). You can use it to ensure no more unparsed data follows in the stream. Parsing checks if stream reached EOF, and raises TerminatedError if not. Building does nothing. Size is defined as 0 because parsing and building does not consume or add into the stream, as far as other constructs see it. :raises TerminatedError: stream not at EOF when parsing Example:: >>> Terminated.parse(b"") None >>> Terminated.parse(b"remaining") construct.core.TerminatedError: expected end of stream crrurrTrRr r rVrzTerminated.__init__cCs|dr tddS)NrKzexpected end of stream)r.r!rr r r rs zTerminated._parsecCr<r&r rr r r rrbzTerminated._buildcCrr&rrr r r rrbzTerminated._sizeofrr r rRr rs  rc@re)RawCopya Used to obtain byte representation of a field (aside of object value). Returns a dict containing both parsed subcon value, the raw bytes that were consumed by subcon, starting and ending offset in the stream, and amount in bytes. Builds either from raw bytes representation or a value used by subcon. Size is same as subcon. Object is a dictionary with either "data" or "value" keys, or both. :param subcon: Construct instance :raises StreamError: stream is not seekable and tellable :raises RawCopyError: building and neither data or value was given :raises StringError: building from non-bytes value, perhaps unicode Example:: >>> d = RawCopy(Byte) >>> d.parse(b"\xff") Container(data=b'\xff')(value=255)(offset1=0)(offset2=1)(length=1) >>> d.build(dict(data=b"\xff")) '\xff' >>> d.build(dict(value=255)) '\xff' cCsNt|}|j|||}t|}t||t|||}t||||||dS)Nr3roffset1offset2r2)rDrrrBr4r)rUr1rrrrrr3r r r rs  zRawCopy._parsec Cs|dur |jjr tdd}d|vr-|d}t|}t||t|}t||||||dSd|vrf|d}t|}|j||||} | durH|n| }t|}t||t|||}t|||||||dSt d)N)rr3)r3rrr2rrz:RawCopy cannot build, both data and value keys are missing) rrrrDr>rrrBr4r") rUrr1rrr3rrrr`r r r rs$   zRawCopy._buildNrr r r r rs rcCs|}t|t|t|S)a Swaps the byte order within boundaries of given subcon. Requires a fixed sized subcon. :param subcon: Construct instance, subcon on top of byte swapped bytes :raises SizeofError: ctor or compiler could not compute subcon size See :class:`~construct.core.Transformed` and :class:`~construct.core.Restreamed` for raisable exceptions. Example:: Int24ul <--> ByteSwapped(Int24ub) <--> BytesInteger(3, swapped=True) <--> ByteSwapped(BytesInteger(3)) )rrrrrr r r ByteSwappedsrc CsDz |}t|t|t|WSty!t|tdtdddYSw)aC Swaps the bit order within each byte within boundaries of given subcon. Does NOT require a fixed sized subcon. :param subcon: Construct instance, subcon on top of bit swapped bytes :raises SizeofError: compiler could not compute subcon size See :class:`~construct.core.Transformed` and :class:`~construct.core.Restreamed` for raisable exceptions. Example:: >>> d = Bitwise(Bytes(8)) >>> d.parse(b"\x01") '\x00\x00\x00\x00\x00\x00\x00\x01' >>>> BitsSwapped(d).parse(b"\x01") '\x01\x00\x00\x00\x00\x00\x00\x00' rKcSrrr&r rr r r r)@rzBitsSwapped..)rrswapbitsinbytesr rrr r r BitsSwapped)s  rcsReZdZdZdfdd ZddZddZd d Zd d Zd dZ ddZ Z S)rIa Prefixes a field with byte count. Parses the length field. Then reads that amount of bytes, and parses subcon using only those bytes. Constructs that consume entire remaining stream are constrained to consuming only the specified amount of bytes (a substream). When building, data gets prefixed by its length. Optionally, length field can include its own size. Size is the sum of both fields sizes, unless either raises SizeofError. Analog to :class:`~construct.core.PrefixedArray` which prefixes with an element count, instead of byte count. Semantics is similar but implementation is different. :class:`~construct.core.VarInt` is recommended for new protocols, as it is more compact and never overflows. :param lengthfield: Construct instance, field used for storing the length :param subcon: Construct instance, subcon used for storing the value :param includelength: optional, bool, whether length field should include its own size, default is False :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes Example:: >>> d = Prefixed(VarInt, GreedyRange(Int32ul)) >>> d.parse(b"\x08abcdefgh") [1684234849, 1751606885] >>> d = PrefixedArray(VarInt, Int32ul) >>> d.parse(b"\x02abcdefgh") [1684234849, 1751606885] Fcrr&)rPrIrVrD includelength)rUrDrrrRr r rV^rzPrefixed.__init__cCsr|j|||}|jr||j||8}t||}|jtur |St|jtur.| |jj S|jt |||Sr&) rDrrrr4rrr/rOrr4rrrr r r rcs  zPrefixed._parsec Csbt}|j||||}|}t|}|jr!||j||7}|j||||t |||Sr&) rrrrrr0rrDrr>) rUrr1rrrlr`r3r2r r r rns zPrefixed._buildcCs|j|||j||Sr&)rDrrrr r r ryrzPrefixed._sizeofcCsDt|}|j|||}|jr||j||8}t|}|||Sr&)rDrDrrr)rUr1rrrqr2rrr r r r|s  zPrefixed._actualsizecCs2|jr|jnd}d|j|||j|fS)Nrz2restream(read_bytes(io, (%s)-(%s)), lambda io: %s))rrDrrr)rUrsubr r r rszPrefixed._emitparsecCs.td|j||dtdd|j||dgS)NrDrEr3)rrr/)rrDr.rr1r r r r'szPrefixed._emitseqr) rrrrKrVrrrrrr'rYr r rRr rICs  rIcsftddtttjdtj}fdd}||_fdd}||_fdd}||_|S) a} Prefixes an array with item count (as opposed to prefixed by byte count, see :class:`~construct.core.Prefixed`). :class:`~construct.core.VarInt` is recommended for new protocols, as it is more compact and never overflows. :param countfield: Construct instance, field used for storing the element count :param subcon: Construct instance, subcon used for storing each element :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises RangeError: consumed or produced too little elements Example:: >>> d = Prefixed(VarInt, GreedyRange(Int32ul)) >>> d.parse(b"\x08abcdefgh") [1684234849, 1751606885] >>> d = PrefixedArray(VarInt, Int32ul) >>> d.parse(b"\x02abcdefgh") [1684234849, 1751606885] r}rIcsd||fS)Nz&ListContainer((%s) for i in range(%s))r^r` countfieldrr r rruz!PrefixedArray.._emitparsecs6t|}|||}t|}|||||Sr&)rDrr)rUr1rrrqrIrrrr r rsz"PrefixedArray.._actualsizecs,td||dtd||dddgS)NrrEr3r)rr/rrrGrrr r r'szPrefixedArray.._emitseq) rrlen_thisr}rIrrr')rrrrrr'r rr PrefixedArrays rcrs)rAaB Restricts parsing to specified amount of bytes. Parsing reads `length` bytes, then defers to subcon using new BytesIO with said bytes. Building builds the subcon using new BytesIO, then writes said data and additional null bytes accordingly. Size is same as `length`, although negative amount raises an error as well. :param length: integer or context lambda, total amount of bytes (both data and padding) :param subcon: Construct instance :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises PaddingError: length is negative :raises PaddingError: subcon written more bytes than entire length (negative padding) Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> d = FixedSized(10, Byte) >>> d.parse(b'\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00') 255 >>> d.build(255) b'\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00' >>> d.sizeof() 10 ctt||||_dSr&)rPrArVr2)rUr2rrRr r rVrzFixedSized.__init__cCsft|j|}|dkrtdt||}|jtur|St|jtur(||jj S|j t |||SNrrp) rr2r r4rrr/rOrr4rrrrr r r rs   zFixedSized._parsec Cst|j|}|dkrtdt}|j||||}|}|t|} | dkr3tdt||ft ||t |t | |Srs) rr2r rrrrrr0r>r2) rUrr1rrr2rlr`r3r>r r r rs   zFixedSized._buildcCs t|j|}|dkrtd|Sr)rr2r rr r r rs zFixedSized._sizeofcCd|j|j|fS)Nz+restream(read_bytes(io, %s), lambda io: %s))r2rrrr r r rr;zFixedSized._emitparsecCs*tddt|jddi|j||S)NrrLrmr )rrSr2rvrr(r1r r r r0r|zFixedSized._emitfulltyperr r rRr rAs  rAcsBeZdZdZdfdd ZddZd d Zd d Zd dZZ S)rLaQ Restricts parsing to bytes preceding a null byte. Parsing reads one byte at a time and accumulates it with previous bytes. When term was found, (by default) consumes but discards the term. When EOF was found, (by default) raises same StreamError exception. Then subcon is parsed using new BytesIO made with said data. Building builds the subcon and then writes the term. Size is undefined. The term can be multiple bytes, to support string classes with UTF16/32 encodings. :param subcon: Construct instance :param term: optional, bytes, terminator byte-string, default is \x00 single null byte :param include: optional, bool, if to include terminator in resulting data, default is False :param consume: optional, bool, if to consume terminator or leave it in the stream, default is True :param require: optional, bool, if EOF results in failure or not, default is True :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises StreamError: encountered EOF but require is not disabled :raises PaddingError: terminator is less than 1 bytes in length Example:: >>> d = NullTerminated(Byte) >>> d.parse(b'\xff\x00') 255 >>> d.build(255) b'\xff\x00' rQFTc,tt||||_||_||_||_dSr&)rPrLrVrKincludeconsumerequire)rUrrKrrrrRr r rV  zNullTerminated.__init__cCs|j}t|}|dkrtdd} zt||}Wn ty&|jr$Ynw||kr=|jr2||7}|jsrK)rUrr1rrr`r r r r<s zNullTerminated._buildcCrr&rrr r r rArbzNullTerminated._sizeofcCs@t|jdkr ttdt|j|j|j|jd|j ||S)NrK) terminatorrr eos_errorr ) r0rKrrrrrrrr(r1r r r r0Ds.zNullTerminated._emitfulltype)rQFTTrr r rRr rLsrLcr)rBa Restricts parsing to bytes except padding left of EOF. Parsing reads entire stream, then strips the data from right to left of null bytes, then parses subcon using new BytesIO made of said data. Building defers to subcon as-is. Size is undefined, because it reads till EOF. The pad can be multiple bytes, to support string classes with UTF16/32 encodings. :param subcon: Construct instance :param pad: optional, bytes, padding byte-string, default is \x00 single null byte :raises PaddingError: pad is less than 1 bytes in length Example:: >>> d = NullStripped(Byte) >>> d.parse(b'\xff\x00\x00') 255 >>> d.build(255) b'\xff' rQcrr&)rPrBrVr>)rUrr>rRr r rV`rzNullStripped.__init__c Cs|j}t|}|dkrtdt|}|dkr||}nGt||}t|}|r:|| d|d|kr:||8}||dkr^|||||kr^||8}||dkr^|||||ksJ|d|}|jturk|St|jtury| |jj S|j t |||S)NrKz(NullStripped pad must be at least 1 byter)r>r0r r7rwrrr/rOrr4rrr) rUr1rrr>r/r3tailunitrHr r r rds(     zNullStripped._parsecCrUr&rVrr r r rzrozNullStripped._buildcCrr&rrr r r r}rbzNullStripped._sizeofcCs4t|jdkr ttddt|ji|j||S)NrK pad_rightr )r0r>rrrrr(r1r r r r0s"zNullStripped._emitfulltypervrr r rRr rBJsrBcrM) RestreamDataa Parses a field on external data (but does not build). Parsing defers to subcon, but provides it a separate BytesIO stream based on data provided by datafunc (a bytes literal or another BytesIO stream or Construct instances that returns bytes or context lambda). Building does nothing. Size is 0 because as far as other fields see it, this field does not produce or consume any bytes from the stream. :param datafunc: bytes or BytesIO or Construct instance (that parses into bytes) or context lambda, provides data for subcon to parse from :param subcon: Construct instance Can propagate any exception from the lambdas, possibly non-ConstructError. Example:: >>> d = RestreamData(b"\x01", Int8ub) >>> d.parse(b"") 1 >>> d.build(0) b'' >>> d = RestreamData(NullTerminated(GreedyBytes), Int16ub) >>> d.parse(b"\x01\x02\x00") 0x0102 >>> d = RestreamData(FixedSized(2, GreedyBytes), Int16ub) >>> d.parse(b"\x01\x02\x00") 0x0102 cs tt||||_d|_dSru)rPrrVdatafuncr)rUrrrRr r rVrzRestreamData.__init__cCs^t|j|}t|trt|}t|tjr|}t|tr't||||}|j|||Sr&) rrr:r;rrrrr)rUr1rrr3rlr r r rs     zRestreamData._parsecCr<r&r rr r r rrbzRestreamData._buildcCrrNr rr r r rrbzRestreamData._sizeofcCr)Nzrestream(%r, lambda io: %s))rrrrr r r rr;zRestreamData._emitparserr r rRr rs  rcr6) ra. Transforms bytes between the underlying stream and the (fixed-sized) subcon. Parsing reads a specified amount (or till EOF), processes data using a bytes-to-bytes decoding function, then parses subcon using those data. Building does build subcon into separate bytes, then processes it using encoding bytes-to-bytes function, then writes those data into main stream. Size is reported as `decodeamount` or `encodeamount` if those are equal, otherwise its SizeofError. Used internally to implement :class:`~construct.core.Bitwise` :class:`~construct.core.Bytewise` :class:`~construct.core.ByteSwapped` :class:`~construct.core.BitsSwapped` . Possible use-cases include encryption, obfuscation, byte-level encoding. .. warning:: Remember that subcon must consume (or produce) an amount of bytes that is same as `decodeamount` (or `encodeamount`). .. warning:: Do NOT use seeking/telling classes inside Transformed context. :param subcon: Construct instance :param decodefunc: bytes-to-bytes function, applied before parsing subcon :param decodeamount: integer, amount of bytes to read :param encodefunc: bytes-to-bytes function, applied after building subcon :param encodeamount: integer, amount of bytes to write :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises StreamError: subcon build and encoder transformed more or less than `encodeamount` bytes, if amount is specified :raises StringError: building from non-bytes value, perhaps unicode Can propagate any exception from the lambdas, possibly non-ConstructError. Example:: >>> d = Transformed(Bytes(16), bytes2bits, 2, bits2bytes, 2) >>> d.parse(b"\x00\x00") b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' >>> d = Transformed(GreedyBytes, bytes2bits, None, bits2bytes, None) >>> d.parse(b"\x00\x00") b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' crr&)rPrrV decodefunc decodeamount encodefunc encodeamount)rUrrrrrrRr r rVrzTransformed.__init__cCszt|jtdr t|}t|jtrt||j}||}|jtur$|St|jt ur2| |jj S|j t |||Sr&)r:rr/r7rr4rrrrOrr4rrrrjr r r rs    zTransformed._parsecCsjt}|j||||}|}||}t|jtr.t ||jkr.t dt ||jft |||S)NzQencoding transformation produced wrong amount of bytes, %s instead of expected %s) rrrrrrr:rrr0rr>rkr r r rs   zTransformed._buildcCs.|jdus |jdur t|j|jkr|jStr&)rrr rr r r rs  zTransformed._sizeofrr r rRr rs  $  rcr6) raE Transforms bytes between the underlying stream and the (variable-sized) subcon. Used internally to implement :class:`~construct.core.Bitwise` :class:`~construct.core.Bytewise` :class:`~construct.core.ByteSwapped` :class:`~construct.core.BitsSwapped` . .. warning:: Remember that subcon must consume or produce an amount of bytes that is a multiple of encoding or decoding units. For example, in a Bitwise context you should process a multiple of 8 bits or the stream will fail during parsing/building. .. warning:: Do NOT use seeking/telling classes inside Restreamed context. :param subcon: Construct instance :param decoder: bytes-to-bytes function, used on data chunks when parsing :param decoderunit: integer, decoder takes chunks of this size :param encoder: bytes-to-bytes function, used on data chunks when building :param encoderunit: integer, encoder takes chunks of this size :param sizecomputer: function that computes amount of bytes outputed Can propagate any exception from the lambda, possibly non-ConstructError. Can also raise arbitrary exceptions in RestreamedBytesIO implementation. Example:: Bitwise <--> Restreamed(subcon, bits2bytes, 8, bytes2bits, 1, lambda n: n//8) Bytewise <--> Restreamed(subcon, bytes2bits, 1, bits2bytes, 8, lambda n: n*8) c2tt||||_||_||_||_||_dSr&)rPrrVdecoder decoderunitencoder encoderunit sizecomputer)rUrrrrrrrRr r rV  zRestreamed.__init__cCs4t||j|j|j|j}|j|||}||Sr&)RestreamedBytesIOrrrrrrclose)rUr1rrrlrr r r r'szRestreamed._parsecCs6t||j|j|j|j}|j||||}||Sr&)rrrrrrrr)rUrr1rrrlr`r r r r-szRestreamed._buildcCs&|jdur td||j||S)Nz7Restreamed cannot calculate size without a sizecomputer)rr rrrr r r r3s zRestreamed._sizeofrr r rRr rs  rcr6) ProcessXoraU Transforms bytes between the underlying stream and the subcon. Used internally by KaitaiStruct compiler, when translating `process: xor` tags. Parsing reads till EOF, xors data with the pad, then feeds that data into subcon. Building first builds the subcon into separate BytesIO stream, xors data with the pad, then writes that data into the main stream. Size is the same as subcon, unless it raises SizeofError. :param padfunc: integer or bytes or context lambda, single or multiple bytes to xor data with :param subcon: Construct instance :raises StringError: pad is not integer or bytes Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> d = ProcessXor(0xf0 or b'\xf0', Int16ub) >>> d.parse(b"\x00\xff") 0xf00f >>> d.sizeof() 2 crr&)rPrrVpadfunc)rUrrrRr r rVRrzProcessXor.__init__cst|j|tttfstdttr tdkr tt|}ttr:dks:t fddt |D}ttr`tdkrMt tks`t ddt t |t t D}|jturg|St|jturu||jjS|jt|||S)N%ProcessXor needs integer or bytes padrKrc3|]}|AVqdSr&r r^rr=r r re_z$ProcessXor._parse..@cs|] \}}||AVqdSr&r r^rpr r r rebrf)rrr:rr;rr0rr7integers2bytesrr2rrcyclerrr/rOrr4rrrrjr r=r rVs"   & zProcessXor._parsecst|j|tttfstdttr tdkr tt }|j ||||}| }ttrGdksGt fddt|D}ttrmtdkrZttksmt ddtt|ttD}t|||S)NrrKrc3rr&r rr=r r retrz$ProcessXor._build..rcsrr&r rr r r rewrf)rrr:rr;rr0rrrrrrrrr2rrrr>rkr r=r ris   & zProcessXor._buildcCrWr&rXrr r r r{rzProcessXor._sizeofrr r rRr r:s  rcsLeZdZdZddeddDZfddZdd Zd d Zd d Z Z S)ProcessRotateLefta Transforms bytes between the underlying stream and the subcon. Used internally by KaitaiStruct compiler, when translating `process: rol/ror` tags. Parsing reads till EOF, rotates (shifts) the data *left* by amount in bits, then feeds that data into subcon. Building first builds the subcon into separate BytesIO stream, rotates *right* by negating amount, then writes that data into the main stream. Size is the same as subcon, unless it raises SizeofError. :param amount: integer or context lambda, shift by this amount in bits, treated modulo (group x 8) :param group: integer or context lambda, shifting is applied to chunks of this size in bytes :param subcon: Construct instance :raises RotationError: group is less than 1 :raises RotationError: data length is not a multiple of group size Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> d = ProcessRotateLeft(4, 1, Int16ub) >>> d.parse(b'\x0f\xf0') 0xf00f >>> d = ProcessRotateLeft(4, 2, Int16ub) >>> d.parse(b'\x0f\xf0') 0xff00 >>> d.sizeof() 2 cs$i|]fddtdDqS)cs$g|]}|>d@|d?BqS)rr r^ramountr r r`r|z'ProcessRotateLeft..)r~)r^r rr r{r|zProcessRotateLeft.rKrcs tt||||_||_dSr&)rPrrVrgroup)rUrrrrRr r rVrzProcessRotateLeft.__init__csrt|j|}t|j|dkrtd|d}|dt|}t|t|dkr2td|dkr7nbdkrLtj|t fddD}nM|ddkrqfdd t Dt fd dt dt|D}n(|ddfd d t Dt fd dt dt|D}|j t ur|St |j tur||j jS|j t|||S) NrK)group size must be at least 1 to be validrr,data length must be a multiple of group sizec3|]}|VqdSr&r r^a translater r rerz+ProcessRotateLeft._parse..cg|]}|qSr r r amount_bytesrr r r`raz,ProcessRotateLeft._parse..c3&|]}D] }||VqqdSr&r r^rry data_intsindicesr r rerac(g|]}||dfqSrKr rrr r r`(c3B|]}D]\}}||>d@||?BVqqdSrNr r^rk1k2amount1amount2r indices_pairsr r re@)rrrr#r7bytes2integersr0rprecomputed_single_rotationsrr~rrr/rOrr4rrr)rUr1rrrr3r rrrrrrrrr rs6     &( zProcessRotateLeft._parsec s\t|j|}t|j|dkrtd| d}|dt}|j||||}|}t |t |dkr@td|dkrEnbdkrZt j |t fddD}nM|ddkrfdd tDt fd dtdt |D}n(|ddfd d tDt fd dtdt |D}t|||S) NrKrrrrc3rr&r rrr r rerz+ProcessRotateLeft._build..crr r rrr r r`raz,ProcessRotateLeft._build..c3rr&r rrr r reracrrr rrr r r`rc3rrr rrr r rer)rrrr#rrrrrrr0rrrr~r>) rUrr1rrrrlr`r3r rr rs4    &( zProcessRotateLeft._buildcCrWr&rXrr r r rrzProcessRotateLeft._sizeof) rrrrKr~rrVrrrrYr r rRr rs %$rcr6) ChecksumaP Field that is build or validated by a hash of a given byte range. Usually used with :class:`~construct.core.RawCopy` . Parsing compares parsed subcon `checksumfield` with a context entry provided by `bytesfunc` and transformed by `hashfunc`. Building fetches the contect entry, transforms it, then writes is using subcon. Size is same as subcon. :param checksumfield: a subcon field that reads the checksum, usually Bytes(int) :param hashfunc: function that takes bytes and returns whatever checksumfield takes when building, usually from hashlib module :param bytesfunc: context lambda that returns bytes (or object) to be hashed, usually like this.rawcopy1.data :raises ChecksumError: parsing and actual checksum does not match actual data Can propagate any exception from the lambdas, possibly non-ConstructError. Example:: import hashlib d = Struct( "fields" / RawCopy(Struct( Padding(1000), )), "checksum" / Checksum(Bytes(64), lambda data: hashlib.sha512(data).digest(), this.fields.data), ) d.build(dict(fields=dict(value={}))) :: import hashlib d = Struct( "offset" / Tell, "checksum" / Padding(64), "fields" / RawCopy(Struct( Padding(1000), )), "checksum" / Pointer(this.offset, Checksum(Bytes(64), lambda data: hashlib.sha512(data).digest(), this.fields.data)), ) d.build(dict(fields=dict(value={}))) cs*tt|||_||_||_d|_dSru)rPrrV checksumfieldhashfunc bytesfuncr)rUrrrrRr r rVs  zChecksum.__init__cCsj|j|||}|||}||kr3tdt|ts|nt|t|ts+|ft|f|S)Nz$wrong checksum, read %r, computed %r) rrrrr$r:r;binasciir)rUr1rrhash1hash2r r r r#s zChecksum._parsecCs&|||}|j|||||Sr&)rrrr)rUrr1rrrr r r r,szChecksum._buildcCrWr&)rrrr r r r1rzChecksum._sizeofrr r rRr rs  * rc2eZdZdZd fdd ZddZddZZS) CompressedaC Compresses and decompresses underlying stream when processing subcon. When parsing, entire stream is consumed. When building, puts compressed bytes without marking the end. This construct should be used with :class:`~construct.core.Prefixed` . Parsing and building transforms all bytes using a specified codec. Since data is processed until EOF, it behaves similar to `GreedyBytes`. Size is undefined. :param subcon: Construct instance, subcon used for storing the value :param encoding: string, any of module names like zlib/gzip/bzip2/lzma, otherwise any of codecs module bytes<->bytes encodings, each codec usually requires some Python version :param level: optional, integer between 0..9, although lzma discards it, some encoders allow different compression levels :raises ImportError: needed module could not be imported by ctor :raises StreamError: stream failed when reading until EOF Example:: >>> d = Prefixed(VarInt, Compressed(GreedyBytes, "zlib")) >>> d.build(bytes(100)) b'\x0cx\x9cc`\xa0=\x00\x00\x00d\x00\x01' Nc stt||||_||_|jdkrddl}||_dS|jdkr*ddl}||_dS|jdkr8ddl}||_dS|jdkrFddl }||_dSddl }||_dS)Nzlibrgzipbzip2lzma) rPrrVr4levelrlibrbz2rcodecs) rUrr4rrrrrrrRr r rVJs"         zCompressed.__init__cCs&|jdvr |j|S|j||jS)Nrrrr)r4r decompressrrmr r r r\^s  zCompressed._decodecCsJ|jdvr|jdus|jdkr|j|S|j||jS|j||jS)Nrr)r4rrcompressrrmr r r r^cs  zCompressed._encoder&rrrrKrVr\r^rYr r rRr r5s rcr) Rebuffereda^ Caches bytes from underlying stream, so it becomes seekable and tellable, and also becomes blocking on reading. Useful for processing non-file streams like pipes, sockets, etc. .. warning:: Experimental implementation. May not be mature enough. :param subcon: Construct instance, subcon which will operate on the buffered stream :param tailcutoff: optional, integer, amount of bytes kept in buffer, by default buffers everything Can also raise arbitrary exceptions in its implementation. Example:: Rebuffered(..., tailcutoff=1024).parse_stream(nonseekable_stream) Ncs"tt||td|d|_dS)N) tailcutoff)rPrrVRebufferedBytesIOrl)rUrrrRr r rV|rzRebuffered.__init__cCs||j_|j|j||Sr&)rl substreamrrrr r r rszRebuffered._parsecCs||j_|j||j||Sr&)rlrrrrr r r rszRebuffered._buildr&rIr r rRr rls rcrG)Lazya Lazyfies a field. This wrapper allows you to do lazy parsing of individual fields inside a normal Struct (without using LazyStruct which may not work in every scenario). It is also used by KaitaiStruct compiler to emit `instances` because those are not processed greedily, and they may refer to other not yet parsed fields. Those are 2 entirely different applications but semantics are the same. Parsing saves the current stream offset and returns a lambda. If and when that lambda gets evaluated, it seeks the stream to then-current position, parses the subcon, and seeks the stream back to previous position. Building evaluates that lambda into an object (if needed), then defers to subcon. Size also defers to subcon. :param subcon: Construct instance :raises StreamError: requested reading negative amount, could not read enough bytes, requested writing different amount than actual data, or could not write all bytes :raises StreamError: stream is not seekable and tellable Example:: >>> d = Lazy(Byte) >>> x = d.parse(b'\x00') >>> x .execute> >>> x() 0 >>> d.build(0) b'\x00' >>> d.build(x) b'\x00' >>> d.sizeof() 1 cstt||dSr&)rPrrVrQrRr r rVrz Lazy.__init__cs tfdd}|S)Ncs0t}tj}t||Sr&)rDrBrr)rGrrr@rrUr1r r executes   zLazy._parse..executer)rUr1rrrr rr rsz Lazy._parsecCs t|r|}|j||||Sr&)rrrrr r r rsz Lazy._buildrIr r rRr rs   rc@s\eZdZdZddZddZddZdd Zd d Zd d Z ddZ eZ ddZ ddZ dS) LazyContainerrcCs(||_||_||_||_||_||_dSr&)_struct_stream_offsets_values_context_path)rUrr1offsetsr\rrr r r rVs  zLazyContainer.__init__cCs||jjvr ||Str&)r_subconsindexesr|r3r r r rks zLazyContainer.__getattr__cCsft|tr |jj|}||jvr|j|St|j|j||jj| |j|j |j }||j|<|Sr&) r:r:rr rrBrrrrrr rUrDrr r r rJs     zLazyContainer.__getitem__cC t|jjSr&)r0rrrTr r r __len__rzLazyContainer.__len__cCr r&)iterrrrTr r r keysrzLazyContainer.keyscfddjjDS)Nc3rr&r r^ryrTr r rerz'LazyContainer.values..rrrTr rTr r\r;zLazyContainer.valuescr)Nc3s|] }||fVqdSr&r rrTr r rerfz&LazyContainer.items..rrTr rTr r}r;zLazyContainer.itemscCs t||Sr&)r__eq__r;r r r rrzLazyContainer.__eq__cCsdt|jt|jjfS)Nz,)r0rrrrTr r r rrVzLazyContainer.__repr__N)rrrrKrVrkrJrrr\r}__iter__rrr r r r rs  rcrM) ra Equivalent to :class:`~construct.core.Struct`, but when this class is parsed, most fields are not parsed (they are skipped if their size can be measured by _actualsize or _sizeof method). See its docstring for details. Fields are parsed depending on some factors: * Some fields like Int* Float* Bytes(5) Array(5,Byte) Pointer are fixed-size and are therefore skipped. Stream is not read. * Some fields like Bytes(this.field) are variable-size but their size is known during parsing when there is a corresponding context entry. Those fields are also skipped. Stream is not read. * Some fields like Prefixed PrefixedArray PascalString are variable-size but their size can be computed by partially reading the stream. Only first few bytes are read (the lengthfield). * Other fields like VarInt need to be parsed. Stream position that is left after the field was parsed is used. * Some fields may not work properly, due to the fact that this class attempts to skip fields, and parses them only out of necessity. Miscellaneous fields often have size defined as 0, and fixed sized fields are skippable. Note there are restrictions: * If a field like Bytes(this.field) references another field in the same struct, you need to access the referenced field first (to trigger its parsing) and then you can access the Bytes field. Otherwise it would fail due to missing context entry. * If a field references another field within inner (nested) or outer (super) struct, things may break. Context is nested, but this class was not rigorously tested in that manner. Building and sizeof are greedy, like in Struct. :param \*subcons: Construct instances, list of members, some can be anonymous :param \*\*subconskw: Construct instances, list of members (requires Python 3.6) cstt|t|tdd|D}t||_tdd|jD|_tddt |jD|_ t dd|jD|_ dS)Ncsrr&r rxr r r rerfz&LazyStruct.__init__..csrr&rrr r r rer{css$|] \}}|jr|j|fVqdSr&rrCr r r res"csrr&rrr r r rer) rPrrVrr}rrrrrr rrrrRr r rVs  zLazyStruct.__init__cCrr&rr3r r r rk rzLazyStruct.__getattr__c Cst||jd|j|j|j|j||ddd }|jd||_t |}d|i}i}t |j D];\}}z|| |||7}t ||Wn tya||||} | ||<|jr[| ||j<t |}Ynw|||d<q-t||||||S)NrrrrrK)rrrrrrrrtrrDrrrrBr rrr) rUr1rrr@r r\rrrr r r rs$,   zLazyStruct._parsec Crrrrr r r r"s*,      zLazyStruct._buildc r)Nrrrc3rr&rrrr r re?rfz%LazyStruct._sizeof..rzrrr rr r:s,zLazyStruct._sizeof) rrrrKrVrkrrrrYr r rRr rs rc@sHeZdZdZddZddZddZdd Zd d Zd d Z ddZ dS)LazyListContainerrcCs.||_||_||_||_||_||_||_dSr&)_subconr_countrrrr )rUrr1rIr r\rrr r r rVGs zLazyListContainer.__init__csrt|trfddt|jDS|jvrj|Stjj|j jj j }|j|<|S)Ncsg|]}|qSr r rrTr r r`Rrz1LazyListContainer.__getitem__..) r:rGr~rrrrBrrrrrr r r rTr rJPs    zLazyListContainer.__getitem__cCs |tjkr|j}|t||Sr&)rmaxsizerrJrG)rUstartstopr r r __getslice__Zs zLazyListContainer.__getslice__cCrr&)rrTr r r r_rzLazyListContainer.__len__csfddtjDS)Nc3rr&r rrTr r recrz-LazyListContainer.__iter__..)r~rrTr rTr rbruzLazyListContainer.__iter__cs.ttkotfddtjDS)Nc3s |] }||kVqdSr&r rr<rUr r refr{z+LazyListContainer.__eq__..)r0rr~rr;r rr res.zLazyListContainer.__eq__cCsdt|j|jfS)Nz*)r0rrrTr r r rhrzLazyListContainer.__repr__N) rrrrKrVrJrrrrrr r r r rDs   rcr6) LazyArraya Equivalent to :class:`~construct.core.Array`, but the subcon is not parsed when possible (it gets skipped if the size can be measured by _actualsize or _sizeof method). See its docstring for details. Fields are parsed depending on some factors: * Some fields like Int* Float* Bytes(5) Array(5,Byte) Pointer are fixed-size and are therefore skipped. Stream is not read. * Some fields like Bytes(this.field) are variable-size but their size is known during parsing when there is a corresponding context entry. Those fields are also skipped. Stream is not read. * Some fields like Prefixed PrefixedArray PascalString are variable-size but their size can be computed by partially reading the stream. Only first few bytes are read (the lengthfield). * Other fields like VarInt need to be parsed. Stream position that is left after the field was parsed is used. * Some fields may not work properly, due to the fact that this class attempts to skip fields, and parses them only out of necessity. Miscellaneous fields often have size defined as 0, and fixed sized fields are skippable. Note there are restrictions: * If a field references another field within inner (nested) or outer (super) struct, things may break. Context is nested, but this class was not rigorously tested in that manner. Building and sizeof are greedy, like in Array. :param count: integer or context lambda, strict amount of elements :param subcon: Construct instance, subcon to process individual elements crr&)rPrrVrI)rUrIrrRr r rVrzLazyArray.__init__c Cs|j}|j}t|r||}d|kstd|ft|}d|i}i}t|D]1} z|||||7}t||WntyQ| |||} | || <t|}Ynw||| d<q't |||||||S)NrrrK) rrIrrrDr~rrBr rr) rUr1rrrrIr@r r\rrr r r rs(   zLazyArray._parsec Crrrrr r r rs  zLazyArray._buildc Crryrrr r r rszLazyArray._sizeofrr r rRr rls  rcrG) LazyBounda Field that binds to the subcon only at runtime (during parsing and building, not ctor). Useful for recursive data structures, like linked-lists and trees, where a construct needs to refer to itself (while it does not exist yet in the namespace). Note that it is possible to obtain same effect without using this class, using a loop. However there are usecases where that is not possible (if remaining nodes cannot be sized-up, and there is data following the recursive structure). There is also a significant difference, namely that LazyBound actually does greedy parsing while the loop does lazy parsing. See examples. To break recursion, use `If` field. See examples. :param subconfunc: parameter-less lambda returning Construct instance, can also return itself Example:: d = Struct( "value" / Byte, "next" / If(this.value > 0, LazyBound(lambda: d)), ) >>> print(d.parse(b"\x05\x09\x00")) Container: value = 5 next = Container: value = 9 next = Container: value = 0 next = None :: d = Struct( "value" / Byte, "next" / GreedyBytes, ) data = b"\x05\x09\x00" while data: x = d.parse(data) data = x.next print(x) # print outputs Container: value = 5 next = \t\x00 (total 2) # print outputs Container: value = 9 next = \x00 (total 1) # print outputs Container: value = 0 next = (total 0) crur&)rPrrV subconfunc)rUr rRr r rVrvzLazyBound.__init__cCs|}||||Sr&)r r)rUr1rrrr r r rszLazyBound._parsecCs|}|||||Sr&)r r)rUrr1rrrr r r rszLazyBound._buildrIr r rRr rs  1rcr) ExprAdapteraJ Generic adapter that takes `decoder` and `encoder` lambdas as parameters. You can use ExprAdapter instead of writing a full-blown class deriving from Adapter when only a simple lambda is needed. :param subcon: Construct instance, subcon to adapt :param decoder: lambda that takes (obj, context) and returns an decoded version of obj :param encoder: lambda that takes (obj, context) and returns an encoded version of obj Example:: >>> d = ExprAdapter(Byte, obj_+1, obj_-1) >>> d.parse(b'\x04') 5 >>> d.build(5) b'\x04' cs0tt||fdd|_fdd|_dS)Nc ||Sr&r rctxr)rr r r) rz&ExprAdapter.__init__..cr"r&r r#rr r r) rrPr!rVr\r^)rUrrrrR)rrr rVzExprAdapter.__init__rr r rRr r!r!cr)ExprSymmetricAdapteru Macro around :class:`~construct.core.ExprAdapter`. :param subcon: Construct instance, subcon to adapt :param encoder: lambda that takes (obj, context) and returns both encoded version and decoded version of obj Example:: >>> d = ExprSymmetricAdapter(Byte, obj_ & 0b00001111) >>> d.parse(b"ÿ") 15 >>> d.build(255) b'' cs0tt||fdd|_fdd|_dS)Ncr"r&r r#r%r r r)rz/ExprSymmetricAdapter.__init__..cr"r&r r#r%r r r) rr&)rUrrrRr%r rVr'zExprSymmetricAdapter.__init__rr r rRr r)sr)cr) ExprValidatora Generic adapter that takes `validator` lambda as parameter. You can use ExprValidator instead of writing a full-blown class deriving from Validator when only a simple lambda is needed. :param subcon: Construct instance, subcon to adapt :param validator: lambda that takes (obj, context) and returns a bool Example:: >>> d = ExprValidator(Byte, obj_ & 0b11111110 == 0) >>> d.build(1) b'\x01' >>> d.build(88) ValidationError: object failed validation: 88 cs"tt||fdd|_dS)Ncr"r&r r# validatorr r r)5rz(ExprValidator.__init__..)rPr*rVrg)rUrr,rRr+r rV3rzExprValidator.__init__rr r rRr r*#r(r*ct|fddS)a7 Validates that the object is one of the listed values, both during parsing and building. .. note:: For performance, `valids` should be a set/frozenset. :param subcon: Construct instance, subcon to validate :param valids: collection implementing __contains__, usually a list or set :raises ValidationError: parsed or build value is not among valids Example:: >>> d = OneOf(Byte, [1,2,3]) >>> d.parse(b"\x01") 1 >>> d.parse(b"\xff") construct.core.ValidationError: object failed validation: 255 cs|vSr&r rr$validsr r r)KrzOneOf..r*)rr0r r/r OneOf8sr2cr-)az Validates that the object is none of the listed values, both during parsing and building. .. note:: For performance, `valids` should be a set/frozenset. :param subcon: Construct instance, subcon to validate :param invalids: collection implementing __contains__, usually a list or set :raises ValidationError: parsed or build value is among invalids cs|vSr&r r.invalidsr r r)ZrzNoneOf..r1)rr4r r3r NoneOfNs r5cst|fddS)a Filters a list leaving only the elements that passed through the predicate. :param subcon: Construct instance, usually Array GreedyRange Sequence :param predicate: lambda that takes (obj, context) and returns a bool Can propagate any exception from the lambda, possibly non-ConstructError. Example:: >>> d = Filter(obj_ != 0, Byte[:]) >>> d.parse(b"\x00\x02\x00") [2] >>> d.build([0,1,0,2,0]) b'\x01\x02' csfdd|DS)Ncsg|] }|r|qSr r )r^r+)r$rr r r`nsz,Filter....r r.r)r$r r)nrzFilter..)r))rrr rr Filter]sr6cs2eZdZdZd fdd ZddZdd ZZS) Slicinga Adapter for slicing a list. Works with GreedyRange and Sequence. :param subcon: Construct instance, subcon to slice :param count: integer, expected number of elements, needed during building :param start: integer for start index (or None for entire list) :param stop: integer for stop index (or None for up-to-end) :param step: integer, step (or 1 for every element) :param empty: object, value to fill the list with, during building Example:: d = Slicing(Array(4,Byte), 4, 1, 3, empty=0) assert d.parse(b"\x01\x02\x03\x04") == [2,3] assert d.build([2,3]) == b"\x00\x02\x03\x00" assert d.sizeof() == 4 rKNcrr&)rPr7rVrIrrstepempty)rUrrIrrr8r9rRr r rVrzSlicing.__init__cCs||j|j|jSr&)rrr8rar r r r\rzSlicing._decodecCsb|jdur|S|jdur|jg|j}|||jd|j<|S|jg|j}|||j|j|j<|Sr&)rrr9rIr8rUrrrr!r r r r^s  zSlicing._encode)rKNrr r rRr r7qs r7cr) IndexingaM Adapter for indexing a list (getting a single item from that list). Works with Range and Sequence and their lazy equivalents. :param subcon: Construct instance, subcon to index :param count: integer, expected number of elements, needed during building :param index: integer, index of the list to get :param empty: object, value to fill the list with, during building Example:: d = Indexing(Array(4,Byte), 4, 2, empty=0) assert d.parse(b"\x01\x02\x03\x04") == 3 assert d.build(3) == b"\x00\x00\x03\x00" assert d.sizeof() == 4 Ncs&tt||||_||_||_dSr&)rPr;rVrIrDr9)rUrrIrDr9rRr r rVs zIndexing.__init__cCs ||jSr&)rDrar r r r\rzIndexing._decodecCs|jg|j}|||j<|Sr&)r9rIrDr:r r r r^s zIndexing._encoder&rr r rRr r;s r;r&rrv)rrrrr rrostempfiler importlibr construct.libconstruct.exprconstruct.versionr/rr r rrrrrrrrrrrrrrrrrrr r!r"r#r$r%r,r4r7r>rBrDrIrLrMrprrrrrrrrrNr[rdrfrirnrtrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrByteShortIntLongrrrrr r SingleDoublerrrrrrrrrrr1r5r7rCrJrNrOrPr rWrFrXr_rprrrrrrHrrrrrrrrrrrrrrrr6r8r>rrHrKrRrQrXrfrgrmrlrwr}r4r~rrrrJrrrrrIrrArLrBrrrrrrrrrrrrrrrr!r)r*r2r5r6r7r;r r r r s`   K >(+JRU                                        5  S ^- M@X>@4+/0,$z DF<:, : = I.5'S F F;1$$" 6  L,DK<4K5ErD7 0/Y(JB   '