o Rŀg@@s^dZddlZddlZddlZddlZddlZddlZddlmZddlm Z ddl Z ddl m Z ddl mZddlmZddlmZed d ZGd d d eZed dedddd edddd edddd edddd edddd edddd edddd edd d!d edd"d#d ed$d%d&d ed'd(d)d ed'd*d+d g e_Gd,d-d-ejZGd.d/d/ejZGd0d1d1ej ZGd2d3d3ZGd4d5d5eZGd6d7d7ZGd8d9d9ZGd:d;d;eZGdd?d?eZ Gd@dAdAZ!GdBdCdCZ"GdDdEdEe!Z#GdFdGdGZ$GdHdIdIZ%GdJdKdKZ&GdLdMdMZ'GdNdOdOZ(GdPdQdQZ)dS)RaBio.Align support for alignment files in the bigBed format. The bigBed format stores a series of pairwise alignments in a single indexed binary file. Typically they are used for transcript to genome alignments. As in the BED format, the alignment positions and alignment scores are stored, but the aligned sequences are not. See http://genome.ucsc.edu/goldenPath/help/bigBed.html for more information. You are expected to use this module via the Bio.Align functions. N) namedtuple)BytesIO) Alignment) interfaces)Seq) SeqRecordField)as_typenamecommentcsZeZdZUdZded<ddZeddZeddZd d Z d d Z fd dZ Z S) AutoSQLTablezJAutoSQL table describing the columns of an (possibly extended) BED format.defaultcCs||_||_||dd<dS)zKCreate an AutoSQL table describing the columns of an (extended) BED format.N)r r )selfr r fieldsrD/var/www/html/myenv/lib/python3.10/site-packages/Bio/Align/bigbed.py__init__MszAutoSQLTable.__init__cCs|dsJ|dd}|dd\}}|dksJ|dd\}}|ds,J|dd}|d|}||dd}|dsIJ|dsPJ|dd}g}|r|d}|d|d}||d|} |d|} | d sJ| dddd\} } | d r| d }| d|} n| } | d vsJt| | | }| |||dd}|s\t |||S) z?Return an AutoSQLTable initialized using the bytes object data.Ntable"();][) intuintshortushortbyteubytefloatcharstringlstring) endswithdecodesplit startswithfindstripindexrsplitrappendr )clsdatatextwordr ir rj field_comment definition field_type field_name data_typefieldrrr from_bytesSs<         zAutoSQLTable.from_bytescCs||dS)z@Return an AutoSQLTable initialized using the string object data.r)r<encode)r0r1rrr from_stringszAutoSQLTable.from_stringcCstdd|D}tdd|Dd}g}|d|j|d|j|d|D]}|jd}|d |d |d |j||jfq-|d d |S)Ncss|] }tt|jVqdSN)lenstrr .0r;rrr sz'AutoSQLTable.__str__..css|]}t|jVqdSr?)r@r rBrrrrDsrz table %s z"%s" z( rz %-zs %-z s "%s" z) )maxr/r r r join)r type_width name_widthlinesr;r rrr__str__s$    zAutoSQLTable.__str__cCst|dS)Nr)rAr=rrrr __bytes__szAutoSQLTable.__bytes__cs2t|trt|}t|j|j|St|Sr?) isinstanceslicesuper __getitem__r r r )rr4r __class__rrrQs   zAutoSQLTable.__getitem__) __name__ __module__ __qualname____doc____annotations__r classmethodr<r>rKrMrQ __classcell__rrrRrr Hs  , r bedzBrowser Extensible Datar%chromz)Reference sequence chromosome or scaffoldr chromStartzStart position in chromosomechromEndzEnd position in chromosomer z Name of item.scorezScore (0-1000)zchar[1]strandz+ or - for strand thickStartz4Start of where display should be thick (start codon)thickEndz1End of where display should be thick (stop codon)reservedz Used as itemRgb as of 2004-11-22r blockCountzNumber of blockszint[blockCount] blockSizesz#Comma separated list of block sizes chromStartsz&Start positions relative to chromStartcs`eZdZdZdZdZ       dfd d Zd d ZddZddZ ddZ ddZ Z S)AlignmentWriterz1Alignment file writer for the bigBed file format.bigBedwb NTrc sR|dks|dkr tdt|||_||_||_||_||_||_||_ dS)aCreate an AlignmentWriter object. Arguments: - target - output stream or file name. - bedN - number of columns in the BED file. This must be between 3 and 12; default value is 12. - declaration - an AutoSQLTable object declaring the fields in the BED file. Required only if the BED file contains extra (custom) fields. Default value is None. - targets - A list of SeqRecord objects with the chromosomes in the order as they appear in the alignments. The sequence contents in each SeqRecord may be undefined, but the sequence length must be defined, as in this example: SeqRecord(Seq(None, length=248956422), id="chr1") If targets is None (the default value), the alignments must have an attribute .targets providing the list of SeqRecord objects. - compress - If True (default), compress data using zlib. If False, do not compress data. Use compress=False for faster searching. - blockSize - Number of items to bundle in r-tree. See UCSC's bedToBigBed program for more information. Default value is 256. - itemsPerSlot - Number of data points bundled at lowest level. See UCSC's bedToBigBed program for more information. Use itemsPerSlot=1 for faster searching. Default value is 512. - extraIndex - List of strings with the names of extra columns to be indexed. Default value is an empty list. rjzbedN must be between 3 and 12N) ValueErrorrPrbedN declarationtargetscompressextraIndexNames itemsPerSlot blockSize) rtargetrorprqrrrtru extraIndexrRrrrs/  zAlignmentWriter.__init__cCs|jdur |j}n|j}t}|j|_|jdur6z |jd|j|_Wnty5tjd|j|_Ynw|j}t||_ t |j |}| |||\}}| t|j| ttj||_| t|||_| ttj||_| t|j||_t |t|jt||||_t|} | t|dtj|t|| ||| |\} } |j!rt"| |j#t$j|_%nd|_%||_&t' | |jd|j&||(|||j&|j|| \} } t| |_)|D]}||_*|j+,t |j+|j|q|-d| t|| t| |-|j| t| |j|ks6J|.||-dt/j0|j1dtj}| |dS)zWrite the alignments to the file stream, and return the number of alignments. alignments - A list or iterator returning Alignment objects stream - Output file stream. Nrr)2rq_HeaderrodefinedFieldCountrpAttributeErrorr r r@ fieldCount _ExtraIndicesrs_get_chrom_usagewritebytessize _ZoomLevelstell autoSqlOffsettotalSummaryOffset_SummaryextraIndicesOffsetchromosomeTreeOffset_BPlusTreeFormatterminrufullDataOffsetcalculate_reductionsto_bytessys byteorder initializewrite_alignmentsrrrFrt_RegionSummaryuncompressBufSizefullIndexOffset_RTreeFormatter_write_zoom_levels zoomLevels fileOffsetchunkssortseektofileioSEEK_END signature)rstream alignmentsrqheaderrp extra_indiceschromUsageListaveSize reductions maxBlockSizeregionszoomListtotalSum extra_indexr1rrr write_files                    zAlignmentWriter.write_filec Csd}d}d}d}d}g} d} d} tj} |D]} | jdj}| jd}| jd}|D]}|| q)||krCtd|d|d|d |d 7}|||7}||kr||kr]td |d |rk| ||| f|d 7}|D] }|j|krvn qmtd |d |d |}t| t |} t |} d}|| krtd|d|d| d|d |dkr||}|| kr|dkrtd|d |} |}q|r| ||| ft j | dd| fddgd} |dkr||}||_ | |fS)NrrErrrrrzend (z) before start (z) in alignment [rrz7alignments are not sorted by target name at alignment [zfailed to find target 'z' in target list at alignment [zend coordinate z bigger than z size of z at alignment [z]'z(alignments are not sorted at alignment [r S)id=i4rrdtype) rmaxsize sequencesr coordinatesupdateMaxFieldSizernr/rFr@nparray_len)r0rrqrrchromId totalBasesbedCountr rkeySize chromSizeminDiff alignmentr\startendrrv lastStartdiffrrrrfs~        z AlignmentWriter._get_chrom_usagecCst}t}t|dkrd|_d|_||fS|j}|j} |j} |d} ||d_ |D]} | dt j } | r;| d} | | krAnq,|d} | d || t j }| rZt ||}nt||}g}g}t||}t| d}|tj}|D]4}tj }|||}|D]%}|||||||jkrt|}|j}||q||7}qqv|t|| dksJ| d|d_|}||d_t||| ||| rt ||}nt|t j }||| ||| ||fS)Nrrscale) rrr@minValmaxValrurrrtr dataOffsetrrr_ZippedBufferedStream_BufferedStream _RangeTreegeneraterbbiResIncrementrrgenerate_summariesrr/rcopyrflushreductionLevel indexOffsetrreduce)rroutputdataSizerrrrru doCompressrtmaxReducedSizeinitialReduction reducedSizerbufferr rezoomedListtreesrdoubleReductionSizetreer summariessummaryrezoomedrrrrrsp 5                  z"AlignmentWriter._write_zoom_levelsc Cs|j}g}|jj}t|dkrtd|dt|dkr!td|jd}|jd}||kr:td|d |d |d kr\|jj}|d krId }nt|dkrWtd|d|||dkrz|j}Wn t ypd }Ynw|dksy|dkrtd|dt |}|||dkr|jd|jdkrd} nd} || |dkrz|j } Wn t y|} Ynw|t | |dkr-z|j } Wn t y|} YnSw| | krtd| d| d | dkr| |ks| |krtd|d|d|d | d| | dkr&| |ks| |kr&td!|d|d|d | d| |t | |d"kr|z|j } Wn t yCd } Yn4w| d#d#} t| d kr^td$d%| Dr^ndt| krld&kronnntd'| d(|| |d)krt|j}t|dkdd*k}|d|}t|}|t ||d+kr|d#d,d%|Dd#|d-kr|jddd.f||}|d#d/d%|Dd#|d0kr|d1krtd2|d3|jd4}|jd5}t|}|t|ksJ|t ||d#||d#d6d%|D|j|dD]2}|j|j}t|t r8||q#t|ttfrI|t |q#|d#tt |q#d7|}||||fS)8Nzalignment target name 'z.' is too long (must not exceed 254 characters)rz.alignment target name cannot be blank or emptyrrzchromStart after chromEnd (z > rrmrE.zalignment query name 'z-' is too long (must not exceed 255 charactersryrizscore (z) must be between 0 and 1000)rr)rr+-z thickStart (z) after thickEnd (zthickStart out of range for :z, thick:zthickEnd out of range for rx,css,|]}dt|kodknVqdS)rrlN)r)rCcolorrrrrD0s*z2AlignmentWriter._extract_fields..lz?Expecting color to consist of r,g,b values from 0 to 255. Got '' r cs|]}t|VqdSr?rA)rCrurrrrD@ rcsrr?r)rCr]rrrrDCrrjzUnexpected value z for bedN in _extract_fieldsexpIds expScorescsrr?r)rCexpScorerrrrDMr )rorvrr@rnrqueryr/r_r|rArarbitemRgbrstripr)allrrrsumrFrG annotationsrpr rNr#mapr=)rrrorowr\r]r^r r_r`rarbrcolorsstepsalignedrerdrfrrexpCountr;valuerestrrr_extract_fieldss                      "              zAlignmentWriter._extract_fieldscCsB|j}d}d}d}d} d} g} |jdurt} nt} d} td}d}d}t|} zt|}Wn ty>|}d}Ynw| |\}}}}|| kr\| durR|}|} |d7}d|d<||kr| }| }|| krn|} | }| || d| d|r| }||}|D] }||||| q| }||_|dur | | fSd}|dkrt|||}| |n||jkr||_|d7}|D]6}||dkr|d d7<||d |d<||dkr|d d7<|d|d 7<||dksq|r|D] }||| q| d7} ||||}| ||d q+) zWrite alignments to the output file, and return the number of alignments. alignments - A list or iterator returning Alignment objects stream - Output file stream. rrNTz=IIIFrrrrr)rtrr _ZippedStreamrstructStructiternext StopIterationr rgetvaluerrtruncate addOffsetSizeoffset_Regionr/raddKeysFromRowpack)rrrrrrtritemIxsectionStartIx sectionEndIx currentChromrrr formatterdoneregionrr\rrr blockStartOffsetrr1blockEndOffsetrurrrrrrYs             z AlignmentWriter.write_alignments)rjNNTrkrlr) rTrUrVrWfmtmoderrrrr rrZrrrRrrgs":NC>rrgc@sbeZdZdZdZdZddZddZdd Zd d Z d d Z ddZ ddZ ddZ dddZdS)AlignmentIteratorzAlignment iterator for bigBed files. The pairwise alignments stored in the bigBed file are loaded and returned incrementally. Additional alignment information is stored as attributes of each alignment. rhbc Cs2t|}|j}|j}||_|j}|j}|j}t|}||t |j } |D]&} | j } | | || j } | | \ } }}}}}}}|_| t jksMJq'||||_| |t|d|d\}||_|jdkrrd|_nd|_| |jt|||_| |jt |||_|||_dS)NQrxrTF)rzfromfilerrr}r{rrreadrformatter_headerrrrunpackrtr _read_autosqlrpr _lengthr _compressedrrrqrr_iterate_index_data)rrrrrr}r{rrr zoomLevelrr1rrunItems startChromId startBase endChromIdendBase endFileOffset dataCountrrr _read_headersL          zAlignmentIterator._read_headercCsN|j|j}|j}|j|_||j||}t|}| |||j|Sr?) rrr}r{rorr(r r<_analyze_fields)rrr autoSqlSizer}r1rprrrr+s    zAlignmentIterator._read_autosqlc Cs d}t|jD]}||j}|||krtd|||fq||kr&g|_t||D]X}||j}||j}d|vrMd|vrMd} |d\}} |}nd} |dvrVt} n|dvr]t } n|d krdt } n |d vrkt } nt d || r{| } | fd d } |j || gq+dS)N) r\r]r^r r_r`rarbrcrdrerfz$Expected field name '%s'; found '%s'rrTF)rrrr )r!r"r#)r#r$r%r&zUnknown field type %scs"|dd}fdd|DS)Nrcsg|]}|qSrr)rCr item_converterrr #zHAlignmentIterator._analyze_fields..converter..)rr))r1r<valuesrr;r converter!sz4AlignmentIterator._analyze_fields..converter)rangeror rn_custom_fieldsr r)r,rrr#rA Exceptionr/) rrr}r{namesr4r r9r8 make_array_r@r<rrrr9s@       z!AlignmentIterator._analyze_fieldsc csFt|jd}|j}|j} z|j}Wnty||j| |j }|j dkr2t |}|ra||d|\}}} |d|d} ||| } || d}||| | dt| fV|s4 |j} | durlYdSt| jD]\} }t|t|krnqqtdz | j| d}Wn ty| }YnwnqbYnw|d}q)NIIITrrFailed to find child node)r rrrrchildrenr|rrr(rr-zlib decompressr*r-r@parent enumerater RuntimeError IndexError)rrrrnoderIr1rr]r^r4r rLr-childrrrr.'sN         z AlignmentIterator._iterate_indexc csPt|jd}|j}|d}|d}|j} z| j} Wnty|| j| | j } |j dkr:t | } |jdkro|| d|\} } }| |krn| |krW||ksc| |krn| |ksc||krn| | || |t| fVnVd}t| }||kr||}|| ||\} } }|}| d|d}| ||}|}| |krqu|| ks||kr| |krqu| |kr||krqu| | ||dt|fV||ksyYn%wd}| D]}|j|jf||fkrq||f|j|jfkrqd}|r|} q | j}|durdSt|jD]\}}t| t|kr nqtdz |j|d} Wn ty$|} Ynwnqq)NrGrTrrFrH)r rrrrrIr|rrr(rr-rJrKrtr*r@r- endChromIxr5 startChromIxr3rLrMrrNrO)rrchromIxrrrr padded_start padded_endrPrIr1 child_chromIxchild_chromStartchild_chromEndr4nr5r  visit_childrQrLr-rrr _search_indexPs           2zAlignmentIterator._search_indexc CsHzt|j}Wn tyYdSw|\}}}}}}|||||||Sr?)rr/r_create_alignment) rrrrr]r^r  dataStartdataEndrrr_read_next_alignments  z&AlignmentIterator._read_next_alignmentcCsL||ddks J|||d}|r|d}ng}|j|}|jdkr-|d} nd} |jdkr9|d} nd} |jdkrt|d } t|d d d t} t|d d d t} t| | krrt d t| | ft| | krt dt| | fd}d}||gg}t | | D] \}}||kr| ||g|}||7}||7}| ||gqt | }t| }n||}t d|gd|gg}|}|dddf|7<td|d}t|| d}||g}| dkr||dddf|dddf<||dkrt d||df||dkr t d||dft||}t||jdkrPi|_t ||jdd|jD]\}}|\} }|||j| <q?|jdkrX|S|d}zt|}Wn t ylYn w|rvt|}||_|jd kr|St|d|_|jd kr|St|d|_|jd kr|S|d|_|S)Nrrrrmrrrrrrrrxz:Inconsistent number of block sizes (%d found, expected %d)zDInconsistent number of block start positions (%d found, expected %d)lengthrrrz/Inconsistent chromStart found (%d, expected %d)rz-Inconsistent chromEnd found (%d, expected %d)ry)r(r)rqrorrfromiterrr@rnzipr/r transposerrrrrrBr# is_integerr_rarbr)rrr]r^r r^r_words target_recordr r`rdre blockStarts tPosition qPositionrru blockStartqSizequery_sequence query_recordrecordsrr3 custom_fieldr@r_rrrr]s                   "       z#AlignmentIterator._create_alignmentcCs|jSr?)r,rLrrr__len__szAlignmentIterator.__len__Nc cs|j}|dur|dus|durtdn2t|jD] \}}|j|kr%nqtd||dur?|dur;d}t|}n td|durG|d}|||||}|D]}|\}} } } } } ||| | | | | }|VqQdS)aIterate over alignments overlapping the specified chromosome region.. This method searches the index to find alignments to the specified chromosome that fully or partially overlap the chromosome region between start and end. Arguments: - chromosome - chromosome name. If None (default value), include all alignments. - start - starting position on the chromosome. If None (default value), use 0 as the starting position. - end - end position on the chromosome. If None (default value), use the length of the chromosome as the end position. Nz5start and end must both be None if chromosome is NonezFailed to find %s in alignmentsrz!end must be None if start is Noner)_streamrnrMrqrr@r\r])r chromosomerrrrTrvr1rr]r^r r^r_rrrrsearchs8    zAlignmentIterator.search)NNN)rTrUrVrWr"r#r8r+r9r.r\r`r]rsrvrrrrr$s, 3)[ \r$cseZdZfddZZS)r cst}t|Sr?)rPrrJrrrr1rRrrrBs  z_ZippedStream.getvalue)rTrUrVrrZrrrRrr Asr c@s$eZdZddZddZddZdS)rcCst|_||_||_dSr?)rrrr)rrrrrrrHs z_BufferedStream.__init__cCsb|j|_t|}|j||j|jkr/|j|j|jd|j ddSdSNr) rrrrrrrrrrritemr1rrrrMs   z_BufferedStream.writecCs.|j|j|jd|jddSrx)rrrrrrrLrrrrVs z_BufferedStream.flushN)rTrUrVrrrrrrrrGs rc@seZdZddZddZdS)rcCsh|j|_t|}|j||j|jkr2|jt|j |j d|j ddSdSrx) rrrrrrrrJrrrrrryrrrr]s   z_ZippedBufferedStream.writecCs4|jt|j|jd|jddSrx)rrrJrrrrrrrLrrrrfs z_ZippedBufferedStream.flushN)rTrUrVrrrrrrr\s rc@s<eZdZdZedZejZdZdZ e ddZ ddZ d S) rz) rrrrrr}r{rrrrz =IHHQQQHHQQIQlrryc Cs|d}tj|ddtjkrd}ntj|ddtjkrd}ntdt|d}t}||_|j }||}| |\ }|_ |_ |_ |_|_|_|_|_|_|_|tjksYJ|j} | d ksd| d krjtd | |S) Nrylittler<big>znot a bigBed file HHQQQHHQQIQrmrjz+expected between 3 and 12 columns, found %d)r(rr<rzrrnr rrrr*rrrrr}r{rrrrbbiCurrentVersion) r0rmagicrrrrr1versionr{rrrr's>   z_Header.fromfilecCs:tjtjtj|j|j|j|j|j |j |j |j |j |j Sr?)rzrrrrrrrrr}r{rrrrrLrrrrMsz_Header.__bytes__N) rTrUrV __slots__r rrrrrrYr'rMrrrrrzls   "rzc@sBeZdZdZedZddZddZddZ d d Z d d Z d S) _ExtraIndex) indexField maxFieldSizerr get_valuez =xxHQxxxxHxxcsd|_d|_t|D] \}}|jkrnq tdd|jdkr%td||_dkr3dd|_dSdkr>d d|_dSfd d|_dS) NrzCextraIndex field %s not a standard bed field or found in 'as' file.r%z+Sorry for now can only index string fields.r\cS|jjSr?)rvrrrrrz&_ExtraIndex.__init__..r cSrr?)rrrrrrrrcs |jSr?)rrr rrrs )rrrMr rnr rr)rr rpr-r;rrrrs(  z_ExtraIndex.__init__cCs*||}t|}||jkr||_dSdSr?)rr@r)rrr rrrrrs   z_ExtraIndex.updateMaxFieldSizecCs ||}||j|d<dS)Nr )rr=r)rrrecordIxr rrrrs z_ExtraIndex.addKeysFromRowcCs(||j||d<||j||d<dS)Nrr)r)rrrstartIxendIxrrrrsz_ExtraIndex.addOffsetSizecCsd}|j||j|jSNr)rrrr)rindexFieldCountrrrrMsz_ExtraIndex.__bytes__N) rTrUrVrr rrrrrrrMrrrrrs  rc@s:eZdZedZddZeddZddZ dd Z d S) r~z=HHQ52xcsfdd|D|dd<dS)Ncsg|]}t|qSr)r)rCr rprrr=sz*_ExtraIndices.__init__..r)rrDrprrrrsz_ExtraIndices.__init__cCs|jjtjjt|Sr?)rrrr@rLrrrrsz_ExtraIndices.sizecCsJ|dkrdS|D]}|j}tdd|fddg}tj||d|_qdS)Nrr z=S)r=u8)rrr)rrrzerosr)rrrrrrrrrsz_ExtraIndices.initializecCsx|jj}t|dkr-||}|j|t||}|||D] }|t|q!dS|j|dd}||dSrx)rrr@rrrr)rrrrr1rrrrrs   z_ExtraIndices.tofileN) rTrUrVr rrrpropertyrrrrrrrr~s   r~c@s.eZdZgdZd ddZddZddZd S) _ZoomLevel)rrrr=cCst|d|_dS)NIxxxxQQ)r rrrrrrrrsz_ZoomLevel.__init__cCs|j|j|j|jSr?)rrrrrrLrrrrMs z_ZoomLevel.__bytes__cCsB||jj}|j|\}}}|dkrt||_||_||_dSrx)r(rrr*rrrr)rrr1rrrrrrr($s z_ZoomLevel.readNr)rTrUrVrrrMr(rrrrrs   rc@sReZdZdZdZedjjeZdddZddZ dd Z e d d Z d d Z dS)rryrrcs$fddttjD|dd<dS)Ncsg|]}tqSr)r)rCr4r|rrr=4r>z(_ZoomLevels.__init__..)rArbbiMaxZoomLevelsrrr|rr3s$z_ZoomLevels.__init__cCs.ddd|D}|ttjt|7}|S)Ncsrr?)r)rCrzrrrrD7rz(_ZoomLevels.__bytes__..)rGrrrr@rwrrrrM6sz_ZoomLevels.__bytes__c CsFttjD]}z |||Wqty ||d=YdSwdSr?)rArrr(r)rrrrrrr(;s  z_ZoomLevels.readcCsxtj}tj|gdd}d}tt||}t|jdj}t|D]}||kr*n |||d<|tj 9}q"|d|S)N))rrr)rrrrr) rrrrrFriinforrAr)r0rrrminZoomresmaxIntresTryrrrrCs    z _ZoomLevels.calculate_reductionscCs6|d}t|dtj}|j}t} tdtjD]z} t|} | |kr%no| }||| _ | dt j } | | |D]} | | q<||}| |||||||| _||| _|tj9}d}d}|D]#} | j|ksv| j|kr| j|}| j}| }|||<|d7}qj|| 7}qj||d=q|| d=dS)Nrrrryr)rrrrrrArr@rrrrrrrrrrrr)rrrrrurt zoomCount reductionrrr rezoomCountr1rrr4rrcurrentSummaryrrrrTs@         z_ZoomLevels.reduceNr)rTrUrVrrrrrrrMr(rYrrrrrrr.s   rc@s8eZdZdZedZejZddZddZ ddZ d S) r validCountrrsumData sumSquaresz=QddddcCs(d|_tj|_tj |_d|_d|_dSNrr)rrrrrrrrLrrrr~s   z_Summary.__init__cCsZ|j|7_||jkr||_||jkr||_|j||7_|j|||7_dSr?r)rrvalrrrupdates  z_Summary.updatecCs|j|j|j|j|j|jSr?)rrrrrrrrLrrrrMsz_Summary.__bytes__N) rTrUrVrr rrrrrrMrrrrrxs  rc@seZdZdZddZdS)r)rrrrcC||_||_||_dSr?)rrr)rrrrrrrr z_Region.__init__N)rTrUrVrrrrrrrs rc@sHeZdZejejZedZej Z ddZ ddZ ddZ dd Z d S) rz =IIIIffffcCsR||_||_||_d|_t||_t||_td|_td|_ d|_ dSr) rrrrrfloat32rrrrr)rrrrr rrrrs     z_RegionSummary.__init__cCsd|j|_|j|j7_t|j|j|_t|j|j|_t|j|j|_t|j |j |_ |Sr?) rrrrrFrrrrr)rotherrrr__iadd__sz_RegionSummary.__iadd__cCsn|j|7_|j|krt||_|j|krt||_t|j|||_t|j||||_dSr?)rrrrrrr)roverlaprrrrrs    z_RegionSummary.updatec Cs*|j|j|j|j|j|j|j|j|j Sr?) rrrrrrrrrrrLrrrrMsz_RegionSummary.__bytes__N)rTrUrVrrrr rrrrrrrMrrrrrs    rc@s$eZdZgdZddZddZdS) _RTreeNode)rIrLr2r3r4r5startFileOffsetr6cCsd|_g|_dSr?)rLrIrLrrrrs z_RTreeNode.__init__cCsD||d7<|d7}|t|krdS|jD]}|||qdSr)r@rIcalcLevelSizes)r levelSizeslevelrQrrrrs  z_RTreeNode.calcLevelSizesN)rTrUrVrrrrrrrrs rc@sLeZdZdZddZeddZddZdd Zd d Z d d Z ddZ dS)r)rootrZfreeListstackrrcCs"d|_d|_g|_||_||_dSrx)rrZrrr)rrrrrrrs  z_RangeTree.__init__c cst|}d}|D]8\}}}|}t||}|dur || zt|}Wn ty0Ynw|jj|kr8n||q!|Vq dSr?)rr(raddToCoverageDepthrrrvr)r0rrr chromNamerrrrrrrs(      z_RangeTree.generatec cs:|j}t|\}}}|j}|j}t||t||||} t||d} || || j |kr>t||t||||} || j krvt|| j t|| j } | dksUJ| | || | 8} | j }| Vt||t||||} || j ksC| | |z t|\}}}Wn t yYn w| j |kr| Vq | VdS)NTrr) rtraverserrrrrrFrrrr) rrrrangesrrrrrrrrrrrrsZ             z_RangeTree.generate_summariescCsL|j}|dur$||jjkr|j}n |jj|kr|j}n|jS|dusdSdSr?)rrzrleftrright)rrrprrrr+2s  z_RangeTree.findcCst|j}||jur-||jur|}||_|j|_||_n3|}|j|_||_|j|_||_n"||jurC|}|j|_||_|j|_||_n |}|j|_||_||_|dkri|j|d}||jurd||_|S||_|S||_|SNrr)r@rrrr)rxyztosmidNoderLrrr restructure<s>     z_RangeTree.restructurecCsg|_z|j}Wn tyt}Ynw|j|_d|_d|_||_d|_|j }|durk |j ||j |jj krM|j}|durL|j}||_nn|jj |j kre|j}|durd|j}||_nndSq/d}n||_ d}||_|j d7_ t|jdkr|jdur|j}||jkr|j}n|j}|dus|jdur||||}d|_d|j_d|j_dSd|_d|_t|jdkrdSd|_|}|j}|jdusdSdSdS)NTFrr)rrpoprO_RedBlackTreeNoderrrzrrr/rrrZr@r)rrzrrcolmqrrradd`st         z_RangeTree.addc Cs|jd}|jd}||kr||}}|||}|dur+t||dd}||dS|j|krh|j|krh|j|krJt|j||j}||_|||j|kr_t||j|j}||_|||jd7_dSt|j ||}|}|}|D]P} || jkrt|| jd}| j}||n|| jkrt| j|| j}|| _||| j|kr| j|krt|| j| j}|| _||| jd7_| j}qw||krt||d}||dSdS)Nrrr)r) rr+_Rangerrrrlistrtraverse_range) rrrrexistingritemsserzrrrrsP              z_RangeTree.addToCoverageDepthN) rTrUrVrrrYrrr+rrrrrrrrs + $ 9rc@ eZdZdZddZddZdS)r)rrrrcCrr?)rrr)rrrrrrrrrz_Range.__init__cCst|j|j|jfSr?)rrrrrLrrr__iter__sz_Range.__iter__N)rTrUrVrrrrrrrrs rc@r)r)rrrrzccsF|jdur|jEdH|jV|jdur!|jEdHdSdSr?)rrrzrrLrrrrs  z_RedBlackTreeNode.traverseccs|jj|kr|jdur|j||EdHdSdS||jjkr3|jdur1|j||EdHdSdS|jdurB|j||EdH|jV|jdurW|j||EdHdSdSr?)rzrrrrr)rrrrrrrs      z _RedBlackTreeNode.traverse_rangeN)rTrUrVrrrrrrrrs rc@sBeZdZdZdddZddZddZd d Zd d Zd dZ dS)rih$rcCsDt|d|_t|d|_t|d|_t|d|_dS)N IIQIIIIQIxxxx?xHIIIIQIIIIQQ)r rr)formatter_nodeformatter_nonleafformatter_leafrrrrrs  z_RTreeFormatter.__init__c Cstdgd}tdgd}||jj}|j|\ }}}}} } } } } |tjks+J|j}|j}|j }|dg|| | | d}|}d} ||j}||\}}|r|j }t |D]"}||j}||\}} } } }}|||| | | ||}| |qY||7} |j }|dur||ksJ|St|j D]\}}t|t|krnqtdz |j |d}Wn ty|}Ynwnqn/|j }t |D]!}||j}||\}} } } }||g|| | | |}| |q|}|d}||jqC) NNonLeaf)rLrIrSr3rRr5rLeaf)rLrSr3rRr5rrrTrHr)rr(r)rr*rrrrrrIrAr/rLrMrrNrOrr)rrrrr1rru itemCountrSr3rRr5r6rtrrrrrP itemsCountedisLeafcountrIr4rrrQrLr-rrrr(s            z_RTreeFormatter.readcCst|}|dkr dSg}|dj}d}d}||krt} || ||} | j| _| _| j| _| j | _ || _ d}| } t |d|D]} || } | j}|| j krRn|d7}qB|}|| _ ||d||D]@} | j| jkrw| j| _| j| _n| j| jkr| j| jkr| j| _| j| jkr| j| _| j | _ qf| j| jkr| j | j kr| j | _ qf||7}||ksd} g}|}|D]}} ||krd}t}| j|_| j|_| j|_| j|_| j |_ | j |_ | j |_ ||nG|d7}| j|jkr| j|_| j|_n| j|jkr | j|jkr | j|_| j|jkr| j|_| j |_ n| j|jkr,| j |j kr,| j |_ |j| || _q|d7}t|dkrF ||fS|}qr)r@rrr/rr2r4rr3rr5rrAr6rLrI)rrurr6rrI nextOffsetoneSizer4rQ startItemendItemr5rz levelCountparents slotsUsedrLrrrrTreeFromChromRangeArrays        !     z(_RTreeFormatter.rTreeFromChromRangeArrayc Cs|j}||krHd}|j|t|j} || |jD]} || j| j| j| j | j | j | j } || q|t |t|j|j jdS|jD]} |||| |d||qKdS)NTr)rrrr@rIrr2r3r4r5rr6rrr rWriteLeaves) rrt lNodeSizercurLevel leafLevelrrrr1rQrrrrs.     z_RTreeFormatter.rWriteLeavesc Cs|}|j} ||krGd} |j| t|j} || |jD]} | | j| j| j| j |} || ||7}q|t |t|j|jj n|jD]} | | |||d|||}qJ| } | |krmtd| d|d|S)NFrz!Internal error: offset mismatch (z vs r)rrrr@rIrr2r3r4r5rrrWriteIndexLevelrrN)rrLru childNodeSizer destLevelrrprevious_offsetrrr1rQpositionrrrrsF      z _RTreeFormatter.rWriteIndexLevelc Cs||||\}}|jtj|t||j|j|j|j || }| ||dur)dSt |t } |j| dd|jj|jj|} |} t|dD]%} | | | | 7} | |dkrd|jj|jj|} |||| d| | |qJ|d} ||| |d| |dS)Nr)rrrm)rr)rrrr@r2r3r4r5rrrrrrrrrrArrr)rrrurtr6rrrr1rr levelOffsetr4rrrrrs8   z_RTreeFormatter.writeNr) rTrUrVrrr(rrrrrrrrrs %sS (rc@s*eZdZdZd ddZddZddZd S) rixrcCs4t|d|_t|d|_|d|_||_dS)N IIIIQxxxxxxxxrz {keySize}sQ)r rr)r fmt_nonleafrrrrrr@s  z_BPlusTreeFormatter.__init__cCs|j}|j}||j}||\}}}}} |tjksJ|j} t |j j |d} t ||d} || j|ks=J|dksCJt dddg} g}d} || j}| |\}}|rt |D]0}|| j}| |\}}}|d}|t|ksJtd|d }t||d }||qbn"g}t |D]}|| j}| |\}}||q|}| ||} |durt|| ksJ|S|j}z|d }Wn ty|j}Ynwnq||qO) NrsIIrxNoderLrITrrarcr)rr)r(rr*rrrr rrformatrrArr(r@rrr/rIrrOrLr)rrrrr1rrurvalSizerrrrrrqrPrrr4keyrrr sequencerecordrIposrLrrrr(Ws^               z_BPlusTreeFormatter.readcCstj}|jdj}|j|}t|}|j}||||||} || |j} t |j j |d} d} ||krFtt d||}| d7} ||ks6t|}| j|| j} | j||j}d}|}t | dddD]f}||}||}t d||}|dkr|}n| }t|| }||}|}|D]<}|||||}t|}|| |||D]}| |d|} || ||7}qt||| j} || q|}qhd}td|D]1}||||}t|}|dkrdS|| ||||t|||j} || qdS)Nr rrrFrT)rrritemsizer@r)rrrr rrrrArrr itertoolsrr)rrrurrrrrrr1rrlevelsbytesInIndexBlockbytesInLeafBlockrrr slotSizePer nodeSizePerindicesbytesInNextLevelBlock levelSizeendLevel nextChildr-blockrZrzrrrrsf          z_BPlusTreeFormatter.writeNr)rTrUrVrrr(rrrrrr=s   8r)*rWrrrr rrJ collectionsrrnumpyr Bio.AlignrrBio.Seqr Bio.SeqRecordrrrr r rgr$r rrrzrr~rrrrrrrrrrrrrrrs.       \    4^V-%J  0\ U