:Qg%ddlmZddlmZmZmZddlmZddlm Z ddddZ ddZ ddZ dd Z dd Zdd Zdd Z dd Z d ddZddZddZy)) annotations)AnyDictList) BeautifulSoup) cells_to_html row_index col_indexcontentct|d}t}|D]}|ddkDs|ddkDr]t|d|d|dzD]B}t|d|d|dzD]%}||f|d|dfk7s|j||f'D|d|df|vs}|d|d}}|Dcgc]}|d|k(s |d|k\s|} }| D]} | dxxdz cc<|j ||f|d|df|vr_|Scc}w)aMove cells to the right if spanned cells have an influence on the rendering. Args: cells: List of cells in the table in Deckerd format. Returns: List of cells in the table in Deckerd format with cells moved to the right if spanned. c|d|dfS)Nyx)rs h/var/www/html/answerous/venv/lib/python3.12/site-packages/unstructured/metrics/table/table_extraction.pyz/_move_cells_for_spanned_cells..s##/?keywhrr)sortedsetrangeaddremove) cells sorted_cellscells_occupied_by_spannedcellijcell_ycell_xccells_to_the_right cell_to_moves r_move_cells_for_spanned_cellsr+se%%?@L # ? 9q=DIM49d3i$s)&;< >tCy$s)d3i*?@>A1v$s)T#Y!77155q!f=> >Cy$s)$(AA!#YS FF-9!cQsVv=MRSTWRX\bRb!!c !c 2 ' S!Q&! ' % , ,ff-= > Cy$s)$(AA ?  "ds& C;4C;=C;c t|d}|jd}|jdg}g}t|D]\}}|jddg}t|D]n\}} ||t | j j ddt | j j dd| jd } |j| pt|S) zConvert html format to Deckerd table structure. Args: content: The html content with a table to extract. Returns: A list of dictionaries where each dictionary represents a cell in the table. z html.parsertabletrthtdcolspanrrowspan)rrrrr ) rfindfindAll enumerateintattrsgettextappendr+) r soupr-rows table_datar$rowr r% cell_datar#s rhtml_table_to_deckerdr@)s - 0D IIg E ==$ DJD/ $3 T4L)%e, $LAy,,Y:;,,Y:;$>> D   d # $ $ ) 44rc g}|Dcgc] }|ddk(s |}}td|D}tt|}|D]h}tt|d|d|dztt|d|d|dz|d|d|d|d|vd}|j |jt |}|Scc}w) zConvert Deckerd table structure to html format. Args: cells: List of dictionaries where each dictionary represents a cell in the table. Returns: A string with the html content of the table. rrc3&K|] }|d yw)rNr).0r#s r z(deckerd_table_to_html..Ss>dS >srrrr )row_nums column_numsrrz cell textz column header)maxrrlistr:r)r transformer_cellsr#first_row_cells header_length header_rowsr?r-s rdeckerd_table_to_htmlrMFs).@cat@O@>o>>MeM*+K ,U49d3i$s).CDEd3icT#Y1F GHcci!#Y+5     + , + ,E LAs B;B;c.t|}t|S)a7Convert html format to table structure. As a middle step it converts html to the Deckerd format as it's more convenient to work with. Args: content: The html content with a table to extract. Returns: A list of dictionaries where each dictionary represents a cell in the table. )r@_convert_table_from_deckerd)r deckerd_cellss r_convert_table_from_htmlrQds*'2M &} 55rcg}|D]#} |d|d|dd}|j|%|S#t$r t}Y&t$r t}Y6wxYw)zConvert deckerd format to table structure. Args: content: The deckerd formatted content with a table to extract. Returns: A list of dictionaries where each dictionary represents a cell in the table. rrr r )KeyError EMPTY_CELL TypeErrorr:)r r=r-r?s rrOrOrsuJ % #"3Z"3Z +I )$ %   #"I #"I #s-AAAct|dS)Nc|d|dfS)Nr r r)r#s rrz#_sort_table_cells..s[0A4 CT/Urr)r)r=s r_sort_table_cellsrXs *"U VVrcg}|D]>}d|vs|ddk(sd|vs t|d}|jt|@|S#t$r)}t d||jiYd}~pd}~wwxYw)aExtracts and converts tables data to a structured format based on the specified table type. Args: file_elements: List of elements from the ground truth file. Returns: A list of tables with each table represented as a list of cell data dictionaries. typeTabler9z$Error converting ground truth data: N)rOr:rX Exceptionprint) file_elementsground_truth_table_dataelementconverted_dataes r,extract_and_convert_tables_from_ground_truthrcs! 3 W G!;'@Q 3!<FO"(../@/PQ 3 #"  3>;-'a b   Month A. 22< " } } Returns: List of extracted cells in a format: [ { "row_index": 0, "col_index": 0, "content": "Month A.", }, ..., ] metadata text_as_htmlzNz*Error converting Unstructured table data: )r8rQr\r])r`valpredicted_cellsrbs rrfrfsqD *  ! !. 1C )3&O@237  @ :1#>?? @s , AA  AcL|djd}d}|r t|}|S)ajExtracts and parse cells from "table_as_cells" field in Element structure Args: element: Example element: { "type": "Table", "metadata": { "table_as_cells": [{"x": 0, "y": 0, "w": 1, "h": 1, "content": "Month A."}, {"x": 0, "y": 1, "w": 1, "h": 1, "content": "22"}] } } Returns: List of extracted cells in a format: [ { "row_index": 0, "col_index": 0, "content": "Month A.", }, ..., ] rqtable_as_cellsN)r8rO)r`rtconverted_cellss rrgrgs20j)--.>?OO5oF rN)r List[Dict[str, Any]])r strreturnrx)r rxrzry)r rxrzrx)r=List[List[Dict[str, Any]]]rzr{)r^rxrzr{)re)r^rxriryrzr{)r`Dict[str, Any]rzzList[Dict[str, Any]] | None)r`r|rzrx) __future__rtypingrrrbs4r$unstructured_inference.models.tablesrrTr+r@rMrQrOrXrcrorfrgrrrrs"""> 45:< 62W#'##8=C( '( 69( ( V,^r