Transformers
Inference Endpoints
File size: 3,487 Bytes
9d9996e
 
 
76cf972
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
---

license: cdla-permissive-2.0
---


# Docling Models

This page contains models that power the PDF document converion package [dockling](https://github.com/DS4SD/docling).

## Layout Model

The layout model will take an image from a poge and apply RT-DETR model in order to find different layout components. It currently detects the labels: Caption, Footnote, Formula, List-item, Page-footer, Page-header, Picture, Section-header, Table, Text, Title. As a reference (from the DocLayNet-paper), this is the performance of standard object detection methods on the DocLayNet dataset compared to human evaluation, 

|                | human   | MRCNN   | MRCNN   | FRCNN   | YOLO   |
|----------------|---------|---------|---------|---------|--------|
|                | human   | R50     | R101    | R101    | v5x6   |
| Caption        | 84-89   | 68.4    | 71.5    | 70.1    | 77.7   |
| Footnote       | 83-91   | 70.9    | 71.8    | 73.7    | 77.2   |
| Formula        | 83-85   | 60.1    | 63.4    | 63.5    | 66.2   |
| List-item      | 87-88   | 81.2    | 80.8    | 81.0    | 86.2   |
| Page-footer    | 93-94   | 61.6    | 59.3    | 58.9    | 61.1   |
| Page-header    | 85-89   | 71.9    | 70.0    | 72.0    | 67.9   |
| Picture        | 69-71   | 71.7    | 72.7    | 72.0    | 77.1   |
| Section-header | 83-84   | 67.6    | 69.3    | 68.4    | 74.6   |
| Table          | 77-81   | 82.2    | 82.9    | 82.2    | 86.3   |
| Text           | 84-86   | 84.6    | 85.8    | 85.4    | 88.1   |
| Title          | 60-72   | 76.7    | 80.4    | 79.9    | 82.7   |
| All            | 82-83   | 72.4    | 73.5    | 73.4    | 76.8   |

## TableFormer

The tableformer model will identify the structure of the table, starting from an image of a table. It uses the predicted table regions of the layout model to identify the tables. Tableformer has SOTA table structure identification,

| Model (TEDS) | Simple table | Complex table | All tables |
| ------------ | ------------ | ------------- | ---------- |
|       Tabula |         78.0 |          57.8 |       67.9 |
|    Traprange |         60.8 |          49.9 |       55.4 |
|      Camelot |         80.0 |          66.0 |       73.0 |
|  Acrobat Pro |         68.9 |          61.8 |       65.3 |
|          EDD |         91.2 |          85.4 |       88.3 |
|  TableFormer |         95.4 |          90.1 |       93.6 |

## References

```

@techreport{Docling,

  author = {Deep Search Team},

  month = {8},

  title = {{Docling Technical Report}},

  url={https://arxiv.org/abs/2408.09869},

  eprint={2408.09869},

  doi = "10.48550/arXiv.2408.09869",

  version = {1.0.0},

  year = {2024}

}



@article{doclaynet2022,

  title = {DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis},  

  doi = {10.1145/3534678.353904},

  url = {https://arxiv.org/abs/2206.01062},

  author = {Pfitzmann, Birgit and Auer, Christoph and Dolfi, Michele and Nassar, Ahmed S and Staar, Peter W J},

  year = {2022}

}



@InProceedings{TableFormer2022,

    author    = {Nassar, Ahmed and Livathinos, Nikolaos and Lysak, Maksym and Staar, Peter},

    title     = {TableFormer: Table Structure Understanding With Transformers},

    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},

    month     = {June},

    year      = {2022},

    pages     = {4614-4623},

    doi = {https://doi.org/10.1109/CVPR52688.2022.00457}

}

```