Can SmolDocling Make Document Parsing More Efficient?

SmolDocling: A Lightweight Vision-Language Model for High-Precision Document Conversion

Digital documents present a significant challenge: accurately converting their rich structure into machine-readable formats. Existing solutions, whether complex pipelines or massive models, often compromise accuracy for efficiency. SmolDocling offers a groundbreaking alternative—a remarkably compact 256M-parameter vision-language model delivering precise, rapid end-to-end document conversion.

Table of Contents:

• The Document Conversion Hurdle
• Introducing SmolDocling: A Novel Approach
• Understanding DocTags: A Universal Markup Language
• Deep Dive: Training Data and Model Architecture
• Performance Comparison: SmolDocling vs. Other Models
• Code Example and Output Visualization
• Conclusion and Future Developments

The Document Conversion Hurdle

Converting diverse document layouts (from business reports to academic papers) into structured data remains a complex task. Key challenges include:

• Layout Variability: Documents exhibit a vast range of styles and formats.
• Opaque Formats: Formats like PDF prioritize printing, hindering semantic parsing.
• Resource Intensive: Traditional methods demand substantial computational resources and intricate tuning.

Introducing SmolDocling: A Novel Approach

SmolDocling tackles these challenges with a unified, end-to-end approach:

• Complete Page Processing: It processes entire document pages simultaneously, eliminating the need for multiple specialized models.
• Compact Design, Powerful Results: Its 256M parameters achieve performance comparable to models many times larger.
• Versatile Multimodal Capabilities: It seamlessly handles diverse document elements: code, tables, equations, charts, and more.

Central to SmolDocling is its innovative markup language, DocTags, a universal standard capturing content, structure, and spatial context.

Understanding DocTags: A Universal Markup Language

DocTags redefine document element representation:

• Structured Vocabulary: Using XML-style tags (inspired by OTSL), it clearly distinguishes text, images, tables, code, etc.
• Spatial Context: Precise bounding box coordinates preserve layout information.
• Unified Representation: Consistent formatting for full pages or individual elements enhances learning and generalization.

Key DocTags include: <img src="/static/imghw/default1.png" data-src="https://img.php.cn/upload/article/000/000/000/174537247742337.jpg" class="lazy" alt="Can SmolDocling Make Document Parsing More Efficient?">

Performance Comparison: SmolDocling vs. Other Models

SmolDocling significantly outperforms larger models in text recognition and document formatting:

Method	Model Size	Edit Distance ↓	F1-score ↑	Precision ↑	Recall ↑	BLEU ↑	METEOR ↑
Qwen2.5 VL	7B	0.56	0.72	0.80	0.70	0.46	0.57
GOT	580M	0.61	0.69	0.71	0.73	0.48	0.59
Nougat (base)	350M	0.62	0.66	0.72	0.67	0.44	0.54
SmolDocling (Ours)	256M	0.48	0.80	0.89	0.79	0.58	0.67

SmolDocling also excels in specialized tasks, achieving high F1-scores and precision in code listing and equation recognition.

Code Example and Output Visualization

[Code examples and visualizations are omitted here due to length constraints. The original input provided these sections.]

Conclusion and Future Developments

SmolDocling demonstrates that smaller models can achieve state-of-the-art performance in document conversion. Its efficient architecture, innovative DocTags format, and comprehensive training strategy establish a new benchmark. While demonstrating strong performance on receipts and acceptable results on other documents, limitations exist due to its memory-efficient design. Future work will focus on improving element localization and multimodal understanding. Public release of the datasets will facilitate further research and collaboration.

The above is the detailed content of Can SmolDocling Make Document Parsing More Efficient?. For more information, please follow other related articles on the PHP Chinese website!