olmocr

# olmOCR **olmOCR** is a toolkit for converting PDFs and other image-based document formats (PNG, JPEG) into clean, readable plain text or Markdown. It is designed for building large-scale LLM training datasets by linearizing documents into natural reading order, even with complex layouts. The project is developed by the AllenNLP team at the Allen Institute for Artificial Intelligence (AI2). ## Key Features - **PDF/Image to Markdown**: Converts PDF, PNG, and JPEG documents into clean Markdown format. - **Natural Reading Order**: Preserves logical reading sequence, even in multi-column or complex documents. - **Cost-Effective**: Less than $200 USD per million pages converted (based on a 7B parameter VLM, requires a GPU). - **Benchmark Suite**: Includes olmOCR-Bench with over 7,000 test cases across 1,400 documents for measuring OCR performance. - **Flexible Inference**: Run locally with GPU, connect to external vLLM servers, or use cloud providers (e.g., Cirrascale, DeepInfra, Parasail). - **Parallel Processing**: Supports multi-node, multi-worker processing for millions of PDFs. - **Docker Support**: Pre-built images with or without the model included (~30GB with model). - **Filtering**: Basic filtering for English PDFs (non-forms, non-SEO spam). ## Installation Install with pip. For GPU support (recommended for local inference): ```bash pip install olmocr[gpu] --extra-index-url https://download.pytorch.org/whl/cu128 ``` For external server usage (no GPU dependencies): ```bash pip install olmocr ``` Optional extras: `beaker` for AI2 Beaker integration, `bench` for benchmark suite. ## Quick Start ### Convert a PDF to Markdown ```bash # Download a sample PDF wget https://arxiv.org/pdf/2502.18443 # Convert it to markdown olmocr ./localworkspace --markdown --pdfs olmocr-sample.pdf ``` ### Convert an Image File ```bash olmocr ./localworkspace --markdown --pdfs random_page.png ``` ### Use Remote Inference Server ```bash olmocr ./localworkspace --server http://remote-server:8000/v1 --model allenai/olmOCR-2-7B-1025-FP8 --markdown --pdfs *.pdf ``` Results are stored as Markdown files in `./localworkspace/markdown/`. You can also use `python -m olmocr.pipeline` instead of the `olmocr` command. ## Command-Line Options Key arguments include: - `--pdfs`: Paths to PDFs (local, S3, or glob patterns). - `--model`: Model identifier (default: `allenai/olmOCR-7B-0725-FP8`). - `--server`: URL of an external vLLM or OpenAI-compatible server. - `--api_key`: API key for external providers. - `--workers`: Number of parallel workers. - `--markdown`: Write output as Markdown. - `--apply_filter`: Apply basic filtering to English PDFs. - `--beaker`: Submit job to AI2 Beaker. - `--guided_decoding`: Enable guided decoding for structured YAML outputs. - `--max_concurrent_requests`: Max concurrent requests to inference provider. ## External Providers olmOCR has been tested with several providers, with costs per million tokens: | Provider | Input $/1M | Output $/1M | Example Command | |----------|-----------|-------------|-----------------| | Cirrascale | $0.07 | $0.15 | `olmocr ./workspace --server https://ai2endpoints.cirrascale.ai/api --api_key sk-XXXX --model olmOCR-2-7B-1025 --pdfs *.pdf` | | DeepInfra | $0.09 | $0.19 | `olmocr ./workspace --server https://api.deepinfra.com/v1/openai --api_key DfXXXX --model allenai/olmOCR-2-7B-1025 --pdfs *.pdf` | | Parasail | $0.10 | $0.20 | `olmocr ./workspace --server https://api.parasail.io/v1 --api_key psk-XXXX --model allenai/olmOCR-2-7B-1025 --pdfs *.pdf` | ## Docker Usage Pull the image with model included (~30GB): ```bash docker pull alleninstituteforai/olmocr:latest-with-model ``` Run a single PDF conversion: ```bash docker run --gpus all alleninstituteforai/olmocr:latest-with-model -c "olmocr /workspace/output --markdown --pdfs /workspace/sample.pdf" ``` For batch processing: ```bash docker run --gpus all -v /input:/input -v /output:/output alleninstituteforai/olmocr:latest-with-model -c "olmocr /output --markdown --pdfs /input/*.pdf" ``` ## Code Overview Reusable components include: - **Prompting strategy** for natural text parsing (buildsilver.py) - **Filtering** by language and SEO spam removal (filter.py) - **SFT Finetuning** code for Qwen2.5-VL (train.py) - **Synthetic data generation** (mine_html_templates.py) - **Pipeline** for processing millions of PDFs with VLLM (pipeline.py) ## Recent Updates - **v0.4.0 (Oct 21, 2025)**: New model release (olmOCR-2-7B-1025-FP8), +4 points on olmOCR-bench, introduces RL training. - **v0.3.0 (Aug 13, 2025)**: New model release (olmOCR-7B-0825-FP8), fixes auto-rotation and blank document hallucinations. - **v0.2.1 (Jul 24, 2025)**: New model release (olmOCR-7B-0725-FP8), +3 points on olmOCR-Bench, faster inference. - **v0.2.0 (Jul 23, 2025)**: Cleaned up trainer code for easier custom training. - **v0.1.75 (Jun 17, 2025)**: Switched from sglang to vllm inference, updated Docker to CUDA 12.8. - **v0.1.68 (May 19, 2025)**: olmOCR-Bench launch (77.4 score), pipeline bug fixes. ## License Apache 2.0. See LICENSE file for details. ## Team Developed by the AllenNLP team at AI2, a non-profit research institute.