paper-qa

PaperQA2 is an open-source Python package and CLI tool designed for high-accuracy retrieval augmented generation (RAG) over scientific literature and other document types. It ingests PDFs, plain text, Markdown, HTML, Microsoft Office files (docx, xlsx, pptx), and source code files, then answers user questions with grounded, in-text citations pointing to specific pages and passages. The system follows a three-phase agentic workflow: (1) Paper Search, where an LLM generates keyword queries to find candidate papers and chunks them into a local full-text search index; (2) Gather Evidence, where the query is embedded, top-k chunks are ranked, each chunk is summarized in context, and an LLM re-scores and selects the most relevant summaries; (3) Generate Answer, where the best summaries are placed into a prompt and an answer is produced. A language agent can invoke these tools in any order, enabling iterative refinement of queries and answers. Key features include automatic metadata fetching from Semantic Scholar, Crossref, and Unpaywall (including citation counts and retraction checks), document metadata-aware embeddings, LLM-based re-ranking and contextual summarization (RCS), and a robust customization interface. The default stack uses OpenAI embeddings and models with a Numpy vector database, but PaperQA2 integrates with LiteLLM to support virtually any LLM provider, including Claude, Gemini, and locally hosted models via llama.cpp. Local embedding models via Sentence Transformers are also supported. The CLI tool `pqa` provides quickstart usage: navigate to a directory of papers and run `pqa ask 'What is PaperQA2?'` to index, search, and answer in one step. Bundled settings (high_quality, fast, wikicrow, contracrow, debug, tier1_limits) offer pre-tuned configurations for different use cases, including contradiction detection and Wikipedia-style article writing. Rate limiting is configurable per model to accommodate API tier restrictions. The Python library offers both synchronous and asynchronous APIs. The `ask` function provides a convenience wrapper, while `agent_query` and direct `Docs` object manipulation allow fine-grained control over document ingestion and querying. The `Docs` object supports adding files, URLs, and code, with methods for evidence retrieval and querying. All prior answers are indexed and stored for later search. Recent updates (December 2025) added multimodal support for tables, figures, non-English languages, and math equations; new model-based PDF readers (Docling and Nvidia nemotron-parse); Microsoft Office document parsing; multimodal contextual summarization where media objects are passed to the summary LLM; and a simplified HTTP stack using httpx. The project moved from semantic versioning to calendar versioning in December 2025. PaperQA2 is licensed under Apache 2.0 and requires Python 3.11+. It depends on libraries including Semantic Scholar, Crossref, Unpaywall, Pydantic, tantivy, LiteLLM, and pybtex. The project is developed by Future House and has been published in peer-reviewed research demonstrating superhuman performance on scientific question answering, summarization, and contradiction detection tasks.