In conversazione con i tuoi documenti.

ThreadLeaf is a local document chatbot with a terminal interface and a web app. It combines Ollama, ChromaDB, hybrid retrieval and a multilingual reranker to answer questions using your PDF, TXT and Markdown documents.

The goal is to keep ThreadLeaf as lightweight as possible so it can run locally on everyday hardware. Development favors small multilingual models, CPU-friendly inference and only the dependencies needed for useful features. Improvements should balance answer quality with memory use and response time, while keeping documents and conversations on your own machine.

Reusable logo exports are in assets/brand: standalone SVGs, transparent PNGs in eight sizes, and a multi-resolution ICO. Download the icon bundle to use them in other applications.

License

Copyright © 2026 Pasquale Esposito. ThreadLeaf's original code, documentation and branding are licensed under the GNU General Public License, version 3 or later (GPL-3.0-or-later). The project's copyright and license notice is in COPYRIGHT.

You may use, modify and redistribute ThreadLeaf, including commercially and for a fee, subject to GPLv3. When distributing covered software or modified versions, preserve the notices and provide the corresponding source as required by GPLv3. Covered derivative works must be distributed under GPLv3; distributing container images or executable packages does not remove the source obligations. Private internal modifications need not be published merely because they are made. Providing network access alone does not trigger AGPL-style source disclosure: this project uses GPLv3, not AGPLv3.

ThreadLeaf is provided without warranty, to the extent permitted by applicable law. The full license controls; this paragraph is a summary. The previous custom commercial restrictions no longer apply to this release.

Third-party dependencies, models and icons retain their own licenses and notices; the GPL grant applies to the original material whose rights the project holds. User documents remain subject to their owners' rights. Keep the attributions below and applicable upstream notices when distributing copies.

Features

• Web chat with Italian and English controls, Markdown answers and source counts with details on demand.
• Document panel with folders, search, selection, uploads, deletion and indexing progress.
• Conversation history with full-message search and SQLite persistence through LangGraph.
• Conversation deletion with confirmation, and an installed-model selector in the composer.
• Page-aware PDF ingestion with optional local Paddle layout detection and JSON archives.
• Terminal chat with Rich formatting, Markdown, source references and retrieval debugging.
• Semantic search combined with BM25 and section-heading matches, followed by optional CPU reranking.
• Document display titles generated by the configured chat model at server startup.

Requirements

• Python 3.11 or newer.
• Node.js 20 or newer and npm for the web interface.
• Ollama installed and running locally.
• Internet access for the initial dependency and model downloads.

Once the dependencies and models are available locally, inference uses local Ollama and ONNX models. Documents are not sent to a hosted model by this application. The web/API is designed for local, single-user use; it has no authentication.

Docker Compose

Install Docker Engine with Compose on Linux, or Docker Desktop on macOS/Windows. Docker builds the frontend and Python runtime, so host Python, Node and Ollama are not needed for this setup. The default image uses Python 3.13 and excludes the optional Paddle dependencies to keep it smaller.

From the repository root, create documents/ and add your files. Build and start:

docker compose up --build -d
docker compose logs -f initialize

The initialization container connects to the Compose Ollama service, downloads the chat and embedding models from config.toml, and prepares the enabled reranker. First startup requires internet access and may take several minutes. The web server starts after initialization finishes. Coreference weights are not downloaded automatically; without them, follow-up questions remain unchanged.

Index files already present in documents/ before starting conversations:

docker compose run --rm threadleaf python ingest.py

Open http://127.0.0.1:4321. Uploads from the UI are indexed automatically. Reindex existing files when changing the embedding model or extraction settings. Avoid ingestion while chatting or changing documents in the UI.

Useful commands:

docker compose build threadleaf                 # Build threadleaf:local without starting it
docker compose logs -f threadleaf              # Application logs
docker compose run --rm threadleaf python cli.py
docker compose run --rm threadleaf python -m scripts.prepare_coreference
docker compose down                           # Stop; retain saved data

The documents/ directory is mounted from your host and supports uploads and deletion. Configuration and system prompt are mounted read-only. Named volumes retain the vector database, conversation history, extraction archives and model files across container rebuilds. Existing host db/, state/, models/ and Ollama downloads are not imported into those volumes. Back up the volumes and documents/ to preserve your data. docker compose down --volumes deletes the named volumes, including saved conversations and downloaded models. Volume mounts assume the default documents/ and db/ paths; update Compose mounts if you change those paths in config.toml. The built image contains code and dependencies, while downloaded model weights stay in volumes.

The container listens on 0.0.0.0:4321, but Compose publishes the web port only on your host's loopback address. The container's listener overrides the server host/port in config.toml; set THREADLEAF_PORT to change the host port. Ollama is reached at http://ollama:11434 over the private Compose network and has no published host port. This setup uses CPU inference; GPU access is not configured. Native Ollama remains an alternative, particularly for macOS GPU use. See Ollama's Docker guide for hardware options.

Optional Paddle image

Enable the [layout] section in config.toml, then build with Paddle support:

INSTALL_LAYOUT=true docker compose up --build -d

In PowerShell:

$env:INSTALL_LAYOUT = "true"
docker compose up --build -d

This adds PaddleOCR, PaddlePaddle, PDFium and Pillow to the image. Initialization downloads PP-DocLayout-M into the model volume. Run ingestion again to apply it to existing documents. Keep INSTALL_LAYOUT set for subsequent Compose commands. Paddle builds require supported platform wheels; the optional image is intended for Linux AMD64, including Docker Desktop using Linux containers on Windows. Apple Silicon users should use the lightweight image or explicitly evaluate AMD64 emulation for Paddle, which can be slower.

Paddle layout detects regions; this implementation still does not perform OCR or reconstruct table cells. The detector disables MKL-DNN to avoid the observed oneDNN/PIR conversion failure. Missing dependencies, weights or inference errors produce warnings and fall back to native page-aware extraction.

Quick start

Clone this repository and open its root directory. Follow the instructions for your operating system, then continue with the shared installation steps below.

Linux

Install Python 3.11+ with virtual-environment support, Node.js 20+ and npm using your distribution's package manager or their official installers. Check the versions:

python3 --version
node --version
npm --version

Install and start Ollama using the official Linux instructions. If Ollama is not already running as a service, run ollama serve in a separate terminal and leave it running while you use ThreadLeaf.

Create and activate the project environment:

python3 -m venv .venv
source .venv/bin/activate
mkdir -p documents

If creating the environment reports that venv or ensurepip is missing, install the virtual-environment package for your Python version and retry.

macOS

Install Python 3.11+ and Node.js 20+ with npm. Check python3 --version, node --version and npm --version in Terminal.

Install Ollama using the official macOS instructions, move the application to Applications and open it. Complete the CLI setup if prompted so that the ollama command is available in Terminal.

From the project root, create and activate the environment:

python3 -m venv .venv
source .venv/bin/activate
mkdir -p documents

Windows (PowerShell)

Install the prerequisites listed above and start Ollama. From the project root:

python -m venv .venv
.\.venv\Scripts\Activate.ps1
New-Item -ItemType Directory -Path documents -Force

Shared installation steps

With the virtual environment active, use python for the commands below on all three platforms. Re-activate the environment in every new terminal session: source .venv/bin/activate on Linux/macOS or .\.venv\Scripts\Activate.ps1 on Windows.

Install the Python dependencies and configure the models:

python -m pip install -r requirements.txt
python setup.py

Setup lets you choose the chat model, embedding model and system prompt. Press Enter to keep the current settings. It downloads the Ollama models and can rebuild the document index. If needed, start Ollama with ollama serve in another terminal.

Create a documents/ directory, add PDF, TXT or MD files, and index them:

python ingest.py

Subfolders are supported. Re-run ingestion after editing documents outside the web app. To enable the configured multilingual reranker, download its files once:

python -m scripts.prepare_reranker

If its local files are unavailable, the application falls back to hybrid retrieval.

Web app

Build the frontend from the repository root:

cd web
npm ci
npm run build
cd ..
python server.py

Open http://127.0.0.1:4321. Startup warms the reranker and generates document titles before serving requests, which can take time with many documents.

Use the vertical tabs to open or collapse Documents and Conversation history. Select the documents to use for the next question; at least one must be selected in the web interface. The document panel mirrors the indexed folder structure. README.txt and REEADME.txt are hidden from this panel.

Uploads support PDF, TXT and MD files up to 25 MB, with an optional destination folder. Each upload is indexed individually and gets a generated display title. Existing files are not overwritten. Deletion removes the file and its indexed passages. Chat pauses while a document operation is running.

Open the settings gear to choose Italiano or English. Interface language does not change existing messages, document titles or the configured answer language. The icon beside settings starts a new conversation. Choose an installed chat model with the selector above the message input; the browser remembers the choice. The trash icon beside a saved conversation asks for confirmation before deletion.

Terminal app

python cli.py
python cli.py --list-models
python cli.py --model qwen2.5:0.5b --k 4 --lang auto --debug

The CLI interface is in English. Answers render as Markdown, with a muted source count below document answers. /sources reveals the filenames for the latest reply. Colors adapt to the terminal and respect NO_COLOR.

Command	Action
/help	Show available commands
/settings	Show the current configuration
/models	List locally installed Ollama models, sizes and model details
/sources	Show document sources for the latest reply
/model NAME	Change the chat model
/k N	Set a positive number of retrieved passages
/lang NAME	Set the answer language; auto follows the question
/debug	Toggle retrieved passages and prompt output
/clear	Clear the screen and show the welcome panel
exit or quit	Leave the chat

The CLI calls RAG directly. Persistent conversations currently apply to the web/API.

python cli.py --list-models lists models without starting a chat. Ollama must be running. The table marks the configured chat and embedding models; /model NAME changes the chat model for the current CLI session. Only installed models are listed. Download another model with ollama pull NAME before selecting it. --lang and /lang control answer language independently of the English controls.

Configuration

Edit config.toml and system_prompt.txt, or run python setup.py again. Restart the app after configuration changes.

Setting	Current configuration
Chat model	llama3.2:1b
Embedding model	nomic-embed-text-v2-moe
Retrieval	24 candidates, 4 final passages
Reranker	Quantized ONNX cross-encoder/mmarco-mMiniLMv2-L12-H384-v1
Answer mode	generated
Answer language	auto
Web server	127.0.0.1:4321

Change llm.answer_mode to extractive to return a model-selected source passage instead of a generated summary. Extractive mode preserves source wording but can still select an irrelevant passage. Generated mode permits summaries and can introduce unsupported claims. Source filenames identify retrieved documents; they do not guarantee that every statement is supported.

Changing the embedding model requires rebuilding the index. Reranking does not. Adjust candidate counts, BM25 weighting, token limits and CPU threads in the config. The lexical tokenizer retains every word, with no hand-written stopword or question-word lists. BM25 weights words using their frequencies in the indexed documents. Heading matches use normalized token overlap without special vocabulary.

For non-interactive setup:

python setup.py --non-interactive --llm qwen2.5:0.5b --ingest

Use --skip-pull to skip model downloads or --prompt-file PATH to replace the system prompt. Run python setup.py --help for all options.

Optional document layout

Ingestion now keeps PDF page boundaries and tries pypdf's layout-preserving text extraction. Both original and layout text are archived under state/document_layout/, alongside detected regions and extraction warnings. Chunks prefer paragraph or line boundaries and carry page and block metadata through hybrid retrieval and reranking into the answer context.

To add local CPU region detection with PP-DocLayout-M:

python -m pip install -r requirements-layout.txt
python -m scripts.prepare_layout
python ingest.py

The layout dependencies are optional and separate from the core requirements. Use a Python version supported by PaddlePaddle's wheels for your platform. For this application's optional Paddle backend, use Python 3.11–3.13; the current Python 3.14 environment can use native extraction, but cannot install the documented PaddlePaddle wheels. See the official installation guide. The model is downloaded explicitly into models/layout/; ingestion does not download it. Configure [layout] in config.toml, or set enabled = false to retain only native page-aware extraction. Reindex existing documents after enabling this feature. The original PDF files remain in documents/.

This first implementation detects regions and extracts their native PDF text. It does not perform OCR or reconstruct table cells. Image-only pages are reported as needing OCR. Native page text remains searchable even when detection misses a region; raw extraction is also retained because layout extraction can omit rotated text. Reading order is a basic geometric approximation, so complex columns still need review. Detected regions without native text remain in the JSON archive. Archives are reused for unchanged files and configuration, and removed when a document is deleted through the application. Extra region and fallback chunks can increase embedding work and index size.

Optional tools: PaddleOCR and PaddlePaddle (Apache-2.0), PP-DocLayout-M (Apache-2.0), pypdfium2 license notices, and Pillow license.

Conversation handling

Source display

Web document answers show an info icon with a source count. Hover, keyboard focus or click opens the source list, with readable document titles and original paths. Click outside or press Escape to close it. Restored replies use the same display. The CLI shows [2 sources · /sources] instead of filenames in every answer.

Small talk and extractive insufficient-evidence replies have no source indicator. A learned linear classifier over the configured multilingual sentence embeddings routes social conversation before retrieval. It trains once per process from complete labeled utterances in threadleaf/data/intent_training.json, then caches its weights. Messages are not matched against greeting or keyword lists. The first request embeds the small training corpus; later requests add one embedding call. Classification failures use RAG, and incorrect predictions remain possible. Small talk uses a separate prompt without document-language hints or context labels, so it can answer directly in the message's language when answer = "auto". Extractive answers expose the selected passage's document only. Generated answers list the retrieved context documents; this doesn't prove every claim is supported.

Follow-up reference resolution

Follow-up questions use a dedicated multilingual coreference model to predict links between mentions in recent conversation and the latest question. For example, a predicted link from "he" to "Alex" makes "What did he study?" search for "What did Alex study?". The remaining wording is preserved. The resolver does not ask Ollama to rewrite the query, use keyword overlap to approve rewrites, or append previous answers to the search query. Hybrid retrieval and reranking remain separate.

Prepare the model once:

python -m scripts.prepare_coreference

The pinned model and tokenizer download is approximately 531 MB and stays in ignored models/coreference/. It runs locally using the existing ONNX Runtime and Tokenizers dependencies, without requiring spaCy or PyTorch. Runtime never downloads missing weights. If the model is missing, fails, finds no link, or the input exceeds its span/token limits, the original question is used unchanged. The resolver uses a bounded recent-history window and makes learned predictions; incorrect links and unresolved references remain possible, especially with ambiguous names, possessives, omitted subjects or long conversations. Its quality must be checked on your own English and Italian follow-up questions.

Set [coreference].enabled = false to disable it. Restart after preparation or configuration changes. Earlier assistant messages supply reference context only, not documentary evidence for the final answer.

Persistence

LangGraph checkpoints persist the full web transcript and selected documents in state/conversations.sqlite3. Only the latest six messages enter the model's conversation context. The browser remembers the current conversation UUID and restores its messages after reloads and server restarts.

History lists conversations newest first, using the first question as the title. Search matches saved user and assistant messages without case sensitivity. Reopening a conversation restores its available selected documents. New conversation starts a separate thread; it does not delete previous chats. Use the trash icon beside a saved conversation to delete it after confirmation. Deleting the active conversation opens a fresh thread. Deleted chats cannot be restored without a backup. Back up state/ to preserve conversation history.

API

The API uses the same local server as the web app.

Endpoint	Purpose
POST /chat	Ask a question; returns a reply, sources and conversation UUID
GET /conversations?q=...	List or search saved conversations
GET /conversations/{thread_id}	Restore a transcript and its document selection
DELETE /conversations/{thread_id}	Delete all saved checkpoints and writes for a conversation
GET /models	List installed chat models for the composer selector
GET /documents	List indexed documents and display titles
POST /documents/upload?source=folder/file.txt	Upload raw file bytes
DELETE /documents	Delete a document using a JSON source
POST /documents/reindex	Retry indexing using a JSON source
GET /documents/operations	Read document-operation progress

Example chat request:

{
  "message": "Summarize this document",
  "lang": "en",
  "sources": ["notes.md"]
}

Reuse the returned thread_id in subsequent requests to continue the conversation. The server owns message history; do not send a history field. Omitting sources uses the entire index; empty or stale selections return HTTP 400. Concurrent document mutations and chat during indexing return HTTP 409.

Development and evaluation

Run the Python checks from the repository root:

python -m unittest discover -s tests -v

The optional live intent-routing regression check requires Ollama and the configured embedding model. It includes English/Italian small talk and mixed greeting/document requests, separate from the router's training utterances:

THREADLEAF_TEST_OLLAMA=1 python -m unittest tests.test_intent_integration -v

On PowerShell, set $env:THREADLEAF_TEST_OLLAMA = "1" before running the same python -m unittest tests.test_intent_integration -v command. The regular test suite skips this live check.

Check and rebuild the frontend:

cd web
npm run check
npm run build

The frontend uses Marked, DOMPurify and Font Awesome from npm. The build compiles TypeScript and generates/copies the browser assets; these generated files are ignored by Git. Keep web/package-lock.json committed and use npm ci for installation. The build also copies the browser dependencies' license texts and ThreadLeaf's license and copyright notice into web/public/vendor/, so deployed assets retain the accompanying notices.

Run python -m scripts.evaluate_rag to compare retrieval methods or add --generate to inspect model answers. Edit rag_eval_cases.json for your own documents. The included Italian legal-document cases require a local codice-civile.pdf, which is not distributed with the repository; they are a small regression sample, not a general accuracy benchmark.

Attributions and upstream licenses

ThreadLeaf uses the following open-source projects. Thanks to their authors and contributors. Links below lead to upstream license texts; bundled components and transitive dependencies retain their own notices.

Tool	Use in ThreadLeaf	Upstream license
Python — Python Software Foundation	Application runtime	PSF-2.0 and incorporated-software notices
Node.js — Node.js contributors	Frontend build runtime	MIT and bundled-component notices
Ollama — Ollama	Local model inference	MIT
Ollama Python — Ollama	Model API client	MIT
ChromaDB — Chroma	Vector index	Apache-2.0
pypdf — pypdf contributors	PDF extraction	BSD-3-Clause
FastAPI — FastAPI contributors	HTTP API	MIT
Uvicorn — Uvicorn contributors	ASGI server	BSD-3-Clause
Rich — Will McGugan and contributors	Terminal rendering	MIT
LangGraph — LangChain	Conversation workflow	MIT
LangGraph SQLite checkpointer — LangChain	Conversation checkpoints	MIT
SQLite — SQLite authors	Local checkpoint storage	Public domain
ONNX Runtime — Microsoft and contributors	CPU reranking	MIT
Tokenizers — Hugging Face	Reranker tokenization	Apache-2.0
Hugging Face Hub — Hugging Face	Reranker downloads	Apache-2.0
NumPy — NumPy developers	Numerical arrays	BSD-3-Clause; bundled components have additional licenses
TypeScript — Microsoft and contributors	Frontend compiler	Apache-2.0
Marked — MarkedJS, Christopher Jeffrey and contributors	Markdown rendering	MIT and included Markdown notices
DOMPurify — Cure53 and contributors	HTML sanitization	Apache-2.0 or MPL-2.0
Font Awesome Free — Fonticons, Inc.	SVG interface icons and rendering code	CC BY 4.0 for icons; MIT for code

Font Awesome icons are converted to SVG markup for the interface; the project uses no Font Awesome font files. The ThreadLeaf logo is an original inline SVG.

Configured models

Model licenses are separate from inference-library licenses. The current default models are credited below; other models selected through Ollama retain their own terms.

Model	Attribution / role	License
llama3.2:1b / Llama 3.2 1B	Meta — configured chat model	Llama 3.2 Community License, with its Acceptable Use Policy
qwen2.5:0.5b / Qwen2.5-0.5B-Instruct	Qwen team — optional chat model	Apache-2.0
nomic-embed-text-v2-moe	Nomic AI — multilingual embeddings	Apache-2.0
cross-encoder/mmarco-mMiniLMv2-L12-H384-v1	Sentence Transformers / Cross-Encoders — multilingual reranking	Apache-2.0
talmago/allennlp-coref-onnx-mMiniLMv2-L12-H384-distilled-from-XLMR-Large	Tal Almagor; AllenNLP / crosslingual-coreference lineage — multilingual mention linking	MIT, as declared in the model card

Repository layout

cli.py                     Terminal interface
server.py                  FastAPI server
setup.py                   Interactive configuration
ingest.py                  Document indexing entry point
threadleaf/                Supporting application modules
  rag.py                   Ingestion and answering
  retrieval.py             Hybrid retrieval
  reranker.py              Local ONNX reranking
  conversation.py          Follow-up question resolution
  coreference.py           Trained ONNX mention linking
  grounded_answers.py      Extractive answer rendering
  chat_workflow.py          LangGraph conversation persistence
  document_management.py   Upload, delete and indexing jobs
  document_titles.py       Generated document display titles
  document_layout.py       Page-aware extraction and optional Paddle regions
  paths.py                 Shared repository paths
scripts/                   Maintenance tools (run with python -m)
  prepare_reranker.py       Model preparation
  prepare_coreference.py    Coreference model preparation
  prepare_layout.py         Paddle layout model preparation
  container_setup.py        Compose model initialization
  evaluate_rag.py           Retrieval evaluation
config.toml                Shared configuration
compose.yaml               Local app, Ollama and initialization containers
Dockerfile                 Frontend build and Python runtime image
LICENSE                    Unmodified GNU GPLv3 license text
COPYRIGHT                  Project notice: GPLv3 or later
requirements-layout.txt    Optional Paddle extraction dependencies
system_prompt.txt          Chat instructions
web/src/                   TypeScript interface
web/scripts/               Browser-asset build scripts
web/public/index.html      Interface markup and styles
tests/                     Python checks and shared test helpers
tests/fixtures/            Sample retrieval evaluation cases
reports/                   Local output and evaluation reports (ignored)

Documents (documents/), Chroma indexes (db/), downloaded models (models/), conversation checkpoints (state/), environments, logs and build output stay local through .gitignore. If you configure different data directories, add those paths to your ignore rules before committing.