{
  "markdown": "# LlamaFarm - Edge AI for Everyone\n\n> Enterprise AI capabilities on your own hardware. No cloud required.\n\n[![License: Apache 2.0](https://img.shields.io/github/license/llama-farm/llamafarm)](LICENSE)\n[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)\n[![Go 1.24+](https://img.shields.io/badge/go-1.24+-00ADD8.svg)](https://go.dev/dl/)\n[![Docs](https://img.shields.io/badge/docs-latest-4C51BF.svg)](https://docs.llamafarm.dev)\n[![Discord](https://img.shields.io/discord/1392890421771899026.svg)](https://discord.gg/RrAUXTCVNF)\n\n**LlamaFarm** is an open-source AI platform that runs entirely on your hardware. Build RAG applications, train custom classifiers, detect anomalies, and run document processing—all locally with complete privacy.\n\n- 🔒 **Complete Privacy** — Your data never leaves your device\n- 💰 **No API Costs** — Use open-source models without per-token fees\n- 🌐 **Offline Capable** — Works without internet once models are downloaded\n- ⚡ **Hardware Optimized** — Automatic GPU/NPU acceleration on Apple Silicon, NVIDIA, and AMD\n\n### Desktop App Downloads\n\nGet started instantly — no command line required:\n\n| Platform | Download |\n|----------|----------|\n| **Mac (Universal)** | [Download](https://github.com/llama-farm/llamafarm/releases/latest/download/LlamaFarm-desktop-app-mac-universal.dmg) |\n| **Windows** | [Download](https://github.com/llama-farm/llamafarm/releases/latest/download/LlamaFarm-desktop-app-windows.exe) |\n| **Linux (x86_64)** | [Download](https://github.com/llama-farm/llamafarm/releases/latest/download/LlamaFarm-desktop-app-linux-x86_64.AppImage) |\n| **Linux (ARM64)** | [Download](https://github.com/llama-farm/llamafarm/releases/latest/download/LlamaFarm-desktop-app-linux-arm64.AppImage) |\n\n---\n\n### What Can You Build?\n\n| Capability | Description |\n|-----------|-------------|\n| **RAG (Retrieval-Augmented Generation)** | Ingest PDFs, docs, CSVs and query them with AI |\n| **Custom Classifiers** | Train text classifiers with 8-16 examples using SetFit |\n| **Anomaly Detection** | 12+ algorithms for batch and streaming anomaly detection |\n| **Tool Calling (MCP)** | Connect models to external tools via Model Context Protocol |\n| **OCR & Document Extraction** | Extract text and structured data from images and PDFs |\n| **Named Entity Recognition** | Find people, organizations, and locations |\n| **Multi-Model Runtime** | Switch between Ollama, OpenAI, vLLM, or local GGUF models |\n\n**Video demo (90 seconds):** https://youtu.be/W7MHGyN0MdQ\n\n---\n\n## Quickstart\n\n### Option 1: Desktop App\n\nDownload the desktop app above and run it. No additional setup required.\n\n### Option 2: CLI + Development Mode\n\n1. **Install the CLI**\n\n   macOS / Linux:\n   ```bash\n   curl -fsSL https://raw.githubusercontent.com/llama-farm/llamafarm/main/install.sh | bash\n   ```\n\n   Windows (PowerShell):\n   ```powershell\n   irm https://raw.githubusercontent.com/llama-farm/llamafarm/main/install.ps1 | iex\n   ```\n\n   Or download directly from [releases](https://github.com/llama-farm/llamafarm/releases/latest).\n\n2. **Create and run a project**\n\n   ```bash\n   lf init my-project      # Generates llamafarm.yaml\n   lf start                # Starts services and opens Designer UI\n   ```\n\n3. **Chat with your AI**\n\n   ```bash\n   lf chat                           # Interactive chat\n   lf chat \"Hello, LlamaFarm!\"       # One-off message\n   ```\n\nThe Designer web interface is available at `http://localhost:14345`.\n\n### Option 3: Development from Source\n\n```bash\ngit clone https://github.com/llama-farm/llamafarm.git\ncd llamafarm\n\n# Install Nx globally and initialize the workspace\nnpm install -g nx\nnx init --useDotNxInstallation --interactive=false  # Required on first clone\n\n# Start all services (run each in a separate terminal)\nnx start server           # FastAPI server (port 14345)\nnx start rag              # RAG worker for document processing\nnx start universal-runtime # ML models, OCR, embeddings (port 11540)\n```\n\n---\n\n## Architecture\n\nLlamaFarm consists of three main services:\n\n| Service | Port | Purpose |\n|---------|------|---------|\n| **Server** | 14345 | FastAPI REST API, Designer web UI, project management |\n| **RAG Worker** | - | Celery worker for async document processing |\n| **Universal Runtime** | 11540 | ML model inference, embeddings, OCR, anomaly detection |\n\nAll configuration lives in `llamafarm.yaml`—no scattered settings or hidden defaults.\n\n---\n\n## Runtime Options\n\n### Universal Runtime (Recommended)\n\nThe Universal Runtime provides access to HuggingFace models plus specialized ML capabilities:\n\n- **Text Generation** - Any HuggingFace text model\n- **Embeddings** - sentence-transformers and other embedding models\n- **OCR** - Text extraction from images/PDFs (Surya, EasyOCR, PaddleOCR, Tesseract)\n- **Document Extraction** - Forms, invoices, receipts via vision models\n- **Text Classification** - Pre-trained or custom models via SetFit\n- **Named Entity Recognition** - Extract people, organizations, locations\n- **Reranking** - Cross-encoder models for improved RAG quality\n- **Anomaly Detection** - Isolation Forest, One-Class SVM, Local Outlier Factor, Autoencoders\n\n```yaml\nruntime:\n  models:\n    default:\n      provider: universal\n      model: Qwen/Qwen2.5-1.5B-Instruct\n      base_url: http://127.0.0.1:11540/v1\n```\n\n### Ollama\n\nSimple setup for GGUF models with CPU/GPU acceleration:\n\n```yaml\nruntime:\n  models:\n    default:\n      provider: ollama\n      model: qwen3:8b\n      base_url: http://localhost:11434/v1\n```\n\n### OpenAI-Compatible\n\nWorks with vLLM, Together, Mistral API, or any OpenAI-compatible endpoint:\n\n```yaml\nruntime:\n  models:\n    default:\n      provider: openai\n      model: gpt-4o\n      base_url: https://api.openai.com/v1\n      api_key: ${OPENAI_API_KEY}\n```\n\n---\n\n## Core Workflows\n\n### CLI Commands\n\n| Task | Command |\n|------|---------|\n| Initialize project | `lf init my-project` |\n| Start services | `lf start` |\n| Interactive chat | `lf chat` |\n| One-off message | `lf chat \"Your question\"` |\n| List models | `lf models list` |\n| Use specific model | `lf chat --model powerful \"Question\"` |\n| Create dataset | `lf datasets create -s pdf_ingest -b main_db research` |\n| Upload files (auto-process by default) | `lf datasets upload research ./docs/*.pdf` |\n| Process dataset (if you skipped auto-process) | `lf datasets process research` |\n| Query RAG | `lf rag query --database main_db \"Your query\"` |\n| Check RAG health | `lf rag health` |\n\n### RAG Pipeline\n\n1. **Create a dataset** linked to a processing strategy and database\n2. **Upload files** (PDF, DOCX, Markdown, TXT) — processing runs automatically unless you pass `--no-process`\n3. **Process manually** only when you intentionally skipped auto-processing (e.g., large batches)\n4. **Query** using semantic search with optional metadata filtering\n\n```bash\nlf datasets create -s default -b main_db research\nlf datasets upload research ./papers/*.pdf                 # auto-processes by default\n# For large batches:\n# lf datasets upload research ./papers/*.pdf --no-process\n# lf datasets process research\nlf rag query --database main_db \"What are the key findings?\"\n```\n\n### Designer Web UI\n\nThe Designer at `http://localhost:14345` provides:\n\n- **Project management** with briefs and quick actions\n- **Visual dataset management** with drag-and-drop uploads\n- **Database & RAG configuration** with built-in query testing\n- **Prompt engineering** with template variables and testing\n- **Interactive chat** with RAG toggle and retrieved context display\n- **Config editor** with syntax highlighting, validation, and auto-completion\n- Switch between visual Designer and raw YAML modes in any section\n\nSee the [Designer Features Guide](docs/website/docs/designer/features.md) for details.\n\n---\n\n## Configuration\n\n`llamafarm.yaml` is the source of truth for each project:\n\n```yaml\nversion: v1\nname: my-assistant\nnamespace: default\n\n# Multi-model configuration\nruntime:\n  default_model: fast\n\n  models:\n    fast:\n      description: \"Fast local model\"\n      provider: universal\n      model: Qwen/Qwen2.5-1.5B-Instruct\n      base_url: http://127.0.0.1:11540/v1\n\n    powerful:\n      description: \"More capable model\"\n      provider: universal\n      model: Qwen/Qwen2.5-7B-Instruct\n      base_url: http://127.0.0.1:11540/v1\n\n# System prompts\nprompts:\n  - name: default\n    messages:\n      - role: system\n        content: You are a helpful assistant.\n\n# RAG configuration\nrag:\n  databases:\n    - name: main_db\n      type: ChromaStore\n      default_embedding_strategy: default_embeddings\n      default_retrieval_strategy: semantic_search\n      embedding_strategies:\n        - name: default_embeddings\n          type: UniversalEmbedder\n          config:\n            model: sentence-transformers/all-MiniLM-L6-v2\n            base_url: http://127.0.0.1:11540/v1\n      retrieval_strategies:\n        - name: semantic_search\n          type: BasicSimilarityStrategy\n          config:\n            top_k: 5\n\n  data_processing_strategies:\n    - name: default\n      parsers:\n        - type: PDFParser_LlamaIndex\n          config:\n            chunk_size: 1000\n            chunk_overlap: 100\n        - type: MarkdownParser_Python\n          config:\n            chunk_size: 1000\n      extractors: []\n\n# Dataset definitions\ndatasets:\n  - name: research\n    data_processing_strategy: default\n    database: main_db\n```\n\n### Environment Variable Substitution\n\nUse `${VAR}` syntax to inject secrets from `.env` files:\n\n```yaml\nruntime:\n  models:\n    openai:\n      api_key: ${OPENAI_API_KEY}\n      # With default: ${OPENAI_API_KEY:-sk-default}\n      # From specific file: ${file:.env.production:API_KEY}\n```\n\nSee the [Configuration Guide](docs/website/docs/configuration/index.md) for complete reference.\n\n---\n\n## REST API\n\nLlamaFarm provides an OpenAI-compatible REST API:\n\n**Chat Completions**\n```bash\ncurl -X POST http://localhost:14345/v1/projects/default/my-project/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}],\n    \"stream\": false,\n    \"rag_enabled\": true\n  }'\n```\n\n**RAG Query**\n```bash\ncurl -X POST http://localhost:14345/v1/projects/default/my-project/rag/query \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"query\": \"What are the requirements?\",\n    \"database\": \"main_db\",\n    \"top_k\": 5\n  }'\n```\n\nSee the [API Reference](docs/website/docs/api/index.md) for all endpoints.\n\n---\n\n## Specialized ML Capabilities\n\nThe Universal Runtime provides endpoints beyond chat:\n\n### OCR & Document Extraction\n\n```bash\ncurl -X POST http://localhost:14345/v1/vision/ocr \\\n  -F \"file=@document.pdf\" \\\n  -F \"model=surya\"\n```\n\n### Anomaly Detection\n\nLlamaFarm supports 12+ anomaly detection algorithms via PyOD, with both batch and streaming modes.\n\n```bash\n# Train on normal data\ncurl -X POST http://localhost:14345/v1/ml/anomaly/fit \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\": \"sensor-detector\", \"backend\": \"ecod\", \"data\": [[22.1], [23.5], ...]}'\n\n# Detect anomalies\ncurl -X POST http://localhost:14345/v1/ml/anomaly/detect \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\": \"sensor-detector\", \"data\": [[22.0], [100.0], [23.0]], \"threshold\": 0.5}'\n\n# Streaming detection (handles cold start, auto-retraining, sliding windows)\ncurl -X POST http://localhost:14345/v1/ml/anomaly/stream \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\": \"live-sensor\", \"data\": {\"temperature\": 72.5}, \"backend\": \"ecod\"}'\n```\n\n**Available backends:** `ecod` (recommended), `isolation_forest`, `one_class_svm`, `local_outlier_factor`, `autoencoder`, `hbos`, `copod`, `knn`, `mcd`, `cblof`, `suod`, `loda`\n\n### Text Classification & NER\n\nSee the [Models Guide](docs/website/docs/models/index.md) for complete documentation.\n\n### Tool Calling (MCP)\n\nGive models access to external tools via the Model Context Protocol:\n\n```yaml\n# In llamafarm.yaml\nmcp:\n  servers:\n    - name: filesystem\n      transport: stdio\n      command: npx\n      args: ['-y', '@modelcontextprotocol/server-filesystem', '/data']\n\nruntime:\n  models:\n    - name: assistant\n      provider: ollama\n      model: llama3.1:8b\n      mcp_servers: [filesystem]\n```\n\nLlamaFarm also exposes its own API as MCP tools for use with Claude Desktop, Cursor, and other MCP clients. See the [Tool Calling Guide](docs/website/docs/mcp/index.md).\n\n---\n\n## Examples\n\n| Example | Description | Location |\n|---------|-------------|----------|\n| **RAG Examples** | | |\n| Large Complex PDFs | Multi-megabyte planning ordinances | `examples/large_complex_rag/` |\n| Many Small Files | FDA correspondence letters | `examples/many_small_file_rag/` |\n| Mixed Formats | PDF, Markdown, HTML, text, and code | `examples/mixed_format_rag/` |\n| Quick Notes | Rapid smoke tests with small files | `examples/quick_rag/` |\n| **Anomaly Detection** | | |\n| Quick Start | Simplest anomaly detection example | `examples/anomaly/01_quick_start.py` |\n| Fraud Detection | Training, saving, loading models | `examples/anomaly/02_fraud_detection.py` |\n| Streaming Sensors | IoT monitoring with rolling features | `examples/anomaly/03_streaming_sensors.py` |\n| Backend Comparison | Compare all 12 algorithms | `examples/anomaly/04_backend_comparison.py` |\n| **Use Cases** | | |\n| FDA Letters Assistant | Regulatory document analysis | `examples/fda_rag/` |\n| Government Planning | Large ordinance documents | `examples/gov_rag/` |\n\nSee [`examples/README.md`](examples/README.md) for setup instructions and the full list.\n\n---\n\n## Industry Use Cases\n\nLlamaFarm is used across industries for document analysis, monitoring, and fraud detection:\n\n- **[Pharmaceutical & Therapeutics](docs/website/docs/use-cases/pharmaceutical-fda.md)** — Analyze FDA correspondence, track regulatory questions\n- **[IoT Sensor Monitoring](docs/website/docs/use-cases/iot-sensor-monitoring.md)** — Real-time streaming anomaly detection with automatic retraining\n- **[Financial Fraud Detection](docs/website/docs/use-cases/financial-fraud-detection.md)** — Multi-stage fraud detection with velocity and behavioral patterns\n\n---\n\n## Development & Testing\n\n```bash\n# Python server tests\ncd server && uv sync && uv run --group test python -m pytest\n\n# CLI tests\ncd cli && go test ./...\n\n# RAG tests\ncd rag && uv sync && uv run pytest tests/\n\n# Universal Runtime tests\ncd runtimes/universal && uv sync && uv run pytest tests/\n\n# Build docs\nnx build docs\n```\n\n---\n\n## Extensibility\n\n- **Add runtimes** by implementing provider support and updating schema\n- **Add vector stores** by implementing store backends (Chroma, Qdrant, etc.)\n- **Add parsers** for new file formats (PDF, DOCX, HTML, CSV, etc.)\n- **Add extractors** for custom metadata extraction\n- **Add CLI commands** under `cli/cmd/`\n\nSee the [Extending Guide](docs/website/docs/extending/index.md) for step-by-step instructions.\n\n---\n\n## Community & Support\n\n- [Discord](https://discord.gg/RrAUXTCVNF) - Chat with the team and community\n- [GitHub Issues](https://github.com/llama-farm/llamafarm/issues) - Bug reports and feature requests\n- [Discussions](https://github.com/llama-farm/llamafarm/discussions) - Ideas and proposals\n- [Contributing Guide](CONTRIBUTING.md) - Code style and contribution process\n\n---\n\n## License\n\nLicensed under the [Apache 2.0 License](LICENSE). See [CREDITS](CREDITS.md) for acknowledgments.\n\n---\n\nBuild locally. Deploy anywhere. Own your AI.\n",
  "bytes": 15374,
  "sha": "64f9ce2dbe46551a746a73d5270c5a924d7c77d12203780b681add51a7d85cbd",
  "repo_slug": "llama-farm/llamafarm",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/skl_llama_farm_llamafarm_code_review_944f71f9/readme"
}