Gigabrain OCR API PRD and Project Specification

Date: 2026-07-01 Owner: Shady Khalifa Project codename: Gigabrain OCR API Target host: gigabrain Primary use case: fast local image-to-text extraction over HTTP

1. Executive Summary

Gigabrain OCR API is a lightweight, GPU-accelerated HTTP service for converting screenshots, invoices, diagrams, handwritten notes, receipts, forms, and general document images into structured text.

The service should run locally on the gigabrain host and expose a simple API that internal tools, agents, scripts, and web applications can call. It should prioritize low latency, predictable output, and local data privacy.

The recommended default engine is PaddleOCR with PP-OCRv6 medium detection and recognition models running through ONNX Runtime GPU. A slower optional document parsing lane can later be added using PaddleOCR-VL-1.6-GGUF served by llama-server for layout-heavy documents, tables, formulas, charts, and diagram interpretation.

2. Problem Statement

Shady wants a fast local OCR endpoint for practical image-to-text workloads:

Screenshots
Invoices
Diagrams
Handwritten notes
Receipts
Forms
Scanned or photographed documents
Multilingual text images

Existing local LLM infrastructure on gigabrain already uses llama-server for GGUF models on GPU. However, general LLM or VLM serving is not the best default for fast OCR. OCR needs specialized models with fast image preprocessing, detection, recognition, confidence scoring, and structured outputs.

3. Goals

3.1 Primary Goals

Provide a quick HTTP API for OCR on gigabrain.
Accept image upload, URL, and base64 inputs.
Return extracted text quickly with confidence scores and bounding boxes.
Use local GPU acceleration where available.
Keep the OCR model loaded in memory between requests.
Support screenshots, invoices, notes, diagrams, receipts, and forms.
Produce outputs useful for downstream automation, search, RAG, and agents.
Provide Docker and docker-compose deployment.
Include health checks and basic observability.

3.2 Secondary Goals

Add a slower smart parsing lane for layout-heavy documents.
Support markdown output for parsed documents.
Support batch OCR.
Support optional PDF page rasterization.
Provide stable schemas for future client integrations.
Provide a path to swap OCR backends without changing the API contract.

4. Non-Goals

This is not a general chat model.
This is not a replacement for existing llama-server LLM endpoints.
This is not a full document management system.
This is not intended to perform semantic reasoning over documents in v1.
This is not intended to guarantee perfect handwritten recognition.
This is not intended to expose a public internet service by default.
This is not intended to store user documents permanently unless explicitly configured.

5. Target Users

5.1 Primary User

Shady, running local tools and agents that need fast OCR from gigabrain.

5.2 Secondary Users

Local automation scripts
Hermes agents
RAG ingestion pipelines
Personal document workflows
Internal tools that need screenshot-to-text or invoice extraction

6. Use Cases

6.1 Screenshot OCR

User uploads a screenshot and receives all visible text with line-level boxes and confidence scores.

6.2 Invoice OCR

User uploads an invoice image and receives raw extracted text. Later versions may add key-value extraction such as invoice number, vendor, date, total, tax, currency, and line items.

6.3 Diagram OCR

User uploads a diagram and receives extracted labels. The fast OCR lane extracts visible text, while the optional smart lane can later describe structure and relationships.

6.4 Handwritten Notes

User uploads a photo of notes and receives best-effort text extraction. Accuracy is expected to vary with handwriting quality, lighting, orientation, and noise.

6.5 Batch Processing

User uploads multiple images and receives per-file OCR results in one response.

6.6 Agent Integration

Hermes or another agent sends an image to the service and receives structured OCR output for downstream reasoning.

7. Recommended Model and Runtime

7.1 Fast Lane: Default OCR

Recommended:

PaddleOCR 3.7 or newer
PP-OCRv6_medium_det
PP-OCRv6_medium_rec
ONNX Runtime GPU
FastAPI
Uvicorn or Gunicorn with Uvicorn workers
NVIDIA container runtime

Why:

PP-OCRv6 is specialized for OCR.
It is much smaller and faster than a VLM.
The medium tier targets server-side accuracy.
It supports many practical OCR scenarios and languages.
It avoids occupying the primary llama-server model slot.

7.2 Smart Lane: Optional Document Parser

Recommended later:

PaddleOCR-VL-1.6-GGUF
llama-server on a separate port
PaddleOCR doc_parser client using llama-cpp-server backend

Use only when the fast lane is insufficient:

Complex layouts
Tables
Formulas
Charts
Seal recognition
Layout-aware document parsing
Markdown export

8. Architecture

8.1 High-Level Architecture

Client sends image to Gigabrain OCR API.

Gigabrain OCR API performs:

Request validation
Image acquisition from upload, URL, or base64
Image normalization
OCR inference through a preloaded PaddleOCR pipeline
Response formatting
Optional output persistence
Metrics and logs

8.2 Components

API Service

FastAPI application exposing OCR endpoints.

Responsibilities:

Accept HTTP requests
Validate inputs
Save temporary files safely
Call OCR engine
Normalize PaddleOCR result shapes
Return stable JSON
Emit logs and metrics

OCR Engine

A singleton object initialized once at startup.

Responsibilities:

Load PaddleOCR pipeline
Load PP-OCRv6 models
Use ONNX Runtime GPU
Run prediction
Return raw OCR result objects

Storage Layer

Default: ephemeral temporary files only.

Optional:

Save uploaded inputs for debugging
Save visualized OCR overlays
Save JSON outputs

Storage must be disabled by default or bounded by retention settings.

Smart Parser Lane

Optional second service path that calls PaddleOCR-VL-1.6-GGUF through llama-server.

Responsibilities:

Parse complex documents
Produce markdown or structured document output
Handle slower, higher-cost requests

9. API Specification

Base URL:

http://gigabrain:8000

9.1 GET /health

Health check endpoint.

Response 200:

{
  "status": "ok",
  "service": "gigabrain-ocr-api",
  "engine": "paddleocr",
  "model_detection": "PP-OCRv6_medium_det",
  "model_recognition": "PP-OCRv6_medium_rec",
  "backend": "onnxruntime",
  "gpu": true
}

9.2 POST /ocr

OCR from multipart image upload.

Request:

Content-Type: multipart/form-data

Fields:

file: required image file
mode: optional, default fast
return_boxes: optional boolean, default true
return_confidence: optional boolean, default true
save_debug: optional boolean, default false

Example:

curl -F "file=@invoice.png" http://gigabrain:8000/ocr

Response 200:

{
  "request_id": "01h...",
  "engine": "paddleocr",
  "mode": "fast",
  "text": "invoice\nacme corp\ntotal: $42.00",
  "items": [
    {
      "text": "invoice",
      "confidence": 0.991,
      "box": [[10, 12], [100, 12], [100, 35], [10, 35]]
    }
  ],
  "pages": [
    {
      "page_index": 0,
      "text": "invoice\nacme corp\ntotal: $42.00",
      "items_count": 3
    }
  ],
  "timing_ms": {
    "total": 84,
    "preprocess": 8,
    "inference": 62,
    "postprocess": 14
  }
}

9.3 POST /ocr/base64

OCR from base64 image.

Request:

{
  "image_base64": "...",
  "filename": "screenshot.png",
  "mode": "fast",
  "return_boxes": true,
  "return_confidence": true
}

Response: same as POST /ocr.

9.4 POST /ocr/url

OCR from URL.

Request:

{
  "url": "https://example.com/invoice.png",
  "mode": "fast",
  "return_boxes": true,
  "return_confidence": true
}

Security requirements:

Disable by default unless needed.
If enabled, protect against SSRF.
Block private IP ranges unless explicitly allowed.
Enforce timeout and max response size.
Restrict schemes to http and https.

Response: same as POST /ocr.

9.5 POST /ocr/batch

OCR from multiple uploaded images.

Request:

Content-Type: multipart/form-data

Fields:

files: one or more image files
mode: optional, default fast

Response:

{
  "request_id": "01h...",
  "count": 2,
  "results": [
    {
      "filename": "a.png",
      "ok": true,
      "text": "...",
      "items": []
    },
    {
      "filename": "b.png",
      "ok": false,
      "error": {
        "code": "unsupported_file",
        "message": "file type is not supported"
      }
    }
  ]
}

9.6 POST /parse

Optional smart parsing endpoint, v2.

Purpose:

Use PaddleOCR-VL-1.6-GGUF through llama-server for layout-aware parsing.

Request:

file upload or base64 image
task: document, table, chart, formula, diagram, spotting
output_format: markdown, json, or text

Response:

{
  "request_id": "01h...",
  "mode": "smart",
  "task": "document",
  "text": "...",
  "markdown": "...",
  "metadata": {
    "backend": "llama-cpp-server",
    "model": "PaddleOCR-VL-1.6-GGUF"
  }
}

10. Output Schema

10.1 OcrResponse

{
  "request_id": "string",
  "engine": "paddleocr",
  "mode": "fast",
  "text": "string",
  "items": ["OcrItem"],
  "pages": ["OcrPage"],
  "debug": "object or null",
  "timing_ms": "object"
}

10.2 OcrItem

{
  "text": "string",
  "confidence": 0.0,
  "box": [[0, 0], [1, 0], [1, 1], [0, 1]],
  "page_index": 0,
  "line_index": 0
}

10.3 ErrorResponse

{
  "request_id": "string",
  "error": {
    "code": "string",
    "message": "string",
    "details": {}
  }
}

11. Functional Requirements

FR-1: Upload OCR

The API must accept image files through multipart upload and return extracted text.

Acceptance:

PNG, JPEG, and WebP are supported.
Invalid files return 400.
Response includes text and items.

FR-2: Base64 OCR

The API must accept base64 image input.

Acceptance:

Valid base64 image returns OCR result.
Invalid base64 returns 400.
Oversized payload returns 413.

FR-3: URL OCR

The API should optionally accept image URLs.

Acceptance:

URL fetching can be disabled by config.
Timeout is enforced.
Max download size is enforced.
SSRF protections are implemented.

FR-4: Batch OCR

The API should accept multiple images in one request.

Acceptance:

Each file gets an independent result.
One failed file does not fail the whole batch unless request validation fails globally.

FR-5: Singleton Model Loading

OCR model must be loaded once at process startup.

Acceptance:

The service does not instantiate PaddleOCR per request.
Health endpoint confirms model readiness.

FR-6: Structured Output

The API must return text, confidence, and bounding boxes when available.

Acceptance:

text is newline-joined in reading order.
items preserve per-line or per-token confidence and boxes.

FR-7: Observability

The service must expose logs and timings.

Acceptance:

Each request has a request ID.
Logs include request ID, duration, input type, and success or failure.
Response includes timing fields.

FR-8: Docker Deployment

The service must run as a Docker container with GPU support.

Acceptance:

Dockerfile exists.
docker-compose.yml exists.
Service starts with docker compose up -d.
GPU runtime can be enabled.

12. Non-Functional Requirements

12.1 Performance

Targets for typical screenshot or invoice image on gigabrain GPU:

Warm p50 latency: under 300 ms preferred
Warm p95 latency: under 1 second preferred
Cold start can be slower due to model loading

Actual targets must be benchmarked on gigabrain.

12.2 Reliability

Service must not crash on malformed files.
Temporary files must be cleaned up.
OCR failures must return structured errors.
Model load failure must make health endpoint fail.

12.3 Privacy

No external API calls for OCR inference.
Uploaded images are not persisted by default.
Debug persistence is explicit and bounded.
URL fetching is disabled by default or heavily restricted.

12.4 Security

Enforce max upload size.
Validate file MIME and decoded image format.
Protect URL endpoint against SSRF.
Bind to LAN or localhost by default, not public internet.
Optional API key authentication for LAN deployments.

12.5 Maintainability

Keep engine behind a small interface.
Keep API schema stable.
Version endpoints if breaking changes are needed.
Add tests for request validation and response normalization.

13. Configuration

Environment variables:

OCR_HOST=0.0.0.0
OCR_PORT=8000
OCR_ENGINE=paddleocr
OCR_BACKEND=onnxruntime
OCR_DEVICE=gpu:0
OCR_DET_MODEL=PP-OCRv6_medium_det
OCR_REC_MODEL=PP-OCRv6_medium_rec
OCR_USE_DOC_ORIENTATION=false
OCR_USE_DOC_UNWARPING=false
OCR_USE_TEXTLINE_ORIENTATION=true
OCR_MAX_UPLOAD_MB=20
OCR_ENABLE_URL_INPUT=false
OCR_URL_TIMEOUT_SECONDS=10
OCR_SAVE_DEBUG=false
OCR_DEBUG_DIR=/data/debug
OCR_REQUIRE_API_KEY=false
OCR_API_KEY=

14. Deployment Specification

14.1 Dockerfile

FROM nvidia/cuda:12.6.3-cudnn-runtime-ubuntu24.04

ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

RUN apt-get update && apt-get install -y \
    python3 \
    python3-pip \
    libglib2.0-0 \
    libgl1 \
    curl \
    && rm -rf /var/lib/apt/lists/*

RUN pip3 install --break-system-packages \
    fastapi \
    uvicorn[standard] \
    python-multipart \
    pillow \
    paddleocr \
    onnxruntime-gpu

WORKDIR /app
COPY app /app/app

EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

14.2 docker-compose.yml

services:
  gigabrain-ocr-api:
    build: .
    image: gigabrain-ocr-api:ppocrv6
    container_name: gigabrain-ocr-api
    restart: unless-stopped
    ports:
      - "8000:8000"
    environment:
      OCR_ENGINE: paddleocr
      OCR_BACKEND: onnxruntime
      OCR_DEVICE: gpu:0
      OCR_DET_MODEL: PP-OCRv6_medium_det
      OCR_REC_MODEL: PP-OCRv6_medium_rec
      OCR_USE_DOC_ORIENTATION: "false"
      OCR_USE_DOC_UNWARPING: "false"
      OCR_USE_TEXTLINE_ORIENTATION: "true"
      OCR_MAX_UPLOAD_MB: "20"
      OCR_ENABLE_URL_INPUT: "false"
      OCR_SAVE_DEBUG: "false"
    volumes:
      - /srv/models/paddleocr:/root/.paddlex
      - /srv/gigabrain-ocr/debug:/data/debug
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

15. Reference Implementation Layout

gigabrain-ocr-api/
  app/
    __init__.py
    main.py
    config.py
    engine.py
    schemas.py
    image_io.py
    security.py
    timing.py
  tests/
    test_health.py
    test_ocr_upload.py
    test_validation.py
    fixtures/
      sample_invoice.png
      sample_screenshot.png
  Dockerfile
  docker-compose.yml
  pyproject.toml
  README.md

16. Minimal FastAPI Implementation Sketch

from fastapi import FastAPI, UploadFile, File, HTTPException
from paddleocr import PaddleOCR
from pathlib import Path
import tempfile
import time
import json
import uuid

app = FastAPI(title="Gigabrain OCR API")

ocr = PaddleOCR(
    text_detection_model_name="PP-OCRv6_medium_det",
    text_recognition_model_name="PP-OCRv6_medium_rec",
    engine="onnxruntime",
    use_doc_orientation_classify=False,
    use_doc_unwarping=False,
    use_textline_orientation=True,
)

@app.get("/health")
def health():
    return {
        "status": "ok",
        "service": "gigabrain-ocr-api",
        "engine": "paddleocr",
        "model_detection": "PP-OCRv6_medium_det",
        "model_recognition": "PP-OCRv6_medium_rec",
        "backend": "onnxruntime",
    }

@app.post("/ocr")
async def run_ocr(file: UploadFile = File(...)):
    request_id = str(uuid.uuid4())
    start = time.perf_counter()

    suffix = Path(file.filename or "image.png").suffix or ".png"

    with tempfile.NamedTemporaryFile(suffix=suffix) as tmp:
        tmp.write(await file.read())
        tmp.flush()

        try:
            results = ocr.predict(tmp.name)
        except Exception as exc:
            raise HTTPException(status_code=500, detail={
                "request_id": request_id,
                "error": {
                    "code": "ocr_failed",
                    "message": str(exc),
                },
            })

    items = []
    text_parts = []

    for page_index, page in enumerate(results):
        data = None

        if hasattr(page, "json"):
            data = page.json() if callable(page.json) else page.json
        elif hasattr(page, "to_json"):
            data = page.to_json()

        if isinstance(data, str):
            data = json.loads(data)

        res = data.get("res", data) if isinstance(data, dict) else {}
        rec_texts = res.get("rec_texts") or []
        rec_scores = res.get("rec_scores") or []
        rec_polys = res.get("rec_polys") or res.get("dt_polys") or []

        for line_index, text in enumerate(rec_texts):
            text_parts.append(text)
            items.append({
                "text": text,
                "confidence": rec_scores[line_index] if line_index < len(rec_scores) else None,
                "box": rec_polys[line_index] if line_index < len(rec_polys) else None,
                "page_index": page_index,
                "line_index": line_index,
            })

    total_ms = round((time.perf_counter() - start) * 1000, 2)

    return {
        "request_id": request_id,
        "engine": "paddleocr",
        "mode": "fast",
        "text": "\n".join(text_parts),
        "items": items,
        "pages": [
            {
                "page_index": 0,
                "text": "\n".join(text_parts),
                "items_count": len(items),
            }
        ],
        "timing_ms": {
            "total": total_ms,
        },
    }

17. Testing Plan

17.1 Unit Tests

Config parsing
Upload size validation
MIME validation
Base64 decoding
URL filtering and SSRF guard
Response normalization from PaddleOCR result objects

17.2 Integration Tests

POST /health returns ok
POST /ocr with sample screenshot returns non-empty text
POST /ocr with sample invoice returns non-empty text
POST /ocr with invalid file returns 400
POST /ocr/batch returns mixed success and failure correctly

17.3 Performance Tests

Benchmark using representative files:

Screenshot 1080p
Invoice scan
Receipt photo
Handwritten note photo
Diagram image

Record:

Cold start model load time
Warm p50 latency
Warm p95 latency
GPU memory use
CPU memory use
Throughput at concurrency 1, 2, 4, and 8

18. Acceptance Criteria

MVP is accepted when:

Service runs on gigabrain through Docker Compose.
GET /health returns ok.
POST /ocr accepts PNG and JPEG uploads.
OCR model is loaded once at startup.
Response includes text, items, confidence, and boxes where available.
Sample screenshot returns useful text.
Sample invoice returns useful text.
Invalid input returns structured errors.
No uploaded files are persisted by default.
README documents install, run, and curl usage.

19. Rollout Plan

Phase 1: MVP Fast OCR API

Implement FastAPI service.
Add PaddleOCR singleton engine.
Add upload endpoint.
Add health endpoint.
Add Dockerfile and docker-compose.yml.
Test on gigabrain GPU.

Phase 2: Usability and Safety

Add base64 endpoint.
Add batch endpoint.
Add optional API key.
Add upload size limits.
Add structured logging.
Add simple benchmark script.

Phase 3: Smart Parser Lane

Run PaddleOCR-VL-1.6-GGUF through llama-server on separate port.
Add /parse endpoint.
Add markdown output.
Add table and formula tasks.

Phase 4: Client Integrations

Add Python client.
Add CLI client.
Add Hermes integration helper.
Add Obsidian ingestion helper if useful.

20. Risks and Mitigations

Risk: PaddleOCR result shape changes between versions

Mitigation:

Pin dependency versions.
Normalize results defensively.
Add tests around sample outputs.

Risk: GPU dependency issues

Mitigation:

Use NVIDIA CUDA runtime base image.
Document driver and CUDA requirements.
Provide CPU fallback for debugging only.

Risk: URL endpoint SSRF

Mitigation:

Disable URL input by default.
Restrict schemes.
Block private IP ranges.
Enforce DNS resolution checks.
Enforce timeout and max download size.

Risk: Large images causing memory pressure

Mitigation:

Enforce upload limits.
Resize or reject huge images.
Add request timeout.

Risk: Handwriting accuracy is inconsistent

Mitigation:

Set expectations clearly.
Add smart parser fallback later.
Benchmark with Shady's real samples.

21. Open Questions

Should the service require an API key on LAN by default?
Which port should be reserved on gigabrain?
Should debug images and JSON outputs ever be persisted?
Should PDF input be supported in MVP or v2?
Should URL input be enabled at all?
Should the service expose Prometheus metrics?
Should it return reading order only, or preserve spatial ordering as separate fields?

22. Recommended Defaults

Engine: PaddleOCR
Detection: PP-OCRv6_medium_det
Recognition: PP-OCRv6_medium_rec
Backend: ONNX Runtime GPU
Orientation classification: off by default
Textline orientation: on by default
URL input: off by default
Debug persistence: off by default
API auth: optional for LAN, required if exposed beyond trusted network
Smart parser: separate v2 lane

23. References

PaddlePaddle Hugging Face organization: https://huggingface.co/PaddlePaddle
PaddleOCR-VL-1.6: https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.6
PaddleOCR-VL-1.6-GGUF: https://huggingface.co/PaddlePaddle/PaddleOCR-VL-1.6-GGUF
PP-OCRv6 collection: https://huggingface.co/collections/PaddlePaddle/pp-ocrv6
PaddleOCR documentation: https://paddlepaddle.github.io/PaddleOCR/latest/en/index.html
PaddleOCR general OCR pipeline: https://paddlepaddle.github.io/PaddleOCR/latest/en/version3.x/pipeline_usage/OCR.html

24. Final Recommendation

Build the MVP as a dedicated FastAPI service around PP-OCRv6 medium with ONNX Runtime GPU. Keep it focused on fast OCR and stable structured output. Add PaddleOCR-VL-1.6-GGUF later as a separate slower parser lane, not as the default path.

This gives gigabrain a practical local OCR appliance: fast for screenshots and invoices, structured enough for agents, private by default, and extensible when layout-aware parsing becomes necessary.

shekohex/gigabrain-ocr-api-prd.md

Gigabrain OCR API PRD and Project Specification

1. Executive Summary

2. Problem Statement

3. Goals

3.1 Primary Goals

3.2 Secondary Goals

4. Non-Goals

5. Target Users

5.1 Primary User

5.2 Secondary Users

6. Use Cases

6.1 Screenshot OCR

6.2 Invoice OCR

6.3 Diagram OCR

6.4 Handwritten Notes

6.5 Batch Processing

6.6 Agent Integration

7. Recommended Model and Runtime

7.1 Fast Lane: Default OCR

7.2 Smart Lane: Optional Document Parser

8. Architecture

8.1 High-Level Architecture

8.2 Components

API Service

OCR Engine

Storage Layer

Smart Parser Lane

9. API Specification

9.1 GET /health

9.2 POST /ocr

9.3 POST /ocr/base64

9.4 POST /ocr/url

9.5 POST /ocr/batch

9.6 POST /parse

10. Output Schema

10.1 OcrResponse

10.2 OcrItem

10.3 ErrorResponse

11. Functional Requirements

FR-1: Upload OCR

FR-2: Base64 OCR

FR-3: URL OCR

FR-4: Batch OCR

FR-5: Singleton Model Loading

FR-6: Structured Output

FR-7: Observability

FR-8: Docker Deployment

12. Non-Functional Requirements

12.1 Performance

12.2 Reliability

12.3 Privacy

12.4 Security

12.5 Maintainability

13. Configuration

14. Deployment Specification

14.1 Dockerfile

14.2 docker-compose.yml

15. Reference Implementation Layout

16. Minimal FastAPI Implementation Sketch

17. Testing Plan

17.1 Unit Tests

17.2 Integration Tests

17.3 Performance Tests

18. Acceptance Criteria

19. Rollout Plan

Phase 1: MVP Fast OCR API

Phase 2: Usability and Safety

Phase 3: Smart Parser Lane

Phase 4: Client Integrations

20. Risks and Mitigations

Risk: PaddleOCR result shape changes between versions

Risk: GPU dependency issues

Risk: URL endpoint SSRF

Risk: Large images causing memory pressure

Risk: Handwriting accuracy is inconsistent

21. Open Questions

22. Recommended Defaults

23. References

24. Final Recommendation