Ingest

Ingest takes one or more documents, chunks them according to the configured voice, embeds each chunk through the corpus’s model, and writes the vectors into the corpus. The same endpoint handles a three-document insert and a 100k-document overnight batch — every request is accepted as a background job (202 Accepted + job_id), and you poll the job to completion. All ingest is asynchronous; all neural search is synchronous.

The basic call

POST /v1/ingest

{
  "corpus_id": "787cfadf…",
  "voice_id": "…",
  "documents": [
    {
      "id": "doc-001",
      "content": "# My document\n\nSome text to embed…",
      "metadata": { "category": "reference", "lang": "en" }
    }
  ],
  "dry_run": false
}

Fields:

Document shape

{
  "id": "doc-001",
  "content": "…the document text…",
  "metadata": { "author": "…", "published": "2026-04-10" },
  "fingerprint": ""
}

• id is the stable, customer-chosen identifier. Re-ingesting a document with the same id replaces the prior version.
• content is plain text or markdown. Binary documents (PDF, DOCX) should be converted upstream — Enscrive does not do file extraction.
• metadata is a string-keyed map. Keys appear verbatim in search filters, so pick keys you actually want to filter by.
• fingerprint is optional. If you supply it, the server uses it to skip re-embedding unchanged content. If you leave it empty, the server computes SHA256(content) server-side — functionally identical for plain text, saves you the compute cost.

Always-async execution

Every ingest request — any payload size — returns 202 Accepted with a JobLaunchResponse:

{ "job_id": "…", "status": "pending", "poll_url": "/v1/jobs/…" }

Poll GET /v1/jobs/{id} for progress and the terminal state. Per-document failures land on the job record (failed_document_ids, warnings, error_message) rather than in the HTTP response. The job internally picks the right embed mechanism — synchronous embed RPC for small chunk counts, a provider batch API above the provider’s threshold — but that choice never blocks your HTTP request.

Legacy request shapes are accepted-and-ignored with a deprecation field in the 202 body:

• "sync": true — formerly forced inline execution. Now ignored; a job is launched.
• Accept: text/event-stream — formerly streamed inline progress. Now ignored; poll the job instead.

Migration note. An older no_batch flag exists on the request shape. It selects the synchronous embed RPC inside the job instead of a provider batch API; it does not make the HTTP request synchronous. It is being retired in favor of an explicit mechanism contract (tracked as J-001); do not wire new code against no_batch.

Fingerprinting and deduplication

Every chunk’s fingerprint (SHA256(chunk_content)) is persisted alongside the vector. Re-ingesting a document with unchanged chunks causes those chunks to be detected as identical and skipped — no provider call, no billing, no Qdrant write. This is what makes enscrive-docs ingest cheap to run in a watch loop.

Practical consequence: if you are in a fast edit-and-refresh cycle while tuning docs, you pay provider tokens only for the chunks that actually changed.

Prepared (two-phase) ingest

POST /v1/ingest-prepared

Use this when your client has already done the chunking. Instead of documents, you send pre-chunked records:

{
  "corpus_id": "…",
  "voice_id": "…",
  "chunks": [
    { "document_id": "doc-001", "chunk_index": 0, "content": "…", "metadata": {} },
    { "document_id": "doc-001", "chunk_index": 1, "content": "…", "metadata": {} }
  ]
}

This skips the voice’s chunking step and goes straight to embed + store. Useful when:

• You have an external segmenter that produces chunks Enscrive’s strategies don’t express.
• You want to checkpoint the chunk boundaries (re-run chunking deterministically across environments, or audit them independently of the service).

Previewing chunking

Before committing to an ingest, you can preview how a document will chunk:

POST /v1/preview-chunking        # exercise the voice's chunking strategy against input text
POST /v1/preview-with-template   # exercise a segmentation template (LLM-driven)

Both return the exact chunks that would be produced — same boundaries, same metadata — but do nothing to the corpus.

LLM-driven segmentation

POST /v1/segment-document

Runs a single-pass LLM segmentation over a document and returns the chunks it proposed. Used as a component of generative segmentation voices, or standalone for offline authoring workflows.

Dry run

"dry_run": true on any ingest endpoint returns what would happen without side-effects:

• Documents are validated (metadata keys, content length)
• Chunking is applied and chunk boundaries returned
• No embedding call, no write to the corpus, no billing

Useful in CI to catch malformed payloads before a real ingest.

Response shapes

Every ingest response (202 Accepted)

{
  "job_id": "…",
  "status": "pending",
  "poll_url": "/v1/jobs/…"
}

The server creates an import_jobs row for every ingest, so there is a uniform audit trail. Poll poll_url for progress, counts, and the terminal state. For the full lifecycle — sub-batch state, provider batch ids, retry semantics — see Batch-sets and Jobs. Deprecated request shapes (sync: true, Accept: text/event-stream) add a deprecation string to this body.

Using enscrive-docs

The typical workflow when documenting a markdown directory is:

enscrive-docs bootstrap            # first-run: creates voice + corpus + ingests
enscrive-docs ingest               # re-ingest (unchanged files skip via fingerprint)
enscrive-docs watch                # dev loop: file-save → re-ingest → browser reload
enscrive-docs reset --yes          # delete + rebuild when the corpus has drifted hard

See the enscrive-docs quickstart for the full tool workflow.

Common errors

For batch-ingest failures that occur after the job is launched (provider rate limits, transient 5xx, partial failure), see Batch-sets and the attempts log — those are job-level failures, not POST /v1/ingest response-level errors.