<!-- Generated by scripts/generate-agent-docs.ts. Do not edit by hand. -->
# Sapien Proof of Quality Docs

Source site: https://sapien-ai-docs-production.up.railway.app

This file concatenates the current public docs into one agent-readable context file.

---

# What is PoQ

URL: https://sapien-ai-docs-production.up.railway.app/start/introduction
Source: docs/current/start/introduction.md

# What is PoQ

**Proof of Quality (PoQ) is a verification layer for AI work.** You define quality as a rubric, a panel of independent validators scores each item against it, consensus resolves one verdict per item, and PoQ seals the rubric, outcome, and level of agreement as a permanent onchain attestation. Instead of asking anyone to trust an internal review process, PoQ records how the decision was reached — and anyone can verify that record.

PoQ sits on top of the evaluation, audit, and review workflows you already run. It doesn't replace your tools or experts; it preserves how quality was judged so the claim can travel with the work.

_Interactive on the website: a work item travels Define (rubric applied) → Validate (independent validators score it and converge to consensus) → Attest (the verdict is sealed onchain in a signed PoQ Report)._

## Why verification matters

A score tells you what was decided, not why. As AI work moves between teams and organizations, customers, partners, auditors, and regulators inherit the conclusion — but rarely the standard, the evidence, or the level of agreement behind it. PoQ attaches that record to the claim itself, so a quality claim no longer depends on the credibility of whoever made it.

:::figure{src="trust-vs-verify" title="The same quality claim with and without its record: a bare score must be taken on faith, while a score carrying rubric, evidence, agreement, and a signature can be checked by anyone" caption="A bare score must be taken on faith. A PoQ-backed score carries its rubric, evidence, agreement, and signature — and anyone can check it."}
:::

## What teams use PoQ for

Any workflow that depends on review benefits from a verifiable record of that review.

:::figure{src="use-case-grid" title="Four places teams use PoQ: labeling and annotation, AI and agent evaluation, expert and audit review, and internal QA" caption="The work differs; the problem is the same. Once a review ends, its reasoning usually disappears — PoQ keeps it."}
:::

## Experts, agents, and scale

Having experts review every item is the gold standard — and it is slow and expensive. PoQ makes the fast path trustworthy instead: panels mix human experts with AI agent validators, [consensus](/validate/consensus-explained) resolves each item, and only split panels escalate to experts. Both kinds of validators see the same rubric and evidence and are scored the same way. The attestation is what makes this automation viable — you don't have to trust the automated review, because you can verify it.

:::figure{src="experts-vs-automation" title="Two ways to review a large batch, compared on cost and time: an expert pass on every item is slow and expensive with trust resting on reputation, while a consensus panel of experts and AI agents with escalation is fast and cheap with trust resting on the attestation" caption="Expert review of everything doesn't scale. Consensus panels with agent validators do — and the signed attestation is why the fast path can be trusted."}
:::

## What you get

You bring intent and a folder of data. PoQ returns a signed [PoQ Report](/validate/poq-report): one verifiable record of what was reviewed, by whom, against what rubric, and with how much agreement.

:::figure{src="poq-steps-output" title="From input to output: you bring data and intent, PoQ runs the pipeline, you get a signed PoQ Report" caption="You bring intent and data; PoQ compiles a spec, builds review items, scores them by consensus, and returns a signed PoQ Report."}
:::

That signed report is a real artifact — open the [example PoQ Report](/example-report/poq-report.html) to browse one now, or run `poq verify` against it offline.

:::details{title="The pipeline, step by step"}

| Input | What happens | What you get |
| --- | --- | --- |
| Your intent | You describe the work in plain language, by writing a [`poq.md`](/build/defining-your-spec) brief or talking it through with the conversational agent | A human-readable project brief |
| `poq.md` | The engine compiles your brief into a spec | An executable, deterministic [`poq.toml`](/build/defining-your-spec) |
| Your data folder | [Ingestion](/build/supported-data-types) reads your files | Datapoints - one row is one review item |
| Datapoints | [Validators](/validate/validation-ui) claim items, inspect the evidence, and answer the rubric | Per-dimension scores, each normalized to `0-100` |
| Scores | [Consensus](/validate/consensus-explained) resolves each dimension and escalates split panels | One verdict and a consensus strength per item |
| Verdicts | Each item's final verdict is sealed onchain and rolled up | A signed [PoQ Report](/validate/poq-report) |
| PoQ Report | Anyone runs `poq verify` | Offline confirmation the report is authentic and unaltered |

:::

## Quickstart

Ready to try it? The [Quickstart](/start/quickstart) creates your first project [in the UI](/start/quickstart#create-a-project-in-the-ui) or [via the API](/start/quickstart#create-a-project-via-the-api) in a few minutes.

## See also

| Page | Description |
| --- | --- |
| [The PoQ workflow](/build/poq-workflow) | The Define, Validate, Attest flow in full |
| [Define your spec](/build/defining-your-spec) | The three ways intent becomes an executable `poq.toml` |
| [Supported data types](/build/supported-data-types) | What files ingestion understands and how datapoints are formed |
| [Validation interface](/validate/validation-ui) | What validators see and how they answer the rubric |
| [How consensus works](/validate/consensus-explained) | Dimensions, strength, and escalation |
| [PoQ Report](/validate/poq-report) | What the report contains and how to verify it offline |

---

# Quickstart

URL: https://sapien-ai-docs-production.up.railway.app/start/quickstart
Source: docs/current/start/quickstart.md

# Quickstart

Create your first PoQ project: bring your data, describe the validations you want, and get your datapoints ready for review. Two paths get you there:

| Path | Use it when |
| --- | --- |
| [In the UI](#create-a-project-in-the-ui) | You want visual previews and the fastest way to a first project |
| [Via the API](#create-a-project-via-the-api) | You want PoQ inside an existing workflow, driven by code or an AI agent |

Both paths follow the same [PoQ workflow](/build/poq-workflow): Define, Validate, Attest. The steps below cover Define. From there, validators score each datapoint against your rubric, and the engine seals the outcome into a signed [PoQ Report](/validate/poq-report).

## Create a project in the UI

| Step | Action |
| --- | --- |
| 1 | Log in to PoQ with your email [here](https://poq.sapien.io/). |
| 2 | From [the Dashboard page](https://poq.sapien.io/dashboard), click the `New Project` button in the upper right corner. |
| 3 | Select the `Custom dataset folder` option, click `Pick folder`, and choose the folder that contains your data artifacts and a `poq.md` file describing the validations you want performed in plain language. |
| 4 | Review the generated validator preview UI and iterate until it gathers the desired data. |

## Create a project via the API

You need an [API key](/integrate/api-keys) and a JSON project spec (a mirror of [poq.toml](/reference/poq-toml)):

| Step | Action |
| --- | --- |
| 1 | Create an API key at [API Keys](https://poq.sapien.io/originator/api-keys). Copy the token, which is shown only once. |
| 2 | Verify connectivity with `GET https://poq.sapien.io/developer/v1/ping`, passing your key as a bearer token. A valid key returns `{ "pong": true }`. |
| 3 | Create a project with `POST https://poq.sapien.io/developer/v1/projects`, sending `name` and `spec` (JSON). |
| 4 | Open an upload session on the new project, declaring each file with `relPath` and `size`. |
| 5 | `PUT` each file's bytes to its presigned `uploadUrl` (no auth header on these PUTs). |
| 6 | Call `process` on the session, then poll `status` until the ingest run finishes. |

The same workflow is available over MCP for AI agents. See the [MCP server](/integrate/mcp) reference and start with its `get_started` tool.

## See also

| Page | Description |
| --- | --- |
| [The PoQ workflow](/build/poq-workflow) | How data flows from project definition to attestation |
| [API keys](/integrate/api-keys) | Key lifecycle, authorization, and security |
| [Developer API](/integrate/developer-api) | Full HTTP endpoint reference |
| [MCP server](/integrate/mcp) | Tool catalog and client setup |

---

# The PoQ workflow

URL: https://sapien-ai-docs-production.up.railway.app/build/poq-workflow
Source: docs/current/build/poq-workflow.md

# The PoQ workflow

Every PoQ project moves through the same three phases: you **Define** the work, validators **Validate** it, and the engine **Attests** to the outcome in a durable report that can be verified.

:::figure{src="poq-workflow" title="The PoQ workflow: Define, Validate, Attest" caption="Phases of a PoQ project."}
:::

The Attest phase ends in a real artifact you can hold: open the [example PoQ Report](/example-report/poq-report.html) (also as [plain text](/example-report/poq-report.txt)) to see what a project produces.

## 1. Define

Project definition turns a dataset into reviewable work.

You start with intent: what needs review, what evidence validators should see, and what "good" means. That intent becomes the `poq.toml` spec that drives the project, in one of three ways, in order of precedence:

- A [spec-compliant `poq.toml`](/reference/poq-toml) you provide is used directly as the source of truth.
- A plain-language [`poq.md`](/build/defining-your-spec) brief (or a `poq.toml` that fails to compile) is handed to the engine, which drafts the `poq.toml` for you.
- With no spec provided, the **conversational agent** walks you through the decisions in a guided chat and drafts the `poq.toml` as you go.

Whichever path you take, the spec defines the whole workload: how inputs are ingested and merged into datapoints, what evidence and instructions validators see, the rubric they score, which validator classes are eligible and how work is routed to them, and when the engine escalates for more review.

## 2. Validate

Validation is the human and agentic expert review step.

Validators claim assigned datapoints, inspect the evidence, and answer according to the rubric. The UI and API present the rubric in several modalities, such as choices and numeric scales, but every answer resolves to a `0-100` score.

Validators do not enter a quality score directly. From their rubric scores the engine derives two independent `0-100` ratings for each datapoint: a **Quality Rating** (the answer the panel landed on) and a **Consensus Strength** (how tightly they agreed). See [How quality is measured](/validate/quality-rating) for how to read the two together.

Consensus Strength drives what happens next. When it is strong enough, the datapoint finalizes. When it is weak, the datapoint can **escalate**, adding validators (often a more senior class) who re-score it. Escalation runs only if your spec defines it; otherwise the datapoint resolves on the answers already collected, with any disagreement recorded. Both ratings are recorded on the datapoint and carried into the report.

See [How consensus works](/validate/consensus-explained) for the mechanics.

## 3. Attest

Attestation turns the resolved scores into a durable artifact.

Once a datapoint has received all required validator inputs, the engine records the final outcome and builds the [PoQ Report](/validate/poq-report), the artifact a customer, auditor, or downstream system can inspect later. It is one signed payload, delivered as readable text, a designed HTML page, JSON, or a bare signature, and it can be [verified offline](/validate/poq-report#verifying-a-report).

From the report, anyone can:

- Inspect what was reviewed
- See how validators answered
- Check the consensus outcome
- Verify the report is authentic and unaltered
- Use the report downstream

---

# Define your spec

URL: https://sapien-ai-docs-production.up.railway.app/build/defining-your-spec
Source: docs/current/build/defining-your-spec.md

# Define your spec

PoQ projects start as intent and become configuration. Every project runs on one artifact: `poq.toml`, the executable, deterministic spec the engine reads. Everything else on this page is a way of arriving at it.

## Three ways to get a spec

You can arrive at `poq.toml` from any of three starting points. When you upload a folder during project creation, PoQ scans it for spec artifacts and picks the path for you, in order of precedence:

| What you bring | What PoQ does |
| --- | --- |
| A finished [`poq.toml`](#poqtoml) | Validates it and uses it directly. No model is involved. |
| A plain-language [`poq.md`](#poqmd) brief | Compiles it into a `poq.toml` draft for you to review. |
| Nothing but your data and your intent | Opens the [conversational agent](#conversational-agent), which interviews you and drafts the spec with you. |

All three end at the same place: a valid `poq.toml` that you review before the project runs.

`poq.toml` wins over `poq.md` when both are present. If the `poq.toml` is invalid and a `poq.md` is present, PoQ compiles the brief instead of blocking you. If it is invalid and there is no brief to fall back on, you get the validation errors and nothing is created.

> [!NOTE]
> You do not have to upload a file. You can paste a brief or a spec into the project-creation flow, and pasted content takes precedence over whatever was in the folder.

## Key information for spec derivation

Whichever path you take, the spec has to settle the same decisions. Knowing them in advance makes every path faster: the agent will raise them as questions, your `poq.md` should describe them in plain language, and your `poq.toml` must declare them as parameters.

| Decision | What it settles | Required? |
| --- | --- | --- |
| **Validation goal** | What a validator decides about one item | Yes |
| **Ingestion** | Which files become datapoints, how they join, which fields are canonical | Yes |
| **Evidence** | What each validator sees, such as markdown, code, images, source links, and facts | Yes |
| **Rubric** | The dimensions scored, each with a scale and answer labels | Yes |
| **Validator count** | How many validators review each datapoint by default | Yes |
| **Reward** | What each validation slot pays, in USD | Yes |
| **Validator classes** | Named tiers or specialists, human or AI, with their own rewards and rules | Only when the work needs them |
| **Qualifications** | Credentials a validator must hold to claim an item | Only when the work needs them |
| **Routing** | Which items go to which class of validator | Only when the work needs them |
| **Escalation** | What happens when a panel disagrees, such as adding reviewers or a senior tier | Only when the work needs them |
| **Consensus weighting** | Whether some validators' answers count for more than others | Defaults to equal weight |
| **Attestation** | How verdicts are sealed and reported | On by default |

Without the required rows, a spec will not validate. The optional rows are where most projects get complicated, so add them only when the work calls for it.

> [!TIP]
> Validator classes and consensus weighting are separate decisions. Adding a senior class puts senior reviewers on the panel. Consensus weighting makes their answers outweigh everyone else's. Most projects want the first without the second.

## Conversational agent

The conversational agent defines a spec with you inside the app, without you writing either artifact by hand. It is the path when you have data and intent but nothing written down.

It works in two stages:

| Stage | What the agent does |
| --- | --- |
| Decide | Reads your uploaded data, proposes the validation goal, evidence, and rubric in plain language, and asks you to confirm what it cannot infer |
| Author | Turns the confirmed decisions into `poq.toml`, then keeps editing that spec as you ask for changes |

Once the agent has produced a spec, you can ask for changes in plain language, such as "make the rubric stricter" or "add a second reviewer". It revises the existing TOML rather than starting over. That same conversational editing is available after the spec is generated, whichever path produced it.

The agent does not ask about everything. Ingestion is settled earlier, when you finalize your data. Validator qualifications and compensation are configured in a later project-creation step rather than in the conversation.

## poq.md

`poq.md` is where the work is described in plain language. Use it when you are bringing a brief you already wrote, or when you would rather author intent outside the app and version it alongside your data.

A brief should answer the [key information](#key-information-for-spec-derivation) above, as far as your project needs it: every required decision, plus the optional ones the work calls for.

Compiling a brief into a spec is a single model pass, so read the draft it produces before you run the project. See example `poq.md` documents [here](/examples/poq-md).

## poq.toml

`poq.toml` is the structured, specification-compliant version of the project intent. Hand-author it when you want precision, repeatability, or a spec under version control.

The app drives the project based on this document. Features defined in the document include ingestion, validation, consensus, routing, and report generation. The [`poq.toml` specification](/reference/poq-toml) is the source of truth for every section, field, type, and default.

Raw rubric answers are ultimately represented as a `0-100` value, even when validators are presented with labels or choices.

See example documents [here](/examples/poq-toml).

## See also

| Page | Description |
| --- | --- |
| [`poq.toml` Specification](/reference/poq-toml) | The source of truth for every field, type, and default |
| [`poq.md` examples](/examples/poq-md) | Worked briefs for common project shapes |
| [`poq.toml` examples](/examples/poq-toml) | Worked specs for common project shapes |
| [Supported data types](/build/supported-data-types) | What files ingestion understands and how datapoints are formed |

---

# Supported data types

URL: https://sapien-ai-docs-production.up.railway.app/build/supported-data-types
Source: docs/current/build/supported-data-types.md

# Supported data types

Ingestion is the single pipeline that turns your uploaded files into datapoints. Whatever you bring (a CSV of labels, a folder of JSON, a bundle of PDFs, or a long Markdown report), ingestion reads it, **normalizes it into one canonical format**, and produces datapoints from that canonical form. Every supported file type is just a different on-ramp to the same pipeline, not a separate, one-off behavior.

## The canonical format

Under the hood, ingestion treats every source as a temporary table:

- **Rows**: one row per item in the source (a CSV line, a JSON file, a Markdown section).
- **Columns**: the fields inside each item (a CSV header, a JSON key, a captured value).

When a project uses more than one source, [joins](/reference/poq-toml/ingestion) line those tables up into one wider canonical table before anything else happens, and `[ingestion.fields]` selects which columns are kept and what they are named. Once ingestion finishes, the temporary tables are discarded and only the canonical rows remain.

The rule that ties everything together:

> **One canonical row = one datapoint.**

Validators never see your original files. They see the datapoints produced from the canonical table, and every downstream section of the spec (evidence, rubric, consensus) operates on that shape alone. This is why the choice of file type matters less than it first appears: CSV, JSON, `file_collection`, and Markdown all resolve to the same rows-and-columns form, and the sections below just describe how each one gets there.

## Supported file types

Ingestion recognizes four source types. Each is a different way of feeding the canonical format, and each has its own rule for how a source becomes canonical rows:

| Source type | File types | How it becomes canonical rows |
| --- | --- | --- |
| [CSV](#csv-csv) | `.csv` | Each row is already a canonical row |
| [JSON](#json-json) | `.json` | Each file, or each element of an unnested array, becomes a row |
| [File collection](#binary-or-opaque-files-jpg-pdf-) | `.jpg`, `.pdf`, and other binary types | Each file becomes a row, unparsed |
| [Markdown](#markdown-files-md) | `.md` | Each section, split on a heading, becomes a row |

### CSV (`.csv`)

The most direct path into the canonical format: a CSV is already rows and columns. One row per item, with an optional header row supplying the column names. Columns and formatting are detected automatically, and each row becomes one canonical row unchanged.

| Case | Supported? |
| --- | --- |
| A single file such as `labels.csv`, with a header row and one row per item | Yes |
| Combining multiple `.csv` files into one source | No: each CSV source reads exactly one file |
| Unusual or inconsistent formatting, such as odd quoting or mixed encodings | No: rows must parse deterministically |

### JSON (`.json`)

Each JSON file becomes one canonical row, with columns read from its object keys, including nested keys, addressed with a dotted path such as `rubric.severity`. If a file instead wraps an array of items under a key, ingestion unnests that array into one row per element, so a single file can expand into many canonical rows.

| Case | Supported? |
| --- | --- |
| One datapoint per file, for example `doc-summary-001.json`, `doc-summary-002.json` | Yes |
| A file holding an array of items, unnested into one datapoint per element | Yes |
| An array nested under a key, such as `report.doc-summaries` | Yes |
| Fields read from nested objects, such as `rubric.severity` | Yes |
| More than one array unnested per source, or an array nested inside another array | No: flatten the inner array (for example, to a CSV) first |

### Binary or opaque files (`.jpg`, `.pdf`, ...)

The `file_collection` upload type ingests a folder of binary files, such as images and PDFs. File contents are not parsed, so each file becomes exactly one canonical row whose column is a reference to the file itself. It is the same rows-and-columns shape as any other source, just with the file as the value rather than parsed text.

| Case | Supported? |
| --- | --- |
| A folder of images, PDFs, videos, or other binary types, matched by a pattern such as `images/*.jpg` | Yes |
| Mixed file types in one pattern, such as `images/*.{jpg,png}` | Yes |
| Pointing at a single file | No: this source type is for folders containing many files |

### Markdown files (`.md`)

Markdown reaches the canonical format the same way every other source does. It is simply split first. A long document is divided into separate sections wherever a chosen heading appears, and **each section becomes one canonical row**, exactly like a CSV line or a JSON file. The section splits are not a special case: they are just the step that turns one `.md` document into the rows the pipeline expects. The section's text lands in a `body` column, and a regex pulls any repeated header parts (an id, a title) into their own columns. See [`markdown_split`](/reference/poq-toml/ingestion#the-markdown_split-input-type) for the full field reference.

| Case | Supported? |
| --- | --- |
| One or more text or report files, divided wherever a heading such as `## Doc-summary 1` appears | Yes |
| Pulling a few extra labeled facts out of each section into their own columns | Yes |
| Dividing sections into smaller pieces again | No: only one level of division is supported |

## Limits

| Limit | Value |
| --- | --- |
| Maximum file size | 5 GB |
| Datapoints per project | 1,000,000 |
| File paths | Must stay inside the project's own folder |

If your data comes in a format ingestion does not yet understand, contact the Sapien team.

## See also

| Page | Description |
| --- | --- |
| [Define your spec](/build/defining-your-spec) | How `poq.md` and `poq.toml` work together |
| [`poq.toml` Specification](/reference/poq-toml) | Field-level reference for `[[ingestion.sources]]`, `[[ingestion.joins]]`, and `[ingestion.fields]` |
| [Quickstart](/start/quickstart#create-a-project-in-the-ui) | Creating a first project |

---

# How consensus works

URL: https://sapien-ai-docs-production.up.railway.app/validate/consensus-explained
Source: docs/current/validate/consensus-explained.md

# How consensus works

Imagine a cooking contest. One dish comes out of the kitchen, and a **panel of judges**
tastes it. Each judge fills out the same scorecard, and somehow all those opinions have to
become **one verdict**: does this dish move on, or not?

That is exactly the problem Sapien solves for every piece of work — one answer, one audit
finding, one labeled image. A group of independent reviewers each give their scores, and the
system turns those scores into a single, trustworthy result. The reviewers are called
**validators**, and the process of turning their scores into one result is **consensus**.

Crucially, a judge on the panel can be a **human expert _or_ an AI agent**. Both are first-class
validators — they see the same rubric and evidence, score the same way, and are routed identically.
A panel can be all human, all AI, or a mix, which is what lets a project move fast on volume and
still bring expert judgment where it matters.

Here is the whole idea in five plain steps.

## 1. The scorecard has several questions

A good judge doesn't just say "I liked it." The scorecard asks separate questions:
*Is it cooked properly? How is the seasoning? How is the presentation?*

In Sapien, each of these questions is a **dimension**. A security review, for example, scores
a finding on its **severity**, its **exploitability**, and how **certain** the validator is.
Each dimension is judged on its own, then combined at the end.

> [!NOTE]
> Some questions matter more than others. The project creator gives each dimension a
> **weight** — "seasoning counts double, presentation is just a tie-breaker." A dimension with
> zero weight is **advisory**: collected for the record, but it never changes the verdict.

## 2. Each judge scores, and we measure how much they agree

Once the judges submit their scores for a dimension, we don't just average them blindly. We
look at how **tightly they cluster**. If everyone scored the seasoning 4 out of 5, that's
strong agreement. If scores are all over the place, that's weak agreement.

That clustering is the **strength** of the consensus — a number from 0% (total disagreement)
to 100% (perfect agreement). Each **dimension** has its own bar the strength has to clear, the
**required strength** (70% is a sensible default) — a tricky question can demand tighter agreement
than an easy one.

> [!TIP]
> Sapien uses the **median** rather than a plain average to find the agreed score. One judge
> who scored a 1 out of pure spite can drag an *average* down to a wrong middle value — but it
> can't move the **median**. The crowd's real opinion wins.

## 3. If the judges are split, call in the experts

What if the panel can't agree? You don't just flip a coin. You bring in **senior judges** —
fewer of them, but more experienced — and their opinion counts for more.

In Sapien this is **escalation**, and it works as a **ladder**. Validators come in **classes**
(think *junior* and *senior*, or *AI panel* and *human expert*) — each class is human or AI, with
its own vote weight. Round one is the base panel. If the item isn't **verified** — any required
question still below its bar — the next rung of the ladder is added: more judges (often a more
senior class) who re-score the whole item. Each step that runs adds its own described mix of
classes, and because senior classes carry a higher **vote weight**, they can tip a split decision.
The ladder keeps climbing until the item is verified or the steps run out.

> [!TIP]
> A common setup is an **AI ensemble** on the base round for speed and coverage, escalating to
> **human experts** only when the agents can't agree — so people spend their time on the genuinely
> contested items.

:::figure{src="consensus-escalation" title="The escalation ladder: a base panel scores every dimension, and while any required dimension is below its bar the next class of judges is added to re-score, until the item is verified or the ladder runs out." caption="The base panel scores every dimension. While any required dimension is below its bar, the next rung adds more judges to re-score — until the item is verified or the ladder runs out."}
:::

## 4. Combine the questions into one verdict

Each dimension now has its own resolved score. The system rolls them up into a single
**aggregate** — a weighted blend, where the heavier questions pull harder. If every
**required** dimension cleared its bar, the item is **accepted**. If even one required
dimension is still unresolved, it isn't.

> [!NOTE]
> Some questions don't always apply. "How good is the fix?" is meaningless if the panel
> already decided the bug isn't real. A dimension can be set to **skip** when another
> dimension resolves a certain way, so it never drags the verdict around.

## 5. What keeps the scores honest

The scores hold up because of *who* judges and *how they're held accountable*, not because of any
single penalty. Three things do the work:

- **A vetted workforce.** Validators are named and qualified before they review — added to a
  project's roster, then granted classes (like "senior reviewer" or "dermatologist") with a
  written justification, so vetting is auditable. Many also **calibrate** against expert reference
  items before live work. Accountability rests on who they are and their workforce contract — no
  funds at risk, no smart contracts.
- **Rewards for good work.** Validators are paid for each completed verification, which aligns the
  incentive with careful, honest scoring.
- **Reputation that follows them.** Each validator's record is tracked across verifications;
  landing near the resolved consensus over time builds standing and access to higher-value work.

> [!TIP]
> The lesson for a validator is simple: **score what you actually believe.** Your reputation is
> built on agreeing with the resolved consensus across many reviews — guessing the crowd or
> rubber-stamping is what erodes the standing that earns you the best work.

## Put it together

| Contest term | Sapien term | What it controls |
| --- | --- | --- |
| Scorecard question | **Dimension** | One thing validators rate |
| How much a question counts | **Weight** | Influence on the final verdict |
| How tightly judges agree | **Strength** | Whether a dimension resolves |
| The bar to pass | **Required strength** | How much agreement is "enough" |
| Calling in expert judges | **Escalation / classes** | What happens when judges split |
| A judge's track record | **Reputation** | Standing and access over time |

That's the entire mechanism: **score each question → measure agreement → escalate if split →
combine by weight → and lean on a vetted, well-rewarded, reputation-driven panel to keep it honest.**

Want to feel how the knobs interact? **[Open the consensus simulator →](/examples/consensus-simulator)**
and configure your own dimensions, validator classes, and thresholds, then watch the verdict
change as you move the votes.

## See also

| Page | Description |
| --- | --- |
| [How quality is measured](/validate/quality-rating) | The other rating: what the Quality Rating means and how to act on it |
| [Validation interface](/validate/validation-ui) | How validators score each dimension on the 0-100 scale |
| [PoQ Report](/validate/poq-report) | The record of the Quality Rating and Consensus Strength this produces |
| [Consensus simulator](/examples/consensus-simulator) | Interactive playground for dimensions, classes, and thresholds |
| [The PoQ workflow](/build/poq-workflow) | Where validation fits: Define → Validate → Attest |

---

# How quality is measured

URL: https://sapien-ai-docs-production.up.railway.app/validate/quality-rating
Source: docs/current/validate/quality-rating.md

# How quality is measured

Validation returns **two independent numbers** for every datapoint, both `0-100`:

- **Quality Rating** — *what* the panel concluded. Directional: `100` is high quality, `0` is low.
- **Consensus Strength** — *how tightly* they agreed, regardless of the answer. See
  [How consensus works](/validate/consensus-explained).

This page is about the Quality Rating, and how to read it against Consensus Strength.

## What the Quality Rating measures

To judge quality, a panel of validators rates each item on a scale — say an AI's report graded from
*Poor* to *Excellent*. The Quality Rating is **where the panel landed**, mapped onto `0-100`. The
rubric decides which end is "good", so the rating carries the *direction* of the answer, not just how
far apart the votes were.

:::figure{src="likert-to-quality" title="A five-point Likert scale from Poor to Excellent laid along a 0-100 track. Five validators pick labels — one at Good, two at Very good, two at Excellent — and the panel's Quality Rating lands at 80, just past Very good." caption="Labels map evenly onto 0–100 (Poor = 0 … Excellent = 100); the Quality Rating is where the panel's picks land — here 80."}
:::

> [!NOTE]
> A high Quality Rating means high on *your* rubric — not that the output is objectively correct or safe.

## How it rolls up

One report is usually scored on several dimensions (accurate, complete, clear), and the rating rolls
up from there:

:::figure{src="quality-rollup" title="A left-to-right rollup: one report's three dimension ratings — Accurate 90, Complete 70, Clear 85 — blend into a single datapoint Quality Rating of 82, and many such reports average into the dataset headline of 78." caption="Per-dimension ratings blend into one score per report (the datapoint); every report's score then averages into the dataset headline."}
:::

## Quality is not the same as agreement

The two numbers are independent. A panel can **agree completely that a report is poor** — high
Consensus Strength, low Quality Rating.

:::figure{src="quality-vs-consensus" title="Two validator panels answering on the same Poor-to-Excellent scale. Both cluster tightly — one entirely on Poor, one entirely on Excellent — so Consensus Strength is high (96) in each, but the Quality Rating is low (6) for the Poor panel and high (96) for the Excellent panel." caption="Both panels agree just as strongly, so Consensus Strength matches — but they agree on opposite things, so the Quality Rating is far apart."}
:::

That independence is why you keep both numbers: Consensus Strength is the one that *acts* — it
decides whether a datapoint is settled or needs another look — while the Quality Rating is only ever
reported.

## Reading the two together: the outcome quadrant

Split each axis at **70** and every datapoint lands in one of four cells:

:::figure{src="outcome-quadrant" title="The outcome quadrant: Quality Rating on the vertical axis and Consensus Strength on the horizontal axis, each split at a display threshold of 70. High consensus (right column) is reliable — Trustworthy when quality is high, Confidently rejected when quality is low. Low consensus (left column) is unsettled and should escalate — Contested when quality is high, Unclear when quality is low." caption="Consensus Strength (horizontal) says whether the result is reliable; Quality Rating (vertical) says the verdict. Right column = settled; left column = escalate."}
:::

**Consensus Strength decides whether to trust the result** (right column = settled); **Quality
Rating decides the verdict** (top row = passed). *Confidently rejected* is a healthy outcome — the
panel reliably caught weak work.

Pushing a datapoint rightward means
[escalating](/validate/consensus-explained#3-if-the-judges-are-split-call-in-the-experts) to more
validators: it raises consensus, but each added reviewer is paid — so escalate only where more
confidence would change a decision.

> [!NOTE]
> The quadrant is just a quick read of the two numbers: the `0-100` Quality Rating and Consensus
> Strength are the real outputs, and `70` is simply where the picture splits "high" from "low".

## Where you'll see it

- **In the app** — on each datapoint, beside its Consensus Strength and outcome.
- **In the CSV export** — a column you can sort and threshold.
- **In the [PoQ Report](/validate/poq-report)** — the signed record, per datapoint and per dataset.

## See also

| Page | Description |
| --- | --- |
| [How consensus works](/validate/consensus-explained) | The other rating: how agreement is measured and how split panels escalate |
| [Validation interface](/validate/validation-ui) | How validators score each dimension on the `0-100` scale |
| [PoQ Report](/validate/poq-report) | The record where the Quality Rating and Consensus Strength are certified |
| [The PoQ workflow](/build/poq-workflow) | Where validation fits: Define → Validate → Attest |

---

# Validation interface

URL: https://sapien-ai-docs-production.up.railway.app/validate/validation-ui
Source: docs/current/validate/validation-ui.md

# Validation interface

The interface shown to a validator renders data from the ingestion phase according to transformations in the project's specification (`poq.toml`). This page describes the evidence and evaluation elements that can be defined via the project's specification.

:::figure{src="validation-interface" title="The validator interface: an evidence panel stacking several artifacts (an image, formatted text, a code snippet, and a source link) rendered from the project's spec, a rubric question answered with a single row of buttons, and a submit control; the chosen button maps to a value on a 0-100 scale." caption="What a validator sees: evidence rendered from the spec — one or more artifacts such as images, text, code, and source links — then one rubric question with a single row of answer buttons and a submit control. Every answer maps to a position on a 0-100 scale."}
:::

## What Sapien can show a validator

Each `[[validation.evidence]]` block in the spec names a renderer `type` and wires it to ingested data. These are the supported renderers — pick one to see what it looks like:

| `type` | Renderer | What the validator sees |
| --- | --- | --- |
| `markdown` | Formatted text | A page of formatted writing (Markdown) — typically the finding narrative. |
| `image` | Image | One image, loaded from a URL field. |
| `json_finding` | Fact sheet | A formatted JSON object, provided inline or by URL. |
| `datapoint_facts` | Fact table | A list of labeled facts, side by side. |
| `source_excerpt` | Source excerpt | A quoted excerpt — text pinned from an exact file in a repository. |
| `source_link` | Source link | A pointer to an exact file, commit, and line. |
| `image_comparison` | Image comparison | Two images at once — side by side, as a slider, or as a difference view. |
| `bounding_box_overlay` | Annotated image | An image with labeled boxes drawn over the important parts. |
| `video_clip` | Video | An inline video player, with an optional cover image and captions. |
| `prompt_response_pair` | Prompt & response | A prompt and the answer it received, sometimes with a second answer for comparison. |

For most of these, the project specification names exactly what to show, for example a particular picture or a particular fact, and Sapien checks that everything required has been provided before anyone sees it. If something is missing when it is time to show a validator, they see a message stating the evidence is missing, instead of a blank space.

## How a validator answers

There is one kind of answer control: a row of buttons, and the validator picks one. There is no typing, no sliders, and no selecting multiple options. Only one button can be picked.

What changes is how many buttons there are and what they say.

- A 5-point or 7-point scale, for example from "Strongly Disagree" to "Strongly Agree", evenly spaced.
- A list of custom labels, for example "Low", "Medium", "High", also spaced evenly.
- A list of custom numbers, as long as it includes the lowest and highest possible score.

Regardless of which type is used, every answer is converted to a number from 0 to 100. The buttons are a way of picking a position on that scale. This is what allows Sapien to compare answers to each other and determine whether reviewers agree, even when the questions look different on screen.

A few rules keep this working.

- Every question needs at least two buttons to choose from.
- If an AI is one of the reviewers, the numbers behind the buttons must be whole numbers, with no fractions. A 5-choice scale always satisfies this, but some other combinations do not.

A question can also be skipped automatically. If the answer is already determined by the data, Sapien fills it in and marks it as skipped instead of asking the validator.

## See also

| Page | Description |
| --- | --- |
| [Define your spec](/build/defining-your-spec) | How `poq.md` and `poq.toml` work together |
| [How consensus works](/validate/consensus-explained) | How 0-100 rubric answers become a verdict |
| [`poq.toml` Specification](/reference/poq-toml) | Field-level reference for `[[validation.evidence]]` and `[[validation.rubric]]` |

---

# PoQ Report

URL: https://sapien-ai-docs-production.up.railway.app/validate/poq-report
Source: docs/current/validate/poq-report.md

# PoQ Report

A PoQ Report is the durable artifact a PoQ project produces once all required validator inputs are in. It answers: **what was reviewed, by whom, against what criteria, and with what level of agreement.**

It is produced in the [Attest](/build/poq-workflow) step as one canonical JSON payload (schema `poq.attestation/v1`), signed once with Ed25519 and delivered as text, HTML, JSON, or JWS — see [Formats](#formats). Text and HTML are **self-verifying**: content and signature travel in one file, verifiable offline.

:::figure{src="poq-report-anatomy" title="Anatomy of a Proof Report: a miniature of the report document. The Result section leads with the two headline numbers — Quality Rating 78 / 100 and Consensus Strength 90% — above a customer-band label. Below them, each remaining section answers one question, and the fenced Ed25519-signed payload travels at the bottom of the same file." caption="One document, front-loaded with the two headline numbers; every section below answers one question, and the signature travels in the same file."}
:::

> [!TIP]
> Hold a real one while you read: the example report as [HTML](/example-report/poq-report.html) or [text](/example-report/poq-report.txt), plus its [signing key (JWKS)](/example-report/poq-report.jwks.json). The project is fictional, but the signature is real — Sapien's production key, the same one published at the `.well-known` endpoint below — and every command on this page works against these files.

## The two headline numbers

**Quality Rating** is how good the quorums judged the work, on your rubric. **Consensus Strength** is how strongly they agreed — confidence in the rating, not a property of the work. Any label (for example `Verified · customer band`) is a customer-defined band over these two numbers; PoQ produces no verdict of its own.

How they're computed: [How quality is measured](/validate/quality-rating) · [How consensus works](/validate/consensus-explained).

## What it asserts — and what it does not

The report certifies that a Validator Pool was assembled per your Assurance Policy, and that a quorum drawn from it — one per datapoint — evaluated the enumerated datapoints against your rubric, reaching the numbers shown. Datapoints added after issuance are not covered.

It is **a record of how the work was validated** — not an assertion that the work is objectively correct, safe, complete, or fit for any purpose. The Quality Rating is relative to the rubric you authored.

## Getting a report

In the Portal, open the project's report page and pick a format from the download menu (originators and admins, once the project has finalized data).

## Verifying a report

Anyone can verify a report **offline**, without trusting Sapien at read time:

:::figure{src="poq-verify-checks" title="How poq verify proves a report offline: the artefact is one file with two layers — the readable render a human reads, and the fenced Ed25519-signed payload — joined by a parse/render round trip. The file flows through five checks: envelope, authenticity (against the publisher's key, fetched independently over TLS from the well-known JWKS endpoint), information, presentation, and fidelity — ending in a VERIFIED verdict, with no call to Sapien at read time." caption="One file, two layers, five checks. The parse/render round trip proves the readable layer carries exactly the signed information; the key arrives on an independent channel, so the whole loop closes offline."}
:::

### Verify one right now

The panel below runs the real verifier — the same Go code as `poq verify`, compiled to WebAssembly — entirely in your browser. Verify the example report, then tamper with its Quality Rating and watch the `fidelity` check fail and name the field. You can also drop in any PoQ report of your own — it checks against Sapien's live published keys automatically.

:::verify-panel
:::

### The signed payload

The payload is canonicalized with JCS (RFC 8785) before signing, so the bytes are deterministic; all numerics are carried as strings; the JWS header's `kid` names the signing key. Any change to a covered value — in the JSON, the text, or the HTML — breaks verification.

The readable formats declare an **envelope** version so a verifier knows the carrier before applying any grammar:

| Envelope | Carrier |
| --- | --- |
| `poq.envelope/text-v1` | Readable render followed by the fenced JWS |
| `poq.envelope/html-v1` | The designed HTML page, embedding the text artefact and tagging visible values |

In text, the compact JWS sits between `-----BEGIN POQ SIGNED PAYLOAD-----` / `-----END POQ SIGNED PAYLOAD-----` fences after the render. In HTML, the same text artefact is embedded in a `<script type="application/poq-attestation+text">` element, and every payload-derived value shown on the page is tagged `data-poq`.

### poq verify

The `poq` binary (`make poq` in the server) verifies any text or HTML artefact:

```bash
poq verify report.html
poq verify report.txt --jwks report.jwks.json
```

It exits `0` verified / `1` failed / `2` usage — so it drops straight into CI. The checks:

| Check | What it confirms |
| --- | --- |
| `envelope` | The artefact declares an envelope version this build supports |
| `authenticity` | The fenced JWS verifies against a trusted published key |
| `information` | Parsing the readable render reproduces **exactly** the signed payload bytes |
| `presentation` | Re-rendering the parsed payload reproduces the readable body byte-for-byte |
| `fidelity` (HTML only) | Every `data-poq`-tagged value shown in the page matches the signed payload |

`information` and `presentation` together prove the readable layer carries exactly the signed information; `fidelity` binds every displayed value to the signature — tamper with a visible number and verification fails, naming the field.

Try it on the [example report](/example-report/poq-report.html):

```bash
curl -sO https://docs.sapien.io/example-report/poq-report.html
curl -s https://poq.sapien.io/.well-known/poq-attestation-keys.json -o sapien-keys.json
poq verify --jwks sapien-keys.json poq-report.html
```

```
  envelope       OK     poq.envelope/html-v1 carrying poq.envelope/text-v1
  authenticity   OK     Ed25519 by poq-signing-key over canonical bytes sha256:243a4aaa…
  information    OK     parse(text) == signed payload — byte-identical
  presentation   OK     render(parse(text)) == text
  fidelity       OK     28 displayed values match the signed payload
VERIFIED — the readable text provably carries exactly the signed information
```

### Trusted keys

A key is trusted if it is pinned into the `poq` binary or supplied at verify time (`--jwks`, or `--key` for a single public key). Each deployment publishes its attestation key as a JWKS over TLS at `/.well-known/poq-attestation-keys.json`, keyed by `kid`. The official Sapien keys:

```
https://poq.sapien.io/.well-known/poq-attestation-keys.json
```

Fetching the key independently of the report closes the loop — after that, verification is entirely offline.

### Rendering a sample

`poq render` produces a report artefact locally in any format, with a sidecar JWKS so you can verify it immediately:

```bash
poq render                        # sample report → report.html (+ report.html.jwks.json)
poq render --format txt            # the text artefact instead
poq render --out - --format json   # canonical payload to stdout

poq verify report.html --jwks report.html.jwks.json
```

`--format` accepts `html | txt | jws | json`; `--out` accepts a file path or `-` for stdout. The [example report](/example-report/poq-report.html) on this page is exactly this output.

## Reference

### Formats

One signed payload, delivered in whichever format you ask for:

| Format | What you get | Best for |
| --- | --- | --- |
| Text (`.txt`) | A readable render followed by the signature, in one file | The default — human-readable and verifiable |
| HTML | The designed, styled report page | Sharing and reading in a browser or PDF |
| JSON | The canonical signed payload | Machine consumption |
| JWS | The bare compact signature | Verifying against your own copy of the payload |

### Report sections

For the exhaustive field-by-field payload, see the [PoQ Report spec](/reference/poq-report).

| Section | Answers |
| --- | --- |
| Scope | Which project and how many datapoints were covered, and how much of the manifest was reviewed |
| Result | The headline Quality Rating, Consensus Strength, and label |
| Scorecard | Per-dimension quality and consensus, with rubric weights |
| Distribution | How quality and consensus scores were spread across datapoints |
| PoQ Record excerpt | A signed, per-datapoint slice showing one datapoint's full quorum and dimension detail |
| Quorum routing | The validator rounds — who reviewed, escalation targets, and where each datapoint settled |
| Configuration | The rubric and the locked `poq.toml` config hash |
| Pool & provenance | Validator pool size and churn, participation, credentials, and independence |
| Chain of custody | Timestamps for config lock, manifest confirmation, the review window, and issuance |
| Attestation | The attestation root, signing key, registry, and how to verify |
| What this asserts | The plain-language statement of what the report certifies |

The **Validator Pool** size counts every validator class eligible for the work, including agentic (AI) validators; **Participation** is reported separately as human and agentic counts.

## See also

| Page | Description |
| --- | --- |
| [PoQ Report spec](/reference/poq-report) | The field-by-field reference for the report payload |
| [PoQ workflow](/build/poq-workflow) | Where the report fits: Define → Validate → Attest |
| [How quality is measured](/validate/quality-rating) | What the Quality Rating means and how to read it with Consensus Strength |
| [How consensus works](/validate/consensus-explained) | How agreement is measured and how split panels escalate |
| [Developer API - REST](/integrate/developer-api) | The public HTTP integration surface |

---

# API keys

URL: https://sapien-ai-docs-production.up.railway.app/integrate/api-keys
Source: docs/current/integrate/api-keys.md

# API keys

Developer API keys enable programmatic access to PoQ by both developers and agents. Use them to access the [PoQ Developer API](/integrate/developer-api) (`/developer/v1`) and the [PoQ MCP server](/integrate/mcp) (`/mcp`).

## What they are

An API key is a bearer token with the prefix `poq_live_`. Each key is scoped to one user within one organization.

| Property    | Detail                                                                               |
| ----------- | ------------------------------------------------------------------------------------ |
| Format      | `poq_live_` + 32 base62 characters (41 chars total)                                  |
| Header      | `Authorization: Bearer poq_live_...`                                                 |
| Scope       | One organization; projects and uploads created via the key live in that organization |
| Attribution | Work performed with a key is attributed to the user who created it                   |

## Creating a key

Open [API Keys](https://poq.sapien.io/originator/api-keys) in the app. Give the key a name and copy the token when it appears. **It is shown exactly once** and cannot be retrieved later. You can still list keys and see their prefix (`poq_live_…`).

Only originators of your **current organization** can create keys. If you are not an originator, ask an organization admin to grant access or create a key for you.

## Managing keys

From the same [API Keys page](https://poq.sapien.io/originator/api-keys) you can list active keys and revoke them by setting an expiry. Revoked keys stop working immediately.

## Authorization layers

A valid key is necessary but not always sufficient.

| Endpoint class               | Requirements                                                                            |
| ---------------------------- | --------------------------------------------------------------------------------------- |
| `GET /developer/v1/ping`     | Valid, non-expired key                                                                  |
| Project and upload endpoints | Valid key **and** the key's creator must still be an originator within the organization |

The originator check runs on every project request, not just at creation time. If the creator loses originator status within the organization, existing keys remain valid for `ping` but can no longer create or manage projects.

## Security

| Practice                                 | Why                                                      |
| ---------------------------------------- | -------------------------------------------------------- |
| Store keys in a secrets manager          | Plaintext keys grant full organization-scoped API access |
| Never commit keys to source control      | Treat a leaked key like a leaked password                |
| Set an expiry for short-lived automation | Limits blast radius if a key is exposed                  |
| Rotate after compromise                  | Revoke the old key in the app, create a new one          |

Presigned upload URLs returned by upload-session endpoints are a separate credential. PUT requests to those URLs need no `Authorization` header.

## What keys are not

| Not this              | This instead                                                           |
| --------------------- | ---------------------------------------------------------------------- |
| Web app access token  | Wallet login at [`https://poq.sapien.io`](https://poq.sapien.io), for browser use               |
| Presigned upload URL  | Returned per file during upload; scoped to one object and time-limited |
| Validator credentials | Validators use wallet auth on app routes, not API keys                 |

## See also

| Page | Description |
| --- | --- |
| [Quickstart](/start/quickstart#create-a-project-via-the-api) | End-to-end workflow with a new key |
| [Developer API](/integrate/developer-api) | HTTP endpoint reference |
| [MCP server](/integrate/mcp) | Tool-based access with the same keys |

---

# Developer API

URL: https://sapien-ai-docs-production.up.railway.app/integrate/developer-api
Source: docs/current/integrate/developer-api.md

# Developer API

The Developer API is the HTTP integration surface for managing PoQ projects and data.

The API is served at [`https://poq.sapien.io/developer/v1`](https://poq.sapien.io/developer/v1).

The [OpenAPI spec](https://poq.sapien.io/developer/v1/openapi.json) is machine-readable, and [interactive docs](https://poq.sapien.io/developer/v1/docs) are available in the browser.

## Authentication

Every request needs a bearer token in the `Authorization` request header:

```
Authorization: Bearer poq_live_...
```

See [API keys](/integrate/api-keys) for key creation and lifecycle. Project and upload endpoints also require the key creator to be an org originator.

## Conventions

**JSON bodies and responses** use camelCase keys.

**Errors** follow a Stripe-shaped envelope:

```json
{
  "error": {
    "type": "invalid_request_error",
    "code": "project_not_found",
    "param": "projectId",
    "message": "Project not found."
  },
  "requestId": "0192f0a0-..."
}
```

`requestId` appears in error bodies and always in the `X-Request-Id` response header. Include it when reporting failures.

**Pagination** on `GET /projects` is keyset-based. Responses include `{ data, hasMore, nextCursor }`. Pass `nextCursor` as `?cursor=` on the next request. `limit` is 1–100 (default 20).

## Endpoints

### System

| Method | Path    | Summary                                              |
| ------ | ------- | ---------------------------------------------------- |
| `GET`  | `/ping` | Verify the key is valid. Returns `{ "pong": true }`. |

### Projects

| Method | Path                    | Summary                                                                                                         |
| ------ | ----------------------- | --------------------------------------------------------------------------------------------------------------- |
| `POST` | `/projects`             | Create an active project from a JSON `spec` and `name`. Returns `projectId`, `specId`, `specVersion`, `status`. |
| `GET`  | `/projects`             | List projects in the key's org, newest first. Query: `limit`, `cursor`.                                         |
| `GET`  | `/projects/{projectId}` | Fetch one project.                                                                                              |
| `PUT`  | `/projects/{projectId}` | Update `name`, `description`, and/or `paused`. At least one field required. Spec and status are immutable.      |

The `spec` object on create is a JSON mirror of the [poq.toml spec](/reference/poq-toml). The project goes live immediately. The spec is compiled and locked at creation.

### Validators

Manage the org's validator roster and per-project assignments. All endpoints are scoped to the API key's organization.

| Method | Path                                 | Summary                                                                                                                                                                                                             |
| ------ | ------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GET`  | `/validators`                        | List the org validator roster, newest first. Query: `limit`, `cursor`. With `?email=` (case-insensitive) it looks up a single validator instead — the response returns `validator` (and null `data`) rather than a page. |
| `GET`  | `/validators/{validatorId}/projects` | List a validator's non-revoked project assignments in the org.                                                                                                                                                     |
| `POST` | `/validators/{validatorId}/assign`   | Assign a validator to one or more `projects`, each optionally granting a `classId` (+ `evidenceNote`). Set `autoEnroll` to promote an existing org member to validator first. Returns `{ assigned, alreadyAssigned, classesGranted }`. |
| `POST` | `/validators/{validatorId}/unassign` | Revoke a validator from `projectIds`. Idempotent. Returns `{ unassigned }`.                                                                                                                                        |

A validator resource carries `userId`, `subject`, `email`, `evmAddress`, `isOriginator`, `isValidator`, and `joinedAt`. Looking up an email that matches no user — or a user outside your organization — returns `validator_not_found`; the API never reveals whether a user exists elsewhere.

### Validator classes

Classes are validator tiers declared in a project's spec (`[validators].classes`). Grant a class to record that a validator qualifies for a tier.

| Method   | Path                                                             | Summary                                                                                                                                                     |
| -------- | --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GET`    | `/projects/{projectId}/classes`                                 | List the classes declared in the project's spec (`id`, `label`, `type`, `rewardCents`, `stakeWei`, `consensusWeight`, `model`). Empty when the spec has no `[validators]` block. |
| `POST`   | `/projects/{projectId}/validators/{validatorId}/classes`        | Grant a class. Body: `classId`, `evidenceNote` (both required). Exclusive — any other active class the validator holds on the project is revoked first.      |
| `DELETE` | `/projects/{projectId}/validators/{validatorId}/classes/{classId}` | Revoke a class grant. Idempotent.                                                                                                                          |

### Invites

Invite people to a project by email. An invite is fulfilled when the invitee logs in.

| Method   | Path                                        | Summary                                                                                                                                                              |
| -------- | ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `POST`   | `/projects/{projectId}/invites`             | Invite an `email`. Optional `classId` (+ `evidenceNote`) pre-stages a class grant; `sendEmail` (default `true`) controls the invite email. Idempotent per (email, project). |
| `GET`    | `/projects/{projectId}/invites`             | List the project's pending + cancelled invites, newest first. Query: `limit`, `cursor`. Accepted invitees appear under `GET /validators`, not here.                 |
| `DELETE` | `/projects/{projectId}/invites/{inviteId}`  | Cancel a pending invite. An already-accepted invite returns `invite_already_accepted`.                                                                              |

An invite resource carries `id`, `email`, `projectId`, `status` (`pending`, `accepted`, or `cancelled`), `assignedAt`, `cancelledAt`, `invitedByEmail`, `emailSentAt`, `createdAt`, and `activeClassIds`.

### Upload sessions

| Method | Path                                                       | Summary                                                                                             |
| ------ | ---------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
| `POST` | `/projects/{projectId}/upload-sessions`                    | Open a session. Declare `files` (`relPath`, `size`). Returns presigned PUT URLs.                    |
| `POST` | `/projects/{projectId}/upload-sessions/{uploadId}/files`   | Add files or refresh URLs for an open session. Manifest is merged, never replaced.                  |
| `GET`  | `/projects/{projectId}/upload-sessions/{uploadId}`         | Session status, declared files, verification counters.                                              |
| `PUT`  | `/projects/{projectId}/upload-sessions/{uploadId}/file`    | One-shot upload: 307 redirect to a presigned URL. Query: `path` (required), `size` (optional).      |
| `POST` | `/projects/{projectId}/upload-sessions/{uploadId}/process` | Verify uploads and start the ingest run. Returns `runId` (202). Optional: `dryRun`, `skipExisting`. |
| `GET`  | `/projects/{projectId}/upload-sessions/{uploadId}/status`  | Poll session lifecycle and the latest ingest run.                                                   |

## Upload workflow

File bytes never pass through the Developer API. The API issues presigned PUT URLs; the client uploads directly to object storage.

| Step | Action                                                                                |
| ---- | ------------------------------------------------------------------------------------- |
| 1    | `POST .../upload-sessions` with the file manifest (`relPath`, `size` in bytes).       |
| 2    | `PUT` each file's bytes to its `uploadUrl` (no `Authorization` header on these PUTs). |
| 3    | Optionally `POST .../files` to declare more files or refresh expired URLs.            |
| 4    | `POST .../process` to verify uploads and start ingest.                                |
| 5    | `GET .../status` until the run reaches `complete` or `failed`.                        |

**Limits:** 5 GiB per file; up to 1000 files per declare call; paths must be relative (no leading `/`, no `..`). Presigned URLs expire after one hour. Request fresh ones via declare-files.

**One-shot redirect:** `PUT .../file?path=data/rows.csv` responds with `307` to the presigned URL before reading the body. Works well with `curl -T`:

```bash
curl -T ./data/rows.csv \
  'https://poq.sapien.io/developer/v1/projects/{projectId}/upload-sessions/{uploadId}/file?path=data/rows.csv' \
  -H 'Authorization: Bearer poq_live_...' \
  --location
```

For large files or unfamiliar HTTP clients, prefer the two-step declare + PUT flow.

## Datapoints

| Method | Path                                    | Summary                                                                                                                                                                    |
| ------ | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `GET`  | `/datapoints/{datapointId}/consensus`   | Fetch a datapoint's terminal consensus outcome: `qualityRatingMean`, `qualityRatingMedian`, `consensusStrength`, and the `consensusDecisionId`. |

The response mirrors the `datapoint.consensus_reached` webhook payload. A datapoint still under validation returns `consensus_not_finalized`; one outside the key's org returns `datapoint_not_found`.

## PoQ Report

| Method | Path                                    | Summary                                                                                                                                              |
| ------ | --------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GET`  | `/projects/{projectId}/proof-report`    | Issue and return the project's signed `poq.attestation/v1` PoQ Report. |

The report is built and signed on demand from the project's finalized consensus data. The response carries the two headline ratings (`qualityRating`, `consensusStrength`), the `attestationRoot`, the `signingKeyId`, the canonical `payload`, and the compact `jws`. The `jws` verifies offline against the published JWKS with no follow-up fetch. Returns `proof_report_not_ready` until the dataset finishes draining, and `proof_report_signing_unavailable` when the deployment has no signing key configured.

## Error codes

Stable codes are part of the public contract. Renaming or removing a code is a breaking change.

### Authentication

| Code                | HTTP | Message                                  |
| ------------------- | ---- | ---------------------------------------- |
| `api_key_missing`   | 401  | Missing or invalid Authorization header. |
| `api_key_malformed` | 401  | Invalid API key format.                  |
| `api_key_invalid`   | 401  | Invalid or expired API key.              |

### Request validation

| Code                     | HTTP | Message                    |
| ------------------------ | ---- | -------------------------- |
| `parameter_invalid`      | 400  | Invalid request parameter. |
| `request_body_invalid`   | 400  | Invalid request body.      |
| `request_body_malformed` | 400  | Malformed request body.    |
| `unknown_endpoint`       | 404  | Unknown endpoint.          |

### Projects

| Code                             | HTTP | Message                                                                  |
| -------------------------------- | ---- | ------------------------------------------------------------------------ |
| `api_key_actor_unavailable`      | 403  | This API key has no user authorized to create projects.                  |
| `not_org_originator`             | 403  | The API key owner is not an originator for this organization.            |
| `project_not_found`              | 404  | Project not found.                                                       |
| `project_already_active`         | 409  | Project is already active.                                               |
| `project_spec_invalid`           | 400  | The project spec is invalid. (Message carries the specific parse error.) |
| `qualification_catalog_conflict` | 409  | A qualification field conflicts with the existing catalog.               |

### Upload sessions

| Code                       | HTTP | Message                                                    |
| -------------------------- | ---- | ---------------------------------------------------------- |
| `upload_session_not_found` | 404  | Upload session not found.                                  |
| `upload_session_not_open`  | 409  | Session is no longer open for new files.                   |
| `upload_session_not_ready` | 409  | Cannot start a run (in flight or already complete).        |
| `upload_no_files`          | 400  | No files were declared.                                    |
| `upload_file_path_invalid` | 400  | A declared path is absolute or escapes the session folder. |
| `upload_id_required`       | 400  | An upload session ID is required.                          |
| `upload_files_missing`     | 409  | Declared files not yet uploaded (message names paths).     |
| `project_spec_missing`     | 409  | Project has no compiled spec.                              |

### Validators and classes

| Code                  | HTTP | Message                                             |
| --------------------- | ---- | -------------------------------------------------- |
| `validator_not_found` | 404  | Validator not found in this organization.          |
| `class_not_in_spec`   | 400  | classId is not declared in the project's validator spec. |

### Invites

| Code                      | HTTP | Message                                                     |
| ------------------------- | ---- | ---------------------------------------------------------- |
| `invite_not_found`        | 404  | Invite not found.                                          |
| `invite_already_accepted` | 409  | Invite has already been accepted and cannot be cancelled.  |

### Datapoints

| Code                      | HTTP | Message                                                |
| ------------------------- | ---- | ----------------------------------------------------- |
| `datapoint_not_found`     | 404  | Datapoint not found.                                   |
| `consensus_not_finalized` | 409  | This datapoint has not reached terminal consensus yet. |

### PoQ Report

| Code                               | HTTP | Message                                                                        |
| ---------------------------------- | ---- | ----------------------------------------------------------------------------- |
| `proof_report_not_ready`           | 404  | No finalized PoQ Report yet; the project's dataset has not finished draining. |
| `proof_report_signing_unavailable` | 503  | PoQ Report signing is not configured for this deployment.                    |

### Server

| Code             | HTTP | Message                |
| ---------------- | ---- | ---------------------- |
| `internal_error` | 500  | Internal server error. |

## See also

| Page | Description |
| --- | --- |
| [MCP server](/integrate/mcp) | Same operations as MCP tools |
| [API keys](/integrate/api-keys) | Creation and authorization |
| [poq.toml spec](/reference/poq-toml) | Authoring the `spec` field |
| [Quickstart](/start/quickstart#create-a-project-via-the-api) | End-to-end walkthrough |

---

# MCP server

URL: https://sapien-ai-docs-production.up.railway.app/integrate/mcp
Source: docs/current/integrate/mcp.md

# MCP server

The MCP server exposes the Developer API as [Model Context Protocol](https://modelcontextprotocol.io) tools over streamable HTTP. Use the same [API keys](/integrate/api-keys) as the REST surface.

The MCP endpoint is served at [`https://poq.sapien.io/mcp`](https://poq.sapien.io/mcp).

The server is stateless, so nothing is managed across requests. Project and upload tools share domain logic with `/developer/v1`; field names, pagination cursors, and error vocabulary match.

## Authentication

Every request needs a bearer token in the `Authorization` request header:

```
Authorization: Bearer poq_live_...
```

See [API keys](/integrate/api-keys) for key creation and lifecycle. Project tools additionally require the key creator to be an org originator, the same rule as the [Developer API](/integrate/developer-api). Unauthenticated requests receive a JSON `401` before any MCP dispatch.

## Client setup

Example for Cursor (`mcp.json`):

```json
{
  "mcpServers": {
    "sapien-poq": {
      "type": "http",
      "url": "https://poq.sapien.io/mcp",
      "headers": {
        "Authorization": "Bearer poq_live_..."
      }
    }
  }
}
```

Other MCP clients (Claude Code, VS Code) use the same URL and header pattern.

## Tools

### Project and upload tools

These mirror the Developer API 1:1. See the [Developer API reference](/integrate/developer-api) and [OpenAPI spec](https://poq.sapien.io/developer/v1/openapi.json) for field-level detail.

| Tool                        | REST equivalent                      | Purpose                                                                          |
| --------------------------- | ------------------------------------ | -------------------------------------------------------------------------------- |
| `ping`                      | `GET /developer/v1/ping`             | Auth probe. Returns `{ "pong": true }`.                                          |
| `create_project`            | `POST /developer/v1/projects`        | Create a live project. Args: `name`, `spec` (JSON).                              |
| `list_projects`             | `GET /developer/v1/projects`         | Paginated list. Args: `limit` (1–100), `cursor`.                                 |
| `get_project`               | `GET /developer/v1/projects/{id}`    | Fetch one project. Arg: `projectId`.                                             |
| `update_project`            | `PUT /developer/v1/projects/{id}`    | Update metadata. Args: `projectId`, plus `name`, `description`, and/or `paused`. |
| `create_upload_session`     | `POST .../upload-sessions`           | Open a session. Args: `projectId`, `files`.                                      |
| `declare_upload_files`      | `POST .../files`                     | Add files or refresh presigned URLs.                                             |
| `get_upload_session`        | `GET .../upload-sessions/{uploadId}` | Session detail and verification counters.                                        |
| `process_upload_session`    | `POST .../process`                   | Verify uploads and start ingest. Returns `runId`.                                |
| `get_upload_session_status` | `GET .../status`                     | Poll session and latest run.                                                     |

### Validator, class, and invite tools

These also mirror the Developer API 1:1. Args use the same camelCase names as the REST bodies.

| Tool                     | REST equivalent                                          | Purpose                                                                                            |
| ------------------------ | ------------------------------------------------------- | ------------------------------------------------------------------------------------------------ |
| `list_validators`        | `GET /developer/v1/validators`                          | Paginated org validator roster. Args: `limit`, `cursor`.                                          |
| `lookup_validator`       | `GET /developer/v1/validators?email=`                   | Look up a single validator by email (case-insensitive). Arg: `email`.                             |
| `list_validator_projects`| `GET .../validators/{id}/projects`                      | A validator's project assignments. Arg: `validatorId`.                                             |
| `assign_validator`       | `POST .../validators/{id}/assign`                       | Assign a validator to `projects`, optionally granting classes. Args: `validatorId`, `projects`, `autoEnroll`. |
| `unassign_validator`     | `POST .../validators/{id}/unassign`                     | Revoke a validator from projects. Args: `validatorId`, `projectIds`.                              |
| `list_project_classes`   | `GET /projects/{id}/classes`                            | Classes declared in a project's spec. Arg: `projectId`.                                            |
| `grant_validator_class`  | `POST .../validators/{id}/classes`                      | Grant a class. Args: `projectId`, `validatorId`, `classId`, `evidenceNote`.                        |
| `revoke_validator_class` | `DELETE .../validators/{id}/classes/{classId}`          | Revoke a class grant. Args: `projectId`, `validatorId`, `classId`.                                 |
| `invite_validator`       | `POST /projects/{id}/invites`                           | Invite a validator by email. Args: `projectId`, `email`, optional `classId`/`evidenceNote`/`sendEmail`. |
| `list_project_invites`   | `GET /projects/{id}/invites`                            | A project's pending + cancelled invites. Args: `projectId`, `limit`, `cursor`.                    |
| `cancel_project_invite`  | `DELETE /projects/{id}/invites/{inviteId}`              | Cancel a pending invite. Args: `projectId`, `inviteId`.                                            |

`lookup_validator` is the one tool that does not map 1:1 to a distinct REST path: the REST surface folds email lookup into `GET /validators?email=`, while MCP exposes it as its own tool. Note also that `invite_validator` corresponds to the REST `POST /projects/{id}/invites` (`createProjectInvite`).

### Results tools

| Tool                      | REST equivalent                        | Purpose                                                                                 |
| ------------------------- | -------------------------------------- | -------------------------------------------------------------------------------------- |
| `get_datapoint_consensus` | `GET /datapoints/{id}/consensus`       | A datapoint's terminal consensus outcome. Arg: `datapointId`.                          |
| `get_proof_report`        | `GET /projects/{id}/proof-report`      | Issue and return the project's signed PoQ Report. Arg: `projectId`.                  |

### Instruction and convenience tools

These have no REST counterpart.

| Tool                           | Purpose                                                                                                                                      |
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- |
| `get_started`                  | Recommended tool sequence for a new project: spec guide → JSON spec → create → upload.                                                       |
| `spec_authoring_guide`         | System prompt and message format for drafting a JSON spec from a `poq.md`. Call before `create_project` when you do not already have a spec. |
| `project_creation_instruction` | Step-by-step upload guidance. Optional `datasetType` (`markdown_split`, `csv`, `json`, `file_collection`) for format-specific tips.          |
| `wait_for_upload_session`      | Polls status internally until `complete`/`failed` or `timeoutSeconds` (1–300, default 60) elapses. Returns `timedOut`.                       |

## Uploading data

Bytes never flow through a tool call. `create_upload_session` and `declare_upload_files` return a presigned PUT URL per file. Upload each file with a plain HTTP `PUT` to that URL (no `Authorization` header), then call `process_upload_session` and poll status.

The REST one-shot `PUT .../file` redirect has no MCP equivalent. It is HTTP transport sugar for `curl -T`. Use `declare_upload_files` instead.

## Errors

Domain failures return tool errors (`isError: true`). The text leads with a stable **camelCase** code:

```
projectNotFound: Project not found.

requestId: 0192f0a0-...
```

Codes are the camelCase spellings of the REST catalog (`projectSpecInvalid` ↔ `project_spec_invalid`, `notOrgOriginator` ↔ `not_org_originator`). Map handling the same way across both surfaces.

Include the `requestId` block when reporting issues. It matches the `X-Request-Id` header on REST responses.

## REST vs MCP

| Concern              | REST                          | MCP                                           |
| -------------------- | ----------------------------- | --------------------------------------------- |
| Transport            | HTTP + JSON                   | Streamable HTTP MCP                           |
| Error codes          | `snake_case` in JSON envelope | `camelCase` in tool error text                |
| One-shot file upload | `PUT .../file` (307 redirect) | `declare_upload_files` + presigned PUT        |
| Spec authoring help  | External (docs, agents)       | `spec_authoring_guide`, `get_started`         |
| Polling              | Client calls `GET .../status` | `wait_for_upload_session` convenience wrapper |

## See also

| Page | Description |
| --- | --- |
| [Developer API](/integrate/developer-api) | HTTP endpoints and error codes |
| [API keys](/integrate/api-keys) | Creation and authorization |
| [Quickstart](/start/quickstart#create-a-project-via-the-api) | End-to-end walkthrough |

---

# poq.md examples

URL: https://sapien-ai-docs-production.up.railway.app/examples/poq-md
Source: docs/current/examples/poq-md.md

# poq.md examples

Plain-language prompts in `poq.md` draft into `poq.toml`. Examples use a Beatles catalog review theme.

## Data & ingestion

### One datapoint per CSV row

:::project-files{title="csv-row" root="csv-row-datapoint" file="poq.md"}
:::

### One datapoint per JSON file

Use `songs/*.json` and dotted paths for nested fields.

:::project-files{title="json-glob" root="json-glob-datapoint" file="poq.md"}
:::

### Image + CSV join

Join track metadata to album cover files on basename.

:::project-files{title="image-join" root="image-csv-join" file="poq.md"}
:::

### Markdown report split

Split liner notes into one datapoint per track heading.

:::project-files{title="markdown-split" root="markdown-split" file="poq.md"}
:::

## Rubric & scoring

### Multi-dimensional rubric

Mix Likert, ordinal, and numeric scales on one track.

:::project-files{title="multi-rubric" root="multi-dimensional-rubric" file="poq.md"}
:::

### Weighted consensus

Genre fit counts twice; confidence is advisory.

:::project-files{title="weighted-consensus" root="weighted-consensus" file="poq.md"}
:::

### Conditional rubric rows

Skip Arrangement in the validator UI when Validity is `False positive` (`consensus_skip.match`; filler votes still count toward consensus).

:::project-files{title="conditional-rubric" root="conditional-rubric" file="poq.md"}
:::

## Validation UI

### Image and text evidence

Cover image first, then markdown caption and album label.

:::project-files{title="image-text" root="image-text-evidence" file="poq.md"}
:::

### Pinned source link

Link to anthology edition, track file, and lyric line.

:::project-files{title="source-link" root="source-code-link" file="poq.md"}
:::

### Structured metadata

Song name, album, and writer in a facts panel.

:::project-files{title="metadata-facts" root="metadata-facts" file="poq.md"}
:::

## Ground truth, prefill & difficulty

### Hidden golden labels

`label` is ground truth; validators never see it.

:::project-files{title="golden-labels" root="golden-labels" file="poq.md"}
:::

### Prefilled hints

Show `label_shown`; validators judge agreement.

:::project-files{title="prefill-hints" root="prefill-hints" file="poq.md"}
:::

### Difficulty metadata

Numeric `difficulty` for routing; stripped from validator UI.

:::project-files{title="difficulty" root="difficulty-metadata" file="poq.md"}
:::

## Validators

### Validator classes

Junior vs Senior music critics with different pay and priority.

:::project-files{title="validator-classes" root="validator-classes" file="poq.md"}
:::

### Attribute-based routing

Iconic tier → 5 critics (2 senior); default → 3.

:::project-files{title="attribute-routing" root="attribute-routing" file="poq.md"}
:::

### Wildcard composition

Fill remaining slots with `class = "*"` at claim time.

:::project-files{title="wildcard" root="wildcard-composition" file="poq.md"}
:::

### Escalation ladder

Add senior critic waves until verified.

:::project-files{title="escalation" root="escalation-ladder" file="poq.md"}
:::

### Class-specific rewards and stakes

Different `reward_usd` / `stake_usd` per critic class.

:::project-files{title="class-pay" root="class-rewards-stakes" file="poq.md"}
:::

### Qualification gates

Require publication and musicology credential before claim.

:::project-files{title="qualification" root="qualification-gates" file="poq.md"}
:::

### Claim time limits

30 minutes per claim.

:::project-files{title="claim-ttl" root="claim-time-limits" file="poq.md"}
:::

### Conflict-of-interest self-decline

Release a claim without penalty when COI applies.

:::project-files{title="coi-decline" root="coi-self-decline" file="poq.md"}
:::

## AI-assisted review

### AI validator panel

Three AI model classes review each track before humans.

:::project-files{title="ai-panel" root="ai-validator-panel" file="poq.md"}
:::

### AI-first with human escalation

AI panel first; contested items escalate to senior critics.

:::project-files{title="ai-escalation" root="ai-human-escalation" file="poq.md"}
:::

### AI-safe scale constraints

Integer-friendly scales when AI critics are present.

:::project-files{title="ai-scales" root="ai-safe-scales" file="poq.md"}
:::

---

# poq.toml examples

URL: https://sapien-ai-docs-production.up.railway.app/examples/poq-toml
Source: docs/current/examples/poq-toml.md

# poq.toml examples

A `poq.toml` file turns uploaded data into review items: what to ingest, what evidence validators see, what questions they answer, and how review slots are assigned. These examples use the current namespaced layout from the [`poq.toml` reference](/reference/poq-toml).

The best first spec is usually small: one source, one review item per row, and a short rubric. Add joins, routing, classes, and AI validators only when the workflow needs them.

## Annotation Validation

Use this when you already have labeled rows and want independent agreement checks. What makes it interesting: it is the smallest useful shape, but still shows the core sequence of data ingested, evidence shown, rubric answered, consensus produced.

:::project-files{title="csv-row" root="annotation-qa" file="poq.toml"}
:::

## Image Annotation Validation

Use this when each review item is a row of structured annotation metadata joined to an image file, common in captioning, classification, segmentation, and other image labeling workflows. What makes it interesting: the CSV is the task root, images attach through a join, and validators see the annotation and image together on one page.

:::project-files{title="csv-image-join" root="image-annotation-qa" file="poq.toml"}
:::

## Markdown Report Review

Use this when one uploaded report file should become many review items, like audit write-ups, policy reviews, incident postmortems, or any document with repeated sections. What makes it interesting: `markdown_split` turns each header match into a row, metadata regex pulls structured fields from the document and sections, and `source_link` sends validators to the cited file and line.

:::project-files{title="markdown-split" root="markdown-report-review" file="poq.toml"}
:::

## Conditional Rubric

Use this when later questions only apply depending on earlier answers. For example, grey out fix-quality scoring when a validator marks an item as a false positive. What makes it interesting: `consensus_skip.match` on a rubric row references another row's `id` and trigger labels; when the validator's own answers match, that row greys out in the UI (filler votes still count toward consensus).

:::project-files{title="conditional-rubric" root="conditional-rubric-triage" file="poq.toml"}
:::

## Model-Prediction Agreement

Use this when a model already emits a prediction for each item and you want to measure how closely an independent human panel lands on it, without letting that prediction anchor the reviewers. What makes it interesting: `[[validation.reported_label]]` with `quality = true` turns the model's prediction into a hidden **distance reference**, so the dimension's Quality Rating becomes how close the panel's consensus sits to the model while Consensus Strength still measures agreement on its own.

:::project-files{title="reported-label" root="model-prediction-agreement" file="poq.toml"}
:::

Because `quality = true`, `model_rating` is stripped from everything validators see, so do not also list it in `[[validation.evidence]]`. The `map` translates the model's `1` to `5` output onto the scale labels, and the mapped value must land on a declared anchor or the dimension falls back to raw-value quality. The `rating` Quality Rating now reads as agreement with the model: 100 when the panel lands exactly on the prediction and lower as it drifts, while Consensus Strength still measures how tightly the panel agreed with itself. See [`[[validation.reported_label]]`](/reference/poq-toml/validation#validationreported_label) for the scoring formula.

## JSON Array Unnesting

Use this when each uploaded JSON file wraps many review items in one array — for example an album file with a `tracks` list. What makes it interesting: `unnest.array_key` turns one file into one review item **per array element**, expanding each element's object fields into row columns.

:::project-files{title="json-unnest" root="json-array-unnest" file="poq.toml"}
:::

## Source Excerpt Review

Use this when validators need to judge a finding against the actual source code or document text, not just a summary. What makes it interesting: `source_excerpt` fetches and displays pinned content from a public repository at a specific commit directly in the task panel.

:::project-files{title="source-excerpt" root="source-excerpt-review" file="poq.toml"}
:::

## Routed Expert Review

Use this when some items need specialists and others can be handled by a general pool. What makes it interesting: validator classes and routes let higher-risk rows get more reviewers and a senior mix without changing the item schema.

:::project-files{title="severity-routing" root="routed-expert-review" file="poq.toml"}
:::

## AI Panel With Human Escalation

Use this when an AI panel can handle first-pass review and humans should focus on contested items. What makes it interesting: each AI class can use a different model or prompt, and escalation steps run in order until the item reaches verified consensus: first one senior human, then a larger human panel if agreement is still insufficient.

:::project-files{title="ai-escalation" root="ai-panel-escalation" file="poq.toml"}
:::

---

# poq.toml spec

URL: https://sapien-ai-docs-production.up.railway.app/reference/poq-toml
Source: docs/current/reference/poq-toml.md

# poq.toml spec

The poq.toml file for each Sapien project is its _system spec_. It turns the data you upload into a working project: the fields on each review item, what validators see, the questions they answer, and how they are scored.

Every `poq.toml` has three phases (ingestion, validation, attestation) plus a validators config and file metadata. Each is documented on its own page below.

## Sections

| Section                                          | Namespace         | What it configures                                                        |
| ------------------------------------------------ | ----------------- | ------------------------------------------------------------------------- |
| [Project](/reference/poq-toml/project)           | `[project]`       | Spec format version (`spec_version`) and optional `tag`.                   |
| [Ingestion](/reference/poq-toml/ingestion)       | `[ingestion.*]`   | Turn uploaded files into datapoint rows: sources, joins, field projection.|
| [Validation](/reference/poq-toml/validation)     | `[validation.*]`  | Task UI, evidence, rubric, and how answers are scored.                     |
| [Validators](/reference/poq-toml/validators)     | `[validators.*]`  | Who reviews: counts, pay, classes, routing, qualification.                 |
| [Attestation](/reference/poq-toml/attestation)   | `[attestation.*]` | Signed PoQ Report export.                                                  |

> **Unknown keys are rejected.** The parser is strict: any key or table that isn't part of the schema (a typo, an extra field, or a leftover from an old flat-layout spec such as `[[inputs]]` or `[stage.*]`) fails at parse time with an error naming the offending key, rather than being silently ignored.

A working spec needs at minimum: `[project]` with `spec_version`, at least one `[[ingestion.sources]]`, `[ingestion.fields]` with a key named `id`, at least one `[[validation.evidence]]`, at least one `[[validation.rubric]]`, and `[validators]`. Every other section is optional.

## The Data Lifecycle: From Files to Review

Before writing your spec, it is helpful to understand how Sapien transforms your files into tasks.

### 1. File Upload

You upload your files (CSVs, JSONs, images, or Markdown) from a project folder.

### 2. Ingestion

When you **Ingest** a project, Sapien runs your `poq.toml` spec to build the review items.

- **The Spreadsheet Model**: Sapien treats every collection of files as a temporary spreadsheet.
  - **Rows**: Each individual item in your file (a CSV line, a JSON file, a Markdown section).
  - **Columns**: The data inside those items (a CSV header, a JSON key, a regex capture).
- **The Mapping**: `[ingestion.fields]` is the wiring — each table key is the column name used everywhere else; each value is `<source_id>.<column>` at ingest time.
  - _Example_: `finding_title = "findings.title"` keeps the JSON `title` property under the name `finding_title`.
- **The Merge (Optional)**: If you have multiple sources (e.g., labels in a CSV and images in a folder), `[[ingestion.joins]]` lines them up into one wider spreadsheet before field projection.

### 3. Datapoints

Once ingestion finishes, the temporary spreadsheets are deleted. What remains are **review items**, stored in a database table.

- **One row = one task**: Each review item is a single row in the database.
- **Persistent data**: The keys you declared in `[ingestion.fields]` are saved on that row.

### 4. Review

When a validator starts a task, they aren't looking at your original files. They are looking at **one specific review item** that was ingested.

- The `[validation.*]` sections tell the UI what to show, what to ask, and how items are scored; the `[validators.*]` sections decide who validates, how many, pay, routing, and qualification gates.

## For developers

**On-disk TOML** in this reference is the **authoring shape** accepted by the parser (`ingestion.*`, `validation.*`, `validators.*`, top-level `[attestation]`). Legacy `[stage.*]` and `[[inputs]]` roots are no longer accepted.

**Compiled runtime JSON** uses its own internal field names, which don't always match the TOML keys 1:1 in spelling. The mapping from the namespaced TOML layout to compile output is 1:1 at the semantic level (no information is added or dropped).

---

# poq.toml — project

URL: https://sapien-ai-docs-production.up.railway.app/reference/poq-toml/project
Source: docs/current/reference/poq-toml/project.md

# `project`

One of the [poq.toml spec](/reference/poq-toml) sections.

The `[project]` header declares which version of the spec format you wrote your file against.

```toml
[project]
spec_version = "1"
```

Today the only accepted value for `spec_version` is `"1"`. Any other value is rejected as soon as the file is read.

| option         | accepts | required | description                                                                                            | example |
| -------------- | ------- | -------- | ------------------------------------------------------------------------------------------------------ | ------- |
| `spec_version` | string  | yes      | Tells Sapien which version of the spec format you wrote. Set to `"1"` — the only value accepted today. | `"1"`   |

---

# poq.toml — ingestion

URL: https://sapien-ai-docs-production.up.railway.app/reference/poq-toml/ingestion
Source: docs/current/reference/poq-toml/ingestion.md

# Ingestion (`ingestion.*`)

One of the [poq.toml spec](/reference/poq-toml) sections.

Everything before review: declare sources, optional joins, and the field projection that becomes each datapoint row. Turn uploaded files into datapoint rows: declare sources, optional joins, and which columns each task carries.

- [`ingestion.sources`](#ingestionsources) (required) — Where your data lives and how Sapien reads it.
- [`ingestion.joins`](#ingestionjoins) (optional) — Combine separate sources before field projection.
- [`ingestion.fields`](#ingestionfields) (required) — Name each column on a task and say which file it came from (e.g. `title = "findings.title"`).

## `ingestion.sources`

`[[ingestion.sources]]` tells Sapien **which files contain your project's data and how to read them**. Every batch of files is its own `[[ingestion.sources]]` block.

**One Primary Source**: Sapien builds your task list from the **first** `[[ingestion.sources]]` block listed in your TOML. If you want tasks from multiple files or folders, you should group them into this first block using a glob.

- **Multiple Markdown files**: Use `path_glob = "reports/*.md"`. Every section in every matching file will become a task.
- **Multiple JSON files (one item per file)**: Use `path_glob = "{folder_a,folder_b}/*.json"`. Every JSON file in both folders becomes one task.
- **JSON report files with an array of items**: Use `path_glob = "reports/*.json"` with `unnest.array_key = "findings"` (or whatever key holds the list). Each array element becomes one task — no manual pre-split into per-item files.

If you create **separate** `[[ingestion.sources]]` blocks (with different `id`s), Sapien assumes they are different _types_ of data that need to be joined together (like joining a CSV of labels to a folder of images). Data in secondary inputs will only appear in your tasks if you explicitly join them to the first input.

```toml

[[ingestion.sources]]
id = "findings"
type = "json"
path_glob = "findings/*.json"
```

The `id` is the label you pick for this batch. Wire columns in `[ingestion.fields]` using that id as the prefix:

```toml
[ingestion.fields]
title = "findings.title"    # "findings" is the source id; ".title" is the column
```

With multiple inputs (CSV + image folder), ids disambiguate sources:

```toml

[[ingestion.sources]]
id = "labels"
type = "csv"
path = "labels.csv"

[[ingestion.sources]]
id = "images"
type = "file_collection"
path_glob = "images/*.jpg"

[ingestion.fields]
diagnosis = "labels.diagnosis"
```

### Input fields

| option       | accepts   | required                                                                    | description                                                                                                                                                                    | example                              |
| ------------ | --------- | --------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------ |
| `id`         | string    | yes                                                                         | Short name for this batch. Used in `[ingestion.fields]` values (`findings.title`), join `left`/`right`, and route `match` keys. Must be unique across `[[ingestion.sources]]`. | `"findings"`                         |
| `type`       | enum      | yes                                                                         | How Sapien reads this batch: `csv`, `json`, `file_collection`, `markdown_split`, or `video_frames`.                                                                                            | `"json"`                             |
| `path`       | string    | no (defaults to `<id>.<ext>` for csv/json)                                  | Single uploaded file when this input is one file. Cannot start with `/` or contain `..`.                                                                                       | `"findings.csv"`                     |
| `path_glob`  | string    | yes for `file_collection`; one of path/glob for `json`, `markdown_split`, and `video_frames` | Pattern matching many files (e.g. `images/*.jpg`). Use instead of `path` for folders.                                                                                          | `"images/*.jpg"`                     |
| `unnest.*`   | see below | optional for `json` only                                                    | For `json` only — explode one array column into many rows. See [JSON array unnest](#json-array-unnest).                                                                       | `unnest.array_key = "findings"`      |
| `splitter.*` | see below | yes for `markdown_split` (at least `splitter.regex`)                        | For `markdown_split` only — full list in [Splitter settings](#splitter-settings).                                                                                              | `splitter.regex = "^## (?P<id>.+)$"` |

### Per-type field set

Each `type` allows a different subset of keys. The compiler rejects forbidden keys with a path-prefixed error (for example `inputs[1].path_glob: only valid on file_collection or json inputs`).

| Key                                                             | csv                            | json                            | file_collection | markdown_split | video_frames |
| --------------------------------------------------------------- | ------------------------------ | ------------------------------- | --------------- | -------------- | ------------ |
| `id`                                                            | required                       | required                        | required        | required       | required     |
| `type`                                                          | required                       | required                        | required        | required       | required     |
| `path`                                                          | optional (defaults `<id>.csv`) | optional (defaults `<id>.json`) | forbidden       | optional       | optional     |
| `path_glob`                                                     | forbidden                      | optional                        | required        | optional       | optional     |
| `file_id_strategy`                                              | forbidden                      | forbidden                       | required        | forbidden      | forbidden    |
| `unnest.array_key`                                              | forbidden                      | optional                        | forbidden       | forbidden      | forbidden    |
| `splitter.regex`                                                | forbidden                      | forbidden                       | forbidden       | required       | forbidden    |
| `splitter.end_regex`, `[[ingestion.sources.splitter.metadata]]` | forbidden                      | forbidden                       | forbidden       | optional       | forbidden    |
| `sample_fps`                                                    | forbidden                      | forbidden                       | forbidden       | forbidden      | **required** |
| `max_frames`, `format`, `clip_seconds`, `overlay_path_glob`     | forbidden                      | forbidden                       | forbidden       | forbidden      | optional     |

`json`, `markdown_split`, and `video_frames` accept **either** `path` (single file) **or** `path_glob` (many files). Setting both is a compile-time error.

### JSON array unnest

By default, each uploaded JSON file becomes **one** review item. Use `unnest.array_key` on a `json` input when each file is a **report object** that wraps an **array of review items** at a known key — ingest explodes that array into **one review item per element** (deterministic key-level splitting; no regex, no interim folder).

```toml
[[ingestion.sources]]
id               = "findings"
type             = "json"
path_glob        = "reports/*.json"
unnest.array_key = "findings"
```

Given `reports/batch-001.json`:

```json
{
  "exportedAt": "2026-06-23T05:15:03Z",
  "findings": [
    { "fingerprint": "F-01", "title": "Unchecked return value", "severity": "medium", "affectedCode": "..." },
    { "fingerprint": "F-02", "title": "Missing access control", "severity": "high", "affectedCode": "..." }
  ]
}
```

Ingest produces **two** review items. Each row's columns are the **fields of that array element**. Root-level scalars outside the array (here `exportedAt`) are **not** copied onto the rows — wire only fields that exist on each element. When the input uses `path_glob`, every row also carries DuckDB's synthetic `filename` column (the source file's URL), the same as a flat `json` glob ingest.

Wire `[ingestion.fields]` using the **exact JSON property names** on each array element (camelCase preserved):

```toml
[ingestion.fields]
id            = "findings.fingerprint"
title         = "findings.title"
severity      = "findings.severity"
affected_code = "findings.affectedCode"
```

| option             | accepts | required | description                                                                                                                                                              | example      |
| ------------------ | ------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------ |
| `unnest.array_key` | string  | yes\*    | JSON key holding the **array of objects** to explode into rows. Dot-separated segments drill into nested objects (`report.items`). Only valid when `type = "json"`. | `"findings"` |

\*Required when you declare `unnest` for this input; omit `unnest` entirely when each JSON file is already one review item.

**Rules:**

- Only valid on `type = "json"`. Setting `unnest.array_key` on `csv`, `file_collection`, or `markdown_split` fails at parse time.
- The value at `unnest.array_key` must be a JSON **array**. An empty array yields zero review items from that file.
- Each array element should be a JSON **object** so its keys can be wired in `[ingestion.fields]`.
- Only each element's own fields become row columns; root-level scalars outside the array are not carried.

### The `markdown_split` input type

Use `markdown_split` when a single Markdown file (or a folder of them) is really **many review items stacked in one document** — an audit with one section per finding, a catalog with one entry per heading, a postmortem with repeated sections. It is the Markdown path into the same spreadsheet model as CSV rows and JSON elements (see [The Data Lifecycle](/reference/poq-toml#the-data-lifecycle-from-files-to-review)): where a CSV gives one row per line and a JSON array gives one row per element, `markdown_split` gives **one row per section**.

**How the split works.** Sapien scans each file top to bottom. Every line that matches `splitter.regex` **opens a new section**; that section runs from its header down to the start of the next matching header — or, if set, to the first [`splitter.end_regex`](#splitter-settings) match — or to end of file. Each section becomes **one row / one datapoint**, exactly like a CSV line.

**What columns a section row has.** Every row carries the section's own text as `body`, plus [built-in columns](#columns-available-after-a-split) like `row_id` and `report_path`, plus one column for each **named capture** in `splitter.regex`, plus any columns you pull with [`metadata`](#splitter-settings). You wire those columns into review-item fields in [`ingestion.fields`](#ingestionfields).

Write the splitter pattern from **header lines in your file** — copy the fixed prefix literally, use `(?P<name>...)` for the parts that change on each row:

| Header line in your Markdown (string) | `splitter.regex`                                          |
| ------------------------------------- | --------------------------------------------------------- |
| `## F-01: Missing rate limit`         | `'^##\s+(?P<id>F-\d+):?\s+(?P<title>.+)$'`                |
| `## F-02: SQL injection`              | _(same regex — one pattern matches every section header)_ |

Optional `splitter.end_regex` — stop the section body before the next header when subsections use the same `##` level:

| Line where the section should end (string) | `splitter.end_regex` |
| ------------------------------------------ | --------------------- |
| `## Next finding`                          | `'^##\s+'`             |

Optional `[[ingestion.sources.splitter.metadata]]` — extract extra columns from a line in the file or in each section.

**Repository row** (document header table) — string → regex:

```
| Repository | my-repo |
'^\|\s*Repository\s*\|\s*(?P<repository>[^|]+?)\s*\|'
```

**File line** (per section) — string → regex:

```
- **File**: `src/auth.ts:L42`
'^-\s*\*\*File\*\*:\s*`?(?P<source_file>.+?):L(?P<line_number>\d+)'
```

```toml

[[ingestion.sources]]
id = "audit_findings"
type = "markdown_split"
path_glob = "reports/*.md"
splitter.regex = '^##\s+(?P<id>F-\d+):?\s+(?P<title>.+)$'
splitter.end_regex = '^##\s+'

[[ingestion.sources.splitter.metadata]]
scope = "document"
column = "repository"
regex = '^\|\s*Repository\s*\|\s*(?P<repository>[^|]+?)\s*\|'
```

#### Splitter settings

These keys apply only when `type = "markdown_split"`. Regex values use **RE2** (Go's `regexp`): linear-time, but **no backreferences or lookaround** — anchors (`^`, `$`), character classes, and named captures `(?P<name>...)` are all supported. Sapien adds multiline matching (`(?m)`) at compile time, so `^` and `$` match at line boundaries — do not prefix `(?m)` yourself. Extracted `metadata` values are trimmed of surrounding whitespace and backticks, so a table cell like `` `a1b2c3d4` `` stores as `a1b2c3d4`.

| option                                    | accepts                                  | required                | description                                                                                                                                                                                           | example                                                                                      |
| ----------------------------------------- | ---------------------------------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------- |
| `splitter.regex`                          | string (regex)                           | yes (`markdown_split`)  | Header pattern that starts each section. Each match becomes one row. **Named captures** (`(?P<id>...)`) become output columns on this input.                                                          | `## F-01: …` → `'^##\s+(?P<id>F-\d+):?\s+(?P<title>.+)$'`                                    |
| `splitter.end_regex`                      | string (regex)                           | no                      | Optional early stop for `body`. After a header match, Sapien scans forward for this pattern; the section ends there instead of at the next header (or EOF).                                           | `## Next …` → `'^##\s+'`                                                                     |
| `[[ingestion.sources.splitter.metadata]]` | array of tables with fields listed below | no                      | Repeat to add extra output columns. Each row uses `scope`, `column`, `regex`, and optional `capture` below.                                                                                           | `[[ingestion.sources.splitter.metadata]]` with `scope = "document"`, `column = "repository"` |
| `scope`                                   | `document`, `section`                    | yes (each metadata row) | `document` — run `regex` once on the full file; value copied to every row from that file. `section` — run `regex` on each split section (`body` span) only.                                          | `"document"`                                                                                 |
| `column`                                  | string                                   | yes (each metadata row) | Output column name. Wire in `[ingestion.fields]` as `repository = "<input_id>.repository"`.                                                                                                           | `"repository"`                                                                               |
| `regex`                                   | string (regex)                           | yes (each metadata row) | Pattern to extract the value. Use a named capture matching `column`, or a single unnamed capture group.                                                                                               | see string → regex examples above                                                            |
| `capture`                                 | string                                   | no                      | Named capture to read from `regex` when it differs from `column`. Defaults to `column`. Use when one regex fills several columns.                                                                     | `"line_number"`                                                                              |
| `list`                                    | bool                                     | no                      | Default `false`. Capture **every** match of `regex` (deduped, first-seen order) into a `VARCHAR[]` column instead of only the first — for array canonical fields such as an evidence block's `paths`. | `true`                                                                                        |

#### Columns available after a split

Each split row exposes three kinds of column, all wired the same way in `[ingestion.fields]` (`<field> = "<input_id>.<column>"`):

- **Named captures** from `splitter.regex` — e.g. `id`, `title` above.
- **Metadata columns** — every `column` you declared under `[[ingestion.sources.splitter.metadata]]`.
- **Built-ins** — always present on every row, listed below.

| Built-in column | Type       | What it holds                                                                                                                                   |
| --------------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| `body`          | `VARCHAR`  | The full section text, from its header through the character before the next section starts (or the `end_regex` cut, or EOF).                    |
| `row_id`        | `VARCHAR`  | Stable per-section id — `<report_path>#<id>` when the splitter has a capture named `id`, otherwise `<report_path>@<start_offset>`. A good `id`.  |
| `report_path`   | `VARCHAR`  | Project-relative path of the file this section came from — useful when `path_glob` spans many files.                                             |
| `source_sha256` | `VARCHAR`  | SHA-256 of the whole source file, identical for every section in it.                                                                             |
| `start_offset`  | `BIGINT`   | Byte offset where the section starts in the file.                                                                                                |
| `end_offset`    | `BIGINT`   | Byte offset where the section ends.                                                                                                              |

Referencing a column that does not exist (no such capture, metadata, or built-in) is a compile-time error naming the missing column, so typos fail fast rather than yielding blank fields.

```toml

[ingestion.fields]
id           = "audit_findings.row_id"    # built-in — one stable id per section
finding_id   = "audit_findings.id"        # named capture from splitter.regex
finding_body = "audit_findings.body"      # built-in — the section text
repo         = "audit_findings.repository" # metadata column
```

### The `video_frames` input type

Use `video_frames` when an uploaded video should become **one review item per frame**. Upload the raw video with your project folder; Sapien splits it server-side, stores every extracted frame durably, and emits one row per frame. You do no ETL: no frame splitting, no CSV, no re-encoding. Extracted frame sets are content-addressed by the video's SHA-256, so re-ingesting the same bytes (or retrying a failed run) reuses the already-extracted frames.

```toml
[[ingestion.sources]]
id   = "frames"
type = "video_frames"
path = "recording.mp4"       # or path_glob = "videos/*.mp4"
sample_fps = 30              # required: 30 fps footage scored at 30 = every frame
```

#### video_frames settings

`sample_fps` is required. The rest are optional, and default to a JPEG frame with a one-second context clip.

There is deliberately no default rate. How densely a video is scored decides both what the audit can catch and what it costs, and neither answer is safe to assume on your behalf — scoring every frame is a choice too, so it gets written down. The frames Sapien keeps are exact native frames carrying their true native indices in `frame_index`, never a re-timed rate. That is what lets a sampled audit line up with annotation files that are themselves indexed by frame number. Because the step is a whole number of frames, a rate the video's frame rate cannot divide evenly lands on the nearest achievable one — asking for 7 frames per second of 30 fps footage keeps every 4th frame, which is 7.5 per second — and an exact half rounds toward more frames, so a run is never sparser than the rate you asked for.

| option              | accepts | default            | description                                                                                                                                                                                                          |
| ------------------- | ------- | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `sample_fps`        | float > 0 | **required** | How many frames per second to score. It is divided into each video's real frame rate to get a whole-frame step, so `sample_fps = 5` keeps every 6th frame of 30 fps footage and every 5th frame of 24 fps footage. To score every frame, set it to the video's frame rate or higher — a rate above the footage's own is clamped to every frame. |
| `max_frames`        | int ≥ 0 | `0` (cap = 50,000) | Per-video ceiling. A video that would exceed the effective cap is refused with a message telling you to set `sample_fps` or `max_frames`; the cross-video total is capped at 200,000 frames.                           |
| `format`            | enum    | `"jpeg"`           | Extracted frame image format: `jpeg` (high quality, ~10× smaller) or `png`.                                                                                                                                           |
| `clip_seconds`      | float   | `1.0`              | Length of the context-clip window emitted per frame as `clip_start_seconds` / `clip_end_seconds`, centered on the frame and clamped to the video bounds.                                                              |
| `overlay_path_glob` | string  | unset              | Glob matching annotation-overlay videos (the same recording with masks or keypoints drawn on). Each overlay pairs with a source video **by filename stem** and fills `overlay_frame_url`.                              |

#### Columns every frame row exposes

| column               | type   | meaning                                                                                     |
| -------------------- | ------ | ------------------------------------------------------------------------------------------- |
| `frame_id`           | string | Stable id: first 12 hex chars of the video's SHA-256 + `_f` + native frame index.           |
| `video_id`           | string | Source filename without extension.                                                          |
| `source_sha256`      | string | SHA-256 of the video file; binds every frame's datapoint to the exact bytes audited.        |
| `frame_index`        | int    | Native frame index in the source video so it steps with `sample_fps`.                           |
| `t_seconds`          | float  | `frame_index / fps`, with fps kept exact (29.97 does not drift).                            |
| `frame_url`          | string | The extracted frame image (presigned for reviewers automatically).                          |
| `prev_frame_url`     | string | The previous kept frame; the first frame carries its own URL so a comparison shows no change. Optional: bind it as `left_image` beside `frame_url` in an `image_comparison` block to catch single-frame annotation dropouts. |
| `overlay_frame_url`  | string | The matching overlay frame, when `overlay_path_glob` matched.                               |
| `video_url`          | string | The original uploaded video.                                                                |
| `clip_start_seconds` / `clip_end_seconds` | float | The context-clip window; bind both to a `video_clip` block's `start_seconds` / `end_seconds` (these accept field names as well as literals). |
| `fps`, `width`, `height`, `duration_seconds` | float/int | Stream metadata.                                                                 |
| `extractor_version`  | string | Extraction pipeline version stamped on every row for reproducibility.                       |

#### Worked example: per-frame video annotation audit

```toml
[[ingestion.sources]]
id   = "frames"
type = "video_frames"
path = "grid_2x2.mp4"
sample_fps = 30

[ingestion.fields]
id        = "frames.frame_id"
frame_url = "frames.frame_url"

[[validation.evidence]]
type  = "image"
title = "This frame"
url   = "frame_url"
```

That is the whole spec: the reviewer sees the frame and scores it. An 85-second 1080p30 video ingests as exactly 2,550 datapoints, one per frame. The project wizard's preview extracts only the first 25 frames so it answers in seconds; the full extraction happens on the real ingest run after project creation.

#### Optional evidence for a frame

Two more blocks exist for when the work needs them — not by default, because each one is another thing the reviewer has to look at on every one of those 2,550 items.

**The previous frame beside this one** catches an annotation that blinks out for a single frame; add it when the rubric is about consistency across frames rather than about one frame on its own.

```toml
[ingestion.fields]
prev_frame_url = "frames.prev_frame_url"

[[validation.evidence]]
type  = "image_comparison"
title = "Previous frame vs this frame"
left_image  = "prev_frame_url"
right_image = "frame_url"
```

**A short clip around the moment** adds motion; add it when a still is genuinely ambiguous. Bind `poster` to `frame_url` so the frame shows at once instead of waiting on the video, and note that the clip only plays if the source is in a format browsers decode (H.264 in an MP4) — extraction reads formats browsers do not, so a clip can extract perfectly and still not play.

```toml
[ingestion.fields]
video_url          = "frames.video_url"
clip_start_seconds = "frames.clip_start_seconds"
clip_end_seconds   = "frames.clip_end_seconds"

[[validation.evidence]]
type          = "video_clip"
title         = "One second around this moment"
video_url     = "video_url"
poster        = "frame_url"
start_seconds = "clip_start_seconds"
end_seconds   = "clip_end_seconds"
```

## `ingestion.joins`

The purpose of `[[ingestion.joins]]` is to **line up related data** from different files so they can be treated as a single task.

Use this section only if you declared more than one `[[ingestion.sources]]` block. It merges your separate data sources into one wider "spreadsheet" before you pick your final columns in `[ingestion.fields]`.

```toml

[[ingestion.joins]]
left = "labels"
right = "images"
left_on = "case_id"
right_on = "file_id"
type = "left"
```

| option     | accepts | required | description                                                                               | example     |
| ---------- | ------- | -------- | ----------------------------------------------------------------------------------------- | ----------- |
| `left`     | string  | yes      | Input id on the left side of the join (usually the table you want to keep all rows from). | `"labels"`  |
| `right`    | string  | yes      | Input id on the right side.                                                               | `"images"`  |
| `left_on`  | string  | yes      | Column on the left batch to match on.                                                     | `"case_id"` |
| `right_on` | string  | yes      | Column on the right batch to match on — often `file_id` for file collections.             | `"file_id"` |
| `type`     | enum    | yes      | Join mode: `left` keeps all left rows; `inner` drops non-matching rows.                   | `"left"`    |

### FROM-root order (required for connected joins)

**In plain terms:** think of building one big spreadsheet by gluing smaller ones together. Start with your **main file** (one row per task — e.g. `labels.csv`) and list it **first**. Each join then **glues a new file onto what you already have**: `left` is the file you already have, `right` is the file you're adding. Always glue new files onto the main one — never the other way around. If you flip it, the tool can't attach your main file and ingest fails.

Ingest builds SQL with the **first** `[[ingestion.sources]]` entry as the `FROM` root (one row per review item — map `id` from this source). Each `[[ingestion.joins]]` row **adds** `right` as a new table; `left` must already be in the FROM chain.

Getting this backwards (e.g. `left = "images"`, `right = "labels"` when labels is primary) produces ingest SQL where the primary table never enters `FROM`.

```toml
# Primary CSV first, then join enrichment
[[ingestion.sources]]
id = "labels"
type = "csv"
path = "labels.csv"

[[ingestion.sources]]
id = "images"
type = "file_collection"
path_glob = "images/*.jpg"

[[ingestion.joins]]
left = "labels"      # primary / already in FROM
right = "images"     # new table
left_on = "image_id"
right_on = "file_id"
type = "left"
```

## `ingestion.fields`

`[ingestion.fields]` is **one flat table** that defines every column on each review item. Each **key** is the column name used everywhere after ingest; each **value** is where the data comes from: `<source_id>.<column>`.

If you used `[[ingestion.joins]]` to merge multiple sources, values can reference columns from any joined source.

After ingest, those columns are stored on each review item in the database. See [The Data Lifecycle](/reference/poq-toml#the-data-lifecycle-from-files-to-review).

**Every project must include exactly one key named `id`.** That key is the unique identifier for each review task.

Ten fields need about ten lines — not thirty repeating two-key blocks.

### Worked example: JSON findings (`finding-004.json`)

Example file: `datasets/test-audit-contract/findings/finding-004.json` in `poq-monorepo`. One JSON file = one review item.

**Step 1 — declare the input** (gives you the `findings.` prefix):

```toml

[[ingestion.sources]]
id = "findings"
type = "json"
path_glob = "findings/*.json"
```

**Step 2 — map JSON keys to review-item columns.** Each row below becomes one line in `[ingestion.fields]`. The value uses the JSON property name after ingest; the key is what you use everywhere else — and what lands in `datapoint.canonical_fields`.

| JSON key in file   | Value in `[ingestion.fields]` | Suggested key       | Value in `finding-004.json`                |
| ------------------ | ------------------------------ | -------------------- | ------------------------------------------- |
| `id`               | `findings.id`                  | `id`                  | `"F-04"`                                    |
| `title`            | `findings.title`               | `finding_title`       | `"Centralization risk — single EOA owner"` |
| `description`      | `findings.description`         | `description`         | (markdown narrative)                        |
| `sourcePath`       | `findings.sourcePath`          | `source_path`         | `"src/Counter.sol"`                         |
| `lineNumber`       | `findings.lineNumber`          | `line_number`         | `7`                                          |
| `proposedSeverity` | `findings.proposedSeverity`    | `proposed_severity`   | `"low"`                                      |
| `repository`       | `findings.repository`          | `repository`          | repo URL                                     |
| `commitSha`        | `findings.commitSha`           | `commit_sha`          | pinned commit hash                           |

```toml

[ingestion.fields]
id                = "findings.id"
finding_title     = "findings.title"
description       = "findings.description"
source_path       = "findings.sourcePath"
line_number       = "findings.lineNumber"
proposed_severity = "findings.proposedSeverity"
repository        = "findings.repository"
commit_sha        = "findings.commitSha"
```

**Step 3 — use these keys in later sections.** Reference `[ingestion.fields]` keys only — never raw `<source_id>.<column>` paths outside that table.

For example, to show the finding's description to a validator:

```toml
[[validation.evidence]]
type = "markdown"
ingestion_field = "description"
```

Or to route tasks based on severity:

```toml
[[validators.routes]]
match = { proposed_severity = "low" }
total = 5
```

You should never reference the original source (like `findings.sourcePath` or `findings.proposedSeverity`) outside of `[ingestion.fields]`.

**CSV + images example** (two inputs, no JSON):

```toml
[ingestion.fields]
id         = "findings.case_id"
image_path = "images.path"
```

| Key (table)  | Value (ingest wiring) | Required | Description                                                                                                                                                                                        |
| ------------ | ---------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| _(each key)_ | string                  | yes      | **Key** — column name used everywhere else (`ingestion_field`, route `match` keys). One key must be `id`. **Value** — `<source_id>.<column>`. After joins, values may reference any joined source. |

---

# poq.toml — validation

URL: https://sapien-ai-docs-production.up.railway.app/reference/poq-toml/validation
Source: docs/current/reference/poq-toml/validation.md

# Validation (`validation.*`)

One of the [poq.toml spec](/reference/poq-toml) sections.

What validators interact with and how their answers are scored: task UI, evidence, rubric, dimension-level consensus outcomes, ground truth, and onboarding. Who does the reviewing — counts, pay, classes, routing, claim, and qualification gates — lives in the [Validators](/reference/poq-toml/validators) section.

- [`validation.evidence`](#validationevidence) (required) — What validators see on the task page.
- [`validation.rubric`](#validationrubric) (required) — Questions validators answer.
- [`validation.answer_key`](#validationanswer_key) (optional) — Hidden correct-answer fields by rubric dimension.
- [`validation.reported_label`](#validationreported_label) (optional) — Provided values by rubric dimension, shown to validators or used as a hidden quality-distance reference.
- [`validation.instructions`](#validationinstructions) (optional) — Onboarding slides. _Currently not rendered._

Values of `ingestion_field = "..."` are always keys from `[ingestion.fields]` unless this doc marks them as literal UI copy. Keys on `source_excerpt` / `source_link` blocks (`repository`, `path`, `commit_sha`, …) also hold `[ingestion.fields]` key names.

## `validation.evidence`

Each `[[validation.evidence]]` block is **one thing validators see** on the task page (an image, markdown text, a source link, a fact table, etc.). Blocks resolve **per-row** values from `[ingestion.fields]` only — do not put static markdown or catalogue text in evidence (use `[[validation.instructions]]` for fixed validator guidance).

When a validator opens a task, each evidence block says which fields to show and how.

Every block starts with `type`. After that, keys on the block either map renderer inputs from `[ingestion.fields]` or provide literal renderer config. Values on the right of `ingestion_field` are always `[ingestion.fields]` key names, not file paths or raw JSON properties; parameterized evidence sections below mark which sibling keys are field keys versus literal/config values.

Optional **`title`** on any evidence block sets the panel heading above that widget. The value is **literal display text** (not an `[ingestion.fields]` key). Omit `title` to render no heading — there are no default headings per type.

Pick the evidence blocks that match your data. Order in the file is display order.

### Worked example: one task page, every evidence type

This example builds on the [JSON findings walkthrough](/reference/poq-toml/ingestion#worked-example-json-findings-finding-004json). First map the columns you need in `[ingestion.fields]`:

```toml

[ingestion.fields]
description       = "findings.description"
image_path        = "images.path"
finding_json      = "findings.payload"
repository        = "findings.repository"
source_path       = "findings.sourcePath"
commit_sha        = "findings.commitSha"
line_number       = "findings.lineNumber"
finding_title     = "findings.title"
proposed_severity = "findings.proposedSeverity"
```

Add **one `[[validation.evidence]]` block per widget**. Order in the file is display order (markdown and images tend to appear first).

```toml

[[validation.evidence]]
type = "markdown"
title = "Finding"
ingestion_field = "description"

[[validation.evidence]]
type = "image"
title = "Dermoscopy image"
ingestion_field = "image_path"

[[validation.evidence]]
type = "json_finding"
ingestion_field = "finding_json"

[[validation.evidence]]
type = "source_excerpt"
repository = "repository"
path = "source_path"
commit_sha = "commit_sha"

[[validation.evidence]]
type = "source_link"
repository = "repository"
path = "source_path"
commit_sha = "commit_sha"
line_number = "line_number"
label = "Open in GitHub"

[[validation.evidence]]
type = "datapoint_facts"
fields = [
  { label = "Title", field = "finding_title" },
  { label = "Severity", field = "proposed_severity" },
]
```

You rarely need every type on one project — pick the blocks that match your data. A melanoma project might only declare the `image` block; an audit might use `markdown`, `source_excerpt`, and `datapoint_facts`.

**How to read a block:** `repository = "repository"` means “read the `repository` column from this task’s row.” The string on the right is a `[ingestion.fields]` key, not a GitHub URL literal.

### Evidence types

| `type`                 | What validators see                   | Wiring pattern                                |
| ---------------------- | -------------------------------------- | ----------------------------------------------- |
| `markdown`             | Rich text (finding narrative)          | `ingestion_field`                               |
| `image`                | Image from a URL column                | `ingestion_field`                               |
| `json_finding`         | JSON payload (inline or URL)           | `ingestion_field`                               |
| `source_excerpt`       | Pinned GitHub file in the panel        | flat keys on the block                          |
| `source_link`          | Link out to GitHub file + line         | flat keys on the block                          |
| `datapoint_facts`      | Label/value fact table                 | `fields = [{ label, field }, …]` on the block   |
| `image_comparison`     | Two images side-by-side / slider       | flat keys on the block                          |
| `bounding_box_overlay` | Image with labelled boxes drawn on it  | `image` + `[[…boxes]]` array on the block       |
| `video_clip`           | Inline video player (optional trim)    | flat keys on the block                          |
| `prompt_response_pair` | Prompt + model response(s) for RLHF    | flat keys on the block                          |

Shared on **all** types: optional `title` (literal panel heading; omit for no heading).

The first three (`markdown`, `image`, `json_finding`) bind a single column with `ingestion_field`. The rest are **structured renderers** that bind several named keys directly on the block (and reject the single-`ingestion_field` form); each is documented below.

### Single-column evidence (`image`, `markdown`, `json_finding`)

One column from the row — set `type` and `ingestion_field`:

```toml

[[validation.evidence]]
type = "markdown"
ingestion_field = "description"
```

| option            | accepts | required | description                                                     | example         |
| ----------------- | ------- | -------- | --------------------------------------------------------------- | ---------------- |
| `type`            | enum    | yes      | `image`, `markdown`, or `json_finding`.                         | `"markdown"`      |
| `ingestion_field` | string  | yes      | The `[ingestion.fields]` key whose cell value feeds the widget. | `"description"`   |
| `title`           | string  | no       | Literal panel heading; omit for no heading.                     | `"Finding"`       |

### Parameterized rich evidence

These renderer types use sibling keys on the evidence block instead of `ingestion_field`. Values marked as **field key** must name a key from `[ingestion.fields]`; values marked as **literal** are copied as UI/config text.

| `type`                 | Required field keys         | Optional field keys | Literal/config keys                                     |
| ---------------------- | ---------------------------- | --------------------- | --------------------------------------------------------- |
| `image_comparison`     | `left_image`, `right_image`  | none                  | optional `left_label`, `right_label`, `mode`               |
| `bounding_box_overlay` | `image`, `boxes[].x/y/w/h`   | `boxes[].score`       | required `boxes[].label`; optional `boxes[].kind`          |
| `video_clip`           | `video_url`                  | `poster`              | optional `captions_url`, `start_seconds`, `end_seconds`    |
| `prompt_response_pair` | `prompt`, `response`         | `alt_response`        | optional `model`, `params`                                 |

```toml

[[validation.evidence]]
type = "image_comparison"
left_image = "before_image_url"
right_image = "after_image_url"
left_label = "Before"
right_label = "After"
mode = "side_by_side"

[[validation.evidence]]
type = "bounding_box_overlay"
image = "image_url"
boxes = [
  { label = "lesion", x = "box_x", y = "box_y", w = "box_w", h = "box_h", score = "box_score" },
]

[[validation.evidence]]
type = "video_clip"
video_url = "video_url"
poster = "poster_url"
start_seconds = 5
end_seconds = 20

[[validation.evidence]]
type = "prompt_response_pair"
prompt = "prompt_text"
response = "model_response"
alt_response = "reference_response"
model = "candidate-v1"
```

For `image_comparison`, `mode` must be one of `side_by_side`, `slider`, or `diff`. For `bounding_box_overlay`, `boxes` must contain at least one row; `label`, `x`, `y`, `w`, and `h` are required on each row. `boxes[].kind` may be either literal text or an `[ingestion.fields]` key.

### `source_excerpt`

Fetches and displays source from a public GitHub repo at a pinned commit (in-panel viewer):

```toml

[[validation.evidence]]
type = "source_excerpt"
repository = "repository"
path = "source_path"
commit_sha = "commit_sha"
```

| option       | accepts | required                 | description                                                                            | example            |
| ------------ | ------- | ------------------------ | ---------------------------------------------------------------------------------------- | ------------------- |
| `type`       | enum    | yes                      | Must be `source_excerpt`.                                                                | `"source_excerpt"`  |
| `title`      | string  | no                       | Literal panel heading; omit for no heading.                                              | `"Source"`          |
| `repository` | string  | yes                      | `[ingestion.fields]` key for repo URL or `owner/repo`.                                   | `"repository"`      |
| `path`       | string  | one of `path` or `paths` | `[ingestion.fields]` key for one file path in the repo.                                  | `"source_path"`     |
| `paths`      | string  | one of `path` or `paths` | `[ingestion.fields]` key whose cell is an **array** of paths (UI shows a file picker).   | `"source_paths"`    |
| `commit_sha` | string  | no                       | `[ingestion.fields]` key for commit hash; UI defaults to `main` when empty.               | `"commit_sha"`      |

Use `path` or `paths`, not both.

### `source_link`

Compact link to a pinned GitHub file (opens in a new tab). Verifies the path exists at the commit before linking.

```toml

[[validation.evidence]]
type = "source_link"
repository = "repository"
path = "source_path"
commit_sha = "commit_sha"
line_number = "line_number"
label = "Open in GitHub"
```

| option        | accepts | required | description                                                                        | example            |
| ------------- | ------- | -------- | ------------------------------------------------------------------------------------ | ------------------- |
| `type`        | enum    | yes      | Must be `source_link`.                                                               | `"source_link"`     |
| `title`       | string  | no       | Literal panel heading; omit for no heading.                                          | `"Open source"`     |
| `repository`  | string  | yes      | `[ingestion.fields]` key for repo URL or `owner/repo`.                               | `"repository"`      |
| `path`        | string  | yes      | `[ingestion.fields]` key for file path in the repo.                                   | `"source_path"`     |
| `commit_sha`  | string  | yes      | `[ingestion.fields]` key for pinned commit hash.                                      | `"commit_sha"`      |
| `line_number` | string  | no       | `[ingestion.fields]` key for line number (GitHub line anchor).                        | `"line_number"`     |
| `label`       | string  | no       | Literal button copy, or a `[ingestion.fields]` key when label text varies per row.    | `"Open in GitHub"`  |

### `datapoint_facts`

Structured label/value rows (severity, id, found-by, etc.). Declare `fields` as an inline array on the block — each element has `label` (UI copy) and `field` (an `[ingestion.fields]` key):

```toml

[[validation.evidence]]
type = "datapoint_facts"
title = "Finding facts"
fields = [
  { label = "Title", field = "finding_title" },
  { label = "Severity", field = "proposed_severity" },
]
```

| option           | accepts         | required                    | description                                                          | example               |
| ---------------- | --------------- | ---------------------------- | ---------------------------------------------------------------------- | ---------------------- |
| `type`           | enum            | yes                          | Must be `datapoint_facts`.                                             | `"datapoint_facts"`     |
| `title`          | string          | no                            | Literal panel heading above the fact table; omit for none.             | `"Finding facts"`       |
| `fields`         | array of tables | yes                           | Ordered rows shown in the fact table. At least one element required.   | `[{ label, field }]`    |
| `fields[].label` | string          | yes (each `fields` element)   | Fixed heading beside the value — UI copy, not a column.                | `"Severity"`            |
| `fields[].field` | string          | yes (each `fields` element)   | `[ingestion.fields]` key whose cell value is shown.                    | `"proposed_severity"`   |

### `image_comparison`

Two images shown together (side-by-side, slider, or diff). `left_image` / `right_image` are `[ingestion.fields]` keys; `mode` is a literal or a per-row field.

```toml

[[validation.evidence]]
type = "image_comparison"
left_image = "before_url"
right_image = "after_url"
left_label = "Before"
right_label = "After"
mode = "slider"
```

| option        | accepts | required | description                                                             | example              |
| ------------- | ------- | -------- | ------------------------------------------------------------------------ | --------------------- |
| `type`        | enum    | yes      | Must be `image_comparison`.                                              | `"image_comparison"`  |
| `left_image`  | string  | yes      | `[ingestion.fields]` key for the left/first image URL.                   | `"before_url"`        |
| `right_image` | string  | yes      | `[ingestion.fields]` key for the right/second image URL.                  | `"after_url"`         |
| `left_label`  | string  | no       | Literal caption or `[ingestion.fields]` key for the left image.          | `"Before"`            |
| `right_label` | string  | no       | Literal caption or `[ingestion.fields]` key for the right image.          | `"After"`             |
| `mode`        | enum    | no       | One of `side_by_side`, `slider`, `diff`, or a `[ingestion.fields]` key.   | `"slider"`            |

### `bounding_box_overlay`

An image with one or more labelled boxes drawn over it. `image` is a `[ingestion.fields]` key; declare each box as a `[[…boxes]]` table whose coordinate keys (`x`, `y`, `w`, `h`, optional `score`) are `[ingestion.fields]` keys.

```toml

[[validation.evidence]]
type = "bounding_box_overlay"
image = "frame_url"
[[validation.evidence.boxes]]
label = "Detection"
x = "box_x"
y = "box_y"
w = "box_w"
h = "box_h"
score = "box_score"
```

| option            | accepts         | required | description                                                        | example                  |
| ----------------- | --------------- | -------- | --------------------------------------------------------------------- | -------------------------- |
| `type`            | enum            | yes      | Must be `bounding_box_overlay`.                                      | `"bounding_box_overlay"`   |
| `image`           | string          | yes      | `[ingestion.fields]` key for the base image URL.                     | `"frame_url"`              |
| `boxes`           | array of tables | yes      | At least one box. Each is a `[[validation.evidence.boxes]]` table.    | see example above          |
| `boxes[].label`   | string          | yes      | Literal box label (UI copy).                                          | `"Detection"`              |
| `boxes[].x/y/w/h` | string          | yes      | `[ingestion.fields]` keys for box origin and size.                     | `"box_x"`                  |
| `boxes[].score`   | string          | no       | `[ingestion.fields]` key for a confidence score.                       | `"box_score"`              |
| `boxes[].kind`    | string          | no       | Literal class name or `[ingestion.fields]` key.                       | `"false_positive"`         |

### `video_clip`

Inline video player, optionally trimmed to a span. `video_url` / `poster` are `[ingestion.fields]` keys; trim bounds are numeric literals.

```toml

[[validation.evidence]]
type = "video_clip"
video_url = "clip_url"
poster = "thumbnail_url"
start_seconds = 12
end_seconds = 30
```

| option          | accepts | required | description                                                | example           |
| --------------- | ------- | -------- | ------------------------------------------------------------ | ------------------- |
| `type`          | enum    | yes      | Must be `video_clip`.                                        | `"video_clip"`      |
| `video_url`     | string  | yes      | `[ingestion.fields]` key for the video URL.                   | `"clip_url"`        |
| `poster`        | string  | no       | `[ingestion.fields]` key for a poster/thumbnail image.        | `"thumbnail_url"`   |
| `captions_url`  | string  | no       | Literal URL or `[ingestion.fields]` key for caption track.    | `"captions_url"`    |
| `start_seconds` | number  | no       | Trim start (seconds).                                         | `12`                |
| `end_seconds`   | number  | no       | Trim end (seconds).                                           | `30`                |

### `prompt_response_pair`

A prompt with one or two model responses — for RLHF / preference tasks. `prompt`, `response`, `alt_response` are `[ingestion.fields]` keys.

```toml

[[validation.evidence]]
type = "prompt_response_pair"
prompt = "user_prompt"
response = "response_a"
alt_response = "response_b"
model = "Model A vs B"
```

| option         | accepts | required | description                                                     | example                  |
| -------------- | ------- | -------- | ------------------------------------------------------------------ | -------------------------- |
| `type`         | enum    | yes      | Must be `prompt_response_pair`.                                     | `"prompt_response_pair"`    |
| `prompt`       | string  | yes      | `[ingestion.fields]` key for the prompt text.                       | `"user_prompt"`             |
| `response`     | string  | yes      | `[ingestion.fields]` key for the primary response.                  | `"response_a"`              |
| `alt_response` | string  | no       | `[ingestion.fields]` key for a second response to compare.          | `"response_b"`              |
| `model`        | string  | no       | Literal label or `[ingestion.fields]` key naming the model(s).      | `"Model A vs B"`            |
| `params`       | string  | no       | Literal text or `[ingestion.fields]` key for generation params.     | `"temp=0.7"`                |

## `validation.rubric`

Each `[[validation.rubric]]` block is **one question** validators answer on the task page — its label, answer buttons (`scale.`), and how that single question is scored.

This section is only about **questions**. Overall finding rollup is fixed in code as strict AND over required dimensions.

```toml

[[validation.rubric]]
id = "severity"
label = "Severity"
prompt = "How severe is this finding?"
scale.type = "ordinal"
scale.labels = ["info", "low", "medium", "high", "critical"]
consensus_weight = 1.0
```

Add one `[[validation.rubric]]` block per question. Reference dimensions elsewhere by `id` (for example route `match` keys, `consensus_skip.match` applicability rules, or ground-truth wiring).

### Rubric dimension fields

Each `[[validation.rubric]]` block sets **question copy**, **scale** (`scale.`), optional **consensus tuning** (`consensus_strength_required`, `consensus_skip.match`), and **consensus weight** on the same dimension.

| option                            | accepts               | required                                       | description                                                                                                                                                                                                                                                                                                                         | example                               |
| --------------------------------- | --------------------- | ---------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------- |
| `id`                              | string                | yes                                            | Internal name for this question — unique within the rubric.                                                                                                                                                                                                                                                                         | `"severity"`                          |
| `label`                           | string                | yes                                            | Heading validators see above the answer buttons.                                                                                                                                                                                                                                                                                    | `"Severity"`                          |
| `prompt`                          | string                | no                                             | Extra question text under the label.                                                                                                                                                                                                                                                                                                | `"How severe is this finding?"`       |
| `tooltip`                         | string                | no                                             | Help text below the buttons (visible, not hover-only).                                                                                                                                                                                                                                                                              | `"Match severity to OWASP guidance."` |
| `role`                            | string                | no                                             | Tag for UI affordances (e.g. `influence_gauge`, `certainty`).                                                                                                                                                                                                                                                                        | `"influence_gauge"`                   |
| `scale.type`                      | enum                  | yes                                            | Answer template: `likert`, `ordinal`, `numeric`.                                                                                                                                                                                                                                                                                    | `"ordinal"`                           |
| `scale.size`                      | int                   | yes (likert only)                              | Likert point count — must be `5` or `7` when `scale.type` is `likert`.                                                                                                                                                                                                                                                              | `5`                                   |
| `scale.labels`                    | array of strings      | yes (ordinal); no (likert); optional (numeric) | Button labels. Must match `scale.size` on likert dimensions if set.                                                                                                                                                                                                                                                                 | `["low", "medium", "high"]`           |
| `scale.values`                    | array of numbers      | yes (numeric)                                  | Stored scores (0–100).                                                                                                                                                                                                                                                                                                              | `[0, 50, 100]`                        |
| `consensus_strength_required`     | number                | no                                             | Minimum consensus strength in `[0, 1]` for `consensus_reached` on this dimension. Defaults to **0.70** when omitted. See [Dimension-level verdict](#dimension-level-verdict).                                                                                                                                                     | `0.85`                                |
| `consensus_skip.match`            | map (row id → labels) | no                                             | Validator-UI applicability guard: referenced dimension ids → trigger labels. When a validator's own answer on every referenced dimension matches, this dimension greys out and becomes optional; unanswered selections are backfilled with the bottom-of-scale filler, which counts toward consensus. Does **not** change the engine outcome. | `{ validity = ["False positive"] }`   |
| `consensus_weight`                | number                | no (default `1.0`)                             | How much this dimension counts in the finding strength blend. `0` excludes the dimension (advisory only).                                                                                                                                                                                               | `1.0`                                 |

Do not rename `consensus_weight` to `importance`. Removed keys `verdict.type`, `verdict.contested_rule.*`, `verdict.match`, and `verdict.verified_threshold` on rubric rows fail parse.

### The scale block (`scale.`)

There is no separate top-level `[scale]` section. On each `[[validation.rubric]]` block, dotted keys (`scale.type`, `scale.size`, …) define the **answer buttons** validators see and the **0–100 number** stored when they pick one.

At compile time the server turns every scale into a list of **(label, value)** pairs. Validators click labels; consensus and storage always use the numeric values. Different dimensions in the same project can use different scale types (e.g. severity as `ordinal`, confidence as `numeric`, diagnosis as 7-point Likert). These values are also the dimension's quality direction — the label mapped to `100` reads as "top quality" — see [Quality Rating vs. `scale.values`](#dimension-level-verdict) below.

#### `scale.type` — which answer template

**Required.** Picks one of three templates:

| `scale.type` | Use for                                                                                                   | Also set on the row                                                                    |
| ------------ | ----------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
| `likert`     | Symmetric agree/disagree or frequency scales (“Strongly disagree” … “Strongly agree”, “Never” … “Always”)   | `scale.size` = `5` or `7`; optional `scale.labels` (must match size)                       |
| `ordinal`    | Your own ordered categories (severity, validity, …)                                                          | **Required** `scale.labels` — one label per button; count sets the number of options      |
| `numeric`    | Explicit 0–100 steps you choose (e.g. certainty 0/25/50/75/100)                                              | **Required** `scale.values`; optional matching `scale.labels`                              |

For `ordinal` and `likert` types, the compiler **auto-generates** evenly spaced 0–100 values from the label count or `scale.size`. For `numeric`, you **declare** the stored scores in `scale.values`.

**AI validator classes:** When any `[[validators.classes]]` row has `type = "ai"`, every rubric scale must resolve to **integer-only** anchors (the AI worker constrains model output to integers per dimension). Practical rules: `likert` with `scale.size = 5` is fine; `scale.size = 7` produces fractional steps and is rejected. `ordinal` label counts must satisfy `(count - 1)` divides 100 (e.g. five labels). `numeric` scales must use integer `scale.values`. See [AI agent classes](/reference/poq-toml/validators#ai-agent-classes) for class fields.

#### Examples by scale type

**Ordinal (multi-bucket)** — labels required; values auto-spaced:

```toml

[[validation.rubric]]
id = "severity"
label = "Severity"
scale.type = "ordinal"
scale.labels = ["info", "low", "medium", "high", "critical"]
```

**Ordinal (binary yes/no)** — still `scale.type = "ordinal"`, but exactly **two** labels:

```toml

[[validation.rubric]]
id = "is_valid"
label = "Is this finding valid?"
scale.type = "ordinal"
scale.labels = ["no", "yes"]
```

**Conditional dimension — grey out when the validator's answer on another dimension matches** — use `consensus_skip.match` on the dependent dimension:

```toml

[[validation.rubric]]
id = "validity"
label = "Validity"
prompt = "Is this finding a real issue?"
scale.type = "ordinal"
scale.labels = ["False positive", "Unlikely valid", "Unclear", "Likely valid", "Clearly valid"]

[[validation.rubric]]
id = "fix_soundness"
label = "Fix soundness"
prompt = "How sound is the proposed fix?"
scale.type = "ordinal"
scale.labels = ["Unsound", "Weak", "Partial", "Mostly sound", "Fully sound"]
consensus_skip.match = { validity = ["False positive"] }
```

When a validator selects **False positive** on Validity, **Fix soundness** greys out in the UI and becomes optional. If they leave it unanswered, the server backfills the bottom-of-scale filler; that vote still counts toward consensus. The engine assigns **`consensus_reached`** or **`consensus_not_reached`** on every dimension — `consensus_skip.match` does not bypass consensus — see [Dimension-level verdict](#dimension-level-verdict).

**Likert** — size required; labels optional:

```toml

[[validation.rubric]]
id = "diagnosis_match"
label = "Does the diagnosis match?"
scale.type = "likert"
scale.size = 7
```

**Numeric** — values required; labels optional:

```toml

[[validation.rubric]]
id = "certainty"
label = "How certain are you?"
scale.type = "numeric"
scale.values = [0, 25, 50, 75, 100]
scale.labels = ["none", "low", "medium", "high", "certain"]

[[validation.rubric]]
id = "confidence"
label = "Confidence Level"
prompt = "How confident are you in this assessment?"
scale.type = "numeric"
scale.values = [0, 20, 80, 100]
scale.labels = ["Guessing", "Low", "High", "Certain"]
```

### Dimension-level verdict

Each `[[validation.rubric]]` block can carry optional `consensus_strength_required` and `consensus_skip.match`. After validators submit, the engine assigns **one of two outcomes per dimension**, then rolls those up into a single finding outcome with a fixed **strict AND** over every required dimension (`consensus_weight > 0`).

#### Dimension outcomes (runtime)

These are computed — not TOML keys you set. The engine produces exactly two:

| outcome                 | Meaning                                                                                                            |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------ |
| `consensus_reached`     | Consensus strength meets the dimension's `consensus_strength_required` (default **0.70**).                         |
| `consensus_not_reached` | Consensus strength is below the threshold, or there is no resolvable consensus label (e.g. no valid votes).        |

There is no `consensus_skipped` engine outcome. `consensus_skip.match` affects only the validator UI (greyed-out, optional dimension); filler votes from self-skipping validators count toward consensus like any other vote.

#### How each dimension is classified

1. If there are no valid votes or no resolvable consensus label on this dimension → `consensus_not_reached`.
2. If consensus strength meets the dimension's `consensus_strength_required` (default **0.70**) → `consensus_reached`.
3. Otherwise → `consensus_not_reached`.

**Consensus strength** is the strongest distance-weighted support for any configured scale label — _not_ a raw vote share. Votes at a candidate label contribute **1.0**, votes at the farthest label contribute **0.0**, and nearer labels contribute partial support by distance; the consensus label is the candidate with the highest weighted support. Ties prefer the label nearest the weighted mean, then the higher scale value.

**Finding rollup is fixed in code** as strict AND: a finding is `verified` only when every required dimension (`consensus_weight > 0`) reaches `consensus_reached`. Dimensions at `consensus_weight = 0` are advisory and never gate the finding. There is no `[validation.verdict]` table — per-dimension `consensus_strength_required` is the only tuning knob.

#### Optional dimension consensus keys

| option                        | accepts               | required | description                                                                                                                                                                                                                                                              | example                             |
| ----------------------------- | --------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------- |
| `consensus_strength_required` | number                | no       | Minimum consensus strength in `[0, 1]` for `consensus_reached` on this dimension. Defaults to **0.70** when omitted.                                                                                                                                                     | `0.85`                              |
| `consensus_skip.match`        | map (row id → labels) | no       | Validator-UI guard only: referenced **dimension id**s → trigger **labels** (scalar or array). Greys out this dimension when the validator's own answers on every referenced dimension match. Filler votes count toward consensus; the engine still classifies the dimension as reached or not. | `{ validity = ["False positive"] }` |

Removed keys (parse error if present): `verdict.type`, `verdict.contested_rule.*`, `verdict.match`, `verdict.verified_threshold` on rubric dimensions, and the `[validation.verdict]` / `[stage.verdict]` tables.

**Quality Rating vs. `scale.values`.** Consensus Strength above measures **agreement** (how tightly the panel clustered) and is direction-agnostic. The **Quality Rating** is the other reported pillar: a directional 0–100 score, the consensus-weight-weighted mean of the scale **values** the panel actually picked. It answers _where on the scale_ the answer landed, so **the label you map to `100` reads as "top quality" and the one at `0` as "low quality."** That direction is set by `scale.values`:

- `numeric`: the direction is exactly the order of your `scale.values`. To make the first label the high-quality end, give it the higher value — e.g. `scale.values = [100, 0]` with `scale.labels = ["A", "B"]` makes **A** top quality; `[0, 100]` makes **B** top quality.
- `ordinal` / `likert`: values are auto-generated evenly across 0–100 by position, so the **last** label is `100` (top) and the **first** is `0`. Order the labels low→high, or switch to `numeric` if you need an order-independent mapping.

Reversing this direction does not change which label wins or whether a dimension reaches consensus (both key off agreement/dispersion) — only the directional Quality Rating flips. To score a dimension by **distance to a reference value** instead of its raw value, use `[[validation.reported_label]]` with `quality = true`. See [Model-Prediction Agreement](/examples/poq-toml#model-prediction-agreement) for a worked spec.

## `validation.answer_key`

Use `[[validation.answer_key]]` when a dataset includes hidden correct answers for one or more rubric dimensions. Each row maps a rubric dimension id to an `[ingestion.fields]` key. Answer-key fields are stripped before validator-facing canonical fields are returned, including AI validator payloads.

Skip when the project has no answer key (for example open-ended preference data).

```toml

[[validation.answer_key]]
dimension = "diagnosis"
field = "correct_label"

[[validation.answer_key]]
dimension = "severity"
field = "expert_severity"
```

| option      | accepts | required | description                                                           | example             |
| ----------- | ------- | -------- | ------------------------------------------------------------------------ | --------------------- |
| `dimension` | string  | yes      | `id` of the `[[validation.rubric]]` dimension this answer belongs to.    | `"diagnosis"`         |
| `field`     | string  | yes      | `[ingestion.fields]` key holding the hidden correct answer.              | `"correct_label"`     |

## `validation.reported_label`

Use `[[validation.reported_label]]` when a dataset includes a value validators should see next to a rubric dimension, such as a model prediction or source-provided value. The validator UI labels this as **Provided value**. By default it never auto-submits a rubric answer, but the flags below can pre-select an option or repurpose the value as a hidden quality reference.

```toml

[[validation.reported_label]]
dimension = "diagnosis"
field = "model_prediction"
default_selection = true            # pre-select the mapped scale option
map = { M = "malignant", B = "benign" }   # stored code -> scale label

# Quality-reference form: the reported value scores the dimension's Quality
# Rating by distance, and is stripped from everything validators can see.
[[validation.reported_label]]
dimension = "severity"
field = "severity_in_report"
quality = true
```

| option              | accepts | required | description                                                                                                                                                                                                                                                                                                                             | example                |
| -------------------- | ------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------ |
| `dimension`          | string  | yes      | `id` of the `[[validation.rubric]]` dimension where the value is shown.                                                                                                                                                                                                                                                                | `"diagnosis"`            |
| `field`              | string  | yes      | `[ingestion.fields]` key whose datapoint value is shown to users.                                                                                                                                                                                                                                                                       | `"model_prediction"`     |
| `quality`            | bool    | no       | Default `false`. Use the value as the **distance reference** for this dimension's Quality Rating (how close the panel lands to it) instead of surfacing it. Implies **server-only**: the field is stripped from every client-facing response and omitted from the returned config, so validators never see what they're scored against. | `true`                   |
| `default_selection`  | bool    | no       | Default `false`. Pre-select the mapped scale option in the answering UI. Validators can still change it; it never auto-submits.                                                                                                                                                                                                        | `true`                   |
| `map`                | table   | no       | Value→label aliases, used when the field's stored encoding doesn't match a scale label directly. Each alias value must be non-empty.                                                                                                                                                                                                    | `{ M = "malignant" }`    |

**How the Quality Rating is scored.** With `quality = true`, each validator's vote is first converted to its **closeness** to the reference, and those closeness values are combined by consensus weight:

`closeness = 100 * (1 - min(1, |vote - reference| / range))`

`range` is the scale span, `max(scale.values) - min(scale.values)`. The dimension's Quality Rating is the consensus-weight-weighted mean of the per-vote closeness, so a validator landing on the reference scores 100 and one at the farthest end scores 0. For example, on a `[0, 25, 50, 75, 100]` scale with the reference at `100` and panel votes of `100, 100, 75`, closeness is `100, 100, 75` and the Quality Rating is **91.7**. Move the reference to `0` with the same votes and it drops to **8.3**, since the panel now sits far from the provided value. Consensus Strength is unaffected either way; only the directional Quality Rating changes.

The reference is resolved against the dimension's scale: the field value is matched through `map`, then to a scale label (case-insensitive), then to an exact `scale.values` anchor. A value that lands on no anchor (for example `33` on a `0/25/50/75/100` scale) does not error; the dimension quietly reverts to raw-value quality. See [Model-Prediction Agreement](/examples/poq-toml#model-prediction-agreement) for a full spec.

`quality = true` is a **hidden reference**, like [`[[validation.answer_key]]`](#validationanswer_key): both strip the field before validators see it, but they answer different questions. An answer key marks the *correct* label for accuracy scoring, while a quality reference scores *how close* the panel lands to a provided value on the Quality Rating pillar.

## `validation.instructions`

`[[validation.instructions]]` holds **onboarding slides** for validators. **Currently not rendered** — the onboarding modal uses hardcoded copy.

Use **TOML literal strings** (`'''…'''`, triple single quotes) for markdown `content` and any other verbatim user or policy text. Literal strings do not process backslash escapes, so Markdown escapes like `\<NONE\>` in tables parse correctly. **Do not** use basic strings (`"""…"""`) for slide bodies — invalid escapes (e.g. `\<`) fail TOML parse with `invalid escaped character`. Reserve basic strings for short values that need escapes (e.g. `delimiter = "\n\n---\n"` under attestation).

```toml

[[validation.instructions]]
type = "markdown"
content = '''
Rate each finding using the OWASP severity definitions.
'''

[[validation.instructions]]
type = "image"
src = "instructions/severity.png"
caption = "Severity ladder"
```

| option    | accepts | required              | description                                  | example                       |
| --------- | ------- | ---------------------- | ---------------------------------------------- | -------------------------------- |
| `type`    | string  | yes                     | `markdown` or `image`.                         | `"markdown"`                     |
| `content` | string  | yes (markdown slides)   | Markdown slide body; use `'''…'''` literals.   | see example above                |
| `src`     | string  | yes (image slides)      | Image path under the project upload prefix.    | `"instructions/severity.png"`    |
| `caption` | string  | no                      | Caption under an image slide.                  | `"Severity ladder"`              |

The same **`'''…'''` literal** rule applies to **`prompt`** on `[[validators.classes]]` when `type = "ai"` — prompts often embed sample rows or policy text with backslashes.

---

# poq.toml — validators

URL: https://sapien-ai-docs-production.up.railway.app/reference/poq-toml/validators
Source: docs/current/reference/poq-toml/validators.md

# Validators (`validators.*`)

One of the [poq.toml spec](/reference/poq-toml) sections.

Who does the reviewing and under what terms: default validator count and pay, validator classes (human or AI), per-item routing and escalation, claim duration, qualification gates, and task-page actions. Read together, these knobs form the project's [Assurance Policy](#assurance-policy).

- [`validators`](#validators) (required) — Default validator count and pay.
- [`validators.classes`](#validatorsclasses) (optional) — Validator personas (human or AI), pay, and consensus weight.
- [`validators.routes`](#validatorsroutes) (optional) — Route items to validator counts and classes.
- [`validators.qualification`](#validatorsqualification) (optional) — Profile fields required before claim.
- [`validators.claims`](#validatorsclaims) (optional) — Claim TTL before a task expires.
- [`validators.actions`](#validatorsactions) (optional) — Task-page actions (e.g. conflict-of-interest self-decline).

## Assurance Policy

The **Assurance Policy** is everything in the spec that determines how much you can trust a verified datapoint. It is not a separate table — it is the existing `validators.*` and rubric knobs read as one concept:

| What it controls             | Knob                                                                                       | Where                                                                                                                          |
| ----------------------------- | -------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| Who reviews                   | classes, `priority`                                                                          | [`[[validators.classes]]`](#validatorsclasses)                                                                        |
| How many, and on which items  | `num_validators`, routes, escalation                                                          | [`[validators]`](#validators), [`[[validators.routes]]`](#validatorsroutes)                               |
| Who may claim                 | qualification gates, claim TTL                                                                | [`[[validators.qualification]]`](#validatorsqualification), [`[validators.claims]`](#validatorsclaims)    |
| How agreement is judged       | class `consensus_weight`, dimension `consensus_strength_required` + `consensus_weight`        | [`[[validators.classes]]`](#validatorsclasses), [Dimension-level verdict](/reference/poq-toml/validation#dimension-level-verdict)                   |

Run-time quality controls (batching, checkpoints, spot audits, calibration) are part of the same policy conceptually but are **not configurable in poq.toml yet**.

Validators are accountable by **identity**: they are named and qualified before they review, so the audit trail and their reputation are on the line.

## `validators`

**Default validator count and pay.** Declare `[[validators.classes]]` and `[[validators.routes]]` separately when you need personas or item-specific routing.

```toml

[validators]
num_validators = 3
reward_usd = "1.00"
```

| option           | accepts | required           | description                                                                                                       | example  |
| ---------------- | ------- | ------------------- | --------------------------------------------------------------------------------------------------------------------- | -------- |
| `num_validators` | int     | yes                  | Default validators per item when no route matches.                                                                    | `3`      |
| `reward_usd`     | string  | yes                  | Default reward per completed review (quoted USD decimal string; compiled to cents).                                   | `"1.00"` |

## `validators.classes`

Under `[validators.*]`, **classes** are validator personas (pay, consensus weight, and optionally an LLM agent) — not rubric answer types.

Each class is filled either by **humans** claiming slots in the portal (`type = "human"`, the default when `type` is omitted) or by an **AI agent** (`type = "ai"`) backed by a synthetic system user and the server AI worker. Routing, composition, and escalation treat both kinds the same — you can compose multiple AI classes on the default route and escalate to a human senior reviewer when consensus is low.

### Class fields

| option             | accepts                                | required                    | description                                                                              | example                            |
| ------------------ | ---------------------------------------- | ----------------------------- | --------------------------------------------------------------------------------------------- | -------------------------------------- |
| `id`               | string                                    | yes                            | Class id used in routes.                                                                       | `"senior"`                             |
| `label`            | string                                    | no                             | Display name in admin UI.                                                                       | `"Senior auditor"`                     |
| `type`             | `human` or `ai`                           | no (default `human`)           | Who fills slots for this class. See [AI agent classes](#ai-agent-classes).                     | `"ai"`                                  |
| `model`            | string                                     | when `type = "ai"`             | OpenRouter model slug — must be one of [Allowed `model` values](#allowed-model-values).          | `"anthropic/claude-sonnet-4.5"`         |
| `prompt`           | string                                     | when `type = "ai"`             | System prompt sent to the model (max 64 KiB); use `'''…'''` literals for verbatim text.          | see AI example below                    |
| `reward_usd`       | string                                     | yes                             | Per-review reward for this class.                                                                | `"10.00"`                                |
| `priority`         | int                                        | yes                             | Tie-break priority (higher = stronger).                                                          | `10`                                      |
| `consensus_weight` | number                                     | no (default `1.0`)             | Per-class vote weight in weighted consensus.                                                      | `1.0`                                      |

Human example:

```toml
[[validators.classes]]
id         = "senior"
label      = "Senior reviewer"
reward_usd = "20.00"
priority   = 10
```

### AI agent classes

Set `type = "ai"` and declare `model` plus `prompt` to fill a class with an LLM instead of human validators. The worker polls unclaimed validations for that class, packages the same evidence and rubric a human would see, calls the model, and submits via the standard validation path (audit and consensus unchanged).

```toml
[[validators.classes]]
id         = "opus-reviewer"
label      = "Opus 4.7 reviewer"
type       = "ai"
model      = "anthropic/claude-opus-4.7"
prompt     = '''
You are an expert reviewer. Score each rubric dimension using only the discrete
anchors provided. Ground every judgment in the evidence; if evidence is
insufficient, pick the midpoint and say so in the rationale.
'''
priority   = 30
reward_usd = "0.00"
```

**Constraints**

- `model` must be an OpenRouter slug from the allowlist below. Compile-time validation rejects unknown slugs and echoes this list in the error.
- `prompt` is required for `type = "ai"` and is sent verbatim as the system message (max 64 KiB).
- `type = "human"` (or omitted `type`) must **not** set `model` or `prompt` — misplaced AI-only fields are rejected.
- AI classes have no wallet. `reward_usd` defaults to `"0.00"` when omitted.
- `consensus_weight` defaults to `1.0` unless you explicitly weight AI votes differently.
- **Rubric scales must use integer-only anchors** when any AI class is declared (see [The scale block](/reference/poq-toml/validation#the-scale-block-scale)).

#### Allowed `model` values

Each slug is an [OpenRouter](https://openrouter.ai/models) model id. All entries support multimodal evidence (images in messages) and strict JSON-schema `response_format` for rubric vectors.

| `model` slug                       | Context (tokens) | Notes                                                                              |
| ------------------------------------ | ------------------ | ------------------------------------------------------------------------------------- |
| `anthropic/claude-haiku-4.5`         | 200,000             | Lowest-latency Anthropic option; good for high-volume first-pass classes.             |
| `anthropic/claude-opus-4.6`          | 200,000             | Prior Opus generation; kept for reproducibility of votes pinned to this version.       |
| `anthropic/claude-opus-4.7`          | 200,000             | Latest Opus; highest-quality reasoning; high-stakes or final-step reviewers.           |
| `anthropic/claude-sonnet-4.5`        | 200,000             | Default for most AI reviewer classes — strong vision, fast, cheaper than Opus.         |
| `google/gemini-2.5-flash`            | 1,000,000           | Fast and cheap with full multimodal support.                                           |
| `google/gemini-2.5-pro`              | 1,000,000           | Long context, strong vision; best when evidence payloads are large.                    |
| `x-ai/grok-4.3`                      | 1,000,000           | xAI flagship; vision + structured outputs; vendor diversity.                           |
| `meta-llama/llama-4-maverick`        | 1,048,576           | Open-weight multimodal flagship; useful for diversity in multi-agent compositions.     |
| `mistralai/mistral-medium-3.1`       | 131,072             | European-hosted multimodal option.                                                     |
| `qwen/qwen3-vl-235b-a22b-instruct`   | 131,072             | Open-weight multimodal (Alibaba); broad provider coverage.                             |
| `openai/gpt-4o`                      | 128,000             | Strong multimodal generalist; diversity in multi-agent compositions.                   |
| `openai/gpt-4o-mini`                 | 128,000             | Cheap, fast multimodal; first-pass or budget-bound classes.                            |

Example route mixing AI panel + human escalation:

```toml
[[validators.routes]]
total = 3
[[validators.routes.composition]]
class = "opus-reviewer"
count = 1
[[validators.routes.composition]]
class = "sonnet-reviewer"
count = 1
[[validators.routes.composition]]
class = "maverick-reviewer"
count = 1

[[validators.routes.escalation]]
add = 1
[[validators.routes.escalation.composition]]
class = "senior"
count = 1
```

Fixture: `server/internal/projectspec/testdata/validators_ai.poq.toml`. Design notes: issue #633; implementation `server/internal/aivalidator`.

## `validators.routes`

Under `[validators.*]`, **routes** assign validator counts and class slots to matching items — not HTTP or API routes.

### Route fields

Use `match` to select items by `[ingestion.fields]` column names. A key may be a **scalar** (exact match) or an **array** (any listed value). Omit `match` on exactly one catch-all route (or use `match = {}`).

```toml

[[validators.routes]]
match = { proposed_severity = ["high", "critical"], image_type = "skin_lesion" }
total = 5
[[validators.routes.composition]]
class = "senior"
count = 2

[[validators.routes]]
total = 3   # catch-all — no match

[[validators.routes.escalation]]
add = 2
```

Route `match` keys must match a `[ingestion.fields]` key. Escalation steps take no `match` — see below.

| option        | accepts         | required                                | description                                                                                                    | example                            |
| ------------- | ----------------- | ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------ | ------------------------------------- |
| `match`       | table             | no (omit on exactly one default route)     | Match review items: `[ingestion.fields]` key → scalar or array of allowed values.                                   | `match = { difficulty = "hard" }`     |
| `total`       | int               | yes                                         | Validators per matching item.                                                                                       | `5`                                     |
| `composition` | array of tables   | yes                                         | Class slots: `{ class, count }`. Use `class = "*"` for any class.                                                    | `{ class = "senior", count = 2 }`      |
| `escalation`  | array of tables   | no                                          | Ordered ladder of extra-validator steps; each fires while the datapoint is not yet verified. No `match` key.         | `add = 2`, `composition = [...]`       |

## `validators.qualification`

Validators fill out **`[[validators.qualification]]`** before claiming items.

```toml

[[validators.qualification]]
key = "audit_firm"
label = "Audit firm"
type = "text"
required = true
```

| option     | accepts | required               | description                                | example        |
| ---------- | ------- | ------------------------ | --------------------------------------------- | ---------------- |
| `key`      | string  | yes                       | Stored profile key (snake_case, unique).       | `"audit_firm"`   |
| `label`    | string  | yes                       | Form question text.                            | `"Audit firm"`   |
| `type`     | enum    | no (default `text`)      | Input type — only `text` today.                | `"text"`         |
| `required` | bool    | no (default `false`)     | Must be filled before claim when `true`.        | `true`           |

## `validators.claims`

Optional **claim duration** — how long a validator has to finish one item after claiming it. Default 5 minutes.

```toml

[validators.claims]
duration_minutes = 45
```

| option             | accepts | required       | description                                               | example |
| ------------------ | ------- | --------------- | ------------------------------------------------------------ | ------- |
| `duration_minutes` | int     | no (default 5)   | Minutes before an unfinished claim expires. Max 24 hours.     | `45`    |

## `validators.actions`

Optional **validator task-page actions**. Today: conflict-of-interest self-decline.

```toml

[validators.actions]
allow_conflict_of_interest_self_decline = true
```

| option                                        | accepts | required | description                                                                       | example                                              |
| ---------------------------------------------- | ------- | -------- | ------------------------------------------------------------------------------------ | ------------------------------------------------------- |
| `allow_conflict_of_interest_self_decline`      | bool    | no       | When `true`, validators can decline a claimed item due to conflict of interest.       | `true`                                                   |
| `conflict_of_interest_decline_button_label`    | string  | no       | Button label override.                                                               | `"Decline due to conflict"`                              |
| `conflict_of_interest_decline_confirm_title`   | string  | no       | Confirmation dialog title override.                                                  | `"Decline this validation?"`                             |
| `conflict_of_interest_decline_confirm_body`    | string  | no       | Confirmation dialog body override.                                                   | `"Use this when you have a conflict of interest."`       |

---

# poq.toml — attestation

URL: https://sapien-ai-docs-production.up.railway.app/reference/poq-toml/attestation
Source: docs/current/reference/poq-toml/attestation.md

# Attestation (`attestation.*`)

One of the [poq.toml spec](/reference/poq-toml) sections.

Signed PoQ Report export — the output phase after review completes: schema, signing key, payload fields, and output format.

## `attestation`

Signed **PoQ Report export** settings. This section controls how the signed YAML attestation is constructed and formatted for download.

```toml

[attestation]
schema_version = "poq.attestation/v1"
signing_key_id = "poq-prod-2024"

[attestation.payload]
include_per_validator_votes = true
include_rationales_inline = false
outputs_alias = "findings"

[attestation.payload.outputs]
report_hash = "audit_report_hash"
severity_in_report = "proposed_severity"
repo_url = "repository"
commit_hash = "commit_sha"

[[attestation.metadata]]
field_name = "report_source"
channel = "portal"
sender = "0xC921CF11A568142223a52C7f0b4AE7023fb3326B"

[attestation.output]
strategy = "aggregated"
delimiter = "\n\n---\n"
fence = "yaml"
```

### Attestation fields

| option           | accepts | required | description                                                                   | example                    |
| ---------------- | ------- | -------- | ----------------------------------------------------------------------------- | ---------------------------- |
| `schema_version` | string  | yes      | Must be `poq.attestation/v1` for the current signer.                          | `"poq.attestation/v1"`        |
| `signing_key_id` | string  | yes      | Matches the server-configured signing key id (e.g. `sapien-prod-ed25519-v1`). | `"poq-prod-2024"`             |
| `payload`        | table   | no       | Nested table for payload options (see below).                                 | `[attestation.payload]`       |
| `metadata`       | array   | no       | Extra signed report header fields (see below).                                | `[[attestation.metadata]]`    |
| `output`         | table   | no       | Download formatting and emit strategy (see below).                            | `[attestation.output]`        |

### The `[attestation.payload]` block

| option                        | accepts | required | description                                                                                                   | example      |
| ----------------------------- | ------- | -------- | ------------------------------------------------------------------------------------------------------------- | -------------- |
| `include_per_validator_votes` | bool    | no       | When `true`, the signed YAML includes each validator's specific answer for every rubric dimension.             | `true`         |
| `include_rationales_inline`   | bool    | no       | When `false`, rationales use opaque references instead of inline text.                                         | `false`        |
| `include_datapoint_id`        | bool    | no       | Tri-state (unset/true/false). When `true`, each finding in the signed payload carries its datapoint id.        | `true`         |
| `outputs_alias`               | string  | no       | Names the main findings array in the YAML. Use `"findings"` for security audits; defaults to `"sub_reports"`.  | `"findings"`   |

### The `[attestation.payload.outputs]` block

Maps attestation fields to **canonical field names** defined in `[ingestion.fields]`.

| option               | accepts | required | description                                                      | example               |
| -------------------- | ------- | -------- | -------------------------------------------------------------------- | ----------------------- |
| `finding_id`         | string  | no       | Field holding the stable per-finding identifier in the report.       | `"finding_id"`          |
| `report_hash`        | string  | no       | Field holding the hash of the original input document.               | `"audit_report_hash"`   |
| `severity_in_report` | string  | no       | Field holding the original tool-reported severity.                   | `"proposed_severity"`   |
| `repo_url`           | string  | no       | Field holding the repository URL (emitted in `audit_target`).        | `"repository"`          |
| `commit_hash`        | string  | no       | Field holding the pinned commit SHA (emitted in `audit_target`).     | `"commit_sha"`          |

### The `[[attestation.metadata]]` block

Used for literal top-level YAML metadata blocks.

| option       | accepts | required | description                                                                    | example               |
| ------------ | ------- | -------- | --------------------------------------------------------------------------------- | ------------------------ |
| `field_name` | string  | yes      | The top-level YAML key name (e.g. `report_source`).                              | `"report_source"`        |
| _(others)_   | any     | no       | Any other keys in the table are emitted as key-value pairs under `field_name`.    | `channel = "portal"`      |

### The `[attestation.output]` block

| option      | accepts | required | description                                                                                                                              | example        |
| ----------- | ------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------ | ---------------- |
| `strategy`  | enum    | no       | How the attestation is emitted: `aggregated` (one signed report) or `in_place` (appended to each input). Omit to use the server default.  | `"aggregated"`    |
| `delimiter` | string  | no       | String inserted between the original document and the attestation in `in_place` mode.                                                      | `"\n\n---\n"`     |
| `fence`     | enum    | no       | Optional markdown fence type. Only `"yaml"` is supported today (wraps the attestation in `yaml ...` ).                                     | `"yaml"`          |

---

# PoQ Report spec

URL: https://sapien-ai-docs-production.up.railway.app/reference/poq-report
Source: docs/current/reference/poq-report.md

# PoQ Report spec

This is the field-by-field reference for the **PoQ Report payload** — the canonical JSON that every PoQ Report is a render of. For the conceptual overview (what a report is, how to read it, and how to verify one), see the [PoQ Report guide](/validate/poq-report).

Every PoQ Report is one canonical JSON payload (schema `poq.attestation/v1`), signed once. The readable formats — text, HTML — are deterministic renders of that payload, so this reference is also the source of truth for every value shown on a report page.

The payload is a single object. Its top-level keys map one-to-one onto the report's sections:

| Key                                          | Type            | Required | Report section                                            |
| -------------------------------------------- | --------------- | -------- | --------------------------------------------------------- |
| [`schema`](#identity)                        | string (const)  | yes      | Header                                                    |
| [`report_id`](#identity)                     | string          | yes      | Header                                                    |
| [`issuer`](#identity)                        | string          | yes      | Header                                                    |
| [`issued_at`](#identity)                     | timestamp       | yes      | Header                                                    |
| [`scope`](#scope)                            | object          | yes      | Scope                                                     |
| [`result`](#result)                          | object          | yes      | Result                                                    |
| [`scorecard`](#scorecard)                    | array           | yes      | Scorecard                                                 |
| [`distribution`](#distribution)              | object          | yes      | Distribution                                              |
| [`poq_record`](#poq_record)                  | object          | yes      | PoQ Record excerpt                                        |
| [`quorum_routing`](#quorum_routing)          | object          | yes      | Quorum routing                                            |
| [`configuration`](#configuration)            | object          | yes      | Configuration                                             |
| [`pool`](#pool)                              | object          | yes      | Pool & provenance                                         |
| [`stake`](#stake)                            | object          | no       | Pool & provenance (Collateral mode only)                  |
| [`custody`](#custody)                        | object          | yes      | Chain of custody                                          |
| [`attestation`](#attestation)                | object          | yes      | Attestation                                               |
| [`assertion`](#assertion)                    | string          | yes      | What this asserts                                         |

The payload is **strict**: `additionalProperties` is `false` at every level, so any key not listed here is rejected. Every required field must be present.

## Value types

Several fields reuse the same primitive types. They are defined once here and referenced throughout.

| Type          | JSON       | Format                                                                                          | Example                                   |
| ------------- | ---------- | ---------------------------------------------------------------------------------------------- | ----------------------------------------- |
| **score**     | integer    | `0`–`100`. A quality or consensus score.                                                        | `82`                                      |
| **weight**    | string     | `"advisory"` **or** a non-negative decimal (`"0"`, `"1"`, `"2.5"`, `"0.5"`). A rubric weight.    | `"1.5"`                                   |
| **wei**       | string     | A non-negative decimal integer, in wei. Used only in Collateral mode.                           | `"1000000000000000000"`                   |
| **hash**      | string     | `sha256:` followed by 64 lowercase hex characters.                                              | `"sha256:9f2b…c1"`                        |
| **timestamp** | string     | RFC 3339 UTC, second precision, `Z` suffix: `YYYY-MM-DDThh:mm:ssZ`.                              | `"2026-07-21T14:03:00Z"`                  |
| **bands**     | object     | Map of score-band label → count. Band labels are `0-9`, `NN-NN` (e.g. `40-49`), or `90-100`.    | `{ "40-49": 3, "90-100": 12 }`            |

> **No floats in the signed bytes.** Every numeric value is carried either as a bounded integer (scores, counts) or as a **string** (weights, wei). The signed payload contains no floating-point numbers, so the canonical bytes are deterministic.

## Identity

The four top-level scalars that identify the report. They render in the report header.

| Field       | Type           | Required | Description                                                                                     |
| ----------- | -------------- | -------- | ---------------------------------------------------------------------------------------------- |
| `schema`    | string (const) | yes      | Always `"poq.attestation/v1"`. Pins the payload to this schema version.                          |
| `report_id` | string         | yes      | Stable report identifier, pattern `PR-YYYY-NNNN-<ALNUM>-NNNN` (e.g. `PR-2026-0042-AB12-0007`).   |
| `issuer`    | string         | yes      | The issuing deployment / organization. Non-empty.                                               |
| `issued_at` | timestamp      | yes      | When the report was issued.                                                                      |

## `scope`

Which project the report covers and how much of it was reviewed. Renders as the **Scope** section.

| Field                | Type    | Required | Description                                                                     |
| -------------------- | ------- | -------- | ------------------------------------------------------------------------------- |
| `project`            | string  | yes      | Project name. Non-empty.                                                         |
| `datapoints_covered` | integer | yes      | Number of datapoints the report certifies (≥ 1).                                |
| `reviewed`           | integer | yes      | Number of datapoints reviewed (≥ 1). At M1 this equals `datapoints_covered` — there is no sampling. |
| `manifest_hash`      | hash    | yes      | Hash of the covered-datapoint manifest.                                          |

## `result`

The headline result. Renders as the **Result** section.

| Field                | Type   | Required | Description                                                                     |
| -------------------- | ------ | -------- | ------------------------------------------------------------------------------- |
| `quality_rating`     | score  | yes      | The overall Quality Rating (`0`–`100`).                                          |
| `consensus_strength` | score  | yes      | The overall Consensus Strength (`0`–`100`).                                      |
| `label`              | object | no       | The headline label, when the rubric produces one.                               |

`label` object:

| Field    | Type   | Required | Description                                                                     |
| -------- | ------ | -------- | ------------------------------------------------------------------------------- |
| `value`  | string | yes      | The label text (e.g. the winning category). Non-empty.                          |
| `source` | string | yes      | Where the label came from (e.g. the rubric dimension that produced it). Non-empty. |

## `scorecard`

Per-dimension quality and consensus, with rubric weights. A non-empty array; renders as the **Scorecard** section. Each element:

| Field                | Type   | Required | Description                                                                     |
| -------------------- | ------ | -------- | ------------------------------------------------------------------------------- |
| `dimension`          | string | yes      | Rubric dimension name (no whitespace).                                           |
| `weight`             | weight | yes      | The dimension's weight — `"advisory"` or a non-negative decimal.                 |
| `quality_rating`     | score  | yes      | Quality Rating for this dimension.                                               |
| `consensus_strength` | score  | yes      | Consensus Strength for this dimension.                                           |

## `distribution`

How scores were spread across datapoints. Renders as the **Distribution** section (histograms).

| Field                | Type  | Required | Description                                                                     |
| -------------------- | ----- | -------- | ------------------------------------------------------------------------------- |
| `quality_rating`     | bands | yes      | Distribution of per-datapoint Quality Ratings, as a band → count map.            |
| `consensus_strength` | bands | yes      | Distribution of per-datapoint Consensus Strengths, as a band → count map.        |

Each **bands** map has at least one entry; band labels are `0-9`, a two-digit range like `40-49`, or `90-100`, and each count is ≥ 1.

## `poq_record`

A commitment to the full per-datapoint PoQ Record, plus an optional one-datapoint excerpt signed inline. Renders as the **PoQ Record excerpt** section.

| Field          | Type   | Required | Description                                                                     |
| -------------- | ------ | -------- | ------------------------------------------------------------------------------- |
| `results_hash` | hash   | yes      | Hash committing to the complete per-datapoint results record.                   |
| `excerpt`      | object | no       | One datapoint's full detail, carried inline and signed with the payload.        |

`excerpt` object:

| Field       | Type           | Required | Description                                                                     |
| ----------- | -------------- | -------- | ------------------------------------------------------------------------------- |
| `schema`    | string (const) | yes      | Always `"poq.record/v1"`.                                                        |
| `shown`     | integer        | yes      | Which datapoint index is shown (≥ 1).                                            |
| `datapoint` | object         | yes      | The excerpted datapoint (see below).                                            |

`excerpt.datapoint` object:

| Field                | Type    | Required | Description                                                                     |
| -------------------- | ------- | -------- | ------------------------------------------------------------------------------- |
| `id`                 | string  | yes      | Datapoint identifier. Non-empty.                                                |
| `canonical_hash`     | hash    | yes      | Hash of the datapoint's canonical content.                                       |
| `quality_rating`     | score   | yes      | This datapoint's Quality Rating.                                                 |
| `consensus_strength` | score   | yes      | This datapoint's Consensus Strength.                                             |
| `settled_round`      | integer | yes      | The round in which this datapoint settled (≥ 1).                                 |
| `escalated`          | boolean | yes      | Whether the datapoint was escalated beyond its first round.                      |
| `quorum`             | array   | yes      | The rounds and votes that decided it (see below). Non-empty.                     |
| `dimensions`         | array   | yes      | Per-dimension detail for this datapoint (see below). Non-empty.                  |

`excerpt.datapoint.quorum[]` — one entry per round:

| Field   | Type    | Required | Description                                                                     |
| ------- | ------- | -------- | ------------------------------------------------------------------------------- |
| `round` | integer | yes      | Round number (≥ 1).                                                              |
| `votes` | array   | yes      | The votes cast this round (see below). Non-empty.                               |

`excerpt.datapoint.quorum[].votes[]` — one entry per validator:

| Field        | Type   | Required | Description                                                                     |
| ------------ | ------ | -------- | ------------------------------------------------------------------------------- |
| `id`         | string | yes      | Validator identifier (pseudonymous). Non-empty.                                 |
| `type`       | enum   | yes      | `"human"` or `"agentic"`.                                                        |
| `model`      | string | no       | Model identifier, for agentic validators.                                       |
| `credential` | string | no       | Credential held, for human validators.                                          |
| `weight`     | string | yes      | The vote's consensus weight — a non-negative decimal. Unlike a rubric weight, `"advisory"` is not accepted here. |

`excerpt.datapoint.dimensions[]` — per-dimension breakdown:

| Field       | Type   | Required | Description                                                                     |
| ----------- | ------ | -------- | ------------------------------------------------------------------------------- |
| `dimension` | string | yes      | Rubric dimension name. Non-empty.                                               |
| `quality`   | score  | yes      | Quality score for this dimension on this datapoint.                              |
| `strength`  | score  | yes      | Consensus strength for this dimension on this datapoint.                         |
| `weight`    | weight | yes      | The dimension's weight.                                                          |

## `quorum_routing`

How validators were assigned across rounds, and where datapoints settled. Renders as the **Quorum routing** section.

| Field       | Type   | Required | Description                                                                     |
| ----------- | ------ | -------- | ------------------------------------------------------------------------------- |
| `rounds`    | array  | yes      | The routing rounds (see below). Non-empty.                                      |
| `execution` | object | yes      | Where datapoints settled (see below).                                           |

`quorum_routing.rounds[]`:

| Field               | Type    | Required | Description                                                                     |
| ------------------- | ------- | -------- | ------------------------------------------------------------------------------- |
| `round`             | integer | yes      | Round number (≥ 1).                                                              |
| `quorum`            | integer | no       | Quorum size targeted this round (≥ 1).                                           |
| `escalation_target` | score   | no       | The consensus-strength threshold below which a datapoint escalates.             |
| `slots`             | array   | yes      | The validator slots offered this round (see below). Non-empty.                  |

`quorum_routing.rounds[].slots[]` — a validator slot. Each slot is **either** a human slot **or** an agentic slot (enforced by the schema's `oneOf`):

| Field               | Type    | Required          | Description                                                                     |
| ------------------- | ------- | ----------------- | ------------------------------------------------------------------------------- |
| `type`              | enum    | yes               | `"human"` or `"agentic"`.                                                        |
| `count`             | integer | yes               | Number of validators in this slot (≥ 1).                                         |
| `vote_weight`       | string  | yes               | Per-vote weight — a non-negative decimal.                                        |
| `credential`        | string  | human slot        | Required credential. **Human slots only.**                                      |
| `instructions_hash` | hash    | human slot        | Hash of the instructions shown. **Human slots only.**                           |
| `model`             | string  | agentic slot      | Model identifier (no whitespace). **Agentic slots only.**                       |
| `prompt_hash`       | hash    | agentic slot      | Hash of the agent prompt. **Agentic slots only.**                               |

A human slot must carry `credential` + `instructions_hash` and must **not** carry `model`/`prompt_hash`; an agentic slot is the mirror image. Mixing the two is rejected.

`quorum_routing.execution`:

| Field             | Type    | Required | Description                                                                     |
| ----------------- | ------- | -------- | ------------------------------------------------------------------------------- |
| `settled_round_1` | integer | yes      | How many datapoints settled in round 1 (≥ 0).                                    |
| `escalated`       | integer | yes      | How many datapoints escalated past round 1 (≥ 0).                                |

## `configuration`

The rubric and the locked config. Renders as the **Configuration** section.

| Field         | Type   | Required | Description                                                                     |
| ------------- | ------ | -------- | ------------------------------------------------------------------------------- |
| `rubric`      | string | yes      | Rubric identifier (no whitespace).                                               |
| `config_hash` | hash   | yes      | Hash of the locked `poq.toml` configuration.                                     |

## `pool`

Validator pool size, churn, participation, and independence. Renders as the **Pool & provenance** section.

| Field                  | Type    | Required | Description                                                                     |
| ---------------------- | ------- | -------- | ------------------------------------------------------------------------------- |
| `size`                 | integer | yes      | Final pool size — every validator class eligible for the work, including agentic (≥ 0). |
| `size_at_start`        | integer | yes      | Pool size when the review window opened (≥ 0).                                   |
| `added`                | integer | yes      | Validators added during the window (≥ 0).                                        |
| `removed`              | integer | yes      | Validators removed during the window (≥ 0).                                      |
| `participated_human`   | integer | yes      | Human validators who cast at least one vote (≥ 0).                               |
| `participated_agentic` | integer | yes      | Agentic (AI) validators who cast at least one vote (≥ 0).                        |
| `credentials`          | string  | yes      | Summary of the credentials represented in the pool. Non-empty.                  |
| `blind_review`         | boolean | yes      | Whether reviews were blind (validators could not see each other's inputs).       |
| `coi_declared`         | integer | no       | Conflict-of-interest declarations collected (≥ 0). Present **iff** COI declarations were collected; `0` means collected, none declared. |

## `stake`

**Collateral mode only.** Present only for events that put stake at risk; omitted otherwise. Renders under **Pool & provenance**.

| Field               | Type | Required | Description                                                                     |
| ------------------- | ---- | -------- | ------------------------------------------------------------------------------- |
| `stake_at_risk_wei` | wei  | yes      | Total stake at risk, in wei.                                                     |
| `slashed_wei`       | wei  | yes      | Stake slashed, in wei.                                                           |
| `rewards_wei`       | wei  | yes      | Rewards paid, in wei.                                                            |

## `custody`

The chain-of-custody timestamps. Renders as the **Chain of custody** section. All five are required timestamps.

| Field                | Type      | Required | Description                                                                     |
| -------------------- | --------- | -------- | ------------------------------------------------------------------------------- |
| `config_locked`      | timestamp | yes      | When the `poq.toml` config was locked.                                           |
| `manifest_confirmed` | timestamp | yes      | When the datapoint manifest was confirmed.                                       |
| `review_opened`      | timestamp | yes      | When the review window opened.                                                   |
| `review_closed`      | timestamp | yes      | When the review window closed.                                                   |
| `issued`             | timestamp | yes      | When the report was issued.                                                      |

## `attestation`

The cryptographic roots and signing key. Renders as the **Attestation** section.

| Field            | Type   | Required | Description                                                                     |
| ---------------- | ------ | -------- | ------------------------------------------------------------------------------- |
| `root`           | hash   | yes      | The attestation root committing to the report.                                  |
| `score_root`     | hash   | yes      | The root committing to the per-datapoint scores.                                |
| `signing_key_id` | string | yes      | Key id (`kid`) of the signing key (no whitespace).                               |
| `registry`       | string | no       | Validator/revocation registry reference. Absent until the registry exists (M4). |

## `assertion`

| Field       | Type   | Required | Description                                                                     |
| ----------- | ------ | -------- | ------------------------------------------------------------------------------- |
| `assertion` | string | yes      | The fixed plain-language statement of what the report certifies. A constant of the schema version; renders as the **What this asserts** section. |

## The signed payload

The object above is signed exactly once and delivered in whichever format you request. The signature and canonicalization are fixed properties of `poq.attestation/v1`:

| Property           | Detail                                                                                   |
| ------------------ | ---------------------------------------------------------------------------------------- |
| Signature          | A single **Ed25519** signature, delivered as a compact **JWS**.                          |
| Canonicalization   | The payload is canonicalized with **JCS (RFC 8785)** before signing — deterministic bytes. |
| Numerics           | All non-integer numeric values are carried as strings; there are no floats in the signed bytes. |
| Key id             | The JWS header carries a `kid` matching [`attestation.signing_key_id`](#attestation).      |

Any change to a covered value — in the JSON, the readable text, or the HTML — breaks verification.

### Formats

The one signed payload is delivered in any of four formats:

| Format | Extension | What you get                                       | Best for                              |
| ------ | --------- | -------------------------------------------------- | ------------------------------------- |
| Text   | `.txt`    | A readable render followed by the fenced signature | The default — human-readable, verifiable |
| HTML   | `.html`   | The designed, styled report page                   | Sharing / reading in a browser or PDF |
| JSON   | `.json`   | The canonical signed payload (this schema)         | Machine consumption                   |
| JWS    | `.jws`    | The bare compact signature                         | Verifying against your own payload copy |

### Envelopes

The two readable formats declare an **envelope** version so a verifier knows how to interpret the carrier before applying any grammar:

| Envelope               | Carrier                                                                                     |
| ---------------------- | ------------------------------------------------------------------------------------------- |
| `poq.envelope/text-v1` | The two-layer text artefact: a readable render followed by the fenced compact JWS.          |
| `poq.envelope/html-v1` | The designed HTML page, which embeds the text artefact and tags every payload-derived value with `data-poq`. |

In the text artefact the JWS sits between `-----BEGIN POQ SIGNED PAYLOAD-----` / `-----END POQ SIGNED PAYLOAD-----` fences. In the HTML the same text artefact is embedded in a `<script type="application/poq-attestation+text">` element. For how to verify either offline, see [Verifying a report](/validate/poq-report#verifying-a-report).

## See also

| Page                                                   | Description                                                        |
| ------------------------------------------------------ | ----------------------------------------------------------------- |
| [PoQ Report](/validate/poq-report)                   | The conceptual guide: what a report is, how to get one, how to verify it |
| [poq.toml spec](/reference/poq-toml)                   | Field reference for the project spec that produces the report     |
| [How consensus works](/validate/consensus-explained)   | The mechanics behind the Quality Rating and Consensus Strength    |

---

# Sapien Tokenomics

URL: https://sapien-ai-docs-production.up.railway.app/reference/tokenomics
Source: docs/current/reference/tokenomics.md

## Proof of Quality

Poor-quality work results in both reputation loss and slashing. This means losing access to future tasks, and part or all of your stake. The result is a system where quality is enforced through aligned incentives, transparent reputation, and distributed oversight.

Proof of Quality is Sapien's core contribution to the infrastructure of AI data. It is a structured signal of trust that emerges from contributor behavior, peer validation, and onchain financial alignment. When tasks require real-world expertise, we verify credentials — that includes doctors, engineers, lawyers, and other professionals. But we do not rely on institutional status alone to enforce quality.

Instead of clawing back earnings after the fact, which is a terrible way to treat contributors, Sapien uses onchain collateral and a transparent review process. Contributors stake before completing work. That stake remains in their control as long as the work meets expectations. Tasks are reviewed by peers with more experience in the same domain, not by anonymous QA teams.

### Staking as collateral

Before a contributor can complete complex tasks, they must lock a portion of their $SAPIEN tokens as collateral. This stake serves as a performance guarantee.

## Supply management and treasury design

The $SAPIEN token has a fixed maximum supply of 1,000,000,000 tokens. There is no inflationary minting, and the protocol includes mechanisms to reduce circulating supply over time through contributor behavior and DAO-governed treasury flows.

### DAO-governed use of protocol revenue

A portion of protocol revenue from enterprise data requests and related services is allocated to the DAO treasury. The DAO may decide how to deploy this capital via onchain governance. Example uses include:

- Grants to support contributors or tooling.
- Community incentive programs.
- Token purchases for redistribution or burning.
- Liquidity provision or strategic reserve formation.

Importantly, there is no automatic burn mechanism, no protocol-level redistribution to token holders, and no implicit or contractual linkage between protocol revenue and token price. All treasury actions are subject to community governance.

## Governance and risk mitigation

The Sapien protocol is designed to operate without centralized control, while preserving enterprise-grade quality, transparency, and security. This is achieved through progressive decentralization of governance and a layered approach to risk management across the technical, economic, operational, and regulatory dimensions of the network.

### Governance structure

Governance of the Sapien protocol is expected to shift gradually toward token holders as the network matures. In its early phases, core development is led by the founding contributors. Over time, responsibility for critical decisions will transition to a decentralized autonomous organization (DAO).

**Governance responsibilities**

- Treasury allocation and grant approval.
- Protocol parameter adjustments (e.g., slashing severity, reward weights).
- Whitelisting new task types or verticals.
- Proposals for upgrades or system migration.
- Onboarding or removal of core contributors.

All governance actions are expected to be executed via onchain voting and tracked transparently. Governance rights are expected to be tied to $SAPIEN holdings, subject to participation thresholds and evolving delegation mechanics.

### Ongoing oversight

To ensure long-term trust and resilience, the protocol commits to:

- Regular security audits and published risk reviews.
- Transparent community metrics dashboards.
- Treasury reports and governance transparency tooling.
- Engagement with third-party legal, operational, and security advisors.

## See also

| Page | Description |
| --- | --- |
| [PoQ workflow](/build/poq-workflow) | How a project moves from definition to attestation |
| [How consensus works](/validate/consensus-explained) | The validation mechanism behind Proof of Quality |
| [Sapien Vault](/validate/sapien-vault) | Staking and collateral in practice |

---

[Download official PDF](https://cdn.sapien.io/tokenomics.pdf)

[Download MiCA Whitepaper PDF](https://cdn.sapien.io/mica_whitepaper.pdf)