poq.toml spec

The poq.toml file for each Sapien project is its system spec. It turns the data you upload into a working project: the fields on each review item, what validators see, the questions they answer, and how they are scored.

Every poq.toml has three phases (ingestion, validation, attestation) plus a validators config and file metadata. Each is documented on its own page below.

Sections

Unknown keys are rejected. The parser is strict: any key or table that isn't part of the schema (a typo, an extra field, or a leftover from an old flat-layout spec such as [[inputs]] or [stage.*]) fails at parse time with an error naming the offending key, rather than being silently ignored.

A working spec needs at minimum: [project] with spec_version, at least one [[ingestion.sources]], [ingestion.fields] with a key named id, at least one [[validation.evidence]], at least one [[validation.rubric]], and [validators]. Every other section is optional.

The Data Lifecycle: From Files to Review

Before writing your spec, it is helpful to understand how Sapien transforms your files into tasks.

1. File Upload

You upload your files (CSVs, JSONs, images, or Markdown) from a project folder.

2. Ingestion

When you Ingest a project, Sapien runs your poq.toml spec to build the review items.

• The Spreadsheet Model: Sapien treats every collection of files as a temporary spreadsheet.
  • Rows: Each individual item in your file (a CSV line, a JSON file, a Markdown section).
  • Columns: The data inside those items (a CSV header, a JSON key, a regex capture).
• The Mapping: [ingestion.fields] is the wiring — each table key is the column name used everywhere else; each value is <source_id>.<column> at ingest time.
  • Example: finding_title = "findings.title" keeps the JSON title property under the name finding_title.
• The Merge (Optional): If you have multiple sources (e.g., labels in a CSV and images in a folder), [[ingestion.joins]] lines them up into one wider spreadsheet before field projection.

3. Datapoints

Once ingestion finishes, the temporary spreadsheets are deleted. What remains are review items, stored in a database table.

• One row = one task: Each review item is a single row in the database.
• Persistent data: The keys you declared in [ingestion.fields] are saved on that row.

4. Review

When a validator starts a task, they aren't looking at your original files. They are looking at one specific review item that was ingested.

• The [validation.*] sections tell the UI what to show, what to ask, and how items are scored; the [validators.*] sections decide who validates, how many, pay, routing, and qualification gates.

For developers

On-disk TOML in this reference is the authoring shape accepted by the parser (ingestion.*, validation.*, validators.*, top-level [attestation]). Legacy [stage.*] and [[inputs]] roots are no longer accepted.

Compiled runtime JSON uses its own internal field names, which don't always match the TOML keys 1