10:I[38308,["937","static/chunks/937-1cb27472c9e24502.js?dpl=dpl_71x4DwML6S8o87Nc7jnqHPQWgCjG","870","static/chunks/app/docs/%5B%5B...slug%5D%5D/page-e1a96eafa5ad2159.js?dpl=dpl_71x4DwML6S8o87Nc7jnqHPQWgCjG"],"DocsShell"] 11:T66f, Get the current state of an alignment. Poll until `status` is `completed`, `failed`, or `cancelled`. Alignment ID. ## Response A completed result contains `status: "completed"`, `improved`, `bestCandidate`, `initialScore`, `bestScore`, `usage`, optional `metrics`, and nullable `artifactPath`. `stage` is one of `starting`, `optimizing`, `scoring`, `finishing`, `committing`, or `done`. `metricCalls / maxMetricCalls` is the useful progress fraction when both values are present; the run may finish before reaching the maximum. 12:Tdf8, Start an asynchronous alignment for a model. Alignment learns from annotated training examples and selects improved instructions using the model's test split. Each question must have string `instructions` for alignment. The more flexible inference contract also permits omitted, null, or other JSON instructions, but those must be replaced with strings before aligning. Criteria descriptions may remain structured objects or arrays. The endpoint returns `202 Accepted`. Poll the URL in the `Location` response header for progress. A key between 8 and 255 characters. Reuse it only when retrying this exact request. Model repository ID. Repository write access is required. Branch to align and, when the run improves the model, update. Current branch head, from 7 to 128 characters. The request returns `409 source_commit_conflict` if the branch has advanced. Contains `maxMetricCalls`, a positive integer up to 1,000,000. This is the maximum number of example-question evaluations available to the run. Optional reflection settings: `model`, positive `minibatchSize` up to 10,000, and integer `seed` from 0 to 2,147,483,647. ## Request example ```json { "branch": "main", "sourceCommitSha": "df5a6fb2d67e4ce6a6ea9d43146d53a4d43f6d37", "budget": { "maxMetricCalls": 500 }, "reflection": { "seed": 42, "minibatchSize": 8 } } ``` ## Response Alignment ID. New commit when the alignment improves the model; otherwise `null`. On completion, includes `improved`, the best candidate instructions, initial and best scores, token usage, optional validation metrics, and the saved artifact path. Live progress with a `stage`, elapsed time, and—when available—metric-call, iteration, candidate, and score counters. Job-level error with `code` and `message`. If the run improves the model, Modaic increments its checkpoint, commits the optimized instructions and alignment artifact, then starts a batch decision at that exact commit for all examples. If it does not improve the model, no model commit is created. 13:T52c, Get the current state of a batch decision. Poll until `status` is `completed`, `failed`, or `cancelled`. Batch decision ID. ## Response Integer `total`, `completed`, and `failed` counts. Retrieve the decisions produced by the batch through each example's [decision history](/docs/api-reference/examples/list-example-decisions). 14:T9ae, Run a repository model over up to 1,000 examples. Supply stored example IDs, inline examples, or `scope: "all"`—exactly one selection mode is required. The endpoint returns `202 Accepted`. Poll the URL in the `Location` response header for progress. A key between 8 and 255 characters. Reuse it only when retrying this exact request. Model repository ID. Repository write access is required. Branch whose model configuration will be used. Commit to run. When omitted, the current branch head is resolved and stored on the batch decision. Between 1 and 1,000 unique stored example IDs. Between 1 and 1,000 inline examples. Each item contains an optional UUID `id` and required JSON `state`. A UUID is generated when `id` is omitted. Set to `"all"` to run every non-deleted example in the model. ## Response Batch decision ID. Exact model revision used by the batch. Integer `total`, `completed`, and `failed` counts. Final counts when the job reaches a terminal state. Job-level error with `code` and `message`. 15:T147b, Run one or more typed questions against a JSON state. Use a supported base model with inline `questions`, or a repository model such as `acme/support-triage`. Repository models load their saved questions. Any questions supplied in the request override saved questions with the same ID and add new IDs for this call. [`POST /v1/systemone`](/docs/api-reference/decisions/systemone) is an alias with the same request, response, authentication, and retry behavior. Optional key between 8 and 255 characters. Reuse it when retrying this exact request. If omitted, each call is treated as a new request. The state to evaluate. For images inside message arrays, use OpenAI-compatible image blocks: `{"type":"image_url","image_url":{"url":"..."}}`. A base model (`mo`, `mo-reasoning`, `modaic/mo`, `modaic/mo-reasoning`, `jev-latest`, or `jev-1.13.0`) or an accessible repository path in `owner/model` form. Repository models only. A branch, tag, or commit SHA. Defaults to the model's default branch. Pin a commit SHA for reproducible decisions. Required for base models. Optional for repository models. When supplied, the object must contain at least one question. Question IDs may be any string, including `""`, and are returned unchanged as answer keys. For every question type, `instructions` is optional and accepts any JSON value, including `null`. Criteria descriptions may be strings, objects, or arrays; objects and arrays can contain nested JSON values. Use `type: "noul"`. `criteria` may be omitted or `null`. `criteria.true` and `criteria.false` are optional descriptions and may be `null`. The answer's `noul` field is the probability of `true` from 0 to 1. Use `type: "choice"`. `criteria` maps non-empty option keys to a description or `null`, with no minimum number of options. The answer includes the selected key, a probability for every key, and `confidence`. Use `type: "score"`. `criteria` is an ordered array of one or more rubric descriptions; individual descriptions cannot be `null`. Levels are zero-indexed: five levels span 0 through 4. The returned score may be fractional. The answer includes the score, legend, probabilities, and confidence. Legend values preserve the string, object, or array descriptions. Repository models only. Groups this decision under an example you identify. When omitted, Modaic derives the example ID from the canonical state so identical states share an example. Repository models only. Set to `false` to skip recording the example and decision. A model's capture sample rate may also choose not to record a call. ## Question example ```json { "state": {"message": "Please cancel and refund my annual plan."}, "model": "mo", "questions": { "intent": { "type": "choice", "instructions": "Classify the customer's primary intent.", "criteria": { "refund": {"description": "The customer wants money returned.", "examples": ["Refund my plan."]}, "cancel": "The customer only wants future service stopped.", "other": null } }, "urgency": { "type": "score", "instructions": "Score how quickly a human should respond.", "criteria": ["not urgent", "routine", "urgent", "critical"] } } } ``` ## Response The model name supplied in the request. One answer for every requested question. Answer IDs and types exactly match the questions. Token usage with integer `input_tokens` and `output_tokens` fields. The captured example. Present for repository models when capture succeeds. The captured decision. Present for repository models when capture succeeds. The repository model checkpoint used for the decision. The resolved revision for a repository model. Whether this repository-model call was recorded. Unknown base models or inaccessible repository models return `404 model_not_found`. An unreadable explicit revision returns `404 revision_not_found`. Invalid questions or answers return `422`. 16:T48f, This route is an alias for [`POST /v1/decision`](/docs/api-reference/decisions/create-decision). It uses the same request and response fields, Modaic authentication, permissions, and error format. Only the route name changes. Optional key between 8 and 255 characters. Reuse it when retrying this exact request. Without a key, each call is a new request. The state to evaluate. A supported base model or accessible repository path, such as `acme/support-triage`. Required for base models; optional for repository models. Supports the same `noul`, `choice`, and `score` questions as the decision route, including empty question IDs, optional JSON instructions, and structured criteria. Repository calls also accept `revision`, `example_id`, and `capture`. See the [decision reference](/docs/api-reference/decisions/create-decision) for all parameters and response fields. 17:T603, Add or update an example's ground truth or reasoning. At least one annotation field is required. The first annotation must include `groundTruth`. Model repository ID. Repository write access is required. Example ID. Expected answers keyed by question ID. `noul` questions use booleans, `choice` questions use option keys, and `score` questions use zero-based numeric indexes into the question's criteria array. Optional explanation for the ground truth. The first annotation assigns a stable `train` or `test` split. Updating the annotation later preserves the existing split. ## Response The merged ground truth and reasoning with the preserved split. 18:T4e9, Get one example from a model repository. Model repository ID. Example ID. ## Response `ingest` for examples added through the examples API; `live` for examples captured from repository-model decisions. Ground truth, optional reasoning, and the stable `train` or `test` split. The newest successful decision, including its model, answers, request, response, checkpoint, revision, source, and timestamp. Returns `404 example_not_found` when the example does not exist in the model. 19:T679, Add up to 1,000 examples to a model. You may provide ground truth during ingest or annotate the example later. Model repository ID. Repository write access is required. Between 1 and 1,000 examples. Caller-assigned ID. A UUID is generated when omitted. The input state that a decision evaluates. Optional annotation with required `groundTruth` and optional `groundReasoning`. Ground truth is an object keyed by question ID. When an example is first annotated, Modaic assigns it a stable `train` or `test` split. Later annotation updates preserve that split. ## Request example ```json { "examples": [ { "id": "31fcc86c-7f49-4daa-a8b1-577db0c8fe76", "state": {"ticket": "I was billed twice."}, "annotation": { "groundTruth": {"needs_refund": true, "priority": "high"}, "groundReasoning": "The ticket reports a duplicate charge." } } ] } ``` ## Response The ingested examples. Each resource includes `id`, `state`, `source`, `imageUrls`, `annotation`, `latestDecision`, `decisionCount`, `createdAt`, and `updatedAt`. `source` is `ingest` for this endpoint. 1a:T458, List examples newest-updated first. Each item includes its current annotation, latest successful decision, and total decision count. Model repository ID. Repository read access is required. One-based page number. Number of examples per page, from 1 to 100. ## Response Examples with `source` (`ingest` or `live`), image URLs, optional annotation, optional latest decision, and timestamps. An annotation contains `groundTruth`, nullable `groundReasoning`, and a stable `split` of `train` or `test`. `latestDecision` is `null` until the model has made a successful decision for that example. 1b:T6b0, Create a private model repository owned by a user or organization. The caller must be allowed to create repositories for `owner`. The user or organization slug. The model slug. It must start with an alphanumeric character and may contain letters, digits, `.`, `_`, and `-`, up to 100 characters. A description up to 1,000 characters. The initial default branch. Up to 20 initial files. Each item has `operation: "upsert"`, a `path` up to 512 characters, and string `contents` up to 1,000,000 characters. ## Response The `modelId` used by model-scoped API routes. Public metadata for the owning user or organization. Model repositories are private. After creation, call [Update model](/docs/api-reference/models/update-model) to set the root model and typed questions, or include a valid `model.json` in `initialFiles`. 1c:T658, List supported base models and your accessible model repositories. Pass an entry's `name` to [`POST /v1/decision`](/docs/api-reference/decisions/create-decision) or [`POST /v1/systemone`](/docs/api-reference/decisions/systemone). Requires a Modaic bearer token with `read:repository` or `write:repository` scope. Personal tokens include your own repositories and those of organizations you belong to. Organization tokens include only that organization's repositories. Archived repositories are excluded. This route has no request parameters. The response lists base models first, followed by repository models sorted by name. A listed repository may still need its model and questions configured before it can run decisions. ## Response The available model entries. Model identifier, such as `modaic/mo`, `jev-latest`, or `acme/support-triage`. Model description. An unset repository description is returned as `""`. Whether this is a supported base model or a model repository. Present for repository models. Use this as `modelId` in model resource routes. A missing or invalid token returns `401`. A token without the required scope returns `403`. 1d:T78e, Update model metadata or commit a new `model.json` configuration. Repository write access is required. Model repository ID. Branch to update. Defaults to the model's default branch. Replace or clear the repository description. Requires repository admin access. Root model used for decisions. Replace the model's typed questions. Question shapes match [`POST /v1/decision`](/docs/api-reference/decisions/create-decision). Commit message, up to 500 characters, for a configuration change. At least one field must produce a change. Updating `model` or `questions` resets the checkpoint to `0` and clears prior evaluation metrics, while preserving the confidence threshold and capture policy. ## Response The current `schemaVersion`, root `model`, `checkpoint`, `questions`, and any saved `metrics` or `capture` policy. Present when the configuration changed. Includes `commitSha`, `branch`, and `previousSha`. A missing model or branch returns `404`. If the branch advances while the update is being committed, the endpoint returns `409`. 1e:T696, Send a Modaic API key as a bearer token on every protected request. ```bash export MODAIC_API_KEY="mdc_..." curl --request GET \ --url 'https://api.modaic.dev/v1/models' \ --header "Authorization: Bearer $MODAIC_API_KEY" ``` Create and revoke keys from the Modaic dashboard. Treat a key like a password: store it in a secret manager, never commit it, and do not expose it in browser code. ## Scopes API keys use read and write scopes for three resource categories: | Scope | Grants access to | | --- | --- | | `read:repository` | List available models and read models, examples, decisions, alignments, and metrics. | | `write:repository` | Run decisions through `/v1/decision` or `/v1/systemone`, and create or update repository-scoped resources. Also grants repository read access. | | `read:organization` | Read organization-scoped resources. | | `write:organization` | Change organization-scoped resources. Also grants organization read access. | | `read:user` | Read resources belonging to the key owner. | | `write:user` | Change resources belonging to the key owner. Also grants user read access. | Organization API keys are restricted to repository-scoped resources owned by that organization. API keys cannot create, list, or revoke other API keys; those operations require an interactive user session. ## Authentication errors - `401 authentication_required` means the bearer header is missing or malformed. - `401 invalid_access_token` means the key is invalid or revoked. - `403 insufficient_token_scope` identifies the missing scope in `detail`. - `403 organization_token_boundary` means an organization key was used outside its permitted resource boundary. 1f:T7cb, Errors use `application/problem+json` and include a stable machine-readable `code` plus a request identifier for support and debugging. ```json { "type": "https://modaic.dev/problems/validation-error", "title": "Unprocessable Content", "status": 422, "detail": "The request did not match the expected schema.", "code": "validation_error", "requestId": "b80926a9-e5c6-4bc7-a5c5-0ef284bf6457", "details": [] } ``` ## Status codes | Status | Meaning | | --- | --- | | `400` | The request is invalid outside normal schema validation. | | `401` | Authentication is missing, invalid, or revoked. | | `403` | The caller lacks a required scope or resource role. | | `404` | The route or an accessible resource was not found. | | `409` | The request conflicts with current state or reuses an idempotency key for a different payload. | | `415` | The request content type is unsupported. | | `422` | A path, query, header, body, or provider response failed validation. | | `429` | The caller exceeded a rate limit. | | `500`, `502`, `503` | A server or upstream dependency failed. Retry only when safe. | ## Idempotency `POST /v1/decision` and its alias `POST /v1/systemone` accept an optional `Idempotency-Key` header. Without it, each call is a new request, so retries are not deduplicated. `POST /v1/models/{modelId}/batch-decisions` and `POST /v1/models/{modelId}/alignments` require the header. Whenever supplied, the key must be between 8 and 255 characters. Reuse the same key when retrying the same logical request. A completed response is replayed. Reusing the key with a different payload returns `409 idempotency_key_reused`. A concurrent decision duplicate may return `409 decision_in_progress`; retry it with the same key after a short delay. ## Request identifiers Every response exposes `x-request-id`. Send your own UUID in the `x-request-id` request header to correlate a call with your logs, or let Modaic generate one. Include this value when contacting support. 20:Ta97, The Modaic API puts your team's decision criteria to work: route tickets, evaluate agent responses, or check content against a policy. Start with a base model, or create a [Model](/docs/models/using_models): a versioned decision primitive that treats the base model and its instructions as one artifact. Run it, measure its decisions, and align its instructions with expert feedback. The base URL is: ```text https://api.modaic.dev/v1 ``` All request and response bodies use JSON unless an endpoint says otherwise. Use [`GET /v1/models`](/docs/api-reference/models/list-models) to discover base models and the model repositories accessible to your token. [`POST /v1/systemone`](/docs/api-reference/decisions/systemone) is an alias for `POST /v1/decision`. Both accept an optional `Idempotency-Key`; include one when you need retries to replay the same response. Batch decisions and alignments still require a key. See [safe retries](/docs/api/errors#idempotency). ## Core workflow Send a state and one or more typed questions to [`POST /v1/decision`](/docs/api-reference/decisions/create-decision). Questions can return a boolean likelihood, a categorical choice, or a score on a rubric. A model stores your root model, questions, checkpoints, and capture policy in a private, versioned repository. Decisions made with a repository model can be captured automatically. You can also ingest examples in bulk, then add ground truth and reasoning. Start an alignment to improve question instructions from annotated examples. An improved alignment creates a new checkpoint that can be evaluated across the full example set. ## Resource identifiers A repository model has two identifiers: - A human-readable path such as `acme/support-triage`, used by `POST /v1/decision`. - A UUID named `modelId` in resource URLs for examples, batch decisions, and alignments. Responses use camelCase for resource fields, such as `repositoryId` and `createdAt`. The decision contract uses snake_case for token usage and optional caller-provided identifiers, such as `input_tokens` and `example_id`. ## Asynchronous operations Batch decisions and alignments return `202 Accepted` with a `Location` header. Poll that URL until `status` is `completed`, `failed`, or `cancelled`. Cancelling a job is idempotent and returns `204 No Content`. ## OpenAPI The current OpenAPI 3.1 document is available at [`https://api.modaic.dev/openapi.json`](https://api.modaic.dev/openapi.json). 21:Te49, > Modaic is the last-mile platform for Decision Models. Run typed decisions, > collect expert feedback, and align models to your criteria and production data. Decision Models are built for high-volume decision tasks. They return typed outputs, probabilities, and confidence scores while running faster and cheaper than general-purpose language models. Modaic takes them from **calibrated in general → calibrated for your use case**. [Learn more about Decision Models](https://docs.typesafe.ai/introduction). ## Quickstart ```typescript TypeScript import { Modaic } from "modaic"; const modaic = new Modaic({ apiKey: process.env.MODAIC_API_KEY!, }); const decision = await modaic.decisions.create({ model: "mo", state: { ticket: "I was charged twice. Please refund one charge.", }, questions: { priority: { type: "choice", instructions: "Choose the support priority.", criteria: { low: "No financial or time-sensitive impact.", normal: "Routine customer issue.", high: "Financial impact or an urgent blocker.", }, }, }, }); console.log(decision.answers.priority); // { choice: "high", probabilities: { ... }, confidence: 0.94 } ``` ```python Python import os from modaic import Modaic modaic = Modaic(api_key=os.environ["MODAIC_API_KEY"]) decision = modaic.decisions.create( model="mo", state={ "ticket": "I was charged twice. Please refund one charge.", }, questions={ "priority": { "type": "choice", "instructions": "Choose the support priority.", "criteria": { "low": "No financial or time-sensitive impact.", "normal": "Routine customer issue.", "high": "Financial impact or an urgent blocker.", }, }, }, ) print(decision.answers["priority"]) # {"choice": "high", "probabilities": { ... }, "confidence": 0.94} ``` ```bash cURL curl --request POST \ --url 'https://api.modaic.dev/v1/decision' \ --header "Authorization: Bearer $MODAIC_API_KEY" \ --header "Idempotency-Key: intro-$(date +%s)" \ --header 'Content-Type: application/json' \ --data '{ "model": "mo", "state": { "ticket": "I was charged twice. Please refund one charge." }, "questions": { "priority": { "type": "choice", "instructions": "Choose the support priority.", "criteria": { "low": "No financial or time-sensitive impact.", "normal": "Routine customer issue.", "high": "Financial impact or an urgent blocker." } } } }' ``` ## What you can do - [**Make decisions**](/docs/api-reference/decisions/create-decision) — Ask typed questions and get structured answers, probabilities, and confidence. - [**Create models**](/docs/api-reference/models/create-model) — Save questions and configuration in a versioned model repository. - [**Collect feedback**](/docs/api-reference/examples/annotate-model-example) — Add examples and corrections from domain experts. - [**Align models**](/docs/api-reference/alignments/start-alignment) — Adapt a model to your criteria and data distribution. - [**Review in the platform**](https://modaic.dev) — Inspect decisions, annotate examples, and manage alignment runs. ## Get started 1. [**Quickstart**](/docs/getting_started/quickstart) — Make your first decision. 2. [**Using models**](/docs/models/using_models) — Create and call a versioned model. 3. [**Aligning your model**](/docs/models/aligning_your_model) — Improve it with annotated examples. 22:Tabf, Make your first typed decision in a few minutes. This quickstart calls a Modaic base model directly, so you do not need to create a model repository first. ## 1. Create an API key Create a key in the [Modaic dashboard](https://modaic.dev/settings/keys), then store it in an environment variable. ```bash export MODAIC_API_KEY="mdc_..." ``` The key needs the `write:repository` scope to create decisions. ## 2. Ask typed questions Send any JSON value as `state`. Each key in `questions` becomes a key in the response's `answers` object. ```bash curl --request POST \ --url 'https://api.modaic.dev/v1/decision' \ --header "Authorization: Bearer $MODAIC_API_KEY" \ --header "Idempotency-Key: quickstart-$(date +%s)" \ --header 'Content-Type: application/json' \ --data '{ "state": { "ticket": "I was charged twice for order 4832. Please refund one charge." }, "model": "mo", "questions": { "needs_refund": { "type": "noul", "instructions": "Should this customer receive a refund?", "criteria": { "true": "A duplicate or invalid charge should be refunded.", "false": "The charge is valid or more information is required." } }, "priority": { "type": "choice", "instructions": "Choose the support priority.", "criteria": { "low": "No financial or time-sensitive impact.", "normal": "Routine customer issue.", "high": "Financial impact or an urgent blocker." } } } }' ``` Modaic returns one typed answer per question and reports token usage. ```json { "model": "mo", "answers": { "needs_refund": { "type": "noul", "noul": 0.97 }, "priority": { "type": "choice", "choice": "high", "probabilities": { "low": 0.01, "normal": 0.05, "high": 0.94 }, "confidence": 0.94 } }, "usage": { "input_tokens": 186, "output_tokens": 42 } } ``` ## 3. Make it versioned Create a model repository when you want to capture production examples, annotate decisions, run alignment, and pin exact revisions. Then call the same endpoint with a model path such as `acme/support-triage` instead of `mo`. ## Next steps - [Decision API](/docs/api-reference/decisions/create-decision) — all question types, capture controls, and response fields. - [Create a model](/docs/api-reference/models/create-model) — create a private, versioned model repository. - [Ingest examples](/docs/api-reference/examples/ingest-examples) — add a dataset and optional ground truth. - [Start an alignment](/docs/api-reference/alignments/start-alignment) — improve instructions from annotated examples. 23:T9c3, Use Modaic for repeatable decisions with a clear output and a human standard. Common examples include classifying content, scoring leads, routing tickets, and reviewing AI outputs. ## What makes a good use case? A task is a good fit when: - **The decision is bounded.** The output is a label, score, selection, or structured field—not an open-ended response. - **The criteria can be written down.** A policy, rubric, or expert judgment defines the correct answer. - **A human can review it.** Experts can confirm or correct the model's answer. - **It repeats at scale.** The same decision applies across many records, messages, documents, or model outputs. Modaic is not intended for open-ended chat, long-form generation, or autonomous workflows without a specific decision to evaluate.

## Types of decision tasks | Type | What it does | Examples | | --- | --- | --- | | **Classification** | Assign a label | Content policy, ticket category, medical code | | **Scoring** | Apply an ordered rubric | Lead quality, risk, severity, answer quality | | **Selection** | Choose an action or item | Approve, escalate, reject, next best action | | **Routing** | Choose a destination | Support queue, claims team, sales owner | | **Matching** | Compare two records | Duplicate detection, entity resolution, candidate fit | | **Verification** | Check against a standard | LLM-as-a-judge, policy compliance, SOP adherence |

## Common use cases | Team | Decisions | | --- | --- | | **AI product** | Evaluate agent outputs, detect hallucinations, route uncertain results | | **Trust and safety** | Classify violations, prioritize reports, apply moderation policy | | **GTM** | Qualify leads, score accounts, review outbound messages | | **Customer support** | Categorize tickets, set priority, choose the right queue | | **Human data** | Grade training data, verify labels, apply quality gates | | **Compliance** | Check policies, review access requests, detect sensitive data | | **Legal** | Classify contract risk, identify clauses, route documents for review | | **Health** | Review notes, check coding, route cases with appropriate oversight | Start with one decision that your team already reviews manually. Define the possible outputs and criteria, then [make your first decision](/docs/getting_started/quickstart). 24:Tb44, Every Modaic Model lives in a private repository. The repository is the Model's versioned source of truth: it records the root model, typed questions, checkpoints, metrics, capture policy, and alignment artifacts. ## Model identity A Model has two identifiers: - **Repository path** — `owner/slug`, such as `acme/support-triage`. Use this path in `POST /v1/decision`. - **Model ID** — a UUID returned when the repository is created. Use it in resource routes for examples, metrics, batch decisions, and alignments. Repository paths are human-readable. Model IDs remain stable if display metadata changes. ## The model configuration The root `model.json` file contains the decision contract for a revision. ```json { "schemaVersion": 1, "model": "mo", "checkpoint": 0, "questions": { "priority": { "type": "choice", "instructions": "Choose the support priority.", "criteria": { "low": "No time-sensitive impact.", "normal": "Routine customer issue.", "high": "Financial impact or an urgent blocker." } } }, "capture": { "sampleRate": 1 } } ``` | Field | Meaning | | --- | --- | | `schemaVersion` | Configuration format. Currently `1`. | | `model` | Root model used to make decisions. | | `checkpoint` | Monotonic version of aligned instructions. New Models start at `0`. | | `questions` | Named `noul`, `choice`, or `score` decision tasks. | | `metrics` | Optional accuracy and calibration measurements for the current checkpoint. | | `capture.sampleRate` | Fraction of live repository-model decisions to record, from 0 to 1. | ## Revisions Branches support ongoing work, tags can name releases, and commit SHAs pin an immutable Model revision. Pass `revision` to the Decision API when you need reproducible behavior; otherwise Modaic resolves the repository's default branch. Batch decisions record their exact `sourceCommitSha`. Alignments also require the current source commit and will refuse to update a branch that has advanced. This keeps evaluation and optimization attached to the Model version they actually used. ## Access roles Repository access has three roles: - **Reader** — view the Model, examples, decisions, metrics, and jobs. - **Writer** — update configuration, annotate examples, and start or cancel batch decisions and alignments. - **Admin** — change repository metadata and permanently delete the Model. Private resources return `404` to callers who cannot access them. ## Related endpoints - [Create model](/docs/api-reference/models/create-model) - [Get model](/docs/api-reference/models/get-model) - [Update model](/docs/api-reference/models/update-model) - [Delete model](/docs/api-reference/models/delete-model) - [Create decision](/docs/api-reference/decisions/create-decision) - [Start alignment](/docs/api-reference/alignments/start-alignment) 25:Tda4, Manually rewriting prompts for every new edge case doesn't scale well. Without a systematic way to evaluate changes, it's hard to know whether a revision improves the task or just fixes the last example you looked at. Alignment turns that trial and error into a repeatable process. Target decisions where the model is unsure, collect feedback from a larger reasoning model or a human expert, then align the Model against a measured baseline. Improved instructions become a new version of the same Model. A [Model](/docs/models/using_models) keeps the base model and its instructions together as one versioned decision primitive. Alignment updates its question instructions, not the base model's weights. When a run improves on the starting version, it creates a new repository commit and increments the Model's checkpoint. ## 1. Collect representative examples Examples can come from captured live decisions, explicit ingestion, or batch decisions. Each example keeps its input state, current annotation, and decision history. Start with low-confidence decisions where the model is unsure. Ask a larger reasoning model or a human expert to review them and provide the expected answer and reasoning. Include repeated mistakes and unclear policy boundaries too. You do not need to label everything; focus feedback where the model needs it most. ## 2. Add ground truth Annotate an example with answers keyed by question ID: - `noul` questions use a boolean. - `choice` questions use one configured option key. - `score` questions use the zero-based index into the question's criteria. Add concise `groundReasoning` when the correct answer depends on a policy or rubric distinction. The first annotation assigns a stable `train` or `test` split; later updates preserve it. Use [`PATCH /v1/models/{modelId}/examples/{exampleId}/annotation`](/docs/api-reference/examples/annotate-model-example) to add or update the annotation. ## 3. Start an alignment Ensure every question has string `instructions` before aligning. Unlike inference, alignment does not accept omitted, null, or other JSON instructions. Structured criteria descriptions are supported. Call [`POST /v1/models/{modelId}/alignments`](/docs/api-reference/alignments/start-alignment) with an `Idempotency-Key` header and: - the branch to update; - its exact current commit SHA; - a maximum metric-call budget; - optional reflection settings. The exact source commit prevents an alignment from overwriting a branch that changed after you prepared the request. If it advanced, the API returns a conflict and reports the current head. ## 4. Evaluate the result Poll the returned alignment URL until its status is `completed`, `failed`, or `cancelled`. Progress reports the current stage, scores, candidate count, and metric-call usage when available. An improving run: 1. commits optimized instructions and the alignment artifact; 2. increments the Model checkpoint; 3. records validation metrics; and 4. starts a batch decision for all examples at the new commit. If the candidate does not beat the starting instructions, the run completes without changing the Model. ## Feedback guidelines - Label genuine disagreements, not only obvious failures. - Keep ground truth consistent across reviewers. - Explain the policy distinction, not how to rewrite the prompt. - Pin decisions and alignments to exact commits when comparing checkpoints. - Re-run representative examples after important schema or rubric changes. 26:T6ae, Confidence helps you decide which Model decisions can proceed automatically and which deserve human review. It is most useful as a ranking and thresholding signal, not as a substitute for measuring agreement against labeled examples. ## Confidence by question type - A `noul` answer returns `noul`, the probability of `true` from 0 to 1. - A `choice` answer returns probabilities for every configured option and a `confidence` value from 0 to 1. - A `score` answer returns probabilities over the score legend and a `confidence` value from 0 to 1. ## Use confidence for review A practical workflow is: 1. Choose a confidence threshold for the decision's risk level. 2. Send decisions below the threshold to a human reviewer. 3. Add ground truth and concise reasoning to corrected examples. 4. Run alignment after you have representative feedback. 5. Re-measure agreement and calibration for the new checkpoint. Higher-risk decisions should use stricter thresholds and appropriate human oversight. Review a sample above the threshold as well, so systematic errors do not remain invisible simply because the Model was confident. ## Metrics Model checkpoints may include: - **Accuracy** — fraction of labeled question answers that match ground truth. - **AUROC** — how well confidence separates correct from incorrect decisions. - **ECE** — how closely stated confidence matches observed correctness. - **Samples** — number of labeled example-question pairs used for measurement. Compare metrics only when the evaluation set and question definitions are compatible. A schema or rubric change can make checkpoint-to-checkpoint numbers look comparable when they are measuring different tasks. 27:Tf1a, A **Model** is a versioned decision primitive: a base model and its instructions, managed as one artifact. Questions, instructions, and criteria define the task. You version, evaluate, and improve the combination together. Use a Model to decide which tickets need escalation, whether content meets your policy, or how well an agent completed a task. ## Beyond manual prompt engineering Manual prompt engineering doesn't scale well. It often becomes an unsystematic loop of tweaking wording, checking a few outputs, and trying again. A change that fixes one example can break another, and spot checks don't show whether the prompt is better across the cases that matter. Modaic makes improvement a repeatable process: target decisions where the model is unsure, collect feedback from a larger reasoning model or a human expert, then [align the Model](/docs/models/aligning_your_model) against a measured baseline. Improved instructions become a new version of the same Model, so you can track what changed and how it performed. ## What a model contains - **Root model** — the base model that evaluates incoming state. - **Questions** — named decision tasks with optional instructions and a fixed answer shape. - **Checkpoint** — the alignment iteration, reset when the root model or questions change. - **Metrics** — optional accuracy and calibration measurements for a checkpoint. - **Capture policy** — the fraction of live decisions saved as examples for review and alignment. Models live in private repositories. The repository path, such as `acme/support-triage`, is the identifier you pass to the Decision API. Model- scoped resource routes use the repository's UUID as `modelId`. Use [`GET /v1/models`](/docs/api-reference/models/list-models) to list supported base models and the repositories accessible to your token. ## Decision question types | Type | Use it for | Answer | | --- | --- | --- | | `noul` | Boolean judgment or likelihood | Probability of `true` from 0 to 1 | | `choice` | Select one configured option | Selected key, option probabilities, and confidence | | `score` | Score against an ordered rubric | Score over zero-indexed levels, legend, probabilities, and confidence | Question IDs become keys in the decision response. Keep IDs stable so stored ground truth and metrics continue to refer to the same task. An empty question ID (`""`) is valid. Choice criteria have no minimum option count, and score rubrics accept one or more levels. See the [decision reference](/docs/api-reference/decisions/create-decision) for the full question shapes and structured criteria examples. ## Create and configure a model Call [`POST /v1/models`](/docs/api-reference/models/create-model) with an owner and slug. The response `id` is the model's `modelId`. Call [`PATCH /v1/models/{modelId}`](/docs/api-reference/models/update-model) to set the root model and typed questions on a branch. Call [`POST /v1/decision`](/docs/api-reference/decisions/create-decision) with the repository path. Pin `revision` to a commit SHA when you need reproducible behavior. ## Live decisions and capture Repository-model decisions can save their state and answers as an example and decision record. The response tells you whether capture occurred and, when it did, returns `example_id` and `decision_id`. Set `capture: false` on an individual request to skip recording it. A model can also define `capture.sampleRate` from 0 to 1 to sample production traffic. Captured examples create the feedback loop: review a decision, add ground truth, then [align the model](/docs/models/aligning_your_model) to compile that feedback into a new checkpoint. 5:["$","$L10",null,{"title":"Cancel alignment","docs":[{"slug":["api-reference","alignments","cancel-alignment"],"source":"\nRequest cancellation of a queued or running alignment. Calling this endpoint\nfor a terminal alignment is a no-op.\n\n\n Alignment ID. Repository write access is required.\n\n\n## Response\n\nReturns `204 No Content`. Poll [Get alignment](/docs/api-reference/alignments/get-alignment)\nto observe the terminal `cancelled` state.\n","title":"Cancel alignment","description":"Request cancellation of a running alignment.","api":"DELETE /v1/alignments/{alignmentId}"},{"slug":["api-reference","alignments","get-alignment-logs"],"source":"\nRead the available alignment log as an array of lines. You can poll this\nendpoint while an alignment is running.\n\n\n Alignment ID.\n\n\n## Response\n\n\n Log lines in display order. The array is empty until output is available.\n\n\n\n Whether any log output is currently available.\n\n\n\n `true` while the alignment has not reached a terminal state.\n\n","title":"Get alignment logs","description":"Read the current log output for an alignment.","api":"GET /v1/alignments/{alignmentId}/logs"},{"slug":["api-reference","alignments","get-alignment"],"source":"$11","title":"Get alignment","description":"Poll an alignment's status, progress, and result.","api":"GET /v1/alignments/{alignmentId}"},{"slug":["api-reference","alignments","list-alignments"],"source":"\nList recent alignment jobs for a model, newest first.\n\n\n Model repository ID. Repository read access is required.\n\n\n\n Maximum number of jobs to return, from 1 to 100.\n\n\n## Response\n\n\n Alignment resources with status, phase, source and result commits, result,\n progress, error, and timestamps.\n\n","title":"List alignments","description":"List recent alignment jobs for a model.","api":"GET /v1/models/{modelId}/alignments"},{"slug":["api-reference","alignments","start-alignment"],"source":"$12","title":"Start alignment","description":"Improve a model's instructions from annotated examples.","api":"POST /v1/models/{modelId}/alignments"},{"slug":["api-reference","batch-decisions","cancel-batch-decision"],"source":"\nRequest cancellation of a queued or running batch decision. Calling this\nendpoint for a terminal job is a no-op.\n\n\n Batch decision ID. Repository write access is required.\n\n\n## Response\n\nReturns `204 No Content`. Cancellation is best-effort; individual decisions\nthat completed before the cancellation remain available.\n","title":"Cancel batch decision","description":"Request cancellation of a running batch decision.","api":"DELETE /v1/batch-decisions/{batchDecisionId}"},{"slug":["api-reference","batch-decisions","get-batch-decision"],"source":"$13","title":"Get batch decision","description":"Poll the status and progress of a batch decision.","api":"GET /v1/batch-decisions/{batchDecisionId}"},{"slug":["api-reference","batch-decisions","list-batch-decisions"],"source":"\nList recent batch decisions for a model, newest first.\n\n\n Model repository ID. Repository read access is required.\n\n\n\n Maximum number of jobs to return, from 1 to 100.\n\n\n## Response\n\n\n Batch resources with status, phase, exact source commit, progress, result,\n error, and timestamps.\n\n","title":"List batch decisions","description":"List recent batch decisions for a model.","api":"GET /v1/models/{modelId}/batch-decisions"},{"slug":["api-reference","batch-decisions","start-batch-decision"],"source":"$14","title":"Start batch decision","description":"Run a repository model asynchronously over stored or inline examples.","api":"POST /v1/models/{modelId}/batch-decisions"},{"slug":["api-reference","decisions","create-decision"],"source":"$15","title":"Create decision","description":"Run typed questions against JSON state with a base or repository model.","api":"POST /v1/decision"},{"slug":["api-reference","decisions","systemone"],"source":"$16","title":"SystemOne alias","description":"Run a decision using the SystemOne-compatible route alias.","api":"POST /v1/systemone"},{"slug":["api-reference","examples","annotate-model-example"],"source":"$17","title":"Annotate example","description":"Add or update ground truth and reasoning for an example.","api":"PATCH /v1/models/{modelId}/examples/{exampleId}/annotation"},{"slug":["api-reference","examples","get-model-example"],"source":"$18","title":"Get example","description":"Get one example with its annotation and latest decision.","api":"GET /v1/models/{modelId}/examples/{exampleId}"},{"slug":["api-reference","examples","ingest-examples"],"source":"$19","title":"Ingest examples","description":"Add examples and optional ground truth to a model.","api":"POST /v1/models/{modelId}/examples"},{"slug":["api-reference","examples","list-example-decisions"],"source":"\nList all decisions recorded for an example, newest first. The response includes\nsuccessful and failed batch records as well as captured live decisions.\n\n\n Model repository ID.\n\n\n\n Example ID.\n\n\n## Response\n\n\n Each record includes `id`, `exampleId`, `commitSha`, `model`, `answers`, the\n complete `request`, nullable `response`, nullable `confidence`, `checkpoint`,\n `revision`, `source` (`batch` or `live`), nullable `jobId`, nullable `error`,\n image URLs, and `createdAt`.\n\n","title":"List example decisions","description":"List every live and batch decision recorded for an example.","api":"GET /v1/models/{modelId}/examples/{exampleId}/decisions"},{"slug":["api-reference","examples","list-model-examples"],"source":"$1a","title":"List examples","description":"List a model's examples, annotations, and latest decisions.","api":"GET /v1/models/{modelId}/examples"},{"slug":["api-reference","models","create-model"],"source":"$1b","title":"Create model","description":"Create a private, versioned Modaic model repository.","api":"POST /v1/models"},{"slug":["api-reference","models","delete-model"],"source":"\nPermanently delete a model. Repository admin access is required.\n\n\n Model deletion cannot be undone. The model, its version history, examples,\n decisions, annotations, and associated media are permanently removed.\n\n\n\n Model repository ID.\n\n\n## Response\n\nReturns `204 No Content` when the model is deleted.\n\nThe endpoint returns `403` when the caller lacks repository admin access and\n`404` when the model does not exist or is not accessible.\n","title":"Delete model","description":"Permanently delete a model and its associated data.","api":"DELETE /v1/models/{modelId}"},{"slug":["api-reference","models","get-model"],"source":"\nGet a model repository that the authenticated caller can access. Inaccessible\nmodels return `404` rather than revealing that a private model exists.\n\n\n User or organization slug.\n\n\n\n Model repository slug.\n\n\n## Response\n\n\n The `modelId` used by examples, batch decisions, alignments, and metrics.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","title":"Get model","description":"Get accessible model metadata by owner and slug.","api":"GET /v1/entities/{owner}/models/{model}"},{"slug":["api-reference","models","list-models"],"source":"$1c","title":"List models","description":"Discover supported base models and model repositories accessible to your token.","api":"GET /v1/models"},{"slug":["api-reference","models","update-model"],"source":"$1d","title":"Update model","description":"Update a model's root model, typed questions, or description.","api":"PATCH /v1/models/{modelId}"},{"slug":["api","authentication"],"source":"$1e","title":"Authentication","description":"Authenticate requests with Modaic API keys and choose the scopes your integration needs.","api":"$undefined"},{"slug":["api","errors"],"source":"$1f","title":"Errors","description":"Handle Modaic API errors, request identifiers, validation failures, and safe retries.","api":"$undefined"},{"slug":["api"],"source":"$20","title":"API overview","description":"Run decisions, collect expert feedback, and align models to your team's criteria.","api":"$undefined"},{"slug":["getting_started","introduction"],"source":"$21","title":"Introduction","description":"Run and align Decision Models for production","api":"$undefined"},{"slug":["getting_started","quickstart"],"source":"$22","title":"Quickstart","description":"Make your first typed decision with the Modaic API.","api":"$undefined"},{"slug":["getting_started","use_cases"],"source":"$23","title":"Use cases","description":"Find repeatable, high-volume decisions that are a good fit for Modaic.","api":"$undefined"},{"slug":["modaic_platform","repositories"],"source":"$24","title":"Model repositories","description":"Version Modaic Models, questions, checkpoints, metrics, and alignment artifacts.","api":"$undefined"},{"slug":["models","aligning_your_model"],"source":"$25","title":"Aligning your model","description":"Move from ad hoc prompt edits to a repeatable process guided by expert feedback and evaluation.","api":"$undefined"},{"slug":["models","confidence_estimation"],"source":"$26","title":"Confidence estimation","description":"Interpret confidence and use it to prioritize review for Modaic Models.","api":"$undefined"},{"slug":["models","using_models"],"source":"$27","title":"Using models","description":"Treat a base model and its instructions as one versioned artifact, and improve decisions through expert feedback.","api":"$undefined"}],"toc":[{"depth":2,"text":"Response","id":"response"}],"markdown":"# Cancel alignment\n\n\nRequest cancellation of a queued or running alignment. Calling this endpoint\nfor a terminal alignment is a no-op.\n\n\n Alignment ID. Repository write access is required.\n\n\n## Response\n\nReturns `204 No Content`. Poll [Get alignment](/docs/api-reference/alignments/get-alignment)\nto observe the terminal `cancelled` state.\n","api":"DELETE /v1/alignments/{alignmentId}","apiGroup":"alignments","apiSidebar":"$L28","children":"$L29"}]

Cancel alignment

Parameters

Response