Examples with the SDK
Examples are known data points (inputs and their labels) used for evaluation and alignment. They can include the ground truth decision and rationale during ingestion or be annotated later.
Setup
from modaic import Modaic
modaic = Modaic()
model = modaic.models.get(workspace="acme", model="support-priority")
Ingest examples
Models returned by models.create, models.get, or models.update expose
model.examples. The model ID is supplied automatically. With AsyncModaic,
await each method. The top-level modaic.examples methods remain available.
created = model.examples.ingest(
examples=[
{
"state": {"ticket": "I was charged twice."},
"annotation": {
"ground_truth": {"needs_refund": True, "priority": "high"},
"ground_reasoning": "The ticket reports a duplicate charge.",
},
},
{"state": {"ticket": "How do I change my profile picture?"}},
],
)
example_id = created.examples[0].id
Supply an id on each input when retries should identify the same example.
When an example first receives ground truth, Modaic assigns a stable train or
test split. Updating the annotation later preserves that split.
List and retrieve
page = model.examples.list(page=1, page_size=30)
example = model.examples.get(page.items[0].id)
print(page.total, page.total_pages)
print(example.state, example.annotation, example.latest_decision)
Annotate and inspect history
updated = model.examples.annotate(
example_id,
ground_truth={"needs_refund": True, "priority": "high"},
ground_reasoning="Confirmed duplicate charge.",
)
history = model.examples.list_decisions(example_id)
for decision in history.decisions:
print(decision.source, decision.answers, decision.error)
Ground-truth values are keyed by question ID: booleans for noul, option keys
for choice, and zero-based numeric levels for score questions.