Skip to content
Amber

Docs

Guided authoring

Draft Tasks from a description of your work and review them in the browser before publishing.

Guided authoring is a web wizard that drafts Tasks for you from a description of your work, so you can measure Model Configurations on your own problems without writing Task folders by hand.

#How it works

  1. Describe the work. Say what you want an Agent to be able to do, and optionally point Amber at your repository: a public https git URL, or a snapshot you upload (a .tar.gz, .tgz, .tar or .zip of at most 100 MB, and 50 MB of files once unpacked, not counting .git).
  2. Accept the Quote. Before anything is spent you get an authoring Quote: the authoring Model's calls and validating every draft, plus the safety margin and the platform fee.
  3. Amber drafts Tasks. An authoring Model writes Task folders: an Instruction, a Dockerfile for the Environment, tests for the Verifier and a reference solution. By default this is the strongest Anthropic Model Configuration.
  4. Amber validates each draft. The oracle Agent must get a Reward of 1 and the nop Agent a Reward of 0. A draft that fails is revised by the authoring Model, for up to three revision rounds; one still failing after that is shown with why it failed.
  5. You review and publish. Read and edit every file in the browser. Edited drafts are validated again before they can be published, which gets its own Revalidation Quote (validation only, no authoring Model calls). Publishing the drafts you choose creates a new private Benchmark Version.

#Before you publish

Treat the drafts as a starting point. Check that each Verifier tests the outcome you care about, that the Instruction does not leak the answer, and that the reference solution does what a good engineer would do. The Task format page lists what makes a Task good.

#Cost

Drafting, revising and validating all use your Wallet, and only after you accept a Quote. The authoring Quote budgets about a first draft and one revision per Task; if it runs out, revising stops early and you keep the drafts so far. You're charged as each round finishes, and the unused part of the Quote goes back to the Wallet when the session's work ends.

#Your repository

Amber treats the repository's contents as untrusted: it reads them to brief the authoring Model and builds them only inside validation Environments. Your repository snapshot and drafts are used only to serve your Workspace.

#Done-for-you

If you would rather not author Tasks yourself, Amber staff can build private Benchmarks for you. This needs you to turn on "allow Amber staff to manage benchmarks" in your Workspace settings, and every action they take is recorded in an audit log.