What AI-supported schools can teach us about agent specs
An illustrative design exercise: define the AI's task, the evidence of progress and the decisions that remain with a person.
The debate around AI-supported schools raises a useful systems question: what, exactly, should the AI be responsible for?
This article uses AI-supported schooling as an illustrative design exercise. It is not a verified description of Alpha Schools' internal architecture, staffing, pricing or educational results.
The exercise is to define a bounded job for the system, the evidence that the job is being done well, and the decisions that remain with a person.
Start with a bounded task
"Help students learn" is a purpose. A specification turns that purpose into a task that can be examined.
For example, an assistant might offer practice questions on a defined topic using an approved curriculum. Its scope could exclude grading consequential assessments, making welfare judgments or changing a student's learning plan without review.
The point is to make the boundary explicit enough that a teacher, parent or system reviewer can question it.
Define evidence of progress
Decide what an acceptable result would look like before choosing how to measure it. In this example, the team might inspect whether the questions match the intended topic, whether explanations are correct and whether a learner can apply the idea independently.
Those checks would be part of an evaluation plan. They would not, by themselves, prove that the system improves education overall.
The same distinction matters for an enterprise agent. Completing a demonstration is one observation; supporting a deployment decision requires evidence tied to the actual job.
Give the person an explicit role
A useful specification names the decisions a person owns. In the school example, that could include resolving uncertainty about a learner's needs, reviewing an inappropriate response and deciding whether the system should continue to be used.
Write down who receives an escalation, what information they receive and what authority they have. The human role should be visible in the operating model and included in the time and resources needed to run it.
Write three decisions before scaling
For an AI agent, I want three decisions in writing:
- What the system should do. Define the positive task, its approved inputs and the limits of its authority.
- What it must not do. Define prohibited actions, failure conditions and the response when a boundary is reached.
- Where a person owns the outcome. Name the reviewer, the escalation route and the decisions that require human approval.
In a manufacturing assistant, this might mean answering from approved procedures, refusing to invent an unsupported setting and directing conflicting instructions to an authorized reviewer.
These decisions do not guarantee a successful deployment. They make it possible to build a testable system and to judge the evidence before expanding its use.
Bring the specification into the release decision
Before adding users, documents or workflows, review the scope and the tests together. Ask which assumptions have changed, which failures remain unresolved and who has authority to approve the next step.
The lesson of the design exercise is simple: scope, evidence and human responsibility belong in the system from the beginning.
For a source-grounded assistant in one manufacturing department, explore AI Tech Magic. For an independent review of an AI agent before release, see Launch Gate.
Virginia Kalesic is the founder of AI Tech Magic. She builds source-grounded assistants for manufacturers and reviews AI systems before production.