# AI Tech Magic > Practical writing by Virginia Kalesic on manufacturing AI, source-grounded assistants, evaluation and production readiness. Public Ghost content for AI and LLM tooling. This file includes a bounded export of public pages first, then recent public posts. Append `.md` to any post or page URL to get the content in Markdown (for example, `/example-post.md`). ## Pages ### About AI Tech Magic URL: https://blog.aitechmagic.com/about/ Last updated: 2026-09-29T08:43:02.000Z AI Tech Magic is about making AI useful in the places where a confident wrong answer has consequences. I'm Virginia Kalesic. I build source-grounded assistants for manufacturers, turning approved manuals, procedures and accumulated operational knowledge into answers people can trace back to the source. ## The experience behind the practice Earlier, I led AI and machine learning for Emerson's measurement business. I built and led a global team and established an AI governance process for reviewing systems before they entered use. My experience also includes program management at Medtronic and Zeiss, and engineering at IBM. That experience shapes the work here: define a useful job, establish the evidence, test the boundaries and make responsibility clear before deployment. ## What I write about These notes cover the work behind a dependable system: clear instructions, representative tests, source quality, refusal behavior, human review and the practical decisions that connect a prototype to daily operations. Some essays use small teaching tools, including Dish Lab, to make those decisions visible. The point is to give people something they can inspect, question and improve. ## Work with me The primary practice is a scoped assistant build for one manufacturing department. I also offer a separately scoped pre-production review for consequential AI agents through Launch Gate. [Explore the manufacturing work](https://aitechmagic.com/?ref=blog.aitechmagic.com) or [see Launch Gate](https://aitechmagic.com/launch-gate?ref=blog.aitechmagic.com). ## Follow the writing Subscribe if you would like new writing by email. You can unsubscribe using the option in a newsletter. For questions about the work, your subscription or personal information, contact info@aitechmagic.com. ## Posts ### What AI-supported schools can teach us about agent specs URL: https://blog.aitechmagic.com/alpha-schools-agent-spec/ Last updated: 2026-09-29T08:34:21.000Z The debate around AI-supported schools raises a useful systems question: what, exactly, should the AI be responsible for? This article uses AI-supported schooling as an illustrative design exercise. It is not a verified description of Alpha Schools' internal architecture, staffing, pricing or educational results. The exercise is to define a bounded job for the system, the evidence that the job is being done well, and the decisions that remain with a person. ## Start with a bounded task "Help students learn" is a purpose. A specification turns that purpose into a task that can be examined. For example, an assistant might offer practice questions on a defined topic using an approved curriculum. Its scope could exclude grading consequential assessments, making welfare judgments or changing a student's learning plan without review. The point is to make the boundary explicit enough that a teacher, parent or system reviewer can question it. ## Define evidence of progress Decide what an acceptable result would look like before choosing how to measure it. In this example, the team might inspect whether the questions match the intended topic, whether explanations are correct and whether a learner can apply the idea independently. Those checks would be part of an evaluation plan. They would not, by themselves, prove that the system improves education overall. The same distinction matters for an enterprise agent. Completing a demonstration is one observation; supporting a deployment decision requires evidence tied to the actual job. ## Give the person an explicit role A useful specification names the decisions a person owns. In the school example, that could include resolving uncertainty about a learner's needs, reviewing an inappropriate response and deciding whether the system should continue to be used. Write down who receives an escalation, what information they receive and what authority they have. The human role should be visible in the operating model and included in the time and resources needed to run it. ## Write three decisions before scaling For an AI agent, I want three decisions in writing: 1. **What the system should do.** Define the positive task, its approved inputs and the limits of its authority. 2. **What it must not do.** Define prohibited actions, failure conditions and the response when a boundary is reached. 3. **Where a person owns the outcome.** Name the reviewer, the escalation route and the decisions that require human approval. In a manufacturing assistant, this might mean answering from approved procedures, refusing to invent an unsupported setting and directing conflicting instructions to an authorized reviewer. These decisions do not guarantee a successful deployment. They make it possible to build a testable system and to judge the evidence before expanding its use. ## Bring the specification into the release decision Before adding users, documents or workflows, review the scope and the tests together. Ask which assumptions have changed, which failures remain unresolved and who has authority to approve the next step. The lesson of the design exercise is simple: scope, evidence and human responsibility belong in the system from the beginning. For a source-grounded assistant in one manufacturing department, [explore AI Tech Magic](https://aitechmagic.com/?ref=blog.aitechmagic.com). For an independent review of an AI agent before release, [see Launch Gate](https://aitechmagic.com/launch-gate?ref=blog.aitechmagic.com). Virginia Kalesic is the founder of AI Tech Magic. She builds source-grounded assistants for manufacturers and reviews AI systems before production. ### Specification: the work before the prompt URL: https://blog.aitechmagic.com/specification-engineering/ Last updated: 2026-09-29T08:31:42.000Z The most useful work often happens before the prompt. It is the work of deciding what you want the system to do, what information it should use, and how you will recognize a result worth using. I think of that work as specification. It gives the model, the prompt and the evaluation a shared job to serve. ## Decide what the system should know and do A specification describes the task before committing to an implementation. It names the relevant context, the intended user, the permitted actions and the conditions under which the system should stop. The model you choose should fit that task. The context you provide and the prompt you write should express the same decisions. This is my approach to specifying an AI system: define the task, operating boundary, evidence and tests before relying on its output. ## Keep the specification when the implementation changes A prompt is one way of expressing a task to a particular model. A specification records what the task actually requires. That distinction gives you something stable to evaluate when you change models, prompts or retrieval methods. The implementation can change while the expected outcome remains explicit. For an operator assistant, the requirement might be to answer from approved procedures, identify the relevant source page and stop when the source does not support an answer. Those requirements are useful regardless of which implementation you test. ## Put process before technology At the start of an AI project, I want the team to name the work, the owner and the evidence that will count as success. The same discipline is useful for a small task in a chat window. If you are asking for a summary, specify its audience, the documents it may use and what must remain visible. If you are asking for a recommendation, specify the decision, the constraints and who makes the final call. ## Specify the task in four lines Before your next piece of AI work, write four short lines: - **Know:** What context, inputs and constraints does the system need? Name them and identify the approved sources. - **Do:** What is the actual job, and what is explicitly out of scope? - **Good:** What would a result you could use look like? Give a concrete example or an acceptance test. - **Edges:** Where should the system stop, what should it leave out, and when should it ask a person rather than proceed? These lines give you a starting specification. For consequential work, develop them into a fuller set of permissions, failure cases and tests before deployment. ## Apply it to a manufacturing assistant **Know:** The current approved procedures for one department, with document versions and access permissions. **Do:** Answer an operator's question and identify the passage supporting the answer. **Good:** The answer is supported by the current procedure, the citation is usable, and representative operator questions meet the agreed acceptance criteria. **Edges:** Stop when information is missing, permissions do not allow access, or conflicting documents require an authorized person to decide. This is an illustrative specification, not a substitute for a plant's safety procedures. The team still needs to define the actual scope, approvals and tests for its equipment and workflow. ## Keep judgment visible The specification records choices about what matters, what is acceptable and who is accountable. Those choices deserve the same attention as the model and interface. For a source-grounded assistant in one manufacturing department, [explore AI Tech Magic](https://aitechmagic.com/?ref=blog.aitechmagic.com). For an independent pre-production review of an AI agent, [see Launch Gate](https://aitechmagic.com/launch-gate?ref=blog.aitechmagic.com). Virginia Kalesic is the founder of AI Tech Magic. She builds source-grounded assistants for manufacturers and reviews AI systems before production. ### Why AI pilots stall before production URL: https://blog.aitechmagic.com/why-ai-agent-pilots-fail/ Last updated: 2026-09-29T08:28:15.000Z A demo answers a narrow question: can this work once? A production decision asks a harder one: under what conditions can people rely on it, and what should happen when those conditions are not met? When I review an AI pilot, I start with that second question. The work is to turn a promising result into an explicit operating boundary, a set of tests and a clear decision about release. ## Define the job Write down who will use the system, what decisions it supports, which documents or tools it may use, and what sits outside its scope. For a manufacturing knowledge assistant, that might mean answering from a defined collection of approved procedures, showing the relevant page, and declining to invent a value that the sources do not contain. That boundary should be understandable to the operator and the person approving the deployment. ## Test the questions people actually ask A useful evaluation set includes routine questions, ambiguous wording, missing information and situations where sources disagree. It also includes questions the system should refuse. Record the expected outcome before changing the implementation. Then evaluate the change against those expectations, including whether the answer remains grounded in the approved source. ## Make the source inspectable An answer should give the reader a way to inspect the evidence behind it. A citation is useful when it points to material that actually supports the answer and that the user is permitted to access. Source versioning matters too. Agree who approves new documents, how retired procedures are handled, and which version is authoritative when two documents conflict. ## Design the handoff to a person "I don't know" needs a next step. Decide when the assistant stops, what context a reviewer receives, and who has authority to resolve the question. For operational work, the assistant's response must sit within the organization's existing safety and approval procedures. Those procedures continue to govern the work. ## Name the owner after launch Before release, identify who maintains the source collection, reviews failed answers, approves changes and responds when the service is unavailable. Base the release decision on a defined scope and acceptance tests, with known limitations written down. The prototype gives the team evidence to evaluate; the acceptance process determines what is ready for use. ## Start with one department AI Tech Magic builds source-grounded assistants around one manufacturing department and an agreed collection of documents. We define scope and acceptance tests before the build. [Explore the manufacturing offer](https://aitechmagic.com/?ref=blog.aitechmagic.com), or [book a 30-minute conversation](https://cal.com/aitechmagic/30-minute-call?ref=blog.aitechmagic.com). For a separately scoped review of an AI agent before release, [see Launch Gate](https://aitechmagic.com/launch-gate?ref=blog.aitechmagic.com).