A staff member opens an email in Front. The AI has placed it in the wrong operational queue. She corrects the tag, moves the conversation, and gets on with her day.
Ordinary software forgets that moment. The same edge case returns next week, and another employee makes the same repair. At small volume this is irritating. Across the more than 600 daily emails handled by Precise Imaging, repeated corrections become a real operating cost.
Our work with Precise Imaging began with email triage: reading incoming messages, applying operational tags, routing work, handling documents, and preparing approved responses. The more interesting problem came next. How could the system retain what staff learned from real work without allowing it to change itself recklessly?
The answer was not model training in the conventional sense. It was an operating loop connecting production usage, human judgment, editable instructions, and regression tests.
How the learning loop works in a governed system
Writing about AI-enabled innovation, former Google CEO Eric Schmidt argues that as experimentation and iteration become cheaper, the constraint becomes how quickly an organization can learn. His conclusion is direct: "those who embrace that loop will outpace those still trying to avoid it."
A learning loop does not mean the production model quietly rewrites its own rules after every correction. One person's preference should not become company policy merely because it was expressed last.
In our design, the system learns through a controlled sequence. Staff identify an error in the tool they already use. The full conversation, original decision, correction, and explanation become evidence. At a regular cadence, that evidence is reviewed for recurring patterns. The smallest useful improvement is proposed, tested, reviewed, and only then released.
The loop is automated where repetition helps and deliberately human where judgment matters. That boundary is what makes learning safer than simply accumulating prompts or retraining on every correction.
Keep the business logic in SKILL.md files
The first design decision was to separate what the software does from what Precise Imaging knows.
Application code handles the plumbing: receiving a Front webhook, reading a thread, calling a model, applying an approved action, recording a trace, and handling failures. A skill file contains the operational instructions: how to interpret a request, which facts matter, what a good result contains, and when the system must stop and send the work to a person.
| Code owns | The skill owns |
|---|---|
| APIs, queues, permissions, logging, and retries | Firm terminology, policy, decision rules, and examples |
| How information moves safely | What the information means operationally |
| Execution and observability | Expected output, exceptions, and human handoffs |
A Markdown skill is readable by an operator, reviewable in a code change, versioned over time, and portable across models. It turns the company's accumulated corrections into an asset the company can inspect and own. It also gives the learning step a bounded place to propose improvements.
We do not expose Precise's actual skill files. They contain private operating logic, language, and exception handling. The reusable lesson is the boundary: keep proprietary business judgment outside the application plumbing.
Build evals before you start improving the system
We learned this in the wrong order. Early in the project, we focused on getting the production workflow to work and treated systematic evals as something we could add later. That delay cost us.
The cost was uncertainty. A rule change could fix the conversation in front of us while changing behavior somewhere else. Verifying releases required more manual checking, debugging took longer, and a convincing example could create more confidence than it deserved.
An eval converts a reviewed example into a durable test. It includes a representative input, the human-approved result, and the conditions that determine whether the system passed. The Precise suite covers ordinary requests, ambiguous language, multi-step threads, document-driven work, dangerous edge cases, and situations where the correct action is to defer to staff.
Production failures are especially valuable. Once reviewed and stripped of unnecessary identifying information, they can become regression cases. The mistake may happen once in live work, but it should remain in the test set for every future version.
This is the practical meaning behind why AI evals matter: they create memory for the system's quality standard.
Capture feedback where humans already work
A learning loop fails when reporting an error requires a second system, a long form, or a meeting with the AI team. The people who see the most useful exceptions are usually busy resolving them.
Precise's team works in Front, a shared CRM inbox. We used that existing interface as the feedback mechanism. When an AI-assisted result needs attention, a staff member can apply the AI:Training tag to the conversation. They can correct the operational tag or output and add an internal comment explaining what was wrong or what should have happened.
The tag says: review this
It creates a visible queue of production examples without interrupting the staff member's normal workflow.
The comment says: here is why
The explanation preserves human reasoning that a corrected label alone cannot capture.
The training record can retain the original conversation, the AI's decision, the corrected outcome, the staff explanation, and the trace of how the system reached its result. A correction is evidence, however, not an automatic instruction. It enters a review queue first.
Add a regular learning step
Collecting feedback is not learning. Someone, or something, has to turn examples into a better shared method.
At a regular cadence, the Precise loop reviews newly flagged Front conversations together with their human comments and system traces. It looks for recurring patterns rather than treating every correction as a new universal rule. When a pattern is credible, the system proposes the smallest change to the relevant skill and identifies the eval cases that should be added or updated.
A person reviews the proposal. They can reject it, narrow it, or approve it for testing. This protects the system from contradictory feedback, one-off preferences, malicious instructions inside an email, and the temptation to overfit a rule to the most recent failure.
The learning step produces a proposed, reviewable change. It does not grant the production agent permission to edit its own instructions.
Re-run the eval suite after every update
The proposed change must solve the target problem and preserve behavior elsewhere. That second requirement is where intuition is least reliable.
Every approved skill edit triggers the relevant eval suite. The tests replay reviewed examples and compare the new output with the expected result. A release can be stopped when it introduces a regression, weakens a human-review boundary, or improves one category by making another less reliable.
- TargetDid the update fix the pattern that prompted it?
- RegressionDid previously correct scenarios remain correct?
- RestraintDoes the system still defer when evidence is incomplete?
- TraceCan a reviewer understand which version produced the result?
- RollbackCan the prior skill version be restored if production disagrees?
Models will change. Vendors will change. The eval set remains the company's definition of acceptable behavior. That is why the durable asset is not a particular model; it is the combination of the firm's skill, feedback history, and tests.
Profit from fewer repeated mistakes
The economics of the learning loop come from repetition. A single corrected email saves little. A correction that prevents a recurring error across a 600-plus-email day can save attention every day thereafter.
The return appears as less manual sorting, fewer repeated corrections, faster handling of routine requests, more consistent responses, and more staff capacity for exceptions that require judgment. It also reduces dependence on the one experienced employee who remembers every unusual case.
The loop does not guarantee that every version is better or that every AI project pays off. It gives the company a disciplined way to find out. Each correction can improve the shared system instead of disappearing into one person's working memory.
That is the compounding advantage: production creates feedback, feedback improves the skill, evals protect the improvement, and the next day's work begins from a stronger baseline.
How a PI firm can build the same kind of loop
The workflow does not need 600 daily emails to justify learning. A PI firm can begin with one repetitive, reviewable process such as after-hours intake, lead follow-up, routine client updates, or records chasing.
- 01Choose one workflowPick work with repeated inputs, visible human corrections, and a measurable outcome.
- 02Write the first skillDocument the trigger, required facts, expected result, exceptions, and human handoff.
- 03Create the eval setStart with reviewed ordinary cases, ambiguity, edge cases, and examples where the AI must stop.
- 04Reuse the staff interfaceAdd a feedback tag, status, or button inside the intake, case, or inbox system people already use.
- 05Review on a cadenceGroup similar corrections, propose the smallest rule change, and have an accountable person approve it.
- 06Gate every releaseRun the complete eval set, inspect failures, version the skill, and keep rollback available.
For intake, staff might flag a conversation when the system missed a competitive-shopping signal, misunderstood case facts, or routed a serious matter too slowly. The correction becomes a reviewed example. The new rule is tested against ordinary inquiries, emotional callers, ambiguous facts, and cases that still require a lawyer's judgment.
This is also why a learning loop should sit inside a governed AI workflow. Access, retention, approval authority, audit history, and human review are part of the system, not paperwork added after deployment.
Frequently asked questions
What is an AI learning loop?
An AI learning loop is a controlled process that captures human corrections, finds recurring error patterns, improves the system's instructions, and tests every change before release. The goal is to prevent the same operational mistake from recurring.
Does an AI learning loop retrain its model automatically?
Not necessarily, and ours does not depend on autonomous model retraining. The loop improves the business instructions, examples, evaluation cases, and workflow around the model. Humans still approve changes and production releases.
Why keep AI business logic in SKILL.md files?
Separating operational instructions from application code makes the firm's rules easier to inspect, review, version, test, and move between models. Code runs the machinery; the skill describes how the business wants the work performed.
How did Precise Imaging staff give the AI feedback?
Staff stayed inside Front, the CRM inbox they already used. They could apply an AI:Training tag to a conversation, correct the operational result, and add a comment explaining what should have happened. That created a reviewable feedback queue without requiring a separate reporting workflow.
How can a PI firm build a similar learning loop?
Choose one bounded workflow, write down its rules, create reviewed examples with expected results, capture corrections in the system staff already use, review feedback on a regular cadence, and require the full eval set to pass before each update reaches production.
Start with a workflow your team can correct.
Possible Minds builds governed AI systems around real operating work, then gives the people doing that work a practical way to teach the system over time.