✍️ Step 2: Assign a Real Task, Expect a Flawed First Draft
Once the assistant has its brief, give it its first real-world assignment, a real input it's never seen before, and see what it produces. The expectation matters here as much as the task itself, expect it to get things wrong. Just like a junior employee, its first attempt will reveal the gaps in its understanding.
|
The Five-Step Process, In Brief
|
5
core components required in the initial job brief
|
|
2 yrs
of building specialized assistants behind Barnard's method
|
|
2–3
measurable success criteria recommended per job brief
|
|
That reframing of a bad first draft as useful data, not wasted effort, is a genuinely healthy way to approach this. A flawed output isn't a sign the process failed, it's the diagnostic step that tells you exactly what needs correcting next, the same way a new employee's early mistakes reveal exactly what training they still need.
|
A flawed first draft is diagnostic information, not a failure, revealing exactly which gaps the AI's instructions still need to close.
|
🗣️ Step 3: Feedback, Not a New Set of Rules
This is where the real coaching begins, and Barnard is specific about the structure that works. Present the critique as, first, a reminder of the input you provided, second, the output the AI generated, and third, your actual critique, stated specifically. His example, "The tone was excellent, but you failed to mention the key statistic about Q3 revenue, which was a critical part of the input. That statistic needs to be in the first paragraph."
"Ask it to rewrite its own instructions to incorporate this feedback."
That last instruction is the subtle, important part, rather than the user rewriting the rules themselves, the AI is asked to update its own instructions based on the specific feedback given. Barnard recommends staying in a single conversation throughout this phase, so the model is building on existing context, adding new rules, refining old ones, and, crucially, learning why the previous version fell short, not just what changed.
|
🔍 Step 4: The Step Almost Everyone Skips
Here's the step Barnard calls the most important, and the one he says most people miss entirely. When you ask an AI to "tweak" its instructions, it can sometimes remove or alter foundational rules in the process of incorporating new feedback, a pattern he calls "instructional drift."
|
Fixing Instructional Drift
| ⚠️ Don't trust the assistant to self-regulate its own updated instructions |
| ⚠️ Compare the newly generated instructions against the previous version directly |
| ⚠️ If a rule is missing, say so explicitly and ask what else may have been dropped |
|
Barnard's own recovery prompt is worth stealing directly, "You have removed the rule about X. That is still a core requirement. Please add it back in, and then tell me what other essential instructions you may have omitted in your update." That second half of the sentence does real work, it turns the correction into an audit, rather than a one-off patch.
|
Comparing updated instructions against the previous version catches rules that quietly disappeared during a "tweak."
|
🧠 AI Spotlight Analysis
The fifth and final step is simply to repeat the cycle, assign, review, critique, and audit, until the AI's understanding becomes more nuanced and its performance more reliable. But Barnard adds one last, genuinely counterintuitive piece of advice at the end, once the instructions are finally right, start a brand new assistant with them, rather than continuing to use the one you've been training.
"The assistant you have been working with will still contain all the inaccuracies, mistakes and back-and-forth from steps 1 to 4. That pollutes the algorithm," Barnard writes, calling this "historical assistant pollution," and describing it as the single thing that makes custom GPTs so frustrating for most people. That's a genuinely useful, easy-to-miss detail, the messy training process itself leaves residue behind, and the clean, refined instructions deserve a clean slate to actually perform on.
💬 Quote of the Week
"The technology is up to the task. My approach isn't."
— Jason Barnard, entrepreneur and founder, Kalicube
That's a genuinely useful reframe for anyone who's given up on a "frustrating" AI assistant after one bad session. The method here isn't exotic, it's structured iteration, treating the model less like software you configure once and more like a new hire you coach through repeated, specific feedback. The upfront patience required is real, but so is the payoff Barnard describes once "historical pollution" is finally cleared out of the picture.
|
💡 Final Thoughts
None of these five steps require any technical skill beyond writing clearly and paying attention, which is exactly what makes this method worth trying before writing off an underperforming AI assistant as simply not good enough. A job brief instead of a vague prompt, an expected flawed first draft, specific feedback instead of a new rulebook, an audit for instructional drift, and a fresh start once the instructions are finally right.
The honest trade-off is time upfront, this is a genuinely more effortful process than typing a single prompt and hoping for the best. But for anyone who's built a custom GPT, fed it documents, and walked away frustrated, the diagnosis worth sitting with is Barnard's own, the technology usually isn't the bottleneck, the one-shot approach to instructing it is.
Have you ever caught "instructional drift" in your own AI assistant without realizing what to call it? Hit reply, we read every response.
|
🔗 Sources and Further Reading
|
❤️ Enjoying AI Spotlight?
If today's edition gave you a fix worth trying on your own AI assistant this week, consider sharing it with a colleague, founder, or friend interested in technology.
Share AI Spotlight →
|
|
|
Thanks for reading AI Spotlight.
Our mission is simple: deliver clear, trustworthy, and actionable AI insights that help professionals stay ahead without the hype.
|
|