The Power of Context
An AI assistant can produce a convincing answer before we have finished understanding the question.
We ask it to review a requirement, suggest a solution, or design a test. Seconds later, we have something coherent. But how much of the situation did the system actually have access to?
Which rules did it know? What did we leave it to guess?
In How AI Chat Systems Work, I explored the layers around a language model. Here, I want to focus on context: the information available to it when it responds.
The knowledge we forget to share
Imagine asking an experienced consultant to review a ServiceNow story based on three sentences. They would probably ask about users, access rules, existing configuration, and previous decisions.
Their experience helps them recognize what is missing. It cannot supply your team's decisions.
An AI assistant faces a similar gap. General knowledge of ServiceNow does not reveal why your team rejected an approach last week.
We remember the meeting. We know which document is outdated. Then we write a short prompt and leave all of that out.
The system can only work with the context made available to it.
The application may supply context through conversation history, documents, retrieved records, and tool results. Its finite context window means earlier material may need to be summarized or omitted. Critical details can be lost along the way. [1]
One word can change the implementation
Consider this requirement:
An engaged user can edit the record.
What does engaged mean? Does access begin when someone is invited, when they accept, or after approval?
Each interpretation could produce different access logic and tests. An assistant that silently chooses one has introduced a business rule nobody agreed to.
One ambiguous word can lead to three different access rules.
Suppose we provide an approved role matrix. It identifies eligible users but says nothing about when editing rights begin. The useful conclusion is precise: eligibility is defined; the event granting access remains unresolved.
That is the behavior I want to encourage in tools such as the ServiceNow Story Refinement Workspace: expose the missing decision and show which tests depend on it.
A polished test matrix has limited value if its expected results rest on an invented rule.
Choose what deserves to be there
Adding documents can help. It can also introduce three versions of the truth.
A requirement may have changed after a meeting. A role matrix may describe an earlier release. A comment may be a suggestion someone never approved.
Useful context needs a clear status: current requirement, confirmed decision, unresolved question, or superseded material. Otherwise, the system must interpret which source to trust.
I would rather supply a current requirement with a clearly identified gap than a folder full of documents whose authority is unclear.
Make the work reviewable
For a story review, I would provide the current story, acceptance criteria, applicable rules, and confirmed decisions. Then give a focused instruction:
Identify ambiguities that could change implementation or testing. Use the attached role matrix as the approved source for eligibility. Return the five most consequential questions and a compact test matrix. Mark expected results that depend on unresolved decisions.
A reusable tool would need to select those references, preserve their source and status, and carry confirmed decisions into subsequent reviews. Managing that context is part of the application's design.
Test it with incomplete stories, conflicting references, and outdated documents. Check whether its conclusions follow from the evidence and whether it preserves uncertainty where a decision is still needed.
Preparing context also asks something of us: distinguish what we know from what we assume, and notice what we have never decided.
Sometimes the difficulty we encounter with an AI system begins in that unfinished thinking.
The power of context is that it gives the system a firmer basis for its work. Preparing it carefully can give us one too.
Source
Prompting with Agentic Techniques
Reference
Comments
Post a Comment