Gartner's guidance to AI leaders in 2026 is blunter than most enterprise teams expect: context engineering is in, and prompt engineering is out. The organisations getting reliable output from AI are not the ones who hired better prompt writers. They are the ones who rebuilt what their AI systems are allowed to see before those systems are asked to reason.
That distinction is now a budget line, an architecture decision, and in Hong Kong, a compliance question. This guide explains what context engineering is, how it works, the four components of an enterprise context layer, and the mistakes that turn the effort into an expensive data project with no measurable output.
What is context engineering?
Context engineering is the discipline of designing what information, tools, and institutional knowledge an AI system assembles into its working memory before it answers. Where prompt engineering optimises how a question is phrased, context engineering governs what the model has access to at the moment of reasoning.
According to Gartner, context engineering means designing and structuring the relevant data, workflows and environment so AI systems understand intent and deliver enterprise-aligned outcomes without relying on manual prompts.
The practical difference is a change of question. Prompt engineering asks: how should I word this? Context engineering asks: what does the model need in front of it right now, and what should it never be able to see?
Why did prompt engineering stop being enough?
Prompt engineering stopped being sufficient because enterprise AI moved from single questions to multi-step work across governed systems. A well-worded prompt cannot fix a model that has been handed the wrong document, a stale policy version, or no record of what happened in the previous session.
The failure mode is specific. When an AI system produces confident output built on incomplete or unapproved source material, the error is not in the phrasing. It is in what was retrieved. Rewording the request produces a differently worded wrong answer.
Gartner predicts that by 2028, context engineering features will be part of 80% of the software tools used to build AI applications, and that these features will improve agentic AI accuracy by at least 30%. That is a statement about where accuracy actually comes from.
There is a governance dimension too. A prompt is not auditable infrastructure. A context layer is. When a regulator or an internal audit committee asks which data an AI system used to reach a decision, the answer has to come from the retrieval and permission design, not from a chat transcript.
How does context engineering actually work?
Context engineering works by assembling a fresh, permission-checked working set for each AI task, then managing that working set as the task runs. Four operations do the work: writing information down, selecting what is relevant, compressing what has grown too long, and isolating what must stay separate.
In practice, this means an AI system no longer receives one large block of text. It receives a curated environment: a set of retrieval paths, a memory store, a scoped list of tools, and rules for what to discard as the task extends.
Anthropic describes one increasingly common technique as just-in-time retrieval, where the system pulls underlying content into context only when it needs it, using lightweight identifiers such as file paths or query strings rather than loading everything up front.
For a business leader, the operational takeaway is that context is a system, not a setting. It has an owner, a cost, and a failure rate.
What are the four components of an enterprise context layer?
An enterprise context layer has four components: retrieval, memory, tool scoping, and compaction. Each answers a different question about what the AI system knows, and each has a distinct owner inside the organisation. Treating them as one undifferentiated data problem is the most common design error.
1. Retrieval: what can the system look up?
Retrieval covers how the system finds source material. Methods include vector search over documents, structured queries against a database, direct file reads, and just-in-time lookups.
--- Owner: data and platform engineering
--- Key control: permission inheritance, so the AI system can never retrieve what the requesting user could not open themselves
--- Failure signal: correct-sounding answers built on superseded documents
2. Memory: what does the system remember?
Memory splits into short-term memory, meaning the conversation and tool results so far, and long-term memory, meaning what persists across sessions such as approved definitions, project conventions, and prior decisions.
--- Owner: the business function using the system
--- Key control: an explicit retention and deletion policy, which under the Personal Data (Privacy) Ordinance matters as much as accuracy
--- Failure signal: the system repeats questions it has already been answered, or remembers something it should have forgotten
3. Tool scoping: what can the system do?
Every tool exposed to an AI system consumes capacity and adds distraction. Scoping means giving each task the smallest set of actions it needs.
--- Owner: security and IT operations
--- Key control: least-privilege authority per task, not per system
--- Failure signal: an AI process that can write to a system where it only needed to read
4. Compaction: what does the system forget on purpose?
As a task runs long, older material has to be condensed so the important parts survive. Anthropic has productised this as automatic context compaction, which summarises earlier portions of a conversation as the working limit approaches.
--- Owner: platform engineering, with business sign-off on what must never be summarised away
--- Key control: a defined list of facts that survive every compaction, such as client identifiers and regulatory constraints
--- Failure signal: quality that degrades the longer a task runs
The reason to separate these four is accountability. When output is wrong, you need to know which of the four failed, and one named person needs to own the fix.
What does context engineering look like in a Hong Kong enterprise?
In Hong Kong, context engineering is where AI ambition meets the Personal Data (Privacy) Ordinance. The context layer is the technical place where data minimisation, purpose limitation, and retention rules are actually enforced, which makes it the most defensible part of an AI programme to present to a board or a regulator.
The regulatory backdrop is now concrete. The Office of the Privacy Commissioner for Personal Data completed compliance checks on 60 Hong Kong organisations in May 2026. Of those, 57 organisations, roughly 95%, used AI in day-to-day operations, and 29, roughly 51%, ran three or more AI systems. The Commissioner found no contravention of the Ordinance, but called for a proactive and structured approach to AI governance.
Three scenarios show how this plays out:
A financial services firm handling client suitability questions. The context layer must retrieve only the current version of the product suitability policy, must never place one client's holdings in front of a query about another, and must keep an auditable record of which policy version informed each answer. That is retrieval design and tool scoping, not prompt wording.
A logistics operator answering shipment exception queries. The system needs live operational data, not last night's export. Long-term memory holds customer-specific handling agreements. Compaction rules must guarantee that the consignment reference and any dangerous-goods constraint survive every summarisation step.
A property management group processing tenant requests. Property management was one of the new sectors covered in the 2026 compliance checks. Tenant records contain personal data with a clear retention limit, so the memory component needs a deletion schedule that matches the organisation's stated retention policy, enforced in the system rather than in a manual process.
In all three, the competitive advantage is not a smarter model. Every competitor can buy the same model. The advantage is that your AI reasons over your governed, current, correctly permissioned institutional knowledge, and theirs does not.
How do you know if your context layer is failing?
A failing context layer produces a recognisable pattern: output that is fluent, confident, plausible, and wrong in ways users only catch if they already know the answer. The symptom is not obvious nonsense. It is quiet inaccuracy that erodes trust until adoption stalls.
Five diagnostic signals worth checking this quarter:
--- Users routinely paste the same background material into the AI system by hand, which means retrieval is not reaching what they need
--- Answer quality falls noticeably as a task or conversation extends, which points to compaction
--- Two teams get materially different answers to the same policy question, which points to competing or stale sources
--- Nobody can state which documents informed a given output, which means there is no audit path
--- Adoption peaked in week three and has been declining since, which is usually a trust problem rather than a training problem
Forrester's 2026 research puts a number on the cost of not measuring this. In its 2026 panel, AI agents without automated evaluation had a 47% rollback rate over the prior year. Agents with full evaluation coverage had a 9% rollback rate. The difference is not model quality. It is whether anyone was systematically checking what the system was working from.
What do organisations get wrong when they attempt this?
The most common error is treating context engineering as a data cleanup project with no defined output. Organisations spend two quarters consolidating documents, produce a governance framework nobody uses, and never connect the work to a single business task with a measurable before-and-after.
Five failure patterns to avoid:
--- Boiling the ocean. Attempting to make all enterprise data AI-ready before any use case ships. Choose one workflow, engineer its context properly, measure it, then extend.
--- Loading everything into the window. More context is not better context. Irrelevant material actively degrades output by competing for the model's attention.
--- Skipping permission inheritance. Building retrieval that ignores who is asking is the fastest route to a data incident, and it is very expensive to retrofit after launch.
--- Leaving compaction to defaults. If the business has never specified which facts must survive summarisation, the system will eventually discard one that mattered.
--- No named owner. When retrieval, memory, tool scoping, and compaction are collectively owned, they are effectively unowned. Each needs one accountable person.
There is also a sequencing mistake worth naming. Context engineering is not a prerequisite for every AI initiative. For a narrow task over a small, clean, current document set, careful prompting and basic retrieval are sufficient. The discipline becomes necessary when AI has to act across systems, remember across sessions, and stand up to audit.
The strategic takeaway
Context engineering reframes the enterprise AI question. The competitive variable is no longer which model you licensed, because your competitors can license the same one next week. It is whether your AI systems reason over current, governed, correctly permissioned institutional knowledge, and whether you can prove it.
For a VP of Operations or Head of Digital Transformation, that is a more defensible investment story than a model upgrade. It is also harder work, spanning data engineering, security, and the business functions that own the knowledge. Anyone who tells you it is straightforward has not enforced permission inheritance across a legacy system.
Start with one workflow. Name an owner for each of the four components. Measure accuracy before and after. Then extend. That sequence is unglamorous, and it is the one that produces results a board will fund again.
Twenty-eight years of Hong Kong technology cycles have taught us that the durable advantage is rarely the newest tool. It is the organisation that built the foundation properly while others were buying licences. We understand AI. We understand you. With UD by your side, AI never feels cold.
Related reading: a framework for enterprise AI accuracy and retrieval quality and the hidden cost of low-quality AI output inside your team.
Reviewed by the UD enterprise AI team.
Where to start
Now that you have the framework, the next step is finding out where your own context layer stands. We'll walk you through every step, from AI readiness assessment to retrieval design, governance controls, deployment and performance tracking, with twenty-eight years of Hong Kong enterprise experience behind every recommendation.
