The Model Isn't the Bottleneck: What Agent Adoption Is Really Missing

In 2023, most people used generative AI the same way: type a sentence into a chat box and wait for a paragraph, a SQL query, or a draft. Today the interaction is changing. You give the AI a goal, and an agent breaks down the work, calls tools, executes, checks, and comes back with a result.

On the surface, this looks like a story about capability. But whether enterprises can actually put it to work often has less to do with the model than with something else. At an event about data and AI, a Google Cloud speaker pointed to a different gap: once agents start doing analysis, producing reports, and taking over part of an analyst's job, what companies lack is a context layer that spells out business definitions, data meaning, and calculation rules. Without that layer, the smarter the model, the more confident its mistakes.

From "write me a query" to "get this done"

The speaker traced the shift. Early on, people asked AI to generate SQL or marketing copy; the pattern was "one question in, one answer out." The more interesting pattern now is the agent: give it an objective, and it decides what to do next along the way.

That shift is already showing up in real work. The speaker cited companies such as Mercari, which uses analytical agents in analysis work. The goal isn't to "generate a query" but to replace part of a process that people used to run: the agent doesn't just return a result; it factors in the user's background and business problem and suggests what action to take next.

He broke the required capabilities into layers. A skill layer handles tool calls, SQL generation, and report writing. An execution harness runs the plan-act-check loop and carries the instructions baked into the system. Beyond that sits the model itself.

But those pieces alone still leave a gap between a demo and real work. The speaker was blunt: this is not enough; something is still missing in practice.

Sometimes the hallucination is a definition problem

He offered a small example. Two departments can use the same word — "sales" — and mean two different numbers.

Marketing may look at the value at the moment a transaction occurs: tax included, counted on an order or cash-receipt basis. Finance follows accounting rules: strip out tax, recognize revenue under the relevant standard, and deduct certain items. Each calculation is "right" in its own context. But if no one has specified which definition is in play, the agent will drift between them and eventually return a number that looks plausible and reconciles with nothing.

Errors like this tend to get labeled "model hallucination." The speaker's diagnosis is different: the problem may not be the model at all, but the fact that the definition was never written into the context. The model doesn't know what your company means by "sales," so it guesses.

The example matters because it explains why so many demos look impressive and then fail in production. Demo questions are isolated and well defined. Real business questions come with an entire set of organizational conventions attached.

PoCs pass; scale is where they hit the wall

Another common trap is the proof of concept. Building the SQL for one scenario and getting a single pipeline to work is usually not hard, the speaker said. The hard part is replicating it across dozens or hundreds of scenarios and handling a high volume of questions around the clock.

Several problems arrive at once:

• Cost: if every request makes a large model re-learn the table structure and business definitions from scratch, token consumption climbs quickly.
• Latency: complex questions require multiple rounds, and users won't wait.
• Governance: who can see which data, and which agents may use which definitions, requires permissions and audit trails.
• Reproducibility: the same question must return the same answer today and tomorrow, from this person and that person.

That changes what a data platform is for. The old platform was mainly built for people to read: humans open reports, look up numbers, run analyses. Now it also has to be a platform where AI can understand and act — where an agent can find the data, grasp what it means, calculate with the right definition, and stay within its permissions.

The context architecture: explaining what the data is

The speaker's answer is a layered design he calls a "context architecture." It isn't complicated, but each layer answers a question an agent will ask:

1. Where is the data? The source layer: warehouses such as BigQuery, plus business systems and data in other clouds.
2. Who can use it? Permissions and governance: which data and which definitions are open to which agents.
3. What is this data? Metadata: tables, columns, business meaning, owners.
4. What does it mean? A semantic layer that maps terms such as "sales," "active user," or "Kansai region" to actual columns and definitions.
5. How is it calculated? Metrics and calculation logic that turn business definitions into executable expressions.
6. How is it used? Finally, agents or APIs deliver all of this to the user.

He emphasized that this architecture came out of practice in Japan. That detail is worth noticing: enterprise AI is deeply local. Organizational habits, data conditions, and compliance requirements differ, so what travels is often not the implementation but the layered way of thinking.

Three kinds of analysis, three approaches

On top of that architecture, the speaker split analysis into three categories, each handled differently:

• Deterministic analysis: fixed definitions, one correct answer — for example, "sales in the Kansai region last month." This should go through the semantic layer and predefined calculation logic, so the result is reproducible.
• Exploratory analysis: questions that arise on the spot and require digging — for example, "which customer segment is churning more lately?" This suits generative AI, with a person exploring alongside it.
• Agentic work: a clear goal but no fixed procedure — for example, "put together an Osaka marketing campaign and verify its results." The agent plans, calls tools, and produces the output.

The first category is the most important. The speaker used orders and payments as an example: a single order can involve multiple payments, refunds, and adjustments. Sales should be calculated once, under explicit rules. Let the model improvise each time, and the same question yields different numbers.

Their approach: the agent first consults a glossary to confirm what "sales" means; the semantic layer then supplies the calculation; finally, the agent generates deterministic SQL. That combines the flexibility of natural language with the reliability of a traditional data stack. The speaker called this the deterministic approach — the model doesn't guess; it calls logic that has already been defined.

A comparison experiment: what context actually buys

To test the architecture, the speaker ran a comparison experiment inside the company.

The experiment used five types of questions, each run 20 times, ranging from simple to complex. Some involved time conditions, some required identifying the right entity, and some — like "sales after audit adjustments" — carried a lot of conditions. One side was a baseline without context; the other was an agent connected to the context architecture.

The results were measured on several dimensions:

• Accuracy: the context-enabled side was clearly better, especially on complex questions that required understanding the business definition correctly.
• Execution steps: the baseline tended to try table after table and column after column, taking detours. The context-enabled side took shorter paths.
• Cost: token consumption and the number of executions both fell.

The speaker also noted a limitation. They tested a lighter model as well; adding context did improve it, but complex questions still needed a stronger model, at a higher cost. In other words, context is not a trick for replacing large models with small ones. It lowers the difficulty of the problem so the model can spend its effort where reasoning is actually required.

That distinction matters: it heads off the misreading that "with context, you don't need a good model."

Managed capabilities: making context operational

Explaining the architecture is one thing; running it is another. The speaker described several Google Cloud data capabilities:

• A serverless warehouse: BigQuery requires no upfront cluster planning, starts quickly, and scales compute as needed.
• Zero-copy access: when business data lives in another cloud or operational system, federated queries and zero-copy access let you analyze it directly — no need to copy it first or wait for a nightly batch. Live operational states such as orders, inventory, and financial transactions can be analyzed as they are.
• Metadata and lineage: tables, columns, upstream and downstream relationships, data quality, and permissions are recorded and queryable, so an agent can attach evidence to its answer.
• Continuous learning: every query and every correction enriches the semantic layer and metadata, improving the next answer.

The last point got special emphasis. The context layer is not a build-it-once project; it is an asset that needs maintenance. Business definitions change, tables change, and agent errors expose holes in the definitions. The system has to learn from those traces of use rather than relying on people to rewrite documentation over and over.

Demo: mortgage refinancing at a fictional bank

The speaker demonstrated with a fictional "Sakura Bank." The scenario was mortgage refinancing: a customer wants to compare options, but the question is vague — "given my situation, which page can calculate this for me?"

The agent takes the question, uses existing definitions and attached information to assess the customer's situation, finds the relevant data, completes the analysis, and presents the aggregated result as a dashboard. One detail in the demo: some customer data sits in another cloud's database and is accessed through zero-copy, without first syncing it into BigQuery.

The result is not the end. The speaker stressed that a data scientist can review the agent's analysis, check the definitions, and correct mistakes. The conclusions and corrections become accumulated knowledge rather than disappearing into someone's chat history. That keeps data assets from becoming person-dependent when people move on, and lets a newcomer see how a number was produced.

The second half of the demo shifts to marketing: a campaign in Osaka requires finding target customers, segmenting the audience, viewing the distribution by prefecture and municipality, generating creative assets, and finally validating performance in a BI tool. The agent strings these together and produces not just copy but evidence that can be reviewed.

Three conclusions

The speaker closed with three points:

1. Don't bet everything on bigger models. A more effective move is to give the model high-quality context. With context in place, both accuracy and cost improve.
2. The context architecture should become a shared foundation for the company. Build that layer first, and dozens, hundreds, or thousands of agents can grow on top of it, instead of every team building its own.
3. Data, compute, and agents should be managed end to end. Companies shouldn't have to assemble every piece themselves; they can focus on business definitions and scenarios.

He also mentioned an event on November 14 at Google's office about AI and data, with customer stories and hands-on sessions. It was an invitation of sorts: once the architecture makes sense, the next step is to bring your own data and try it.

A final note

The real value of the talk wasn't another new term. It was turning "AI is hard to deploy" into a problem you can actually work on: the model isn't lacking intelligence; the company hasn't made "what we mean by this word" machine-readable.

Definitions, calculation rules, permissions, lineage — work that sounds like traditional data governance — become more important in the agent era, not less. People can infer from context. Agents can't; they execute the definition you give them. Give them the right one and they are dependable colleagues. Give them the wrong one and they scale the error to everyone who comes asking for an answer.