The assembly problem: why an integrated business AI operating system beats a Databricks plus Claude build

Why connecting Databricks and Claude isn't enough: the real enterprise AI challenge is the contextual layer, governance, and orchestration in between.

Vaughan Emery
Vaughan Emery

May 13, 2026

10 min read
The assembly problem: why an integrated business AI operating system beats a Databricks plus Claude build

There is a quiet assumption running through most enterprise AI conversations right now, and it deserves to be examined before anything else. The assumption is that if you take a best-in-class data platform and connect it to a best-in-class language model, you will end up with a best-in-class AI capability. Buy Databricks for the data. Buy Claude for the intelligence. Wire them together. Ship.

It is a reasonable-sounding proposition. Both are genuinely excellent products. Databricks has built one of the most capable data and analytics platforms in the market, and Claude is among the most capable reasoning models available. On paper, the sum should be formidable.

The trouble is that enterprise AI is not an addition problem. It is an integration problem. And the gap between those two things is where most build efforts quietly stall.

Key Takeaway

The model and the data platform are the parts you can buy. The contextual layer, governance, orchestration, observability, and interface are the parts you would have to build, and they are precisely what determines whether your AI can solve problems or only answer questions. The build decision is not about the anchors. It is about who owns the connective architecture in between.

The premise worth questioning

The build-it-yourself case rests on a specific belief: that the hard parts of enterprise AI are the data platform and the model, and that everything between them is plumbing. Get the two anchors right, the thinking goes, and the connective work is a matter of engineering hours.

This is worth taking seriously, because for a certain class of problem it is true. If your goal is to let an analyst ask questions of a governed dataset and get well-reasoned answers back, a Databricks and Claude pairing can absolutely get you there. The model reasons over the data you hand it, returns a thoughtful response, and a competent team can stand that up.

But notice what that workflow actually is. A person forms a question. A person decides which data is relevant. A person interprets the answer and decides what to do about it. The AI is doing the reasoning in the middle, and the human is doing the framing on both ends. That is AI that answers questions, and it is valuable.

The problem is that answering questions is not what the organizations pushing hardest on AI actually want anymore. What we consistently see customers reaching for is something different in kind: AI in critical-thinking roles, AI running analytical work, AI automating the judgment-laden parts of a workflow rather than just the retrieval-heavy parts. They want AI that solves problems. And the distance between those two ambitions is not a distance you cross with more engineering hours on the same architecture. It is a different architecture.

What the build path actually asks of you

Let us be fair and concrete about what a Databricks plus Claude build involves, because the shortcomings only become visible once you trace the full path.

You start with your data in Databricks, which is a strong place to start. Then you need the model to be able to reason over that data in a way that reflects how your business actually works, which means you need to give the model context. Not just the rows and columns, but what they mean, how they relate, which definitions are authoritative, what a given metric signifies in your particular organization versus what it might mean elsewhere. That context does not live in Databricks in a form a model can consume. You build it.

Then you need governance that travels with the AI. It is not enough for Databricks to enforce access policies on the data warehouse if the model, once it has retrieved data, can surface it to a user who should never have seen it. The permissions that govern a human analyst have to govern the agent acting on that analyst’s behalf, and they have to hold across every step of a multi-step workflow. That enforcement layer does not exist between the two products. You build it.

Then you need orchestration, because a problem-solving workflow is not a single call to a model. It is a sequence: retrieve, reason, act, check, retrieve again, decide. Something has to manage that loop, maintain state across it, handle the failures gracefully, and know when the work is done. You build it.

Then you need observability, because the moment AI is making operational decisions rather than drafting answers for a human to vet, you need to see what it did, why it did it, and whether it did it correctly. You need an audit trail that satisfies a compliance officer and a debugging surface that satisfies an engineer. You build it.

Then you need an interface that a non-technical person can actually use, because the entire value of putting AI into broad operational roles evaporates if only the data team can operate it. The frontline manager, the underwriter, the operations lead, the finance analyst, none of them are going to write notebooks. You build it.

Each of these is a real system. Each requires design, construction, testing, and ongoing maintenance as both underlying products evolve on their own release cycles. And critically, none of them is a one-time cost. Databricks ships changes. Claude ships new model versions with different behaviors. Every layer you built to bridge them is now a layer you own the maintenance of, forever, against two moving targets you do not control.

The contextual layer is the actual product

Here is the reframing that changes the whole calculation. In an enterprise AI system, the language model is not the intelligence. The language model is raw reasoning capacity. The intelligence is what you feed it: the full context of the business, the complete data ecosystem, and the permission to act on both.

An LLM that does not know your business is a brilliant consultant on their first morning, before anyone has told them anything. It can reason beautifully about a problem it does not understand and reach a confident, well-argued, wrong conclusion because it was missing the three things everyone in the room already knew. The reasoning was never the constraint. The context was.

This is why we believe the contextual layer is the thing that actually has to be built well, and why we believe it cannot be an afterthought bolted between two products that were not designed to produce it. For AI to operate in the broad, autonomous, problem-solving roles that organizations want, the model needs to know the full context of the business, reach the complete data ecosystem rather than a curated slice of it, and function in genuinely autonomous roles where it can learn and iterate on hard problems rather than answering one question and stopping.

At Datafi, this is the global business contextual layer, and it is the foundation the entire operating system is built to produce and maintain. It is not a connector. It is the thing the connectors serve.

What an integrated operating system does differently

An operating system, in the original computing sense, is the thing that lets applications share resources without every application having to reimplement memory management, file access, and hardware control. Nobody writes a word processor by first writing a disk driver. The operating system provides the common substrate so the application can be about the actual work.

The Datafi Business AI Operating System applies that same logic to enterprise AI. The connective architecture that a build effort would have to construct and maintain is, in an integrated system, the substrate. It is already there, designed to work together, maintained as one thing.

The contextual layer is native, not assembled. The system’s reason for existing is to develop and maintain the global business contextual layer that complex agents and workflows depend on. The data ecosystem, its meaning, and its relationships are understood by the platform rather than reconstructed by hand for each new use case.

Governance travels with the AI through Cyber, the governance framework. Policies and controls are not a separate enforcement layer you hope holds at every step. They are woven through the runtime, so a governed data access model applies to the agent exactly as it applies to the human, at every step of every workflow. This is what makes compliance-ready AI a property of the system rather than a project you run alongside it.

Orchestration is built for autonomous, multi-step work through Runtime, the agent runtime. The retrieve-reason-act-check loop that problem-solving workflows require is what the runtime is for. It maintains state, sequences the steps, and manages the work through to completion rather than returning a single answer and stopping.

Observability is native through Control Tower. When AI moves into operational decisions, you can see what the agents did and why, with the audit trail a compliance officer needs and the visibility an operator needs, without building a monitoring stack from scratch.

And the Business Chat is designed for the non-technical user from the start, because the entire premise is AI in broad roles across the enterprise. If the value depends on the underwriter, the operations lead, and the finance analyst being able to use it directly, the interface cannot be an artifact the data team operates on their behalf.

One more thing worth stating plainly: this integration does not mean lock-in to a single model. Datafi is deliberately LLM-agnostic. The reasoning engine is a component you can choose and change; the contextual layer, governance, orchestration, and interface persist across whichever model you run. Integration and ownership are separable. You get an integrated system without surrendering your ability to swap the parts that improve fastest.

The fair case for building

It would be a disservice to pretend the build path is never right, because sometimes it is, and credibility requires saying so.

If your ambition genuinely stops at AI that answers questions for a technical audience, a Databricks plus Claude build can be a sound choice, particularly if you already have deep platform expertise on staff and a data team with spare capacity. If your use cases are narrow, well-bounded, and unlikely to proliferate, the connective architecture you have to build stays small, and the maintenance burden stays manageable. And if control over every layer of the stack is itself a strategic requirement for reasons specific to your business, then owning the assembly is a feature, not a cost.

There are organizations for whom each of these holds. The build path is not a mistake for them. It is a fit.

But these conditions are more specific than they first appear, and they tend not to survive contact with success. Narrow use cases expand the moment the first one works and the rest of the business notices. The technical audience gives way to demand from non-technical teams who want the same capability. The small connective layer grows every time a new workflow needs context, governance, or orchestration the previous one did not. The build that was right for the pilot becomes the thing the platform team is drowning in maintaining by the time it reaches the tenth workflow. What was a fit at the start is a liability at scale, and scale is the entire point of enterprise AI.

Where the decision actually lands

The honest version of this comparison is not that Databricks and Claude are the wrong products. They are excellent products. The honest version is that the choice is not really between two vendors and one build. It is between two philosophies about where the hard problem lives.

The build philosophy says the hard problem is the anchors, and the middle is plumbing you can construct. The integrated philosophy says the anchors are the parts you can buy, and the middle is the actual product, the contextual layer and everything that serves it, which is far too load-bearing to be assembled by hand and maintained forever against two moving targets.

Our conviction, formed from working directly on the problem of actioning data rather than merely querying it, is that the second philosophy is the one that survives contact with what organizations actually want. Because what they want has moved. They are no longer asking for AI that answers questions from a curated dataset for a technical user. They are asking for AI that solves problems, in autonomous and analytical roles, for every employee, governed and compliant by default, learning from the full context of the business against the complete data ecosystem. That is not a capability you reach by connecting two products. It is a capability you reach by building the connective architecture as the product, which is precisely what an integrated business AI operating system is.

The right question to carry into the decision, then, is not which model and which platform. It is who owns the twelve feet in between, and for how long. Answer that honestly against your real ambitions, and the choice tends to make itself.


Next in this series: how the global business contextual layer is constructed, and why it is the difference between an agent that reasons about your business and one that merely reasons.

Datafi is an applied AI software company building the Business AI Operating System for mid-enterprise organizations. Our vertically integrated data and AI stack gives every employee governed, compliance-ready access to AI that solves problems rather than just answering questions. Learn more at datafi.co.

ShareCopied!
Vaughan Emery

Written by

Vaughan Emery

Founder & Chief Product Officer

Continue Reading

All articles

Transform your enterprise with AI

See how Datafi delivers results in weeks, not years.

Interested in investing in Datafi?

Request a Demo

See how Datafi can transform your business AI strategy in a personalized walkthrough.