Your AI Isn’t Failing Because of the Model

From the Arrayo Insights Desk

Most companies disappointed with their AI results blame the model. The model isn’t the problem.

Over the past several months, we’ve had the same conversation repeatedly with data leaders across financial services and life sciences. A team has invested heavily in AI tooling, but the outputs still can’t be trusted. Dashboards that look right but calculate wrong. Chatbots that confidently return incorrect answers. Reports requiring manual verification before anyone will act on them.

The instinct is to switch models or add more compute. Neither fixes it.

The missing layer

When you point an LLM at a data warehouse, it sees column names and table relationships. It does not see what those columns mean.

In financial services, “exposure” can mean current exposure, potential future exposure, netted exposure, or regulatory exposure, depending on who’s asking. In life sciences, “patient start,” “screen failure,” and “cycle time” mean different things across clinical, commercial, and operational teams. In many organizations, “revenue” has multiple definitions that are technically correct but contextually different.

The LLM doesn’t know any of this. So it guesses.

This is the semantic layer problem. The semantic model is the layer that sits between raw data and business meaning. It’s where you define your metrics, relationships, and business vocabulary. Without it, AI has no stable reference point for what your business actually is.

Think of the LLM as short-term working memory. Fast and fluent, but it forgets your business the moment the session ends. The semantic model is long-term memory. It holds your definitions, your calculation logic, your context. That’s what makes AI outputs consistent and auditable, regardless of who’s asking or what tool they’re using.

What the benchmarks show

Research from dbt Labs, AtScale, data.world, and others points in the same direction: LLMs perform materially better when business semantics are made explicit rather than inferred from raw schemas. The exact results vary by dataset and setup, but the pattern is consistent. Context beats guessing.

Marco Russo of SQLBI, one of the most respected voices in data modeling, is direct about it: “AI performs significantly better when paired with carefully-designed semantic models.” His point is worth sitting with — it means the work of defining your business logic properly isn’t something AI replaces. It’s something AI depends on.

What building it actually looks like

You don’t build a semantic model for your entire organization at once. The most effective approach is domain by domain: regulatory reporting, risk analytics, investment data, clinical operations. Each domain has its own metrics, relationships, and vocabulary. Build the semantic model for one domain and you create a reusable foundation. New reports can take days instead of months, and AI has a much better chance of querying it accurately. When the next AI tool arrives, the foundation is already there.

One thing worth saying clearly: a semantic model improves consistency and trust, but it doesn’t automatically solve data quality issues, stale pipelines, conflicting source systems, or governance gaps. Those still need to be addressed. What it does is ensure that when the data is right, AI can reason over it correctly, and that the business logic isn’t reinvented from scratch every time a new tool or team asks a question.

The reframe

What firms that have invested in this layer are realizing is that they didn’t just build better dashboards. They built AI infrastructure.

Defining the right metrics, relationships, and business vocabulary is the structural work that often gets skipped in the rush to deploy AI. It’s also what makes every AI capability built on top of it reliable from day one. Every new use case, every new tool, every new team member inherits the same trusted foundation.

The companies that will get durable value from AI aren’t the ones who adopted it fastest. They’re the ones who built the right foundation first.

The foundation doesn’t have to take years to build. A well-scoped semantic model for a single domain — regulatory reporting, risk analytics, investment data — can be built in weeks. You start where the business need is clearest, prove the value, and grow from there.

If your AI roadmap depends on trusted business data, the question isn’t just which model you’re using. It’s whether your business meaning is modeled well enough for AI to rely on it.