Why LLM observability is venture’s next major play

The AI revolution continues to impact all aspects of the financial services industry. Enterprises are racing to deploy AI, pouring billions into models, agents and infrastructure, trying to keep up with the rapid pace of change and gain competitive advantages against peers.

Amid these rapid advancements, we at Citi Ventures want to pause and highlight a pattern we have seen occur before with other disruptive innovations, such as the internet and smartphone: with great technological change comes new secondary markets that are just as transformative as the root catalyst.

Today, as enterprises move from AI experimentation to production, they need tooling to handle issues like hallucinations, latency, token usage, safety risks, model drift, agent behavior and overall application reliability. These collective needs have given rise to one of the most critical and fastest-growing layers of the new AI stack, known as LLM observability, AI observability and LLMOps monitoring.

This is not the traditional monitoring of logs and metrics but instead a new paradigm built to address the unique, complex and often unpredictable nature of AI systems. For enterprises, especially in regulated areas like finance, getting this right is a strategic imperative. Meanwhile, for investors, LLM observability represents a ground-floor opportunity in a category poised for explosive growth.

A multi-billion dollar market in the making

The LLM observability market is poised for healthy growth. It is currently valued at just under $2 billion in 2025 but projected to surge to more $9 billion by 2030, a CAGR of over 36%.

This is just the beginning. This growth is a derivative of the colossal expansion in enterprise AI spending, which is expected to climb from approximately $115 billion in 2026 to over $275 billion by 2031. We see LLM observability acting as a necessary addition to this spend. Just as Datadog captured 1-2% of the cloud infrastructure market it monitored, AI observability platforms are positioned to capture a significant share of the AI application market, a market far more complex and failure-prone than traditional cloud infrastructure. At full-scale production, this additional spending on top could represent a $10 billion to $25 billion market.

The enterprise dilemma: Why startups are the answer

As enterprises deploy AI agents and complex, multi-step agentic workflows, they face a barrage of new, high-stakes questions that traditional tooling cannot answer:

• Trust and Safety: Why did the model hallucinate or produce a toxic output? Are we violating compliance policies?
• Performance: Which prompts are causing failures? Is the quality of our RAG retrieval degrading?
• Cost: Which workflows are driving our token costs? How can we optimize routing and caching?
• Governance: Can we provide a complete audit log for a model's decision? Is there a human in the loop at the right junctures?

These are not trivial challenges. They require a new breed of specialized, AI-native tooling. While the hyperscalers and incumbents are moving to bundle basic features, the speed, neutrality and depth required by enterprises, especially those operating in multi-model, multi-cloud environments, create a massive opening for focused startups. Enterprises cannot afford to be locked into a single provider's ecosystem, and they need platforms that prioritize deep evaluation and governance over simple tracing. This has fueled the emergence of startups building specialized observability platforms, including companies such as Langfuse (acquired by Citi portfolio company Clickhouse) and Galileo (acquired by Cisco).

Navigating the risks

The path forward is not without peril. Startups in this space face the classic "death from above and below" squeeze.

1. Death from Above: Incumbents like Datadog and Dynatrace are bundling LLM features, leveraging their 30,000+ customer relationships.
2. Death from Below: Open-source tools like Phoenix are commoditizing the basic tracing layer, making it difficult to charge for simple instrumentation.

However, the winning thesis is that the core tracing layer has no durable moat. The companies that will survive and thrive will win on the layers above: on evaluation depth, sophisticated governance features and the ability to close the loop between observing a failure and fixing it.

Why we're excited about AI observability

For a large financial institution, the need for robust LLM observability is non-negotiable. Enterprises require auditability, compliance monitoring, PII leakage prevention and secure AI routing. This is a must have, not a nice-to-have.

While the market is crowded and the risks are real, the underlying need is undeniable. Enterprises will not bet their business on black-box AI systems running without accountability or traceability. The startups that can bring light to how LLMs perform, their outputs and overall costs will become a foundational, indispensable layer of the modern enterprise. They will be the "Datadog for AI," and for venture investors, that's an opportunity too large to ignore.