What’s Preventing LLMs From Prevailing in Business Analytics?

Generative AI models, like OpenAI’s O-series, DeepSeek R1, and Gemini, are increasingly capable of addressing analytics-related tasks. In theory, they can help solve many of the problems business analysts face. And yet, even the most forward-leaning analysts tend to use these models only for basic idea generation. The gap between that potential and actual day-to-day use is a wide one.

Why? The core issue is integration. Business analytics doesn’t happen in a vacuum. It happens inside a complex, context-rich environment. Analysts still face challenges that LLMs aren’t yet equipped to fully handle:

The Context Problem

A business can be thought of as a self-contained system. Every analysis must fit within the constraints of that system: its data, logic, goals, and tools. When analysts turn to LLMs, they have to first discover the data, understand it, then prompt the model with that context. Despite all this effort, the model’s context window, in even the most advanced systems, often isn’t large enough to maintain continuity across an entire session.

Data Processing Remains Manual

LLMs aren’t designed to process large volumes of data. At best, they can absorb small amounts and suggest code or queries. The analyst must then run that code in a separate execution environment, interpret the result, and return to the model for the next step. This creates a fragmented, manual loop instead of a seamless reasoning process.

Lack of Visual Reasoning

Visualization is essential for analysts to make sense of data. Current multimodal models aren’t trained to generate or interact with visualizations in ways that support analysis. As a result, reasoning over visuals remains out of reach.

No Foundation for Collaboration

There is no shared framework for collaborative analytics using LLMs. Findings can’t be easily transferred or built upon across teams. Each new user has to start from scratch — reloading context, rephrasing prompts, and retracing steps. This makes it difficult to create continuity or collective understanding.

The limitations outlined above aren’t temporary usability issues. They highlight a structural gap between LLMs and the needs of business analysts.

Rethinking the Environment Around LLMs

Bridging this gap may require more than agentic features layered on top of existing tools — it demands a new kind of analytics environment entirely. Here’s what that would entail:

• Environments that support LLM reasoning LLMs need structure, memory, and access, not just prompts. Building a system that supplies the right context, data, and feedback loop is foundational to enabling more meaningful reasoning.

• Moving beyond hard-coded workflows Shift away from rigid, pre-defined build processes and let LLMs apply their reasoning abilities dynamically within a business context.

• Breaking Tool SilosRemove rigid boundaries between BI tools, data warehouses, ETL systems, and integration platforms. Let the reasoning process move freely across layers.

• Scalable Data InteractionDefine an interaction model that lets LLMs work with large-scale data while preserving continuity and insight, even across slow, iterative workflows.

• Interactive Visualizations Allow LLMs to not just generate text responses, but visuals, too. More importantly, allow for interaction with them and participate in a visual discovery process.

• Governed Insight SharingGovern, evaluate, and manage insights from Gen AI as shareable assets not only for collaboration among analysts but also to enable a continuous learning process for the LLM model itself.

Ideally, this platform would be built as a neutral, platform-agnostic layer that can integrate with multiple base models, adapting as the underlying LLM ecosystem evolves and avoiding vendor lock-in in the process.

From Tools to Agent Networks: The Evolving Role of Gen AI in Business Analytics

The adoption of Gen AI in business analytics is following a natural progression from isolated tools, to intelligent agents, to interconnected agent networks. Each stage brings analysts closer to systems that can understand context, reason through complexity, and support decision making beyond what traditional tools allow.

Tool: Purpose-Built Capabilities

In the current landscape, Gen AI tools allow analysts to perform specific tasks using natural language. These tools take a narrow function, like generating a query or summarizing a dataset, and wrap it in a simple interface powered by LLMs. The result is an efficiency boost, especially for analysts without technical backgrounds. With the right toolset, analysts can move faster, handle tasks they previously depended on others for, and experiment more freely, even without writing code.

Agent: Context-Aware Guidance

Agents go a step further. Rather than executing a single command, an agent acts like a domain expert — one that understands best practices, can hold a conversation, and iteratively works with the analyst to arrive at better outcomes. It’s not just about answering questions, but about co-reasoning through business problems.

Agent Network: Coordinated Problem Solving

A network of agents can take on more complex challenges. Instead of a single expert, this model brings together multiple specialized agents, each responsible for a different domain or layer of the problem. The network works by:

• Following business hierarchies. Problems are divided and conquered through a structure aligned to how organizations operate.
• Sharing context. A shared knowledge base helps all agents operate with consistent logic and memory, including a track record of decisions.
• Evolving toward better outcomes. Through iterative reasoning, the network moves from narrow, local solutions to more optimized, system-wide decisions.
• Keeping humans in the loop. Complex decisions still benefit from human oversight and intentionality.

The industry, in its current state, is still early in this evolution, but gradually emerging from the discovery phase of what Gen AI can bring to analysts. Most Gen AI adoption among analysts today is happening at the tool level — basic utilities like text-to-SQL or auto-discovery functions. But momentum is building. As more agentic applications enter the fray, the next logical step is to build a platform that connects these efforts, tying tools, agents, and data together into an environment that can grow alongside advances in base LLMs.

Knowledge Is The Foundation That Gen AI Can’t Do Without

Knowledge is the core enabler of any Gen-AI-driven analytics workflow. LLMs have the unique ability to break through traditional knowledge silos, parsing information that lives in people’s heads, on local spreadsheets, inside documents, and across various systems in the modern data stack. Whether structured or unstructured, this fragmented data can be brought together and reasoned over in ways that go beyond what even the most sophisticated graph models can visualize.

But knowledge isn’t just for retrieval or context-building. It plays a direct role in guiding reasoning. With the right knowledge inputs, Gen AI can begin to replace hard-coded business logic, enforce policy-based behavior, and train agents that act appropriately within an enterprise environment. Reinforcement learning techniques can further refine this behavior over time, turning static capabilities into adaptable, compliant systems.

The Case for a Neutral Knowledge Layer

Without a strong knowledge foundation, Gen AI tools and agentic applications lose much of their value. And yet, this potential remains largely underutilized. Why, you ask? Two major reasons:

• Misconceptions about centralization. It’s a common misconception that a platform like Snowflake — by storing large amounts of data — inherently holds most of a company’s knowledge. But data volume doesn’t equal context or meaning.
• Limits of public LLMs. Businesses are understandably reluctant to expose sensitive knowledge to public models. Enterprise knowledge is proprietary, constantly changing, and subject to strict governance.

There’s a clear need for a neutral layer in the Gen AI stack — one that focuses specifically on knowledge management. This layer would:

• Observe and consolidate fragmented knowledge across files, tools, and systems
• Empower agents with contextual awareness and enable faster deployment of new ones
• Support continuous learning to improve model behavior over time
• Provide governance controls to ensure knowledge remains secure, accurate, and compliant

Compared to public knowledge, enterprise knowledge comes with additional challenges like security, privacy, and confidentiality requirements that must be respected. But for Gen-AI-first platforms, this isn’t necessarily a limitation — it can be a design opportunity.

Rethinking the Toolset for Business Analysts

Every skilled business analyst has a personal toolkit made up of a mix of purpose-built apps, custom scripts, and hand-crafted Excel templates. These tools have long played a critical role in improving efficiency, delivering higher-quality outputs, and navigating complexity without requiring deep technical skills.

But as Gen AI becomes more capable, a question emerges: Are these traditional tools still relevant? If LLMs can reason through problems and generate solutions on demand, what happens to the value of pre-built logic? And how do the providers of these tools justify their place in a Gen-AI-first environment?

Tools are essentially domain knowledge on how to solve a particular problem in a given environment using specific technologies. This will be always valuable regardless of the development of LLMs, but the nature of the tools needs to evolve in order to stay relevant for analysts. 

To remain useful, tools must break from their current limitations:

• They can’t remain black boxes with rigid, tightly coupled solutions.
• They shouldn’t rely on pre-defined parameters, configurations, and workflows.
• They need the flexibility and intelligence to adapt, refine, and respond during conversations with analysts or even with other agents.

Tool providers may no longer command a premium for packaging hard-coded solutions. But by rethinking how tools are developed, they can unlock new value:

• Focus on domain knowledge. Build tools at higher velocity that bring specialized expertise, not pre-set outputs.
• Let LLMs handle adaptability. Shift logic tuning and contextual adjustments to the model layer.
• Commoditize execution. Offload the mechanics to open platforms, reducing the overhead of proprietary runtimes.
• “Intelligentize” existing tools. Tool vendors can add reasoning capabilities to existing offerings, allowing them to interact with LLMs rather than operate as isolated endpoints.
• Rearchitect platforms. To support this new class of tools, platforms must evolve to enable dynamic integration, agent collaboration, and smoother transitions through changing business needs.

This opens the door to faster development cycles, smarter tools, and platforms that can respond to constant business and technical change without requiring full-scale migrations every time something shifts.

The tool providers who succeed will be those that extract the most value from their existing knowledge bases and repackage it into systems that evolve. Their platforms will need to support this new generation of intelligent tools: modular, adaptable, and built for dynamic environments.

Rather than clinging to static configurations, they’ll enable seamless transitions across shifting strategies, new technologies, and changing business needs, creating long-term value for analysts, enterprises, and providers alike.

Good Agent, Bad Agent: What Quality Means in Agentic Apps

As agentic apps gain momentum, we’re seeing a surge of new “agents” enter the market — some that even allow users with no technical background to build their own agents, using natural language alone. While accessibility is a step forward, it also raises a critical question: what actually makes a good agent?

Not all agents are created equal. There’s a growing risk of oversimplification — turning agent design into little more than writing business logic in plain language. That approach underutilizes the potential of today’s reasoning-capable models from OpenAI, DeepSeek, and Gemini 2.0.

What a Good Agent Should Offer

A high-quality agent does more than just follow instructions. It establishes an intelligent environment in which an LLM can reason effectively and improve over time. Specifically, it should:

• Use an LLM as its core reasoning engine. The agent supplies domain context and execution pathways, while the model handles the decision-making logic.
• Guide reasoning, not hardcode it. Instead of scripting fixed flows, a good agent shapes how the model thinks — influencing outcomes while allowing flexibility.
• Expose and execute the model’s outputs. Once the model generates a reasoning path, the agent interprets and runs it in the appropriate environment.
• Track and learn from behavior. Agents should generate, evaluate, and retain past processes, forming a memory that strengthens over time.
• Create a feedback loop. The agent continuously feeds results and outcomes back to the model, enabling reinforcement learning and progressive improvement.
• Support continuous learning. Quality doesn’t come from static scripts. It comes from iteration, feedback, and adaptation within the enterprise environment.

When agents fall short, it’s often not the fault of domain experts. Designing and delivering a high-quality agent requires infrastructure: a platform that can translate business knowledge into reasoning contexts, support execution, and manage learning over time. The responsibility lies with those building the frameworks, not just the individuals creating agents.

Rethinking Gen AI for Hard-Core Data Analytics

Text-to-SQL has become the most common entry point for applying Gen AI to data analytics. It’s relatively straightforward, cost effective, and easy to integrate into existing workflows. But it’s not the endgame. 

While text-to-SQL offers practical value, it doesn’t reflect the full range of what LLMs can do. It frames the model as a translator, converting natural language into structured queries, but bypasses the model’s capacity to reason through a problem from first principles. That reasoning ability becomes especially important when tackling complex analytical tasks that go beyond simple query generation.

Even today, LLMs are capable of sequencing multi-step analytical workflows. For example, a well-sequenced model can:

1. Generate a mathematical formulation for a business or technical problem.
2. Translate that formulation into an executable plan that fits the computing environment.
3. Convert that plan into runnable code in languages like SQL or Python — often the simplest step.

The value lies in the reasoning process, not just the final query. And that value often gets lost when Gen AI is used as a one-step translation tool.

What Happens as the Stack Evolves?

The temporary edge that text-to-SQL provides — affordability and speed — is already being challenged. Hardware is getting more powerful (thanks to next-gen GPUs from NVIDIA), models are becoming more efficient (from players like DeepSeek), and new infrastructure is emerging to better connect LLMs with data at scale (innovations like LlamaIndex).

Looking ahead, it remains to be seen who will be the eventual winners of this transformation. It could be an evolved form of cloud data warehouses, a new wave of open-source big data platforms with native natural language interfaces, or a middle-layer innovation that drives computing towards the established solutions.

Ultimately, the goal is to give analysts more control, not less. The future isn’t about replacing tools with static automation, but democratizing tool creation itself, letting analysts work directly with Gen AI to build what they need, when they need it. Tool providers, in turn, will shift their focus back to foundational methodology, enabling interaction rather than enforcing rigid logic.

A New Automation Landscape With Gen AI

OpenAI’s Operator is an early glimpse into what Gen-AI-powered automation could become — combining multimodal understanding with reasoning to make digital tasks feel fluid and intuitive. It’s an exciting development, but also a disruptive one. For enterprise architects, it raises fundamental questions about how software is built, deployed, and maintained.

Operator hints at a future where interaction shifts away from rigid APIs and UI workflows toward natural-language interfaces that mimic human behavior. Instead of every application needing to “act human,” the model itself becomes the intelligent layer, interpreting user intent, executing tasks across systems, and learning from the results.

This reframes the role of many existing automation tools. Agent-based solutions built with static logic may struggle to keep pace with this model-first approach. It raises an uncomfortable question: If the model can reason better than a pre-scripted agent, why maintain the agent at all?

Implications for Incumbents

For traditional automation platforms like UiPath, the Operator-style model represents a superior solution, converting manual, mechanical builds into adaptive, learning-driven processes. To remain relevant, these platforms will likely need to rethink their foundations.

But this new model isn’t without its gaps. The current Operator paradigm still falls short in several areas critical to enterprise automation:

• Extracting consistent, repeatable processes from fluid natural language interactions
• Balancing governance and democratization in how automation is created and applied
• Making context-aware decisions aligned with enterprise-specific logic
• Supporting continuous learning and collaboration between users, systems, and models
• Capturing institutional knowledge and feeding it back to the model to improve future automation

It’s still unclear whether every enterprise will need to train its own model to support these needs, or whether a generalized framework can adapt dynamically.

What Comes Next

The automation problem won’t be solved by LLMs alone. But a new automation architecture is emerging — one where business processes are treated as evolving knowledge assets. These processes can be learned from, reasoned over, and continuously refined through LLM interaction.

The automation problem won’t be solved by LLMs alone. But a new automation architecture is starting to take shape — one where business processes are treated as knowledge. These processes can be captured, structured, and provided to LLMs as input, and models can learn from them. 

What’s still missing is a common platform that supports enterprises across industries in managing this transition. Just as the cloud defined the last era of digital transformation, Gen AI may define the next. Building a foundation that helps businesses move into this era could be the next major leap after digitalization.

In this future:

• Workflows won’t be hard-coded DAGs. Instead, they’ll be generated dynamically, based on business strategy, context, and planning.
• Natural language won’t just define tasks. It will define the logic behind them, enabling systems to adapt even as goals change.
• Models will assist with the unknown. Gen AI’s greatest strength may lie not in repetitive tasks, but in helping teams navigate the problems no one has solved before.

Choosing the Right Model for Your Business

Selecting the right Gen AI model is more than just a technical decision; it has implications across cost, adaptability, governance, and long-term relevance. The choice is a difficult and nuanced one. Here are key considerations when evaluating models for enterprise use:

1. Understand Model Differences

Not all models are built for the same tasks. Some are general-purpose, while others are optimized for reasoning, summarization, or code generation. A better public understanding of how models differ and what use cases they support is still emerging, and often underdocumented.

2. Consider Operational Costs

Running a model isn’t free. Specialized or domain-specific models often offer significantly lower inference costs compared to larger general-purpose ones. Fine-tuning also varies widely in complexity and expense depending on the base model architecture.

3. Account for Development Overhead

It’s a misconception that public or open-source models are the finished product. Deploying them in a business setting still requires significant adaptation.

There are generally two ways to tailor a model for enterprise use:

• Reinforcement Learning. This approach can gradually align a model with company-specific behaviors. Providers like DeepSeek are making this more affordable, but the process still lags behind real-world change and is not yet easy to repeat at scale.
• Agent Development. Another route is to build agents on top of the model, embedding business knowledge into workflows and logic. This offers flexibility, but also brings risks: Agents can quickly become outdated as base models evolve, and current agent frameworks still lack full reflection of enterprise environments and continuous learning capability.

No “One Model to Rule Them All”

Given the pace of change, it’s unlikely that a single model will meet every business need. More realistically, we’ll see the rise of specialized — and in some cases, personalized — models tailored to specific use cases, domains, or organizational constraints.

It’s still too early to know whether an intelligent orchestration layer will be needed to manage dynamic, high-stakes decisions across these models. But the industry is clearly moving toward a multi-model, multi-agent future that reflects the complexity of real-world business environments, rather than trying to abstract them away.

Beyond Hype, Toward a Real Gen AI Operating Layer

The conversation around Gen AI in business analytics has moved past experimentation. Analysts are no longer asking whether LLMs are capable — they’re confronting the harder question of how to make them work in practice.

Across tools, agents, models, and platforms, the common thread is clear: reasoning alone isn’t enough. Business environments demand memory, structure, governance, and adaptability. These qualities don’t emerge from model output alone, but from the systems that surround and support it.

What’s needed now is less about prompting and more about architecture. The organizations that make this shift from isolated Gen AI features to an environment that captures knowledge, guides reasoning, and learns over time won’t just use Gen AI — they’ll build with it, evolve alongside it, and define what the next era of enterprise intelligence looks like.

Will you be among them?