"GenAI has fallen short": Can Databricks' compound models perform better?

Generative AI has “fallen short” in the delivery of business intelligence (BI) and large language models “struggle” to offer accurate and useful information, Databricks has warned. 

The data and AI firm said that LLMs often fail when dealing with the "messy" data and imperfect human language that inevitably arise when performing data analysis in the real world.

Databricks, founded by the creators of Apache Spark, has now released a product called Databricks AI/BI which aims to address the shortcomings of Gen AI and introduce self-service data analysis into the enterprise. 

It uses compound AI models that deploy an “ensemble” of agents to answer business questions and generate answers that are (hopefully) useful and non-hallucinatory. 

In a blog, Databricks wrote: “With the excitement around LLMs, the BI industry started a new wave of incorporating AI assistants into BI tools… Unfortunately, while these offerings are promising in concept and make for impressive product demos, they tend to fail in the real world. When faced with the messy data, ambiguous language, and nuanced complexities of actual data analysis, these ‘bolt-on’ AI experiences struggle to deliver useful and accurate answers.”

Databricks AI/BI is described as an "intelligence product built from the ground up to deeply understand your data's semantics and enable anyone to analyze data for themselves.”

“For the last 30 years, business users have been given reports and dashboards to answer the data questions they have," it wrote. "However, as businesses evolve, these users rely on scarce and overworked data professionals to create new visualizations to answer new questions.

“The reality is that it's not enough to just point an LLM at a database schema and do text-to-SQL, because the schema itself is missing a lot of knowledge, like definitions of business processes and metrics, or how to handle messy data. The other approach is to capture this understanding in formal semantic models, but they require significant up-front investment, can't capture all the nuances, and are impractical to keep up-to-date as data and business processes evolve.”

The best semantic model “lives in people’s heads” and “comes pouring out” whenever they interact with systems to run queries, create dashboards, and perform analyses, Databricks continued.

It hopes AI/BI can allow this information to gush out from inside users’ brains into a giant well of actionable business intelligence. 

Entering the compound

The Databricks compound AI system extracts insights from across its platform, including ETL pipelines, lineage, and other queries. 

Despite its warnings about bolt-on AI capabilities, the new Databricks solution features a conversational interface called Genie which learns from data and semantics continuously based on human feedback.

The data firm said it could answer a broad range of business questions using its reasoning capabilities, while still providing certified answers for query patterns set out by data teams.

Additionally, Databricks has introduced AI/BI Dashboards, an AI-powered, low-code solution that "includes all the conventional BI capabilities you'd expect out-of-the-box for answering a fixed set of business questions.”

“These features make AI/BI a significant step towards true self-service BI, significantly broadening the range of analytics that everyday users can perform,” Databricks wrote. “Additionally, AI/BI integration with Databricks' Data Intelligence Platform ensures unified governance, lineage tracking, secure sharing, and top-tier performance at any data scale.”

Each AI agent has a specific “narrow but important task”, such as planning, SQL generation, and result certification. 

"Together, they provide reasoning capabilities far beyond any individual, monolith model,” Databricks claimed.

The system is designed to continuously learn and adapt its performance by incorporating human feedback.

So for instance, when a customer churns, the lessons learned from that failure will be used to address relevant queries to drive better outcomes in future. 

AI/BI can also draw on data from across the Databricks platform, such as ETL pipelines and popularity statistics.

"We believe compound AI systems that can draw insights about your data from its full lifecycle will be transformative to the world of business intelligence," Databricks concluded. "The initial release of AI/BI represents a first but significant step forward toward realizing this potential. The system will become smarter over time as usage ramps up and the system evolves.”

The rise of compound AI models

In a paper called The Shift from Models to Compound AI Systems, experts from Berkeley Artificial Intelligence Research said the “intense focus” on LLMs is fading.

“State-of-the-art AI results are increasingly obtained by compound systems with multiple components, not just monolithic models,” they wrote. 

The team added: “Compound AI systems will likely be the best way to maximize AI results in the future, and might be one of the most impactful trends in AI in 2024."

AI/BI Dashboards are now generally available on AWS and Azure. They are also in public preview on Google Cloud Platform (GCP). All AWS and Azure customers can use Genie in public preview, with availability on GCP expected soon.