# What Are Large Database Models? AI for SQL Data

## Executive summary

Large Database Models (LDMs) represent a significant advancement in applying AI to enterprise data by bringing semantic capabilities directly into SQL and relational databases. Unlike Large Language Models (LLMs), which train on general text, LDMs are trained specifically on selected tables or views within a structured database. This allows organizations to unlock the estimated 99% of critical business data—often locked behind encryption and access controls—without needing to move it.

## Key takeaways

- LDM Functionality vs. Traditional SQL: Traditional methods require data scientists to manually write rigid SQL filters (e.g., `where age is between 20 and 40`) and move data to an analytics platform, which is slow and expensive. LDMs use vector representations learned from co-occurring values across columns to perform semantic queries, eliminating the need for manual field selection or guessing constraints.
- Core LDM Capabilities: LDMs enable advanced querying capabilities such as finding customer similarity (finding customers 'most similar' to a given ID), identifying unusual transactions (fraud detection), and exploring product relationships, all executed via standard SQL against the database itself.
- Commercial Availability: IBM launched the first LDM-based database product, 'SQL Data Insights,' which ships as part of DB2 for ZOS. A follow-up version, 'SQL Data Insights Pro,' extends this approach to unstructured text and adds incremental model refresh.

## Technical details

- LDM Architecture (The Five Steps): 1. **Selection:** Choose a table/view in the relational database. 2. **Classification:** Classify columns as categorical, continuous, or key identifiers. 3. **Tokenization & Embeddings:** Every value is converted into a token and then an embedding (a vector). For numeric data, values are first 'binned' using clustering algorithms to group numerically close values, treating them as equivalent tokens. 4. **Row Sentence Creation:** Each row becomes an unordered sentence (bag of words), pairing column names with their respective tokens. 5. **Training & Exposure:** A self-supervised neural network reads these sentences to learn vectors for every unique token. The trained model is then exposed through SQL, allowing semantic searches.
- Embeddings and Vector Space: An embedding converts a word or value into a vector (a list of numbers). Values with similar meanings or co-occurring patterns in the data will result in vectors that point in a similar direction within the vector space. This allows the model to understand relationships without explicit field constraints.
- Data Handling Efficiency: LDMs process data locally, running queries 'in place' against the database, avoiding the costly and insecure practice of extracting records and moving them to external analytics platforms.

## Practical implications

- Reduces reliance on data scientists to manually write complex, rigid SQL queries for AI insights.
- Significantly reduces the cost and complexity associated with moving large volumes of sensitive enterprise data outside its protected environment.
- Enables real-time semantic analysis (e.g., fraud detection or product similarity search) directly within existing relational database infrastructure.

## Topics

Large Database Models, AI/ML, SQL, Data Engineering, Vector Embeddings, Relational Databases (RDBMS), Large Database Models (LDMs), IBM AI Newsletter

Source: https://www.youtube.com/watch?v=uU1EP9_4qBU