Distil Labs Enables Rocketgraph’s Private AI on IBM Power with Small Language Models

Your Data Never Leaves Your Enterprise: Faster, Greener, and with More Secure AI-Powered Graph Querying

In an era where data breaches make headlines daily and regulatory compliance grows ever stricter, enterprise customers face a fundamental dilemma: how to leverage the power of modern AI while maintaining absolute control over sensitive data. Today, Rocketgraph, IBM, and Distil Labs announce a breakthrough solution using Small Language Models (SLMs) that delivers performance on par with Large Language Models (LLMs) for graph analytics without a single byte of customer data ever leaving the enterprise perimeter all whilst running 10x faster and using 100x less energy than cloud-based LLMs.

The Privacy Paradox in Enterprise AI

Many Rocketgraph customers running graph analytics on IBM Power hardware have been asking for AI-powered natural language querying capabilities. They've seen the impressive demonstrations of ChatGPT and Claude translating plain English into complex database queries. But there's a catch that's been a blocker for regulated industries, government agencies, and security-conscious enterprises. Using these cloud-based LLMs means sending potentially sensitive query patterns, schema information, and business logic to external servers.

Add to this are both financial and environmental hidden costs of cloud-based LLMs. Each query costs money, takes seconds to process, and consumes significant energy in remote data centers. For organizations running thousands of queries daily, these costs compound quickly.

For organizations handling financial transactions, healthcare records, defense contracts, or intellectual property, the privacy issue alone is often legally insurmountable. When you factor in the performance and sustainability concerns, the need for a different approach becomes clear.

Enter SLMs: The Enterprise AI Revolution

Small Language Models (SLMs) represent a fundamental shift in how enterprises can deploy AI. Unlike their massive LLM cousins that require entire data centers to run, SLMs are compact, specialized models typically ranging from 1-10 billion parameters (compared to popular closed-source LLMs that are rumored to be north of 1 trillion parameters).

The key insight: SLMs can match or exceed LLM performance on specific tasks while being many orders of magnitude smaller, faster, and more efficient. Think of it as the difference between using a Swiss Army knife versus a specialized surgical instrument, when you know exactly what you need to accomplish, the specialized tool is superior.

A Fundamentally Different Approach: Specialized SLMs for Enterprise Security

Instead of trying to work around the privacy limitations of cloud-based LLMs, we've taken a completely different approach. By partnering with Distil Labs, we've developed a specialized SLM that:

  • Runs entirely within your infrastructure - The SLM operates on your IBM Power hardware, behind your firewall, under your security policies
  • Never phones home - No telemetry, no cloud connectivity required, no data leakage risk
  • Achieves 85% of Claude 4 performance - Despite being many orders of magnitude smaller
  • Executes 10x faster - Sub-second query translation vs several seconds for cloud LLMs
  • Uses 100x less energy - A few watts vs kilowatts for large model inference
  • Specializes in one thing - Translating natural language to Rocketgraph-compliant OpenCypher queries

How We Built Enterprise-Grade Privacy Into SLMs

The Training Data Challenge

The key insight was that we could train an SLM with 8B parameters using publicly available information and synthetic data, meaning the SLM itself contains no customer-specific information. This dramatic size reduction is what enables the 10x speed improvement and 100x energy savings. Here's how we did it:

  1. Started with Rocketgraph documentation - Public information about our OpenCypher variant
  2. Created synthetic schemas - Translated 900+ schemas from public Neo4j datasets into Rocketgraph-compatible formats
  3. Generated 15,000+ training examples - All validated against the Rocketgraph platform
  4. Fine-tuned IBM Granite 3.3 8B - An SLM small enough to run efficiently on-premise

The Technical Innovation

The challenge wasn't just about privacy but also accuracy. Rocketgraph uses a specific variant of OpenCypher that differs from standard Cypher. For example, where standard Cypher might use:

MATCH (d)-[r:EdgeType]->() RETURN d, count(r) AS count

Rocketgraph's idiomatic approach is:

MATCH (d) RETURN d, outdegree(d, EdgeType) AS count

Our SLM had to learn these Rocketgraph-specific patterns without ever seeing actual customer queries or schemas during training.

What SLMs Mean for Your Enterprise

Complete Data Sovereignty

  • Your queries never leave your data center
  • Your schema remains confidential
  • Your business logic stays proprietary
  • Compliance teams can sleep soundly

Deployment Simplicity

The SLM integrates directly with your existing Rocketgraph installation on IBM Power hardware. No complex networking, no firewall exceptions, no data governance reviews for external services. It's your SLM, running on your hardware, analyzing your data, under your complete control.

The Performance and Sustainability Revolution of SLMs

Speed That Changes How Teams Work

When SLMs run locally on optimized hardware, the difference is dramatic:

  • Query translation in milliseconds, not seconds - No network latency, no API queuing
  • Instant feedback loops - Analysts can iterate and refine queries in real-time
  • Batch processing becomes viable - Process thousands of natural language queries without API throttling
  • Consistent sub-second response times - No variability from cloud service congestion

Our benchmarks show the SLM translating complex natural language queries to Cypher in under 200ms on IBM Power hardware, compared to 2-5 seconds for cloud-based LLMs (including network overhead).

Real-World Impact: SLMs in Banking

Consider a financial institution analyzing transaction patterns for fraud detection. Their analysts need to query complex relationship graphs containing:

  • Customer personal information
  • Transaction histories
  • Account relationships
  • Suspicious pattern indicators

With our SLM solution, an analyst can simply ask: "Show me all transactions over $10,000 involving accounts opened in the last 30 days that have connections to flagged entities."

The SLM translates this to precise Rocketgraph Cypher in under 200 milliseconds, executes it locally, and returns results without any data exposure risk. This speed is critical for fraud detection where every second counts. Compare this to cloud-based LLMs where:

  • Network round-trip alone adds 50-100ms
  • Query processing takes 2-5 seconds
  • API rate limits might delay batch analysis

For a fraud team running hundreds of investigative queries per hour, SLMs mean faster investigations, lower costs, and complete data privacy.

The Technical Deep Dive: How Distil Labs Makes SLMs Possible

The breakthrough came from Distil Labs' expertise in knowledge distillation, which involves teaching SLMs to replicate the capabilities of larger models for specific tasks. This process doesn't just make models smaller; it makes them dramatically more efficient. Instead of requiring 10,000+ hand-labeled examples (which would likely contain sensitive information), their platform enabled us to:

  1. Generate high-quality synthetic training data from documentation and public schemas
  2. Validate all examples programmatically against Rocketgraph
  3. Fine-tune the SLM to understand query variations ("List all," "Find all," "Show me all")
  4. Optimize specifically for Rocketgraph's OpenCypher variant
  5. Compress the SLM to run efficiently on CPUs without GPU acceleration

Getting Started: Your Data, Your Control, Your SLM

For Rocketgraph customers on IBM Power hardware, deploying your own SLM is straightforward:

  1. The SLM runs directly on your existing infrastructure
  2. No external dependencies or API keys required
  3. Integration with your current Rocketgraph installation
  4. Full support from the combined Rocketgraph, IBM, and Distil Labs team

The Future of Enterprise AI: Specialized SLMs Leading the Way

This collaboration between Rocketgraph, IBM, and Distil Labs represents more than just a product release, it's a blueprint for how enterprises can adopt AI without compromising on security, speed, or sustainability. By embracing SLMs over LLMs for specialized tasks, and by keeping computation local, we've proven that organizations don't have to choose between innovation and responsibility.

In a world where data is the most valuable asset and environmental responsibility is paramount, SLMs represent the future of enterprise AI. Your graph analytics can be intelligent, secure, fast, and sustainable.

Ready to explore how SLMs can transform your graph analytics without the privacy, performance, or sustainability trade-offs? Contact our team to learn how you can deploy your own specialized SLM in your enterprise environment.

For technical documentation on SLM deployment and implementation details, visit https://docs.distillabs.ai/getting-started/overview. For security audits, performance benchmarks, and sustainability metrics, reach out to our enterprise team.

Heading 1

A Content Management System (CMS) is a software application that allows users to create, manage, and modify content on a website without the need for specialized technical knowledge. It provides a user-friendly interface for managing digital content, making it accessible to a wider audience.

Heading 2

One of the primary benefits of using a CMS is the ease of use. Users can easily add, edit, and delete content without needing to write code. Additionally, CMS platforms often come with built-in SEO tools, templates, and plugins that enhance functionality and improve user experience.

“A good CMS empowers users to manage their content effectively.”

Heading 2

Heading 3

Heading 4

Heading 5
Heading 6

normal text

bold

italics

up

down

AAAAaaa

aaaa

Image description

Heading 1

A Content Management System (CMS) is a software application that allows users to create, manage, and modify content on a website without the need for specialized technical knowledge. It provides a user-friendly interface for managing digital content, making it accessible to a wider audience.

Heading 2

One of the primary benefits of using a CMS is the ease of use. Users can easily add, edit, and delete content without needing to write code. Additionally, CMS platforms often come with built-in SEO tools, templates, and plugins that enhance functionality and improve user experience.

“A good CMS empowers users to manage their content effectively.”

Heading 2

Heading 3

Heading 4

Heading 5
Heading 6

bold

italics

up

down

Image description

Code snippet

Code goes here
> xd
}

ABCDE
1
DatasetTeacher (↑)
Trained Student (↑)
Base Student (↑)Seed Student (↑)
2
Ecommerce0.9050.91 +/- 0.01NA0.88
3
TREC0.850.92 +/- 0.005NA0.81
4
Mental Health0.820.85 +/- 0.01NA0.80
5
Banking770.910.895 +/- 0.02NA0.76
6
PII Redaction0.85 +/- 0.020.82 +/- 0.010.54 +/- 0.030.73 +/- 0.01
7
HotpotQA0.93 +/- 0.010.91 +/- 0.010.8 +/- 0.010.85
8
Roman Empire QA
0.98 +/- 0.010.99 +/- 0.010.91 +/- 0.020.93 +/- 0.01

Code snippet

{
"input":"Please verify account with SSN 987654321 and send update to alex@x.io.",
"output": {
  "redacted_text":"Please verify account with SSN [SSN] and send update to [EMAIL].",
  "entities":[
    {"replacement_token":"[SSN]","value":"987654321"},
    {"replacement_token":"[EMAIL]","value":"alex@x.io"}
    ]
  }
}