Real-Time LLM Firewall Benchmarks: Security without Latency

When building enterprise LLM pipelines, security is often viewed as a trade-off against speed. Developers fear that adding input scanners and safety filters will introduce visible latency, degrading the interactive user experience.

At LLM Bastion, we designed our firewall pipeline from the ground up for extreme performance. Let’s examine the raw performance data, testing methodologies, and results of our latest v1.4 release.

---

Benchmark Setup

All tests were performed under the following parameters:

• Model Engine: GPT-4o-mini & Claude 3.5 Sonnet.
• Payload Size: 1,500 average tokens (untrusted email text).
• Scanner Concurrency: 250 requests/sec simultaneous peak load.
• Host Infrastructure: Distributed Edge nodes (PC Spécialiste cluster).

Performance Metrics

Below is a breakdown of scanner latency across different protection levels:

Scanner Component	Protection Level	Mean Latency (ms)	P99 Latency (ms)
Injection Classifier	Standard	4.8 ms	7.2 ms
Semantic Drift Check	Extended	6.2 ms	9.5 ms
Leakage Obfuscator	Strict	3.1 ms	4.9 ms
Full Security Suite	Max Guard	14.1 ms	18.2 ms

As shown in the table, even under Max Guard (activating every single heuristic and classification classifier), the average overhead is under 15 milliseconds. This is imperceptible to users compared to standard LLM generation times (which often range from 800ms to 2,000ms).

---

Real-world Integration

Integrating the high-speed gateway into your existing TypeScript / React pipeline is straightforward. Here is an example using the official @llmbastion/sdk package:

import { LLMBastionShield } from '@llmbastion/sdk';

// Initialize the high-speed security gateway
const shield = new LLMBastionShield({
  apiKey: process.env.BASTION_API_KEY,
  environment: 'production',
  failOpen: false // Safety-first fallback
});

async function handleChatRequest(userPrompt: string, untrustedContext: string) {
  // Scan untrusted context before model ingestion
  const { isSafe, sanitizedPrompt, incidentReport } = await shield.scan({
    prompt: userPrompt,
    context: untrustedContext,
    protectionLevel: 'strict'
  });

  if (!isSafe) {
    console.error("🚨 Intrusion attempt blocked:", incidentReport.reason);
    throw new Error("Safety check failed. Request aborted.");
  }

  // Forward safely to model
  return await callOpenAIModel(sanitizedPrompt);
}

Architectural Innovations

How do we keep scanning speeds so fast?

1. Lightweight Tokenizers: Our initial classifiers do not call large LLMs themselves. They use specialized high-velocity classification models running directly on Edge nodes.
2. Short-circuit Pipelines: If a request passes initial high-confidence indicators, it bypasses heavier structural evaluation, resolving instantly.
3. Rust Runtime: The core parsing engine is compiled to native Rust binaries, running safely without JavaScript garbage collection overhead.