What Vector Databases Actually Do
Vector databases store and retrieve high-dimensional numerical representations of unstructured data. If that sounds abstract, here's the operational reality: your maintenance logs, regulatory filings, equipment manuals, and incident reports are text. Large language models convert that text into vectors—arrays of 768 or 1536 floating-point numbers that capture semantic meaning. Vector databases let you search those vectors by similarity, not keywords.
We've deployed vector databases across utilities for retrieval-augmented generation systems, equipment troubleshooting assistants, and regulatory compliance search. The difference between keyword search and vector search is the difference between finding documents that contain "transformer failure" and finding documents about the same operational problem described as "step-down unit trip" or "distribution equipment fault."
In our SCADA integration work, vector databases became the memory layer that let AI assistants recall relevant procedures, past incidents, and equipment specifications without re-reading millions of documents on every query. That's the core value: semantic memory at operational speed.
Why Energy Operations Need This Now
Energy utilities sit on decades of unstructured operational knowledge—maintenance logs going back to the 1980s, engineering drawings in proprietary CAD formats, tribal knowledge captured in email threads. Traditional databases can't search this effectively. Full-text search finds keywords but misses context. Graph databases excel at relationships but struggle with semantic similarity.
Vector databases fill the gap between structured relational data and pure keyword search. When an engineer asks "what caused the capacitor bank failure at Substation 7 last winter," you need semantic understanding. The relevant incident report might describe it as "reactive power compensation equipment outage during extreme cold," and vector search finds it.
For NERC CIP compliance, we've used vector databases to implement "ask your compliance library" systems. Instead of manually searching through CIP-002 through CIP-014 standards, engineers query in plain language. The vector database retrieves the relevant sections, which a local LLM uses to generate contextualized answers. No compliance data leaves the facility, no third-party API calls, full audit trail.
The air-gapped requirement drives much of our vector database work. You can't send SCADA data or equipment specifications to OpenAI's API for embedding and search. You need local embedding models like sentence-transformers running on-premises, with vector storage that respects data boundaries.
Core Capabilities That Matter in Production
Embedding Storage and Retrieval
Vector databases store embeddings—those 768 or 1536-dimensional arrays—with metadata. The metadata is critical. In our deployments, we tag every vector with source document, timestamp, equipment ID, facility code, and classification level. When the vector database returns similar vectors, you need that metadata to trace back to the original maintenance log or engineering specification.
Retrieval happens via approximate nearest neighbor search. Exact nearest neighbor search on high-dimensional vectors is computationally prohibitive at scale. ANN algorithms like HNSW (Hierarchical Navigable Small World) trade perfect accuracy for speed—typically 95-99% recall with 10-100x faster queries. For operational use cases, 97% recall is sufficient. You're finding relevant documents, not calculating spacecraft trajectories.
Hybrid Search
Vector similarity alone isn't always enough. Sometimes you need semantic search combined with exact keyword matching or metadata filters. If an engineer asks about transformer failures, but only wants results from a specific substation in the last 90 days, pure vector search won't handle the temporal and location constraints efficiently.
Hybrid search combines vector similarity with traditional filtering and keyword search. Weaviate has strong built-in hybrid capabilities. With Qdrant, we implement filtering via payload queries before vector search. The architecture matters: filter first, then vector search on the reduced set, not the other way around.
Multitenancy and Access Control
In utility environments, not every engineer should access every document. Generation control room operators shouldn't see transmission maintenance logs. NERC CIP Critical Cyber Assets require stricter access than non-critical systems. Your vector database needs multitenancy—logical separation of data by facility, department, or clearance level.
Qdrant handles this through collections and payload-based filtering. We create separate collections per facility or implement row-level security via payload tags. It's not database-level authentication—you still need application-layer authorization—but the database must support efficient filtered queries without scanning the entire vector space.
The Open-Source Landscape: Actual Differences
Qdrant: Our Default Choice
Qdrant is Rust-based, fast, and designed for production RAG systems. We deploy it most often because it balances performance with operational simplicity. Single-node Qdrant handles 10-50 million vectors comfortably on modest hardware. The Docker deployment is straightforward, the API is clean, and it doesn't require Kubernetes or distributed system expertise to run reliably.
Qdrant's payload filtering is excellent for our metadata-heavy use cases. Every maintenance log vector carries 15-20 metadata fields. Filtering by equipment type, facility, date range, and classification before vector search keeps query latency under 50ms even with 20 million vectors. The quantization support—storing vectors as 8-bit integers instead of 32-bit floats—cuts memory usage by 75% with minimal accuracy loss.
Weak points: Qdrant's distributed mode exists but isn't as mature as Milvus. For single-facility deployments, that doesn't matter. For multi-region utilities needing active-active replication, you'll fight Qdrant's architecture.
Weaviate: When You Need Hybrid Search Built-In
Weaviate's killer feature is native hybrid search and built-in vectorization. You can send raw text to Weaviate, and it handles embedding generation internally using sentence-transformers or OpenAI models. For rapid prototyping, that's convenient. In production, we prefer explicit embedding pipelines for observability and control, so this matters less.
We've used Weaviate where the application team needed BM25 keyword search tightly integrated with vector similarity. The hybrid fusion algorithm is solid. But Weaviate's resource footprint is heavier than Qdrant—expect 2-3x memory usage for the same dataset. The GraphQL API is elegant but unfamiliar to teams used to REST.
Milvus: Trillion-Scale Ambitions We Don't Have
Milvus is built for massive scale—billions to trillions of vectors across distributed clusters. For hyperscalers and recommendation engines, that's compelling. For utilities, it's overengineered. Our largest single-facility deployment has 50 million vectors. Even a multi-site utility rarely exceeds 200 million.
Milvus requires etcd, MinIO, and Pulsar for distributed operation. That's three additional systems to deploy, monitor, and secure. The operational complexity doesn't justify itself unless you're operating at a scale we've never encountered in energy. If you're running a global asset management platform indexing satellite imagery across 50 countries, consider Milvus. For NERC CIP-compliant RAG at a utility, you're adding failure modes for capabilities you won't use.
ChromaDB: Prototyping and Embedded Use
ChromaDB shines for local development and embedded applications. The API is dead simple, deployment is pip install, and for datasets under 1 million vectors, it performs fine. We use it for proof-of-concept work and demo systems.
In production, ChromaDB's single-server architecture and file-based storage become limitations. No native replication, basic access control, limited observability. It's SQLite for vector databases—excellent for what it is, but you outgrow it as requirements expand. If your RAG system will run on an engineer's laptop or a single edge device, ChromaDB is appropriate. If it's backing a multi-user application, you'll migrate to Qdrant or Weaviate within six months.
Neo4j: Not a Vector Database, But Often Adjacent
Neo4j appears in this context because knowledge graphs and vector embeddings solve complementary problems. Graph databases excel at explicit relationships—this transformer connects to that bus, this maintenance procedure references that equipment standard. Vector databases excel at semantic similarity—finding conceptually related documents.
We've deployed systems where Neo4j stores the equipment hierarchy and maintenance workflow graph, while Qdrant stores embedded maintenance logs and procedures. When an engineer queries about a piece of equipment, Neo4j identifies related assets and procedures via graph traversal. Qdrant retrieves semantically similar historical incidents. The LLM synthesizes both into a coherent answer.
Neo4j now supports vector indexes natively, but it's a secondary feature. If your primary need is relationship traversal, use Neo4j. If your primary need is semantic search over unstructured text, use a dedicated vector database. Don't compromise on the core capability.
Deployment Reality: What Actually Breaks
Embedding Model Drift
Your vector database is only as good as your embedding model. If you embed your entire document corpus with sentence-transformers/all-MiniLM-L6-v2, then six months later switch to a different model, your vectors are incompatible. You must re-embed everything. We've seen teams discover this after indexing 5 million documents.
Pick an embedding model with care. We use sentence-transformers/all-mpnet-base-v2 for general text and nomic-embed-text for longer documents. Lock the model version. Plan for re-indexing capacity if you must upgrade.
Query Performance Tuning
ANN search has parameters—HNSW's ef_construct and M values, quantization settings, shard distribution. Default settings work for demos. In production, you'll tune based on your query patterns and latency requirements.
We've seen query latency drop from 400ms to 50ms by tuning HNSW parameters and enabling scalar quantization in Qdrant. The tooling for this tuning is immature—expect to run benchmarks yourself. The vector database vendors don't have energy-specific guidance.
Backup and Disaster Recovery
Vector databases are stateful. If your Qdrant instance dies, you've lost your vector index. Rebuilding from source documents takes hours to days depending on corpus size and embedding throughput. We run nightly snapshots to S3-compatible storage and maintain warm standby instances for critical systems.
Disaster recovery testing revealed that re-indexing 20 million documents takes 8 hours with our embedding pipeline. That's acceptable for planned maintenance, unacceptable for outage recovery. Hence the snapshots.
The Verdict
Vector databases are infrastructure, not magic. They enable semantic search and RAG by storing and retrieving embeddings efficiently. For energy operations, they're the memory layer that lets AI assistants recall relevant operational knowledge without re-processing millions of documents on every query.
Qdrant is our default recommendation for utilities. It's fast, operationally simple, handles 50 million vectors comfortably on single-node deployments, and the payload filtering aligns with metadata-heavy energy use cases. Deploy it in Docker, point your embedding pipeline at it, and you'll have a production-grade vector store running in a weekend.
Use Weaviate if hybrid search and built-in vectorization matter more than resource efficiency. Use ChromaDB for prototypes and embedded applications. Avoid Milvus unless you have distributed system expertise and genuinely need multi-billion vector scale. Combine with Neo4j when you need both relationship traversal and semantic similarity.
Vector databases won't fix bad data or replace domain expertise. They're a tool that makes unstructured operational knowledge searchable. Deploy them where semantic understanding adds value—equipment troubleshooting, regulatory compliance search, maintenance procedure retrieval. Ignore the hype, measure the latency, and keep your embeddings model versioned. The technology works when you treat it like infrastructure, not innovation theater.