Vector databases for a team that has never used one

Written By: BrainSoft In AI & Research

Somebody on the team reads that the new search feature should be semantic, and suddenly there is a vector database in the backlog. Nobody has run one before. The docs assume you already know what an embedding is, what an index type means for recall, and why your query returns eight near-identical results. It is not that hard, but the order of operations matters.

Start with a local instance of a small engine, embed a few hundred real documents, store the vectors with their metadata, and query them. Do that once end to end before you pick a hosted provider or a production schema, and the rest of the decisions get much easier.

What a vector database actually does

An embedding model turns a piece of text into a list of numbers, usually a few hundred to a couple of thousand of them. Texts with similar meaning end up with vectors that sit close together. A vector database stores those lists and answers one question fast: which stored vectors are nearest to this one? That is the whole product. Everything else, filtering, replication, backups, is normal database work layered on top.

The consequence is that you cannot eyeball the data. A wrong row in Postgres looks wrong. A bad embedding looks like a plausible float array and silently returns mediocre results. Budget time for evaluating retrieval quality, not just for getting the service to start.

Set it up in five steps

1. Pick an engine you can run locally. For a first project, Qdrant, Weaviate, or pgvector inside Postgres all work. If your data already lives in Postgres and the corpus is under a few million chunks, pgvector avoids adding a service to your stack. If you expect heavy filtering plus large scale, a dedicated engine is worth the extra container.
2. Choose an embedding model and fix it. Write the model name and its dimension into your config, not into a comment. Changing the model later means re-embedding everything, so treat it as a schema migration.
3. Chunk your documents sensibly. Split on paragraph or heading boundaries, keep chunks in the few-hundred-token range, and store the source document id and a title on every chunk. Overlap of one sentence between chunks is usually enough.
4. Create the collection and insert. Batch your inserts. Most clients handle a few hundred vectors per request comfortably, and batching is the difference between minutes and hours on a real corpus.
5. Query, then measure. Write twenty questions you know the answers to, run them, and read the results by hand. That small set is your regression suite for every later change.

A minimal pgvector setup looks like this. The dimension has to match your model exactly, so check it first.

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE chunks (
  id bigserial PRIMARY KEY,
  doc_id text NOT NULL,
  content text NOT NULL,
  embedding vector(1536)
);

CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);

Inserting is plain SQL. Pass the vector as a string literal in the format pgvector expects.

INSERT INTO chunks (doc_id, content, embedding)
VALUES ('faq-12', 'Reset your password from the account page.', '[0.021,-0.113,0.447,...]');

And the query orders by distance, so smaller is closer with cosine distance. Ask for more rows than you plan to show; you will want to filter and deduplicate afterwards.

SELECT doc_id, content, embedding <=> $1 AS distance
FROM chunks
ORDER BY embedding <=> $1
LIMIT 20;

Mistakes that cost the most time

• Embedding the query with a different model or different preprocessing than the documents. Similarity scores go strange and nothing else looks wrong.
• Chunks that are too large. A 2000-token chunk about three topics retrieves for none of them well.
• Forgetting to store the source text. You will want to show it, re-chunk it, or re-embed it, and refetching from the original system is often impossible.
• Trusting the default index parameters at scale. HNSW build settings trade memory and build time against recall; revisit them when the corpus grows.
• No metadata filters. Real applications almost always need to restrict by tenant, language, or document type, and filtering changes which index strategy makes sense.

If you would rather have someone sit with the team for the first iteration, that is the kind of work we do through our services.

When you outgrow the local setup

The signals are concrete: index builds no longer fit in a maintenance window, p95 query latency drifts past your budget, or you need replication and snapshot restores you do not want to hand-roll. At that point move to a managed instance of the same engine rather than switching engines mid-project. Keeping the client code and query shape identical makes the migration a configuration change.

Frequently asked questions

#AI & Research