Custom Embedders

Overview

sovseal extracts vector embeddings from memories to run semantic searches. By default, client SDKs download and run a local Transformers.js ONNX embedding pipeline (using the 384-dimensional all-MiniLM-L6-v2 model) in-memory on the device.

If you are self-hosting the server and want to perform embeddings extraction server-side, you can configure alternative local or managed embedders.

Dimension Alignment Invariant

[!IMPORTANT] Vector Dimension Constraints

The vector dimension configured in your vector database collection MUST match the output dimension of your embedder.

The default sovseal MCP server vector dimension is 384 (matching all-MiniLM-L6-v2).

Changing to a different model (e.g. text-embedding-3-small with 1536 dimensions) requires recreating the vector collections or mapping the database fields.

Configuring Embedder Providers

1. Ollama (Local Embeddings)

Run your embedding pipeline locally without sending data to third-party endpoints.

embedder:
  provider: ollama
  config:
    model: nomic-embed-text
    host: http://localhost:11434

2. Vertex AI (Managed Google Cloud)

Ideal for enterprise workloads deployed within Google Cloud Platform.

embedder:
  provider: vertexai
  config:
    model: textembedding-gecko@003
    project_id: your-gcp-project-id

3. OpenAI (Managed Cloud)

Sourced via API key.

embedder:
  provider: openai
  config:
    model: text-embedding-3-small
    api_key: ${OPENAI_API_KEY}

Extending Shipped Providers

The default self-hosted Docker image includes bindings for openai and gemini embedder clients. Heavy machine learning frameworks (e.g., PyTorch and SentenceTransformers, which add ~2 GB to image sizes) are excluded by default.

To deploy a heavy local embedder model (like sentence-transformers) on your self-hosted API container:

Step 1: Add Package to Requirements

Add the dependency to your server/requirements.txt file:

sentence-transformers>=2.2.0

Step 2: Register Provider in API Router

Add the package mapping to the BUNDLED_EMBEDDER_PROVIDERS registry inside the server runtime:

# server/main.py
BUNDLED_EMBEDDER_PROVIDERS = ["openai", "gemini", "sentence-transformers"]

Step 3: Rebuild the Container

Recompile and boot your local image stack:

docker compose build --no-cache
make up

On this page