Facet
DeutschInstall

Local search with Ollama

Find the answer in your product. Locally.

Connect FACET to Ollama, index your records and find the source passages relevant to a business question.

About 10 minutes to read; model download and indexing take additional time

Your coding agent

This selection applies to website examples. It does not configure a connection on your machine.

Prepare Ollama

Install Ollama using its official instructions. Start the Ollama application or, for a command-line installation, run ollama serve in a separate terminal. If the service already runs, keep the existing one.

Download an embedding model:

sh
ollama pull embeddinggemma

Check the OpenAI-compatible endpoint FACET will use. This is a direct HTTP request, so it needs no placeholder API key:

sh
curl http://localhost:11434/v1/embeddings \  -H 'Content-Type: application/json' \  -d '{"model":"embeddinggemma","input":"An expired discount code must be rejected."}'

Look for a JSON response containing data and an embedding array. The actual numbers depend on the model. An endpoint error must be resolved before indexing the project.

Connect FACET

Open the project's existing .facetrc.yaml and merge this top-level block. Do not replace registry or other project settings:

yaml
embedding:  provider: openai  base_url: http://localhost:11434/v1  model: embeddinggemma

Here openai selects the compatible HTTP request format; it does not mean that the request goes to OpenAI. FACET appends /embeddings to base_url. Do not use Ollama's separate /api/embed endpoint for this configuration.

Leave api_key_env unset for ordinary local Ollama. If it is already set for another provider, remove that setting for this local configuration. The local API does not require authentication. Some SDK examples use ollama as a dummy key because the SDK requires a value; FACET's HTTP client does not.

FACET's provider: local means its built-in word-overlap search, not Ollama. The block above deliberately selects a model-backed HTTP endpoint.

Work with an agent

Use a prompt that names the question and the evidence you expect:

text
Find the documented behaviour for expired discount codes in this project.Use FACET retrieval if available. Read the matching records and their scenarios.Cite the source files and distinguish documented promises from passing test results.If the sources do not answer the question, describe the gap instead of guessing.

When your agent application is connected to the project's facet mcp server, FACET exposes a tool named rag_query. This is an example of its tool name and arguments, not a shell command or a guarantee that your client is connected:

json
{  "tool": "rag_query",  "arguments": {    "query": "What happens when a customer uses an expired discount code?",    "limit": 5,    "kind": "record"  }}

The response includes source paths and text. The agent can then use the server's feature_get tool when it knows the feature id. MCP tool labels may have a client-specific prefix. The MCP process must run in the same configured project and inherit any environment variables needed by the embedding endpoint.

Without that MCP connection, an agent with shell permission can run:

sh
facet rag query "What happens when a customer uses an expired discount code?" --limit 5 --json

If no retrieval index exists, direct text search remains useful:

sh
rg -n -i 'expired|discount|voucher' docs/facets

rg is a separate command-line tool. It matches words, not meaning; adjust the search terms and directory to the project's actual vocabulary and layout. The human or agent must still read the matching files.

Understand access keys

For the local setup above, no token needs to be obtained, generated or pasted anywhere.

If your organization puts an OpenAI-compatible endpoint behind a proxy that requires a Bearer token, its operator supplies that token. An Ollama cloud API key comes from the Ollama account's API-key settings; it is not automatically a credential for your organization's proxy. This tutorial uses neither cloud inference nor an authenticated proxy.

For such a separately configured authenticated endpoint, FACET supports:

yaml
embedding:  provider: openai  base_url: https://embeddings.example.invalid/v1  model: your-approved-embedding-model  api_key_env: FACET_EMBEDDING_TOKEN

That hostname and model are placeholders, not a working service. Replace them only with the values supplied by your endpoint's operator. Set FACET_EMBEDDING_TOKEN in the environment of the FACET process through your usual secret-management method. FACET reads that variable and sends Authorization: Bearer ... only when a non-empty key is available. It does not create or obtain the token for you. Custom authentication schemes are not configured by this field.

Resolve common problems

  • Connection refused: start the local Ollama service on the machine running FACET. In a container or remote session, localhost is that environment, not necessarily your desktop.
  • Model not found: check the model name and complete ollama pull embeddinggemma before retrying.
  • 404 or an unexpected response: use the API root http://localhost:11434/v1. FACET adds /embeddings itself.
  • No records to index: change into the project containing the records, or use FACET's global --root option. This guide does not create or backfill records.
  • No index: run facet rag index with the same project configuration before querying.
  • Results use word overlap: check that embedding.provider is openai and rebuild the index with the configured model. The built-in local provider is lexical.
  • 401 from a protected endpoint: check the endpoint's own authentication requirements and the environment of the process. Do not add a token to ordinary local Ollama just to imitate an SDK example.

Sources and further reading