Semantic Search

Overview

The Logrite Semantic Search API wraps OpenSearch’s neural search capabilities with a clean REST interface. It uses sentence-transformers (all-MiniLM-L6-v2) to generate 384-dimensional embeddings and performs k-NN similarity search.

Setting	Value
Technology	Node.js + TypeScript + Express
Model	huggingface/sentence-transformers/all-MiniLM-L6-v2
Embedding Dims	384
Host port	9100 (search.logrite.ai · dev-search.logrite.ai)
Repo	Deployd-io/logrite-semantic-api
Runs as	A docker compose container on each VM

API Endpoints

Method	Path	Description
GET	/health	Health check
GET	/api-docs	Swagger UI
GET	/api/v1/indexes	List the caller’s workspace indexes (plus shared ones) with has_knn flag
GET	/api/v1/models	List registered embedding models (/models/deployed for the live one)
POST	/api/v1/search	Neural search on a specific index
POST	/api/v1/search/all	Combined neural + keyword search across all indexes

Debug mode

Append ?debug=1 to any request, or send X-Debug: 1 as a header, to opt the call into diagnostic logging:

• The request-logger middleware prints index=, bodyKeys=, and debug= on the inbound line so a 502 can be correlated to “which index was being touched” and “what shape of query”.
• The error handler attaches a _debug object to the response body containing upstream.method, upstream.url, upstream.status, and osReason so the caller (typically the Logrite-UI debug console) can render the upstream OpenSearch failure without spelunking through pod logs.
• The Logrite-UI surfaces this _debug block automatically when window.__logriteDebugToggle(true) is active.

No row data, no query text, no credentials are ever included in the _debug block — only metadata and OpenSearch’s own error.reason field.

Index Discovery

/api/v1/indexes returns every index the caller may read — the caller’s workspace and the shared indexes, semantic or not:

[
  {
    "name": "payment_queries_semantic",
    "has_knn": true,
    "document_count": 5420
  },
  {
    "name": "payment_queries",
    "has_knn": false,
    "document_count": 5420
  }
]

Combined Search

/api/v1/search/all searches:

• Semantic indexes (neural k-NN) for indexes with has_knn: true
• Non-semantic indexes (keyword BM25) for indexes with has_knn: false
• Returns combined results with fields like traceId where available

An empty answer says why when nothing was searched (2026-10-01). When no index at all was searched, the response gains an additive warnings array instead of a bare empty 200 that reads exactly like “nothing matched”:

code	When
NO_WORKSPACE	the caller carries no workspace, so no index could be searched
NO_SEMANTIC_INDEX	no *_semantic index on the cluster holds any documents
NO_SEMANTIC_INDEX_IN_WORKSPACE	populated semantic indexes exist, but none is the caller’s or shared — the others are never named or counted per tenant

Each entry is {code, message}, and the two index messages point at POST /api/v1/search/hybrid over the caller’s log family (log-analytics-<ws>-*). No existing field changes.

Tenant scoping — the legacy compliance-logs index

compliance-logs (bare) is the shared index the MCP server wrote before compliance logs went per-workspace (compliance-logs-<ws>). Its documents carry no workspace field and the name has no workspace segment, so it used to fall through to “no owner, readable”. It is now refused for /indexes listing, search and the :name routes. A wildcard starting with compliance-logs must name the caller as a whole segment (compliance-logs-<ws> or compliance-logs-<ws>-*), because the generic -<ws>* allowance would also match another workspace whose id begins with the caller’s. The backend’s OpenSearch proxy refuses the same name.

Authentication

The search routes (/api/v1/search*, /api/v1/embeddings/*), /api/v1/indexes and /api/v1/models take a session JWT or a Logrite lgp_live_ API key (validateCredential), and every query is scoped to the caller’s workspace — /indexes lists the caller’s workspace plus shared indexes, and /indexes/{name} runs the same scope check. Until 2026-10-01 /indexes and /models were JWT-only and answered a key the search route accepted with 401 jwt malformed, which reads as a broken key. Registering, deploying or undeploying a model still needs the Owner, Admin or Maintainer role, which an API key does not carry — so model mutations stay closed to a key. /health, POST /health/ensure and /api/v1/status are open.

When /health reports model: false

curl -s https://search.logrite.ai/health
# {"status":"success","data":{"status":"degraded","checks":{"opensearch":true,"model":false},…}}

model: false means no embedding model is deployed in OpenSearch ML Commons. The service stays up, but neural search has nothing to embed queries with, so POST /api/v1/search/all answers 200 with an empty result ("results":[],"total":0, "indexes_searched":[]) rather than an error — and the MCP semantic_search tool reports 0 hit(s). An empty answer here is not evidence that the logs do not contain the phrase. (On dev, POST /api/v1/search against one named index still returned a match for the same query, so the single-index route degrades less than search/all does.)

State on 2026-10-01: both estates report model:false

search.logrite.ai/health and dev-search.logrite.ai/health both answer degraded with model: false. On dev, GET /api/v1/models lists two registered copies of all-MiniLM-L6-v2, both in model_state: DEPLOY_FAILED, and GET /api/v1/models/deployed answers 404 No deployed embedding model found. Semantic search returns nothing until a model is deployed.

How the model is provisioned now

There is no opensearch-mgr, no /root/logrite/cli/… and no model-health cron any more — they belonged to the host destroyed on 2026-08-03 (cli/model-health-check.js survives in MasterControl but nothing schedules it). Provisioning is the service’s own code:

• src/services/modelConfig.ts pins the model: huggingface/sentence-transformers/all-MiniLM-L6-v2, version 1.0.1, TORCH_SCRIPT — not the default 1.0.2, whose TorchScript used an op (aten::scaled_dot_product_attention) the cluster’s libtorch lacked.
• src/services/modelRecovery.ts (ensureModelDeployed) is the recovery sequence: if a model is deployed, do nothing; if one is REGISTERED/UNDEPLOYED, deploy it; if one is DEPLOYING, wait; otherwise register a new 1.0.1 copy and deploy it. If the cluster cannot be read it does nothing rather than register a duplicate.
• It runs on demand, when something needs an embedding, or when asked:

# 1. inspect (session JWT)
curl -s https://dev-search.logrite.ai/api/v1/models/deployed -H "Authorization: Bearer $JWT"
curl -s https://dev-search.logrite.ai/api/v1/models          -H "Authorization: Bearer $JWT"

# 2a. run the service's own recovery sequence (open route)
curl -s -X POST https://dev-search.logrite.ai/health/ensure
#    200 + ensureResult.action (already_deployed | deployed_existing | registered_and_deployed)
#    503 + ensureResult.error when it could not

# 2b. or deploy one specific registered model (JWT, Owner/Admin/Maintainer)
curl -s -X POST https://dev-search.logrite.ai/api/v1/models/<model_id>/deploy \
  -H "Authorization: Bearer $JWT"

A copy in DEPLOY_FAILED is neither redeployed nor reused by /health/ensure — the sequence moves on and registers a fresh copy. If that copy fails the same way, the cause is in the cluster (read the failed model’s record for the error), and the manual register-and-deploy procedure with the OpenSearch admin user is in Troubleshooting under Embedding model in DEPLOY_FAILED state.

Integration with MCP

The MCP server’s semantic_search tool calls this API with the caller’s own credential, so results are limited to that workspace’s indexes.

How a search matched (2026-10-09)

search and hybridSearch responses carry mode: semantic, hybrid or keyword. An index whose mapping has no knn_vector field is searched by keyword (BM25), and the response says so:

{"mode": "keyword",
 "warnings": [{"code": "KEYWORD_FALLBACK",
   "message": "This index has no embeddings, so the search matched words, not meaning."}]}

Collector log indexes stay keyword. Upload datasets created since 2026-10-09 carry embeddings and answer hybrid/semantic; older uploads stay keyword and say so.

Keyword recall, ranked by meaning (semantic-api 84e95dc, 2026-10-14). On an index with no embeddings /search/hybrid over-fetches 50 BM25 candidates, groups them by template (digits and hex ids folded), embeds the question and up to 24 distinct texts with the deployed model in one batch, and orders the groups by cosine similarity. Measured on dev: ~80 ms a text on the CPU model, so ~2 s a search. Any failure — no model, a 4 s deadline, a malformed reply — keeps the keyword order. The response says which happened:

{"mode": "keyword", "ranking": "meaning",
 "warnings": [{"code": "KEYWORD_FALLBACK",
   "message": "This index has no embeddings, so results were found by shared words and then ranked by meaning."}]}

ranking: "words" keeps the original warning.

Meaning recall from a template catalogue (semantic-api f2063f1). Ranking alone still needed a shared word: “user could not sign in” never reached Login failed. Real embeddings mean reindexing (index.knn is fixed at creation; dev’s largest indexes are 1–2.7 M documents), so instead semanticTemplates.ts samples each searched index (600 lines + 600 error-like lines), folds them into up to 150 distinct templates, embeds them once in the background (9–26 s on dev) and keeps them for 6 hours. A search embeds the question, takes the 3 templates nearest in meaning (cosine ≥ 0.45 on text with ids, emails, addresses and numbers removed — calibrated on dev: right matches 0.46–0.71, wrong ones up to 0.44; at 0.35 a database question pulled login failures), fetches their lines by phrase and reranks them with the word matches. Until the catalogue is ready the search is the word search. Structured lines are embedded by their msg/message/error fields ({"level":…,"msg":"Login failed"} scored 0.24 raw and 0.71 as its message), and a template’s phrase is whole words only. The answer carries recall: "words" | "words+meaning". Measured on dev: “user could not sign in” → Login failed (0.71); “credential rejected by the directory server” → scheduler failed authentication for …; 0.65–1.5 s a search.

⚠ The catalogue is a SAMPLE: a template too rare to be sampled is found only by its words. OpenSearch’s ML node answers 429 past its own concurrency limit; every step then falls back to the word search.

A pattern (log-analytics-3103-*) runs the neural part only when every index it covers has embeddings. When only some do, every index is searched by keyword and the warning says why: “Some indexes this pattern covers have no embeddings, so every index was searched by words, not meaning.” Before semantic-api 36723aa9 one embedded index marked the whole pattern semantic and the neural query matched nothing on the collector indexes, so the search returned 0 hits.

• query must be 1–1000 characters on search, hybrid and batch; otherwise 400 naming query.
• A query the engine cannot run (too many clauses after tokenisation) answers 400 {"code":"QUERY_REJECTED","message":"The search engine could not run this query (too many terms after tokenisation). Shorten the query to a phrase."}. A generic “all shards failed” is mapped this way only when the upstream answered a 4xx other than 429, so an overloaded cluster is not blamed on the query.
• GET /api/v1/embedding/model → {"modelId": "…", "dimension": 384, "pipeline": "logrite-upload-embedding"}. At boot (and lazily per deployed model id) the API writes the ingest pipeline logrite-upload-embedding = {"processors":[{"text_embedding":{"model_id":"…","field_map":{"message":"text_embedding"},"ignore_failure":true}}]}.
• The AI service creates each new upload dataset index with index.knn: true, index.default_pipeline: "logrite-upload-embedding" and text_embedding as a 384-d knn_vector, read from that route (cached 10 min). When the route is unreachable it creates a plain index (keyword only) and logs once.

Close-out fix wave 2 (2026-10-09)

• Hybrid uses a normalization search pipeline. hybridSearch runs with a temporary search pipeline (min_max normalization, weighted arithmetic_mean combination), so keyword and vector scores are on one scale (semantic-api 4cd4f1a).
• Results are deduplicated. The same document never appears twice in one result list.
• Embedding fields are excluded. Every search route sets _source excludes for text_embedding and k-NN fields, so vectors never reach callers.

Tenant fence and operator token (2026-10-10)

Documented from semantic-api main at e9ac9893.

Index targets. index, every indexes[i] and every batch queries[i].index must be a string matching ^[a-z0-9*?_.+-]+(,[a-z0-9*?_.+-]+)*$; anything else (an array, an object, a path) is a 400 before any upstream call. A token containing /, \, #, % or .. is never allowed, and every OpenSearch path the service builds encodes each index name, so a body like "index": "x/../log-analytics-3103-otlp/_search#" can no longer reach another workspace’s index.

Workspace wildcards end at a segment. A pattern is allowed only when its workspace segment is the caller’s and ends there: log-analytics-2905-* and log-analytics-2905 are allowed for workspace 2905, log-analytics-29* is not (it would match 2905 and 2906 alike), and a pattern whose leading literal names another workspace is refused. A trailing _semantic is stripped before the workspace segment is read.

Default index. A search or batch with no index uses the caller’s first configured index after workspace scoping, or answers 400 index required. GET /indexes returns only the caller’s configured_indexes, and the unauthenticated /status no longer lists indexes.

Shared-model routes need the operator token. POST /api/v1/models/register, /models/:id/deploy, /models/:id/undeploy, /indexes/reindex, /indexes/neural/warmup and /indexes/neural/clear-cache act on the model every workspace shares, so a workspace Owner or Admin role is no longer enough. They need a valid credential and the header X-Semantic-Admin-Token equal to the service’s SEMANTIC_ADMIN_TOKEN (compared in constant time); without it the answer is 403. The service refuses to start in production (NODE_ENV=production, which dev also sets) when SEMANTIC_ADMIN_TOKEN is unset. /reindex and /neural/* are now registered before /:name*, so the warmup route reaches its own handler.

Errors and limits.

Upstream / case	Answer
no deployed model	503 (ModelError)
OpenSearch 401/403	502 UPSTREAM_AUTH
no response / ECONNABORTED	504
OpenSearch 429	503 with Retry-After: 5
index_not_found	404 INDEX_NOT_FOUND
any other upstream 4xx	502

• Batch routes (/search/batch, /embeddings/batch) spend one rate-limit token per item.
• An invalid API key is cached as invalid for 5 s (keyed by its sha256), and introspections that miss the cache are limited to 30 a minute per peer (SEMANTIC_RATE_LIMIT_INTROSPECT; 429 with Retry-After).
• Queries are trimmed and must be 1–1000 characters on /search, /search/all, /search/hybrid and batch items; from + k must not exceed 10000, and with dedupe on from must be below 1000.
• Time expressions in a query are matched case-insensitively, at a word boundary, and an invalid date (day > 31, month > 12, 31 Feb) is not treated as a time.
• A hybrid search with a time range puts the range in hybrid.filter; a kNN search puts it in neural.text_embedding.filter with k equal to the page size.