Log Analysis API
Overview
Section titled “Overview”Send one log file, get back findings. One call, one answer — the response carries the completed analysis, not a job id to poll.
There are two routes, and which one you use depends on whether the caller holds a credential:
| Route | Credential | Files per request | Shape |
|---|---|---|---|
POST /api/v1/log-analyses | API key (lgp_live_), session JWT, or legacy partner key | 1 | Synchronous by default — the reply IS the analysis |
POST /api/v1/log-analytics/{name} | None, with {name} registered by an operator — or an API key (lgp_live_, since 2026-10-07) | up to 1000 | Synchronous by default since 25 Sep 2026 — 202 with a batchId only past 16MB or 10 images |
Either route takes a file that fits in one request. A file that does not is uploaded in parts through the same credential — see A file bigger than one request.
Served by AI Service on port 9300.
Getting a name for the open route
Section titled “Getting a name for the open route”The {name} in /api/v1/log-analytics/{name} is not something a request chooses. A Logrite
operator registers a name against your workspace, per estate: a name registered on dev does
not exist on production. Measured 2026-10-01, the demo name acme answers 200 on
dev-aiservice.logrite.ai/api/v1/log-analytics/acme/limits and 404
{"status":"error","error":{"code":"NOT_FOUND","message":"Not found"}} on
aiservice.logrite.ai — exactly what any unregistered name answers.
To get one, ask Logrite support (or your Logrite operator) to register a name for your
workspace on the estate you will call. The examples on this page write it as
<your-registered-name>. If you hold an API key you do not need a name at
all: use POST /api/v1/log-analyses.
Authenticate
Section titled “Authenticate”Use a Logrite API key. Create one in the app under Sandbox → API keys — it starts
with lgp_live_, and you choose its lifetime when you create it (30 days, 90 days, 1 year or
no expiry; creating one needs the Admin or Owner role). Send it as a bearer token:
curl -s -X POST "https://aiservice.logrite.ai/api/v1/log-analyses" \ -H "Authorization: Bearer lgp_live_…" \ -H "x-filename: app.log" --data-binary @app.log- The key must be in
Authorization: Bearer. Sent asX-API-Keyit is not read, and the route answers401 missing_bearer. - A key expires at the end of the lifetime chosen, and a revoked or rotated-away key stops
working at once. Both answer
401("That API key was not accepted."). Create or rotate a key in Sandbox → API keys; the AI Service holds no key material and asks the backend about every key, so a revocation takes effect here immediately. - The key acts as the person who created it, in the workspace it was created in. Every analysis it submits is scoped to that workspace, and no request field can change it.
Two other credentials are accepted on the same route:
- Session token — the JWT the application itself uses. Short-lived, so not suited to a server-to-server integration.
- Partner key (legacy) — a static
secret:workspaceId:labelentry an operator configures in the AI Service’sLOG_ANALYSIS_API_KEYS. It is checked first, reaches only this route and its workspace, and is kept for integrations issued one beforelgp_live_keys existed. New integrations should use an API key.
Without a credential, the open route relies on {name} being a name an operator
registered against a workspace: an unregistered segment is a 404, so the URL simply does
not exist. A route that accepted a bare workspace id with no credential would let any caller
write into any tenant by typing its number.
API keys on the upload routes (2026-10-07). The POST /api/v1/log-analytics/{name} family
also accepts an lgp_live_ key as Authorization: Bearer …. Before, the offered credential went
through the JWT-only guard and a valid key answered 401 INVALID_CREDENTIAL (“jwt malformed”) —
measured on dev by the cortex_consistency.py setup. The key is now checked with the backend like
on every other key route (503 if the backend cannot be asked), and the workspace comes from the
key.
Analyse a file
Section titled “Analyse a file”curl -X POST "https://aiservice.logrite.ai/api/v1/log-analyses" \ -H "Authorization: Bearer <API_KEY>" \ -F "file=@./application.log"A raw body works too, with the name in a header — useful for pipes:
curl -X POST "https://aiservice.logrite.ai/api/v1/log-analyses" \ -H "Authorization: Bearer <API_KEY>" \ -H "X-Filename: application.log" \ --data-binary @application.logAny content type is accepted, including none. Plain text, newline-delimited JSON, a JSON array of records, and CSV are all read; the body is read as bytes whatever it declares itself to be.
A gzipped file is decompressed first, detected by its magic bytes rather than its
name, and the size limits apply to the DECOMPRESSED bytes — so a file cannot be gzipped
past the ceiling. Zip, 7z, xz and zstd are refused by name with 415 UNSUPPORTED_FORMAT
rather than read as though they were text.
Several files in one request
Section titled “Several files in one request”curl -X POST "https://aiservice.logrite.ai/api/v1/log-analytics/<your-registered-name>" \ -F "file=@./app-1.log" -F "file=@./app-2.log" -F "dataset=nightly-batch"One request is one batch. The reply is a 202 carrying the batchId before any analysis
starts, and the work continues after the connection closes:
{ "batchId": "BATCH-06e15fd9", "status": "RUNNING", "statusUrl": "/api/v1/log-analytics/<your-registered-name>/BATCH-06e15fd9", "pollAfterMs": 2000, "files": [ { "fileId": "FILE-aef3e7", "filename": "app-1.log", "sizeBytes": 258, "datasetId": "log-analytics-3952-nightly-batch", "status": "QUEUED", "fileUrl": "/api/v1/log-analytics/<your-registered-name>/BATCH-06e15fd9/FILE-aef3e7" } ]}Poll statusUrl for the whole set or a fileUrl for one file. Files complete
incrementally, so a finished file is readable while the rest are still running.
Every entry carries fileId, fileUrl, filename, sizeBytes and datasetId in
every state — nothing a caller is already reading changes key when a file succeeds. A
finished entry adds analysisId, findings and summary.
What a 202 is answered before
Section titled “What a 202 is answered before”The acknowledgement is written once the upload has been read — parsed, decompressed, counted against the limits — and before any document is converted to text. That matters because converting is the slow part: a screenshot or a PDF is transcribed by a vision model, seconds per file.
So an async caller is refused on arrival for anything knowable on arrival — an empty
body, a truncated multipart, an unsupported encoding, too many files, too many images,
a file over the per-file ceiling, a bad dataset key. Those are still a 4xx on the
POST.
A file that cannot be READ is reported differently in the two modes, and it has to be: asynchronously the response has already gone.
| the upload | Prefer: respond-sync | Prefer: respond-async |
|---|---|---|
| one file, unreadable | 415 / 422 on the POST | 202, then that file’s entry is FAILED with the same code and sentence |
| several files, one unreadable | 200; that entry is FAILED, the rest are analysed | identical, behind the 202 |
The code and the message are the same either way — only where you read them moves. A
fault on our side is never folded into a file’s entry in either mode: it fails the
batch, and statusUrl says so.
Naming a dataset
Section titled “Naming a dataset”dataset=<key> pins every file in the request — and every later request using the same
value — to log-analytics-<workspaceId>-<key>. On a raw body, send X-Dataset instead.
The key is normalised, never rewritten past recognition: lower-cased, with runs of
anything outside a-z0-9 becoming a single dash. Nightly Batch and nightly-batch are
the same dataset. Two rules keep the result predictable:
- A key longer than
maxDatasetKeyLength(64 after normalisation) is400 INVALID_DATASET. It is not shortened. Truncating it would put two keys that differ only past the limit into one dataset. - A
datasetfield and anX-Datasetheader naming different destinations is400 DUPLICATE_FIELD, exactly like the field sent twice with two values. Send one, or send both with the same value.
files[].datasetId on the acknowledgement is always where the events will go, so the
destination never has to be inferred.
A file bigger than one request
Section titled “A file bigger than one request”Every request to this service passes a 100 MiB body ceiling it does not own, and a single-request upload cannot be larger than that however generous the service’s own limits are. Since 25 September 2026 a file that does not fit is uploaded in parts. Each part is its own request, so the ceiling applies to a part and never to the file.
The protocol is identical on both surfaces — only the prefix differs:
| Surface | Prefix | Credential |
|---|---|---|
| Log analysis | /api/v1/log-analytics/{workspace}/uploads | none; {workspace} is operator-registered |
| Cortex attachments | /api/v1/attachments/uploads | session JWT |
| Verb | Path (under the prefix) | What it does |
|---|---|---|
POST | /sessions | Open a session. 201 carries uploadId, partSize, totalParts, expiresAt and the three urls to use next |
PUT | /sessions/{id}/parts/{n} | Upload part n. 1-based. Answers the part plus which parts are still missing |
GET | /sessions/{id} | Which parts have landed. Cheap, safe to poll — this is the resume call |
POST | /sessions/{id}/complete | Assemble and hand off. All-or-nothing |
DELETE | /sessions/{id} | Give up and free the disk. Idempotent, so a client may call it blind |
Open a session with filename and totalBytes; partSize, sha256, contentType and
(log analysis only) dataset are optional.
Retrying a part cannot corrupt the file
Section titled “Retrying a part cannot corrupt the file”A part is addressed by index and overwrites, so the same index may be sent any number of times and the worst a duplicate costs is bandwidth. The dangerous design is an appending one, where a retry the client was unsure about duplicates bytes and shifts everything after it — a corruption that looks like a successful upload.
Two digests, doing different jobs:
X-Part-Sha256on aPUTis verified on arrival. A mismatch is422 PART_DIGEST_MISMATCHand the part is removed, not merely reported — a recorded part is one assembly will trust. Re-send that part alone.sha256at session open covers the whole file, and nothing is accepted unless the assembled bytes match it.
A wrong-sized part is refused at the PUT (400 PART_SIZE_MISMATCH), not at assembly.
Parts are fixed-size except the last, which is the remainder; told now a client bug costs
one part, told at complete it costs the whole upload.
Log analysis over this door is always asynchronous
Section titled “Log analysis over this door is always asynchronous”The single-request endpoint lets a caller ask to wait (Prefer: respond-sync). This one
cannot honour that and does not pretend to: a file large enough to need chunking takes
longer to analyse than any proxy in front of this service will hold a connection open, so
a synchronous answer here would be a promise kept only for the files that never needed
the door. complete returns the same 202, batchId and poll headers the existing async
path returns, so a caller already handling that handles this unchanged.
Everything after assembly is the existing pipeline called with the existing
arguments — a chunked .gz decompresses exactly as a posted one does, and Word, PDF and
screenshots are extracted by the same code. There is no second implementation of any of
it.
Two things this door gets right by refusing early
Section titled “Two things this door gets right by refusing early”- A dataset name is validated at session open, not at
complete. Being told after sending a gigabyte that the destination name was invalid is the worst possible moment to learn it. - A refusal means the same thing on both doors. The attachment surface translates its
three service errors into the protocol’s type, so an over-quota file is
429here exactly as it is on the single-request POST. Untranslated they arrive as naked500s, and two doors that disagree about why a file was refused is a worse bug than either answer alone.
Analyse an index you already have
Section titled “Analyse an index you already have”Most questions in this product are asked about logs that are already stored — they
arrived through a collector, an earlier upload, or the ingest shim. Requiring those to be
re-uploaded before the analyser could read them was a limit of the transport, not of the
analysis. POST /api/v1/log-analyses/dataset removes it.
curl -X POST "https://aiservice.logrite.ai/api/v1/log-analyses/dataset" \ -H "Authorization: Bearer <API_KEY>" \ -H "Content-Type: application/json" \ -d '{"dataset":"log-analytics-2905","tailHours":24,"limit":2000}'The response is the same report as a file analysis, and every shaping parameter
(?shape=, ?summary=, ?depth=, ?q=) works here too.
| Field | Default | Meaning |
|---|---|---|
dataset (or index) | — | The index to analyse. Required. One concrete name. |
tailHours | 0 (no window) | Analyse only the last N hours. Max 336. |
limit | 2000 | How many records to read. Max 10000. |
Nothing is written back. The events came out of an index that already holds them;
re-ingesting would duplicate your data and bill you storage for a read. dataset.eventsStored
is 0 on this route by design, and the status is still COMPLETED — a read that stores
nothing is not a shortfall.
The analysis is stored, though, which is a different thing from the events. An id in
a response is a promise it can be read back, so the analysisId resolves on
GET /api/v1/log-analyses/{id} — including for an index that held nothing, because that
id was handed out too. It reads back only for the workspace that ran it.
The workspace comes from your credential, never from the request. The index name is
chosen by the caller, so the name alone can never be trusted. An index that does not
belong to the calling workspace is 403 DATASET_FORBIDDEN — it is never answered with an
empty report, because “no findings” and “not yours” must never look alike. A wildcard or a
comma-separated list is 400 DATASET_INVALID: a pattern cannot be ownership-checked
before it expands.
An index that returns no records is a 200 carrying the reason in warnings, not an
error — an empty window is an answer. When the read hits its limit, a warning says so,
so a capped read is never mistaken for full coverage.
The response
Section titled “The response”{ "analysisId": "ANL-8f31ab27", "status": "COMPLETED", "file": { "name": "application.log", "sizeBytes": 1048576, "format": "text", "eventsParsed": 15234 }, "dataset": { "id": "log-analytics-2905-application-log-8f31ab", "eventsStored": 15234, "eventsRejected": 0 }, "summary": { "eventsAnalyzed": 15234, "threatsDetected": 3, "securityFindings": 7 }, "findings": [ { "id": "F-1001", "type": "THREAT", "severity": "HIGH", "category": "BRUTE_FORCE", "title": "Repeated failed authentication attempts", "evidence": { "eventCount": 412, "firstSeen": "2026-08-27T02:14:09.000Z", "samples": ["2026-08-27 02:14:09,001 [http-7] WARN a.AuthFilter - Failed login for admin from 203.0.113.9"] }, "detectedBy": "rules" } ], "warnings": []}| Field | Meaning |
|---|---|
status | COMPLETED, or PARTIAL when the file was larger than one analysis covers |
dataset.id | Handle for the retained events — the same handle the application UI lists them under |
findings[].type | THREAT, SECURITY, COMPLIANCE, OBSERVABILITY |
findings[].severity | CRITICAL, HIGH, MEDIUM, LOW, INFO — worst first |
findings[].evidence.samples | Verbatim log lines, secret-redacted, length-capped, and de-duplicated |
findings[].detectedBy | rules for a deterministic detector, model for the AI correlation pass |
warnings | Non-fatal degradations: truncation, retention unavailable, AI pass skipped |
Every count and timestamp in a finding is measured from the file. The AI pass may only cite evidence the detectors produced, so a finding never carries a number that was not counted — no finding can claim more events than the file holds.
One call, one report
Section titled “One call, one report”The open route answers a synchronous POST with the whole report — no batch id, nothing to poll:
curl -X POST "https://dev-aiservice.logrite.ai/api/v1/log-analytics/<your-registered-name>?wait=true" \ -F "file=@./Portal_181.log" \ -F "dataset=prod-app-logs"There is a dedicated path for it too, which is synchronous by definition:
curl -X POST "https://dev-aiservice.logrite.ai/api/v1/log-analytics/<your-registered-name>/analyze" \ -F "file=@./Portal_181.log"The response is ordered for reading, and JSON.stringify preserves that order:
status → summary → overview → critical → high → medium → low → infoAn empty severity section is omitted, not shipped as [].
{ "status": "COMPLETED", "summary": { "headline": "103,720 events over 2026-08-18 14:13 → 2026-08-21 19:06 UTC (3d 4h), 1,767 error-level (1.7%). Worst of 10 findings: CRITICAL — Injection or traversal payload observed in requests.", "highlights": [ "103,720 events analysed, 2026-08-18 14:13 → 2026-08-21 19:06 UTC (3d 4h).", "Levels: INFO 50,097 · DEBUG 49,325 · ERROR 1,767.", "1,767 error-level events (1.7%).", "10 findings — CRITICAL 2, HIGH 3, MEDIUM 5.", "CRITICAL: Injection or traversal payload observed in requests (19 events).", "Repeats 3,233x (non-error): e.g. [DBG] License heartbeat : 5f3c9a2b-… - Failure" ], "files": [{ "name": "Portal_181.log", "sizeBytes": 10485792, "eventsParsed": 103720, "dataset": "log-analytics-3103-prod-app-logs" }], "timeRange": { "from": "2026-08-18T14:13:22.631Z", "to": "2026-08-21T19:06:58.360Z" }, "errors": 1767, "errorRate": 0.017, "findings": { "total": 10, "bySeverity": { "CRITICAL": 2, "HIGH": 3, "MEDIUM": 5, "LOW": 0, "INFO": 0 }, "byType": { "THREAT": 4, "SECURITY": 2, "COMPLIANCE": 1, "OBSERVABILITY": 3 } }, "topIssues": [ /* the worst three, so a caller has an answer without reading all of them */ ], "topPatterns": [ /* recurring shapes, with a REAL example line and its own first/last seen */ ] }, "overview": "…", "critical": [ { "id": "F-1001", "severity": "CRITICAL", "events": 19, "evidence": ["…"], "recommendation": "…" } ], "high": [ { "id": "F-1004", "evidence": [], "evidenceSameAs": "F-1001" } ]}sizeBytes is the upload, and byType counts the whole batch
Section titled “sizeBytes is the upload, and byType counts the whole batch”Two numbers in the batch summary described the wrong thing until 24 September 2026, and both were plausible enough to be believed:
sizeBytesreported the length of the transcription, not of the file. A batch of ten screenshots of 19–38 KB each answered301,217,244— which reads as a truncated upload, not as a different quantity. It is the uploaded bytes now, for every file whose content was extracted (image, PDF, Word, Excel, PowerPoint).findings.byTypewas the first file’s tally.totalandbySeveritywere recomputed across the batch andbyTypewas not, so a ten-image batch reportedtotal: 14besidebyType: {OBSERVABILITY: 1}.byTypenow sums tototal, always.
What it deliberately does not repeat
Section titled “What it deliberately does not repeat”| Removed | Because |
|---|---|
batchId / fileId / fileUrl / statusUrl | there is nothing to come back for |
limits (2,012 bytes) | identical on every response, and readable at /limits. Still attached to every rejection |
metrics per finding | totalEvents is summary.eventsAnalyzed once per finding; matchedEvents is events renamed; ratio is those two divided |
threatsDetected and its three siblings | findings.byType states the same four numbers |
| repeated evidence | measured on a real 10MB upload, 7 of 10 findings carried samples byte-identical to an earlier finding’s — the rules engine and the model quoting the same lines, ~40% of the findings payload. The later finding keeps its count and bounds and carries evidenceSameAs |
Measured on the deployed build: 15.1 KB → 6.0 KB for the same file.
Keeping the batch envelope
Section titled “Keeping the batch envelope”?shape=batch returns the previous shape unchanged, so an integration written
against files[0].findings has one flag rather than a rewrite:
curl -X POST ".../log-analytics/<your-registered-name>?wait=true&shape=batch" -F "file=@./app.log"The asynchronous default is untouched: a POST with no wait/Prefer from a
non-interactive client still answers 202 with a batch id, and the two GET reads
still return the batch and the per-file analysis exactly as before.
The urls are absolute
Section titled “The urls are absolute”statusUrl and fileUrl used to come back as bare paths
(/api/v1/log-analytics/<your-registered-name>/BATCH-…), so a caller who copied one out of the
response could not use it without already knowing a host the response never told
them. They are now absolute, resolved from X-Forwarded-Proto / X-Forwarded-Host
— this service sits behind Cloudflare and Coolify, where req.protocol is http
and the socket host is a container address. With no host header at all the relative
path is returned rather than an invented origin: a wrong absolute url is worse,
because it looks usable.
Asking for less than the whole report
Section titled “Asking for less than the whole report”A caller that wants a narrower body says so. Each parameter can be sent as a query
string, a multipart field, or a header (X-Summary, X-Depth), in that precedence
— a browser form has nowhere to put a query string, and a curl --data-binary pipe
has nowhere to put a field.
| Parameter | Values | Effect | Doors |
|---|---|---|---|
shape | report, findings, batch | findings returns the findings alone; batch is the pre-September envelope | both |
summary | none, brief (default), full, digest | Chooses the WORDING of overview. brief is the counted bullets — health, level breakdown, main failure, busiest hour, one action. full is the plain-English explanation: what each finding is, how big it is against the window, the wall-clock window of a burst, one real log line with its file line number, and what to do — held to 1,000 words, and it returns the overview and nothing else. digest is the per-category blocks. none drops the key | both |
depth | titles, standard (default), full | titles drops evidence and recommendations; full keeps everything | both |
q | free text | The view is read from the question | keyed only |
narrow | true (default), false | false reads the question but does not act on it — see below | keyed only |
full and brief swapped in September 2026
Section titled “full and brief swapped in September 2026”full used to return execSummary’s counted bullets. Measured on one real file,
?summary=full came back 762 bytes and ?summary=brief came back 10,089 — because
full means “the overview and nothing else”, so it returned the least of the three
levels, and what it returned was a bullet list. Reported from Postman:
Why it not gives the exact plain english explaintion to the API which we except like gemeni, deepseek, etc. other AI chat models? It should give that in response, correct for summary=full and this response should be there on the summary=bried
So the two swapped. The bullets are the short form and are at brief, which is still the
default, so no existing integration changes shape — only the wording of overview at
the explicitly-requested full. digest is untouched.
A browser is floored at brief whatever the question resolves to, because the application
renders its own explanation in the closing Summary with the log lines folded into a
collapsible card; serving the API explanation as the lead as well would print the same
reading twice.
The application’s chat composer sends depth=full, because that surface is read by
a person rather than parsed by a program.
An unrecognised value is 400 BAD_PARAM naming what was accepted, never a silent
fallback to the default — a caller that misspells a parameter must not be told its
request worked. tailHours=1e9 is rejected rather than clamped for the same reason:
analysing the whole file because the number was silly is a different answer from the
one asked for.
q is the one a chat surface uses: “just the findings” resolves to the findings
view, “give me a summary” to the overview, “in-depth, critical only” to the whole
report narrowed to critical. A question it cannot classify returns the default view
rather than guessing.
A severity filter says what it removed
Section titled “A severity filter says what it removed”Narrowing to one severity deletes the other sections, and an empty result then reads
exactly like a healthy system — to the one caller who asked about critical issues.
So the narrowing reports itself in warnings[]:
Showing critical findings only. This analysis also found 4 high, 3 medium — askagain without a severity in the question to see them.“No critical findings” and “nothing wrong” are different answers, and the response has to be able to tell them apart.
The severity words a question can use
Section titled “The severity words a question can use”| Section | Words that resolve to it |
|---|---|
critical | critical, crit, blocker |
high | high, severe, major |
medium | medium, moderate |
low | low, minor |
info | info, informational, note |
info used to resolve to low, from when INFO was not a section of its own. It is
one now — and it is where a healthy file keeps its only finding — so “show me the
info findings” was narrowing to a different set of findings and presenting them as
the ones asked for.
A filter that matches nothing does not apply
Section titled “A filter that matches nothing does not apply”any critical insights? asked of a file whose only finding is an INFO note used
to return a report with no findings in it at all: the question was answered by
deleting the answer. A severity in a question is a request to lead with that
severity, not an instruction to discard the analysis when none of it matches.
So the filter narrows where narrowing leaves something, and steps aside where it
would leave nothing. The body then carries every section, view.severities is []
because no narrowing happened, and warnings[] says which question was answered:
No critical findings were raised. Showing everything this analysis did findinstead — 1 info.A file that genuinely has critical findings is unaffected: the filter applies, the other sections are dropped, and the first note above is the one you get.
Reading the question without acting on it — narrow=false
Section titled “Reading the question without acting on it — narrow=false”Stepping aside when a filter matches nothing fixes the empty report, but it leaves
the filter in charge whenever it matches something: ask any critical insights?
of a file with one critical and forty other findings and forty are withheld. For a
surface that is read rather than parsed, that is the wrong default — the reader
asked what is critical, not to be shown less.
narrow=false keeps the whole body and reports the question separately, so the
severity decides what to put first, not what to withhold:
"view": { "sections": ["overview", "summary", "findings"], "severities": [], "asked": ["critical"], "source": "keywords"}asked is what the question NAMED. severities is what was actually removed on
the strength of it. They are different facts and were the same field: with the
filter suppressed, severities is always [], and a surface that read it to say
“showing critical” would have had nothing to read.
The application’s chat composer sends narrow=false on every call. A program that
wants the filter simply omits the parameter — the default is unchanged, so nothing
that integrates today behaves differently.
Synchronous by default, a batch past the ceilings
Section titled “Synchronous by default, a batch past the ceilings”Both routes answer on the connection — 200 with the report — unless the upload passes
one of two ceilings, in which case they answer 202 with a batchId and a statusUrl to poll.
How this default came about is in the history.
The two ceilings
Section titled “The two ceilings”| Upload | Answer |
|---|---|
≤ syncMaxBytes (16MB) and ≤ syncMaxImages (10) | 200 with the report |
| past either | 202 with a batchId and a statusUrl |
Both are published on GET …/limits and both are env-tunable
(LOG_ANALYSIS_SYNC_MAX_BYTES, LOG_ANALYSIS_SYNC_MAX_IMAGES). Neither is a refusal —
the analysis runs either way; they only decide where the caller reads the result.
Keeping the old contract
Section titled “Keeping the old contract”One header. Prefer: respond-async (or ?wait=false) returns the batchId however small
the upload — the V1 contract byte for byte. Prefer: respond-sync / ?wait=true waits
however large it is, and ?shape=batch keeps the envelope shape. An explicit choice wins
in both directions, and Prefer outranks ?wait.
The per-turn index ceiling (C2)
Section titled “The per-turn index ceiling (C2)”GET /api/v1/log-analyses/limits also publishes limits.maxIndexesPerTurn (default 8,
LOG_ANALYSIS_MAX_INDEXES_PER_TURN) and limits.turnsPerMin (default 10,
LOG_ANALYSIS_TURNS_PER_MIN). POST /dataset takes one index per request — the fan-out
over several indexes is the client’s job — and the route’s own rate budget is the product
of those two.
🔴 AND THE PRODUCT WAS NOT THE BUDGET. The route asked for
createRateLimit(turnsPerMin × maxIndexesPerTurn), and createRateLimit clamps to
config.rateLimitPerMin — default 60. So /limits published 10 × 8 = 80 and the limiter
enforced 60: seven and a half turns. A client doing exactly what /limits told it to do was
refused part-way through its eighth question, and the answer came back with the report half
missing — which is C2’s own symptom, produced by C2’s own fix. Publishing the ceiling removed the
mirrored constant and left an unapplied one, and the second is harder to see: the caller is no
longer guessing, it has been told a number and refused on it.
The route now uses createBurstRateLimit, whose whole reason for existing is a bucket where one
logical operation is legitimately many requests — a reader turn fanned out over N indexes, the same
shape as a chunked upload’s 64 PUTs. It is unclamped by design, on its own bucket, and every call
stays bounded by its own sync and byte budgets. datasetTurnBudget() derives all three numbers in
one place and limits.datasetRequestsPerMin publishes the budget that is actually enforced, so a
client can check the arithmetic instead of trusting it.
⚠️ The unit test could not see this. It imported the constants and asserted the product, which is
true whichever limiter the route is wired to. tests/api/datasetTurnBudgetEnforced.test.ts spends
the whole published budget through the real mounted app; against the clamped helper it fails at
“turn 8, index 5”.
⚠️ That product is why the number is published rather than copied. The browser composer
held its own MAX_FALLBACK_TARGETS, bound to the server’s copy by nothing but a comment in
another repository. Raise the client cap alone — the obvious fix for “only 4 of my 7 indexes
were analysed” — and the budget is unchanged, so the cost per turn doubles against a fixed
ceiling and 10 turns a minute quietly becomes 5. The symptom gets strictly worse while
looking like a fix.
⚠️ The fields are nested under limits, not at the top level. The first client to read
them read the top level, got undefined on every successful response, and fell back to its
hard-coded default — a fetch that succeeds and yields nothing usable is indistinguishable at
the call site from one that failed. A client that cannot read them must degrade to its own
default, never to an unbounded fan-out.
Images are counted separately from bytes, and that is the load-bearing part. Ten
screenshots are a few MB and ten vision calls: trivially under any byte ceiling and the
slowest thing this endpoint does. A byte figure structurally cannot see that request.
countImages() is shared with the TOO_MANY_IMAGES refusal so the two cannot disagree
about what an image is the first time the sniffer learns a format.
Likely trigger: the warnings that came first (2026-10-07)
Section titled “Likely trigger: the warnings that came first (2026-10-07)”A failing log is often explained by what it said before it failed. Every analysis now
looks for one such precursor (findPrecursor in rootCause.ts):
- The failure is the largest error-level shape — the error-or-worse cluster with the most events.
- A candidate is a warning/activity shape (lines grouped by template, so thread or host variants count together) that is at least half WARN, and whose first line is at or before the first line of that failure and no more than 10 minutes earlier.
- It must be big enough to matter: a count ≥ 2× the file’s error count, or
≥ 25 % of the file’s WARN lines — and, since ai-service
8654ef4, at least 5 lines. - It must still be running when the failure starts: its last line no more than 10 s
before the failure’s first. Before, three
login failedwarnings that had ended 53 s earlier (and had already caused their own lockout) were named the trigger of twoMySQL server has gone awayerrors in another subsystem — 3 of 3 WARN lines clears the 25 % test on a 10-line file. - The earliest candidate wins; on a tie in first-seen time, the larger count.
Other rule fixes from the same upload (ai-service 8654ef4)
Section titled “Other rule fixes from the same upload (ai-service 8654ef4)”ELEVATED_ERROR_RATEnames the top ERROR-level shape. The ranked failure list also holds lines that only read as failures (login failedat WARN), and the finding titled “30% of events failed, most often:<a WARN line>” with WARN samples as its evidence.- SSNs are their own finding: detector
government_id, categoryGOVERNMENT_ID_EXPOSURE,COMPLIANCE, HIGH, from a single line (\d{3}-\d{2}-\d{4}orssn=/social security number:followed by a digit;ssn=[REDACTED]and 3-3-4 phone numbers are not hits). It used to be one arm ofsensitive_data, whose floor of 3 counts emails too, so one SSN beside one email raised nothing and the model filed the SSN under “personal data in authentication logs” at MEDIUM.sensitive_datakeeps cards, DOB and emails. MySQL server has gone away/lost connection to … servermatchdependency_failure.- A lockout is a finding on its own (detector
account_lockout,THREAT/BRUTE_FORCE/ HIGH, one line:account locked,locked out,too many failed attempts; unlocks andlockout policy …settings excluded). Three failed logins and a lockout stayed underauth_failure’s floor of 5, so the lockout appeared only when the model pass answered — on a re-test where it returned no JSON, not at all. Same key asauth_failure, so they merge. - Model categories: a finding saying two incidents are separate or unrelated maps to
CORRELATED_ACTIVITY(it wasBRUTE_FORCE, on the word “lockout”); SSN wording maps toGOVERNMENT_ID_EXPOSURE, so it dedupes with the rule finding. Since ai-service8424767the separate-incident wording is read before the model’s own category is trusted (the model itself wroteBRUTE_FORCEon “multiple independent failure modes”). - Dedupe at equal severity keeps the rule finding (
8424767). The merged list is sorted by severity, type, then count, and the first finding pertype:categorywins — so a model restatement that counted more lines (an email and an SSN, count 2) displaced the one-line SSN rule finding. A model finding now holds a key over a rule only when strictly more severe.
A precursor produces three things:
| where | what |
|---|---|
| headline | opens the summary: Likely trigger: … (N warnings from HH:MM:SS UTC), followed by … (M errors from HH:MM:SS UTC). |
| finding | category: "LIKELY_TRIGGER", type: "OBSERVABILITY", detectedBy: "rules"; severity HIGH when the failure has ≥ 100 events, else MEDIUM. evidence carries eventCount, firstSeen, lastSeen and up to 3 samples; metrics adds warningEvents, followingErrorEvents and leadSeconds |
| evidence pack | the model pass sees a trigger:0 row (likely trigger (starts before the main failure): …) plus the top 3 warning/activity shapes, even when the file is failing |
Before this, warning shapes reached the model only when nothing failed, so the model saw
the errors and blamed their own subject. On the Loghub Hadoop_2k.log sample (level
census INFO 1040, WARN 808, ERROR 150, FATAL 2) the trigger is 476 WARN
Client: Address change detected … msra-sa-41 lines from 18:05:27, followed by 147
RMContainerAllocator: ERROR IN CONTACTING RM. from 18:06:01 — the address change is the
cause, the RM errors its consequence.
Error counts use the log’s own level words
Section titled “Error counts use the log’s own level words”The headline’s error clause counts error-or-worse lines (ERROR, ERR, FATAL,
CRITICAL, SEVERE) and breaks them down in the words the file used, largest first:
152 errors (150 ERROR, 2 FATAL; 7.6%). A level the file never wrote never appears — an
earlier build printed 152 SEVERE for that Hadoop log, which contains no SEVERE line.
Same file, same analysis (2026-10-07)
Section titled “Same file, same analysis (2026-10-07)”Uploading the same bytes twice used to create another dataset copy and run the model
pass again — which is not repeatable, so the same 22-line file came back with 4 findings,
then 3. On the multipart upload routes (POST /api/v1/log-analytics/{name} and its
/analyze form) a file is now answered from the stored analysis when all of these hold:
- same workspace and same
contentSha256(SHA-256 of the file bytes); - the stored analysis was made under the current
ANALYSIS_VERSION(2026-10-07.1; a release that changes what an analysis says bumps it); - it is still within retention, finished
COMPLETED, and is notpartial; - the request asks for the plain analysis of the whole file — no time window, no tail, no
dataset=key (those requests are analysed fresh, and their results are never reused).
That file’s entry then has status: "reused", the existing analysisId and
datasetId, the same findings, and reusedFrom: { analysisId, createdAt }. No index is
written and no model call is made. The entry keeps the filename of this upload. The
single-file POST /api/v1/log-analyses route records its uploads (so they are listed and
deletable, below) but always analyses.
A file attached in Cortex chat follows the same rule (ai-service c8b13f8, prod re-test
P-3). Before writing events, the attachment hashes its bytes and reuses a live dataset this
workspace already holds for them, with no new index; two identical attachments in flight share
one write. When it does write a dataset it leaves a record listed by
GET /api/v1/log-analyses as {id: <attachmentId>, datasetId, fileName, bytes, findings: null, status: "RETAINED"} — deletable with DELETE /api/v1/log-analyses/<attachmentId> (index,
register entries and record), and reused by a later identical upload, which is analysed into
that dataset without a copy. Before this, every attachment wrote a new index that no list showed.
To analyse it again anyway, send the multipart field force=true (or header x-force: true):
curl -X POST "https://aiservice.logrite.ai/api/v1/log-analytics/<your-registered-name>" \ -F "file=@./app.log" -F "force=true"Two more changes make a fresh analysis steadier: the model pass answers through a strict
json_schema response format, and rule and model findings are sorted before they are
de-duplicated, so the same inputs keep the same survivor. The model pass’s findings are also
kept in the Redis cortex:logfindings cache (24 h), so the same evidence replays the same model
findings (see AI models).
partial: true — rule findings only
Section titled “partial: true — rule findings only”When the model pass is skipped — its time budget was spent on the file, the call failed,
or no model is configured — the analysis carries partial: true with the deterministic
findings alone and the reason in warnings. A partial analysis is never reused: the
next identical upload is analysed again. The Sandbox labels it
“AI review skipped (timed out); rule findings only”.
Limits and errors
Section titled “Limits and errors”Every limit is published, enforced, and named in the rejection. Read the live numbers rather than trusting this page:
curl "https://aiservice.logrite.ai/api/v1/log-analyses/limits" -H "Authorization: Bearer <API_KEY>"curl "https://aiservice.logrite.ai/api/v1/log-analytics/<your-registered-name>/limits" # no credentialBoth return maxFilesPerRequest, maxBytesPerFile, maxBytesPerRequest,
maxEventsPerFile, maxRequestsPerMinute, maxConcurrentAnalyses,
maxConcurrentBytes, resultRetentionHours, maxFieldsPerEvent, acceptedFormats,
extractedFormats, maxImagesPerRequest, acceptedEncodings and the dataset-naming
settings. acceptedFormats and extractedFormats are separate on purpose — see
Word, Excel, PowerPoint, PDF and screenshots. The rate limit reported is the
EFFECTIVE one, after the service-wide cap. The same object is attached to every success
and every 4xx of the route the request was made on — the two routes do not have the
same ceilings.
| Status | Code | When |
|---|---|---|
400 | EMPTY_BODY | No body at all |
400 | NO_FILE / EMPTY_FILE | No file part, or one with no bytes |
400 | INCOMPLETE_UPLOAD | The multipart body ended before its closing boundary |
400 | INVALID_FILENAME | X-Filename is not valid percent-encoding |
400 | INVALID_DATASET | A dataset key that cannot become a name, or one over the length limit |
413 | TOO_MANY_IMAGES | More images than maxImagesPerRequest; each is a vision call |
415 | UNSUPPORTED_FORMAT | A format that cannot be read — the message names what it actually is |
422 | NO_TEXT_EXTRACTED | The format was understood but held no text (empty document, scanned PDF) |
400 | DUPLICATE_FIELD | A field sent twice with different values — a form field and its X- header count as the same field |
400 | CORRUPT_ARCHIVE | Gzipped, but it would not decompress |
400 | INVALID_IDEMPOTENCY_KEY | Outside [A-Za-z0-9._:-]{1,128} |
401 | — | Missing, unknown or expired credential (keyed route only) |
403 | NO_WORKSPACE / IP_NOT_ALLOWED | Credential not bound to a workspace; caller address not registered |
404 | NOT_FOUND | Unknown or expired id, an unregistered workspace segment, or another tenant’s id |
409 | IDEMPOTENT_REQUEST_IN_FLIGHT | Your own retry overtook your first request |
422 | IDEMPOTENCY_KEY_REUSED | That key was already used for a different upload. Nothing was analysed |
413 | FILE_TOO_LARGE / BATCH_TOO_LARGE / TOO_MANY_FILES | Over a published ceiling |
415 | UNSUPPORTED_FORMAT | An archive format this endpoint does not read |
429 | RATE_LIMITED | Over maxRequestsPerMinute; carries Retry-After |
503 | ANALYSIS_CAPACITY | At the concurrency ceiling. Nothing was accepted — send the same upload again. Carries Retry-After |
Nothing a caller can put in a request produces a 5xx. A 500 means we broke.
A file above maxEventsPerFile is analysed up to that point and the response says so in
status (PARTIAL) and warnings — it never reports on a prefix silently.
A fault in one file does not fail the batch
Section titled “A fault in one file does not fail the batch”A breach is charged to the thing that caused it. A breach of the request refuses the whole call; a fault in one file fails that file and the rest are still analysed.
| Breach | Scope | Result |
|---|---|---|
NO_FILE, TOO_MANY_FILES, BATCH_TOO_LARGE | the request | Nothing is accepted — no batchId, nothing analysed |
EMPTY_FILE, FILE_TOO_LARGE, UNSUPPORTED_FORMAT, UPLOAD_NOT_BINARY | one file | That file is dropped and named; the others are analysed. The call still answers 200 |
With one file the two are the same thing. A single-file upload that breaches a
per-file limit is still refused outright with its usual 400 / 413 / 415, because
there is nothing left to answer with — every existing single-file contract is unchanged.
Where the dropped file shows up depends on the shape, and this is the part to write your client against — a per-file fault never changes the HTTP status, so a caller that only checks the status code will not notice one:
| Shape | status | Where the failed file appears |
|---|---|---|
?shape=batch | PARTIAL | Its own entry in files[], status: "FAILED", with error.code and error.message |
| default (the report) | COMPLETED | warnings only — "zero.png was not analysed. \"zero.png\" is empty…". The report’s summary.files[] lists the files that were analysed |
On the batch shape you always get one entry per file you sent. A batch that returned
three results for four uploads would be the answer nobody downstream retries; four
entries, one of them FAILED with a reason, is one you can act on. On the report shape
there is no per-file status to carry, so the sentence in warnings is the whole record
of it — read warnings, not just status.
Pacing a poller
Section titled “Pacing a poller”429 and 503 both carry Retry-After in seconds alongside retryAfterSeconds in the
body, and the two always agree. A 202 — from the POST and from every read after it —
carries Retry-After, X-Poll-After-Ms, and pollAfterMs in the body. Use them; the
interval is estimated from the size of what you actually sent.
Every response also carries X-Request-Id. Send your own and it is echoed, so one id
spans both sides of a failure report.
Retrying safely
Section titled “Retrying safely”Send Idempotency-Key on any POST. A second request carrying the same key within the
retention window returns the original response — the same id, byte for byte —
instead of starting a second analysis. Keys are scoped per workspace, because two
partners will both pick 1.
curl -X POST "https://aiservice.logrite.ai/api/v1/log-analytics/<your-registered-name>" \ -H "Idempotency-Key: nightly-2026-08-28" -F "file=@./app.log"A key belongs to one upload. Sending the same key with a different file — or a
different filename, or a different dataset — is 422 IDEMPOTENCY_KEY_REUSED, and
nothing is analysed. Use a new key for a new upload; reuse one only to retry the
identical request.
The check is on the files themselves, not on the raw request bytes. A multipart body carries a randomly generated boundary, so hashing the bytes would refuse every legitimate retry — re-sending the same file with a fresh boundary is still recognised as the retry it is.
Behind more than one replica
Section titled “Behind more than one replica”Reservations and async results are shared through OpenSearch, so a retry or a poll that
lands on a different replica than the one that took the original request is answered from
that request’s record rather than starting a second analysis or 404ing a live id. The
claim is atomic in the store, so two simultaneous requests carrying one key cannot both
begin — the loser is told 409 IDEMPOTENT_REQUEST_IN_FLIGHT, which it can retry.
If the shared store is unreachable the endpoint keeps answering with per-process state. The cost is cross-replica agreement, never a wrong answer: an outage in bookkeeping must not fail an upload.
When the connection drops before an answer
Section titled “When the connection drops before an answer”A client-side ECONNRESET — Postman renders it “Could not send request — Error:
read ECONNRESET” — is a TCP reset, not an HTTP status. It carries no body and no code,
so it looks identical whether the endpoint refused the upload, the edge dropped it, or
the network between you and Cloudflare broke. It is not a rejection from this API: every
refusal this endpoint makes arrives as a status with a named code.
Two things make an upload survive it:
- Do not hold the connection open for work you can poll for. Screenshots and PDFs
are transcribed before the analysis starts, so a
Prefer: respond-synccall is held for as long as that takes — seconds per image.Prefer: respond-asyncreturns a202with astatusUrl; poll that instead. - Send an
Idempotency-Key. After a reset you cannot tell whether the request arrived, so a blind retry risks a second analysis and a second dataset. With a key, the retry returns the original answer if the first one landed and starts the work if it did not — see Retrying safely above.
Log formats it reads
Section titled “Log formats it reads”Logrite instruments 28 languages, each through that language’s normal logging
library, and every one of those has its own default line format. The parser is
measured against all of them — scripts/log-analysis-language-corpus.js, 35
formats in each library’s documented default layout, pinned as a test so a 29th
language cannot be added to the platform without it failing.
Structured records (JSON, NDJSON, CSV) are read field-by-field. Plain text is read by dialect. In both cases no line is ever dropped: a line the parser cannot date or classify is still stored with its text, because it is still evidence.
| Written by | Read |
|---|---|
| logback, log4j2, slf4j (Java, Kotlin, Scala, Groovy) | timestamp, thread, level, logger |
Python logging, loguru | timestamp, level |
| winston, pino, bunyan (JS, TS, Vue, Svelte) | timestamp, level — including pino’s numeric levels |
| Serilog console and compact/CLEF, NLog, Microsoft.Extensions.Logging (C#, VB.NET, F#) | timestamp, level — CLEF’s @t / @l / @mt |
| logrus, zap, zerolog, slog (Go) | timestamp, level, in both text and JSON |
| env_logger, tracing (Rust) | timestamp, level, target |
Monolog (PHP), Timber/logcat (Kotlin), spdlog (C++), package:logging (Dart) | timestamp, level |
std.log (Zig), NSLog (Objective-C), os.Logger (Swift), Log4perl, futile.logger (R), LuaLogging, Elixir Logger, syslog | whatever the format carries |
Also read: RFC 5424 and RFC 3164 syslog, Common Log Format (nginx, Apache), logfmt, epoch seconds and milliseconds, and sub-second precision from three to nine digits.
Severity
Section titled “Severity”A severity written in uppercase is read from anywhere in the line. Any other
casing is read only next to a delimiter that marks it as a field — [error],
level=error, E/Tag:. This is deliberate: error is an ordinary English word,
and an INFO line reading “no error occurred” must not be stored as an ERROR.
Numeric levels are matched exactly against pino’s 10–60 scale. A number off that scale is not given a severity — syslog’s severities run 0–7, nothing in a record says which convention is in use, and guessing produces a severity nobody wrote.
Records that carry a time but no date
Section titled “Records that carry a time but no date”log4j2’s stock pattern, Serilog’s console sink and Elixir’s Logger all write a
time and no date. Those events take their day from the nearest dated line in the
same file, and only failing that from the date the file was received. When that
happens the events carry timestampInferred: true and the response warnings
say where the day came from.
A stack trace is one event
Section titled “A stack trace is one event”A line that does not open a log entry belongs to the entry above it. Read line-by-line, a .NET exception log reports one event per stack frame — which inflates the total, deflates the error rate, and leaves every frame undated and levelless, findable by no time window and no severity filter.
Measured against a real customer estate — 311 files, 532 MB, 12 services:
| line-by-line | folded | |
|---|---|---|
| events carrying a timestamp | 71.6% | 100.0% |
| events carrying a level | 71.6% | 100.0% |
exception_*.log events | 81,596 | 1,526 |
| …and its reported error rate | 1.9% | 98.9% |
The fold is never silent — it is named in warnings with both numbers:
exception_2026-02-02.log: 56,917 of 58,007 lines were continuation lines — stack frames and wrapped messages belonging to the entry above them — and were folded into it. The file holds 1,090 log entries.
Bounds, because both directions of getting this wrong are real. A file where nothing opens an entry is returned untouched — five of the 28 supported formats carry no timestamp and no level in the line, and folding those would produce one event holding the whole file. An entry stops absorbing after 200 lines and the overflow continues as ordinary events, so a misread format cannot swallow a file and nothing is dropped either way.
Prose quotes the file, not a placeholder
Section titled “Prose quotes the file, not a placeholder”Repeated failures are clustered by masking the varying parts of a line. That key is the right thing to group on and the wrong thing to print, and it was being printed:
Most repeated failures:
<ts> -<n>:<n> [DBG] License heartbeat : <uuid> - Failure (x3,233)
The leading -<n>:<n> is not even a masked quantity — it is the timezone offset
-05:00. And that line is [DBG]: a debug line was leading the report as the
estate’s top failure. Now:
Most repeated errors (e.g.):
[ERR] Unhandled exception. System.NullReferenceException at Portal.Api.Handler (x210)
A real line from the file, labelled e.g. — a cluster of 3,233 covers 3,233
different ids, and quoting one without that word claims one id failed 3,233
times. Error-level shapes sort first; anything else is named for what it is.
Word, Excel, PowerPoint, PDF and screenshots
Section titled “Word, Excel, PowerPoint, PDF and screenshots”Since 23 September 2026 these endpoints read the same formats Cortex chat does. People send an Excel export of a query log, a PDF incident report, or a screenshot of a stack trace — so the text is extracted first and the analysis runs on that.
curl -X POST "https://aiservice.logrite.ai/api/v1/log-analytics/<workspace>?summary=full" \ -H "Prefer: respond-sync" \ -F "file=@./incident-report.pdf" \ -F "file=@./query-timings.xlsx" \ -F "file=@./error-screenshot.png"Nothing changes for an ordinary log. text, json, ndjson, csv and source files are
passed to the parser byte for byte, exactly as before.
| You send | What happens | files[].source.format |
|---|---|---|
.log .txt .json .ndjson .csv + code | read directly, untouched | (absent) |
.docx | paragraph text from the OOXML body | Word document |
.xlsx | every sheet as TSV, shared strings resolved | Excel workbook |
.pptx | slide text plus speaker notes | PowerPoint deck |
.pdf | the text layer | PDF |
.png .jpg .jpeg .gif .webp .bmp .heic .tiff .avif | transcribed by a vision model | image |
.doc .xls .ppt (pre-2007) | 415 — re-save as the modern format | — |
.zip .7z .rar, executables, SQLite | 415, naming what it is | — |
The format is decided by the bytes, never the extension or the Content-Type you
send — a .docx renamed .log is still read as Word.
Images are capped separately, and lower
Section titled “Images are capped separately, and lower”Each image is a vision-model call, so maxImagesPerRequest (default 20) is far below
the 1000-file limit. Going over is refused with 413 TOO_MANY_IMAGES — never quietly
trimmed to the first 20.
How long several images take, and the ceiling that is not ours
Section titled “How long several images take, and the ceiling that is not ours”Images are transcribed concurrently, four at a time, and a batch in which every file is under a megabyte is also analysed four at a time. Both bounds are deliberate: the provider meters 10 concurrent calls in aggregate and 5 per model across every caller, and a batch holding any file over a megabyte still runs strictly one at a time, because a large log holds its whole parsed event list in memory while it is analysed.
Measured through Cloudflare against dev on 24 September 2026, before and after:
| images in one request | serial | concurrent |
|---|---|---|
| 1 | 22.8s | 10.7s |
| 2 | 19.9s | — |
| 4 | 40.0s | — |
| 6 | — | — |
| 10 | 106.0s | see below |
A second consequence of serial work is quieter. The synchronous time budget (80s) starts
when the request arrives, and transcription spends it before the analysis begins — so on a
large image set the model pass is skipped for the later files and the report says so in
warnings: “AI correlation was skipped: this analysis used its time budget on the file
itself.” Findings are still returned; they are the deterministic ones only.
A large image is re-encoded to fit, not refused for its size
Section titled “A large image is re-encoded to fit, not refused for its size”An image over the byte budget a vision model is sent is re-encoded until it fits — JPEG first, then progressively smaller on the long edge — and the response says what was done. Nothing is refused for its size unless three attempts have already failed.
Measured on dev, 24 September 2026: one macOS screenshot, saved five ways.
| file | size | before | now |
|---|---|---|---|
shot.gif | 303 KB | read | read |
shot.heic | 254 KB | read | read |
shot.jpg | 653 KB | read | read |
shot.bmp | 30 MB | refused | read |
shot.tiff | 30 MB | refused | read |
⚠ BMP was advertised and had never once been decoded — the same trap, one
layer down. bmp is in extractedFormats, the sniffer maps 42 4d to
image/bmp, and there is a human name ready for it, but libvips ships no BMP
loader in the prebuilt binary: sharp(bmp) answers “Input buffer contains
unsupported image format”. The size refusal was hiding it, and hiding it
plausibly — BMP is uncompressed, so a BMP really is enormous, and the file really
was over the limit. It would have failed at any size. It is now decoded directly
(BI_RGB and BI_BITFIELDS at 24 and 32bpp), and the raw pixels go through the same
encoder as everything else.
One picture, opposite outcomes, decided by a container format the sender did not choose deliberately: BMP and TIFF store pixels uncompressed, so the same screen is forty times the size. The refusal then asked them to “attach a smaller version or a cropped screenshot” — the one job the service is better placed to do itself.
⚠ The gate was in the wrong place, which is what made it unfixable by tuning the number. It ran before normalisation, so it measured the file as sent rather than the bytes that reach the model — and normalising a 30 MB BMP produces a PNG of a few hundred kilobytes. Raising the limit to 10 MB would have moved the boundary and kept the bug.
Resolution is spent only after fidelity, in that order, because the job is reading text off a screenshot. Halving the long edge halves the height of every glyph and is what actually loses a log line; JPEG at quality 80 leaves text crisply legible at a fraction of the size. So a budget that plain re-encoding can meet costs no pixels at all.
The sizes are measured, not calculated: bytes after encoding cannot be predicted from pixel count — a screenshot of dense text and a photograph of the same dimensions differ by an order of magnitude, and PNG’s advantage on flat UI colour reverses on photographic noise. The ladder encodes, looks, and steps down only if it must.
Which model reads the picture (updated 2026-10-07)
Section titled “Which model reads the picture (updated 2026-10-07)”Since 2026-10-07 the image transcription and the findings pass both run on one pinned model. There is no fallback model: a failed call is retried on the same model, then sent to the same model on the provider’s discounted flex lane. A cross-model ladder could answer the same upload differently depending on which rung was up — and, as measured below, a model that cannot see replaces a refusal with fabrication. See AI models for the pinned model and the lanes.
The measurements below are what led there; the multi-model ladder they describe is retired.
Every model id the provider serves was probed with one image of known content on
24 September 2026. 16 of 22 transcribed it perfectly; 6 could not see it — and not one
of the 6 returned an error. Five said so in prose (“I don’t see any image attached”),
which the ladder treats as a failed rung and climbs past. The sixth returned five invented
lines of plausible monospace text. Capability is also not inferable from the family name:
one glm-5.3-flash sees, plain glm-5.3 does not.
Ranked over 14 images × 3 repetitions each, scored against ground truth taken from each image’s own source rather than from another model’s reading:
| vision model | recall | precision | perfect runs | run-to-run consistency |
|---|---|---|---|---|
gemma-4-31b | 97.7% | 98.8% | 36 of 42 | 92.9% |
glm-5.3-flash | 96.5% | 98.4% | 37 of 42 | 71.4% |
deepseek-v4.1-flash | 97.7% | 89.8% | 26 of 42 | 50.0% |
All three recover the line that names the failure 100% of the time, so none of them loses the diagnosis. They differ in what they add: the lowest-precision model returned 17 lines for an 11-line screenshot and 7 for a 5-line one, and an invented line becomes an evidence row indistinguishable from a transcribed one.
The instability is an interaction with this service’s own transcription prompt, not a property of the model — under a bare “transcribe this” prompt the same model reaches 99.2% precision. A ranking measured against one prompt does not transfer to another.
A reasoning model needs room to think AND to answer
Section titled “A reasoning model needs room to think AND to answer”The findings pass is capped at LOG_ANALYSIS_FINDINGS_MAX_TOKENS (4000). That number
is not arbitrary and the previous one — a hard-coded 1500 — caused a measured regression
worth stating plainly.
Some models on this provider emit reasoning_content before writing any answer. Measured
on a single seven-row evidence table, two repetitions each:
| model | completion tokens | reasoning_content |
|---|---|---|
gemma-4-31b | 87–96 | none |
deepseek-v4-flash | 161–172 | none |
deepseek-v4.1-flash | 590–1,351 | 1,741–5,100 characters |
On a larger evidence table the reasoning crosses the ceiling and the answer is truncated, so the completion arrives empty.
⚠ A benchmark must make the call this service makes. The model that caused this had been ranked at 100% correct — by a harness that sent no output ceiling, and so measured a request the service never issues. The same caution applies to the vision ranking above and its prompt.
A format we understand but cannot read is refused
Section titled “A format we understand but cannot read is refused”An empty document, or a scanned PDF with no text layer, comes back 422 NO_TEXT_EXTRACTED
naming the file. It is not analysed as zero events and reported COMPLETED.
Every image format, and two decoders
Section titled “Every image format, and two decoders”Verified end to end against dev on 23 September 2026 — one screenshot converted to
each format and sent through this route: .png .jpg .jpeg .gif .bmp .webp
.tiff .heic .avif all returned 200 with a transcript.
HEIC needs saying separately. sharp’s format list advertises heif and sharp cannot
decode it: the prebuilt libvips ships no HEVC decoder, which shows up as
source: bad seek to 260187 on a real file and not as a missing-format error. So AVIF
(AV1) goes through sharp and HEIC (HEVC) goes through heic-convert. HEIC has been the
iPhone default since 2017 and TIFF is what scanners emit, so before this the two formats
most likely to arrive from a phone or a scanner were the two that were refused.
“The service was busy” is not “your image is unreadable”
Section titled ““The service was busy” is not “your image is unreadable””The vision ladder used to try four models. They share one account, and the provider meters concurrency across all of them at once:
429 Concurrent limit reached for deepseek-ai/DeepSeek-V4.1-Flash: 5/5 slots in use429 Aggregate concurrency limit reached: 10/10 total slots in use across all modelsThat second limit breaks the ladder’s premise. When the account is saturated — or Cloudflare 502s the gateway — all four rungs fail within milliseconds of each other, and the walk down consumes four more slots on the way. The response then reported the union of four identical infrastructure errors as a fact about the picture:
This image could not be read after trying deepseek-v4.1-flash, gemma-4-31b, qwen3.6-35b-fast, kimi-k3-fast … Describe what it shows, or paste the text from it.
Handed those exact bytes on demand, all four models read the file. Since 23 September a failure is classified before it is acted on: a 429, 502, 503, 504, timeout or reset retries the same model after 600 ms then 1800 ms inside the existing 90-second budget, because slots free up in seconds; only a failure the image itself caused moves down the ladder. A reason that cannot be classified is treated as permanent, which is the conservative direction.
When every attempt failed transiently the message says so, and says the opposite thing:
The image itself looks fine — every attempt failed because the vision service was busy or unreachable, not because of anything in the picture. Send the same file again in a few seconds.
“Re-crop your screenshot” and “wait ten seconds” are opposite instructions, and only one of them was ever given.
A decode complaint is a claim about the bytes the provider RECEIVED
Section titled “A decode complaint is a claim about the bytes the provider RECEIVED”The same brownout has a second face, and it looks nothing like the first. One request on dev, eleven seconds, from the container log:
13:50:46 deepseek-v4.1-flash 502 …the origin web server returned an invalid or INCOMPLETE response… (retried ×3)13:50:53 gemma-4-31b 400 Failed to load image: cannot identify image file13:50:56 qwen3.6-35b-fast 400 same13:50:57 kimi-k3-fast 400 samecannot identify image file is PIL’s words for “these bytes are not a picture”, so it
was classified permanent — the conservative reading — and three rungs ended the walk
without one retry between them. But it is the provider’s account of what arrived at
its decoder, and a truncated upload produces exactly that error. “Invalid or
incomplete response” is the same event seen from the other direction. Handed the
byte-identical payload — same prompt, same max_tokens, same 2200 px normalised PNG —
four minutes later, every rung read it: 3127, 2030, 2068 and 2466 characters.
So the service now checks the claim instead of believing it. Before an image is
sent, a decoder on the host reads the exact bytes that will go on the wire
(sharp(...).stats(), a full pixel walk). The verdict has three states and the third
one carries the design:
| local verdict | meaning | a cannot identify 400 is then |
|---|---|---|
true | a decoder here read these bytes | transient — retried on the same rung |
false | our decoder refused them too | permanent — corroborated, not disputed |
| absent | no decoder installed on this deployment | permanent — unchanged behaviour |
The absent case is the point. “Nobody asked” is not “we checked and it is fine”; reading it as a dispute would make every genuinely corrupt upload retry three times before failing with the same message.
Two details are load-bearing. The probe decodes rather than reading the header — a
truncated PNG keeps a perfectly valid header, so a header read answers “yes, 3584×2164”
about a file whose pixel data stops a third of the way in. And it runs at failOn: 'truncated', not the 'none' used for conversion (which reconstructs what it can and
reports success, defeating the whole check) and not the stricter 'warning' (every
macOS screenshot carries an iCCP chunk, and a colour-profile complaint is not a decode
failure).
Claims about the model rather than the bytes — image too large, does not support image input, unsupported media type — stay permanent whatever the local decoder says.
A valid image can still be too large.
When the dispute is what ended the walk, the closing sentence reconciles the two facts rather than leaving the reader holding both:
The file itself is a valid image — it was decoded here before it was sent, and the same bytes read fine on every model minutes either side of this. The vision service rejected the upload it received, which is what a truncated or interrupted transfer looks like from its end. Send the same file again.
The same message again — and this time the bytes really were broken
Section titled “The same message again — and this time the bytes really were broken”The section above is true and was not the whole story. Days later the identical warning came back, on the identical file, from a build that already carried the fix:
Screenshot 2026-09-10 at 1.03.15 PM.pngwas not analysed … no text could be read from it … (Request failed with status code 400 — the provider said: Failed to load image: cannot identify image file<_io.BytesIO object>). Describe what it shows, or paste the text from it, and I can work with that.
That last sentence is the tell. It is the permanent advice, and a disputed decode
cannot produce it — so bytesVerified had not come back true, so a decoder on the host
had looked at those bytes and refused them too. The guard was not failing. It was
reporting, correctly, that the upload was not a picture.
The request log said why, and had been saying it all along:
bytes=1554581 ← the three files as the service received them 6,056 (logs.txt) + 19,390 (.md) = 25,446 leaving 1,529,135 for a screenshot that is 878,552 bytes on disk1,529,135 is exactly Buffer.byteLength(png.toString('utf8')). The client had read the
file as UTF-8 text and re-encoded it before sending: every byte that is not valid UTF-8
became the three-byte U+FFFD, the magic arrived as ef bf bd 50 4e 47 instead of
89 50 4e 47, and 331,777 replacement characters — 65% of the file — replaced the image.
Replaying those exact bytes through the public API reproduced the owner’s response word
for word.
The conversion is lossy and nothing downstream can undo it. Hundreds of distinct byte values collapse onto one character. So the service detects it and says so, in the sniffer, ahead of the signature table:
{ "code": "UPLOAD_NOT_BINARY", "statusCode": 400 }This upload arrived corrupted: it was sent as TEXT rather than as binary data, so roughly 67% of its bytes were replaced with the Unicode replacement character in transit. The service received 1,529,135 bytes for this file — compare that with its size on disk, which will be SMALLER, because every byte that is not valid UTF-8 becomes three. That conversion is lossy, so the original PNG image cannot be recovered from what arrived. The file on your disk is almost certainly fine — this is the sending client, not the file. Attach it as a FILE rather than as a text value (with curl:
-F "file=@/path/to/file"), then send it again. If that client cannot be made to send binary, encode it instead:base64 -i /path/to/file -o file.b64and upload file.b64. Base64 is plain ASCII, so it survives the very conversion that destroyed this upload, and this endpoint decodes it back to the original bytes.
The byte count is in the message because the diagnosis is otherwise uncheckable.
“This is the sending client, not the file” is correct and unverifiable from the caller’s
chair, and the first version of this message said only that — so the same request was
sent again, twice, unchanged. A UTF-8 round trip can only ever grow a binary file, so
the received count beside one ls -l settles whose side the damage happened on without
anyone having to be believed.
The service was ruled out by replay, not by reading its own parser. The multipart
reader’s comment says it is byte-safe, which is exactly the claim under suspicion. So the
same 878,552-byte file went through the live public endpoint in the shape PostmanRuntime
uses — chunked transfer encoding, a per-part Content-Type: image/png, its User-Agent —
and came back transcribed, logged bytes=22748 via=postman/Postman. Same client shape,
same tunnel, same route, correct bytes.
⚠ bytes= in that log line is measured AFTER extraction, not off the wire. A
878,552-byte screenshot that transcribes cleanly is recorded as bytes=3561 — the size of
the text read out of it. The original size survives only when extraction failed, which
is why the damaged upload is the one whose true arrival size appears there. Reading it as
a wire measurement will make a healthy upload look like a 250x shrink.
Three choices in that are deliberate.
It is not in the image path, even though that is where it surfaced. A .pdf, .docx
or .xlsx sent the same way is damaged identically; those formats merely begin with ASCII
magic (%PDF, PK), so they survive their first bytes and fail later and far less
legibly. The sniffer is the one place every format and both upload routes pass through.
It is a 400, not a 415. “Unsupported format” is the wrong diagnosis for a file this endpoint reads perfectly well — the request was malformed, and a partner branching on the code has to be able to tell those apart.
Two conditions, because either alone is wrong. A wrecked PNG or JPEG signature is
conclusive: an intact PNG never begins with U+FFFD. But %PDF and PK are the first
bytes of every intact PDF and Office file, so a magic match only names the format in the
message. And density alone would refuse a log full of mojibake — real text that went
through a bad encoding upstream, dense in U+FFFD, and a file the caller is entitled to
have analysed. So the damage must also look binary: a NUL byte in the scanned region,
which every mangled PNG, PDF and Office file carries and ordinary text does not.
Refusing that log would have been this fix causing the exact harm it was written to end.
One more thing was wrong for exactly one deploy, and it is the same failure in miniature. The scan is bounded at 256KB, so on a 1.5MB screenshot holding 331,777 replacement characters the message read “roughly 67% of its bytes (58,362 of them)” — a count of the window stated beside a percentage, which reads as a count of the file. The share survives truncation; that is what a proportion is for. The count does not, and now appears only when the scan covered the whole file.
Uploads carried as text that can be put back
Section titled “Uploads carried as text that can be put back”The refusal above is the last resort, not the first response. UTF-8 mangling is the one version of “sent as text” that destroys the file; the others are reversible exactly, and the endpoint reverses them before anything else looks at the bytes. Nothing needs to be declared and no parameter turns it on.
| carried as | example | recovered |
|---|---|---|
| base64 | iVBORw0KGgoAAAANSUhEUg…, wrapped or not | exactly |
a data: URI | data:image/png;base64,iVBORw0… | exactly |
| latin-1 text | a client that read the file with latin1/binary encoding | exactly |
| UTF-8 text | the case above | never — refused |
So this works, through any client that can send a text file at all:
base64 -i "Screenshot.png" -o shot.png.b64curl -X POST "$BASE/api/v1/log-analytics/$WORKSPACE" \ -H 'Prefer: respond-sync' -F "file=@./shot.png.b64"The upload is named shot.png in the response — the .b64 suffix is a statement about
transport, not part of the filename — and the acknowledgement records what happened:
[log-analytics/public] BATCH-… accepted files=1 … recovered=[shot.png (base64)] […]Base64 is the door to recommend to a client that cannot be fixed, and the reason is mechanical rather than a matter of preference: base64 is pure ASCII, so the UTF-8 round trip that replaces 65% of a PNG leaves it byte-for-byte identical. It arrives intact through the same broken client, with nothing changed at that end but the file picked.
Latin-1 and UTF-8 mangling look identical to a person and are not the same event.
Both arrive inflated and unreadable. A latin-1 round trip maps 0x00–0xFF onto
U+0000–U+00FF and back with nothing lost; a UTF-8 round trip collapses hundreds of
byte values onto one replacement character. The size says which: the 878,552-byte
screenshot above would be 1,285,602 bytes through latin-1 and 1,529,135 through
UTF-8, and it arrived at 1,529,135. One of those is recoverable and the endpoint recovers
it silently; the other is the refusal.
⚠ Decoding runs before every other decision, and the order is load-bearing. gzip
detection, the archive refusal and the Office-container probe all read leading bytes, and
on an encoded upload those bytes belong to the encoding. Run the other way round, a
base64’d .gz is never decompressed and a base64’d .zip is never refused — both are
handed on as text and analysed as though the base64 were a log, which produces a
confident report over an encoding rather than an error.
⚠ The risk here runs opposite to the rest of the upload path. Everywhere else a
mistake refuses something readable; here a mistake rewrites a file that arrived
perfectly, then analyses whatever it happened to decode to. So an implicit decode
must land on a signature the platform already recognises, and one that lands on nothing
is abandoned with the file left exactly as it arrived. A log full of base64 — a JWT, a
signature, a dumped payload — is therefore left alone, because it decodes to nothing
identifiable. Only an explicit declaration (a data: URI, a .b64 name) is trusted
without that, because there the caller has said so.
One response, two renderers — and only one of them had been fixed
Section titled “One response, two renderers — and only one of them had been fixed”Re-running the same screenshot on its own surfaced the other half. The summary was right and the paragraph beneath it was not:
| field | said |
|---|---|
summary.headline | “113 transcribed lines of a transcribed image, not log events.” ✓ |
overview (?summary=full) | “113 events were read. … 20 of the 113 events read carry it — 18% of the window.” ✗ |
The summary and the plain-English narrative are different renderers of the same
facts, and the first fix reached one of them — the shorter one. ?summary=full now
counts in the unit the file holds, drops the health clause and the span entirely (a
picture has no time axis and no error rate), and measures a finding’s share of the
file rather than of the window.
The scope is null the moment ANY file in the upload is a log: the merged counts then
cover the log files only, so event is the correct noun for what is left and a document
noun would mislabel it in the other direction. An upload holding several different
document kinds — a write-up beside a screenshot — counts in records, the same neutral
word the merged headline already uses.
A document is analysed, not counted as traffic
Section titled “A document is analysed, not counted as traffic”A file that is not a log — a written incident report, an XML document, the text read off a screenshot — is still read, still analysed and still reported with its own findings. Nothing is refused and nothing is dropped. What changed on 23 September 2026 is what the numbers around it claim.
The line parser never fails: it keeps any line it cannot classify, so every sentence of a markdown write-up became a record. A 259-line incident report measurably came back as:
"259 events, 1 error-level (0.4%).""Repeats 4x (non-error): e.g. ---"The 0.4% is one sentence containing the word CRITICAL, divided by the number of
sentences — a claim about a population made over a document. The --- is a markdown
horizontal rule reported as a recurring failure shape. The same upload’s batch total read
“424 events across 3 files” over a log holding 45 entries.
Now:
45 events across 1 log file, 0 error-level. 4 findings. 2 further files are documentsrather than a log (382 records), analysed and reported on their own.| log file | document, markup or transcript | |
|---|---|---|
| counted in | events | lines, elements, transcribed lines |
| error rate | reported | never — there is no population to take a share of |
| recurring patterns | reported | empty, not reworded |
summary.eventsAnalyzed | summed into the upload’s total | counted in summary.documentRecords instead |
| cross-file correlation and the batch timeline | included | left out |
documentRecords is absent when an upload holds no such file, so an all-logs response is
byte-identical to before. Per-file numbers are unchanged — eventsAnalyzed, levels and
errors stay exactly as measured on each file, and files[].file.contentKind names what
it turned out to be. Only the merged totals and the wording changed.
Turning it off
Section titled “Turning it off”LOG_ANALYSIS_RICH_FORMATS=false sends every such file back as a 415 naming its format.
That is not a return to the old behaviour — the old behaviour was neither reading nor
refusing them.
GET /limits publishes all of this: extractedFormats lists what is converted,
separately from acceptedFormats, and maxImagesPerRequest sits beside the file count.
Text encoding is detected, not assumed. UTF-8, UTF-16 and UTF-32 are all read, with
or without a byte-order mark. That matters more than it sounds: PowerShell’s Out-File
and > redirection write UTF-16LE by default, so a Windows operator who pipes a log to a
file produces one without choosing to. Read as UTF-8 that file does not fail — it yields
a NUL between every character and a confident report over nonsense. When the file was not
UTF-8 the response says so in warnings.
One file per request on the keyed route. Several files in one call is what the batch
route is for; sending more here is refused with 413 TOO_MANY_FILES rather than analysed
in part and reported whole.
Correlating across files
Section titled “Correlating across files”Since 4 September 2026 a batch does a second pass once every file has finished, and
reports what the files say together that none of them says alone. It arrives as
crossFindings[], beside the files[] you already read.
{ "batchId": "BATCH-06e15fd9", "status": "COMPLETED", "summary": { "filesAnalyzed": 4, "eventsAnalyzed": 1284119, "timeRange": { "from": "2026-09-02T08:55:11Z", "to": "2026-09-02T10:31:48Z" }, "correlationsFound": 2, "correlationBasisUsed": ["SHARED_SIGNATURE", "TIME_WINDOW"] }, "crossFindings": [ { "id": "X-1001", "scope": "CROSS_FILE", "severity": "CRITICAL", "category": "CREDENTIAL_EXPOSURE", "title": "The same failure appears in 2 files", "files": ["exec-agent.log", "db.log"], "correlation": { "basis": "SHARED_SIGNATURE", "confidence": "HIGH", "signature": "Server=<id>;User Id=<id>;Password=…", "contributors": [ { "fileId": "FILE-aef3e7", "filename": "exec-agent.log", "findingId": "F-1001", "eventCount": 200 }, { "fileId": "FILE-b21c04", "filename": "db.log", "findingId": "F-1001", "eventCount": 212 } ] }, "evidence": { "eventCount": 412, "firstSeen": "…", "lastSeen": "…", "samples": ["…"] }, "detectedBy": "rules" } ], "warnings": [], "files": [ "…unchanged…" ]}Nothing you already read changed shape. Four fields are new on the envelope —
crossFindings[], warnings[], summary.timeRange, summary.correlationsFound and
summary.correlationBasisUsed[] — and one is new on every per-file finding:
files[].findings[].fileId. Per-file ids restart at F-1001, so four files in a batch
all publish an F-1001. id alone never said which file a finding was in; now
(fileId, id) does. A cross-finding uses an X- prefix, so it can never collide with either.
What counts as a link
Section titled “What counts as a link”All five bases the schema declares are now produced, derived from the finished per-file findings and nothing else. The analysis itself is not involved, so a defect here cannot change what a single file reports.
| Basis | Confidence | What it means |
|---|---|---|
ID_JOIN | HIGH | The same request, trace, span or transaction id in two files. Since 9 September 2026. The only basis that does not require the two findings to be of the same kind, which is the point of it: a checkout timeout in the browser log and a connection-pool exhaustion in the service log are different categories, different severities and one request. Reported first, because it is the only basis that says the files recorded the same unit of work. |
SHARED_SIGNATURE | HIGH | The same masked failure shape in two files. The parts that vary between two occurrences of one error are replaced before comparison, and so are the timestamp, the level and the emitting service — the service is exactly what differs between two files describing one fault, so leaving it in would mean this link could only ever match files from the same service. What remains must still be at least 24 characters, so a generic tail cannot become a HIGH-confidence link. Needs no timestamps at all. |
TIME_WINDOW | MEDIUM | Findings of the same type and category whose windows share a common instant, overlapping by at least 50%. Both sides must be MEDIUM or worse. |
ACTOR | MEDIUM | The same named principal or client address in two files. Since 9 September 2026. Like ID_JOIN it does not require the findings to be of the same kind — but a shared actor is one person, not one unit of work: an account generates thousands of unrelated requests in a day, so this is a lead to follow and never more than MEDIUM. Only the four detectors that group by actor contribute one, all of them security or threat detectors. |
SHARED_CATEGORY | LOW | The same kind of finding in two files, with no shared window and no shared shape. Dropped entirely when something stronger already covers that category. |
A time overlap is never HIGH. Coincidence is cheap in logs: two files reporting errors in
the same minute is the most ordinary thing in any upload, and calling that one incident would
make the whole section worth ignoring. window and overlapPct are there so you can judge
it yourself.
Findings whose windows merely touch are not linked, and LOW findings never form a time link — they co-occur constantly.
ACTOR was the last basis named in the schema and not produced. It was absent for a plumbing
reason rather than a semantic one — the actor breakdown lived on the internal signals and
stopped there — and it now travels the same route the identifiers do, as
evidence.actors[] on the findings that have one.
What it refuses is the point of it: a loopback or unspecified address (127.0.0.1, 0.0.0.0,
::1, 255.255.255.255) and the names a logger writes when it has no principal (-,
unknown, anonymous, null, none, system, guest). Those appear in almost every server
log, so joining on one would link every file in a batch to every other — the widest possible
false positive.
When something stronger already covers the same two files, the actor agreement travels with it
as alsoMatchedBy rather than as a second, weaker row saying the same thing.
Which finding carries the evidence
Section titled “Which finding carries the evidence”Both identifiers[] and actors[] are accumulated by the rule pass, over every event a
detector matched. One finding per type:category is published, ranked by severity — so a HIGH
model finding regularly outranks the MEDIUM rule finding that did the accumulating. The
survivor now inherits the fields it lacks, because otherwise a decision about which finding a
reader sees would decide which evidence exists, and a batch could lose its only cross-file
link to a tie-break. A finding that never had either — a genuinely model-only one — is still
joinable: its verbatim samples are scanned for both.
Joining on an identifier
Section titled “Joining on an identifier”A line carrying requestId=, request_id, X-Request-ID, correlationId, traceId,
spanId, transactionId or a W3C traceparent header contributes that value to the finding
it belongs to, and the finding publishes what it was seen with:
"evidence": { "eventCount": 412, "samples": ["…"], "identifiers": ["request:req-8f31c2", "trace:4bf92f35…"]}correlationId and requestId are read as one kind — a correlation id is the request id
as it crosses a service boundary, and keeping them apart would refuse exactly the join this
exists to make. trace, span and transaction stay distinct, so two files sharing a hex
string for two different reasons are not linked.
What is deliberately not treated as an identifier: the placeholders a logger writes when
it has none (none, null, unknown, all-zeros), a short run of digits (retry-id=7 is a
counter, and collides across unrelated files constantly), and anything under six characters.
A missed join degrades to the clock-overlap and category bases; a wrong one puts two unrelated
services in one incident at HIGH, which is the failure this whole section exists to avoid.
An ID_JOIN names one contributor per file — a file that raised four findings about one
request is one contributor to that request’s story — and orders them by when each file first
recorded the id. That order is the path the work took, and the recommendation sends you to
the last layer it reached rather than the first: the entry point holds the symptom, and a
failure in the deepest layer is what the earlier files are reporting the consequences of.
A cluster that fails to form is degraded, never deleted
Section titled “A cluster that fails to form is degraded, never deleted”Until 9 September 2026 a TIME_WINDOW cluster that could not reach the 50% floor was
discarded whole. Measured: three files carrying one request id reported the real chain, and
the same three plus one unrelated file — a nightly batch job in the same minute, with its own
request id — reported nothing from that basis at all. The unrelated file entered the cluster,
pulled the common intersection under the floor, and took the other three down with it, after
which the SHARED_CATEGORY grouping naming the batch job was the only answer left. An added
file did not add noise; it deleted a finding the caller had seen a moment earlier.
A failed cluster is now retried on its subsets, and what the retry excluded is named in
warnings[]:
Excluded billing.log from the elevated error rate window: it recorded a finding of the same kind in the same period but shares no common window with the others, so it is not part of that finding. The remaining 3 files are reported together.
When no subset qualifies either, that is said too, and the group falls back to the category basis — which is a weaker claim, stated as one.
Ordering and narrowing what you read back
Section titled “Ordering and narrowing what you read back”GET /{workspace}/{batchId} takes these. They reshape the stored result and never change
what was analysed.
| Parameter | Values | Default |
|---|---|---|
sort | severity, time | severity |
order | desc, asc | desc |
include | all, cross, files | all |
minSeverity | CRITICAL … INFO | INFO |
minConfidence | HIGH, MEDIUM, LOW | LOW |
basis | csv of the bases above | all |
type | csv of THREAT, SECURITY, COMPLIANCE, OBSERVABILITY | all |
category | csv of finding categories | all |
detectedBy | all, rules, model | all |
file | csv of filenames or fileIds | all |
trimBelow | a severity — below it a finding keeps its count and loses its detail | none |
limit / offset | page crossFindings; max 500 | 100 / 0 |
fileLimit / fileOffset | page files[]; max 500 | 200 / 0 |
# the cross-file section alone, worst first — a few KB however many files you sentcurl "https://aiservice.logrite.ai/api/v1/log-analytics/<your-registered-name>/BATCH-06e15fd9?include=cross"
# a timeline instead of a ranking, and drop the detail on anything under HIGHcurl "…/BATCH-06e15fd9?sort=time&trimBelow=HIGH"Three behaviours to rely on:
- An unrecognised value is a
400 BAD_PARAM, never a fallback. A filter that is silently ignored hands you a complete, successful, wrongly-scoped answer, and nothing in the response tells you it happened. summaryis never recomputed by a filter. It keeps describing the batch that was analysed. What the filter did is reported separately underview—crossFindingsMatchedagainstcrossFindingsReturned,filesMatchedagainstfilesReturned. Otherwise narrowing toCRITICALwould let you readsummary.findings: 1and conclude the upload found one thing.sort=timeputs undated findings last in both directions. Putting “we could not read a timestamp” first is the least useful thing an ascending read could hand you.
verbosity=concise is a different projection with its own merge rules. Combining it with a
non-default filter is a 400 rather than one of the two being quietly dropped.
Reading the evidence on a cross-file finding
Section titled “Reading the evidence on a cross-file finding”Each entry in correlation.contributors[] carries its own sample — one verbatim
line from that file — and evidence.samples takes one line from each contributing file
before a second from any. On a three-layer upload that gives you the traversal directly:
worksoft-portal-ui.log 73,577 events 09:22:29 [ERROR] portal.ui - GET /api/execution/EXEC-88431/status 503 after 30014msworksoft-exec-agent.log 63,980 events 09:21:19 [ERROR] agent.executor - ExecutionServiceException: result store unavailableworksoft-certify-backend.log 57,582 events 09:20:13 [ERROR] certify.db - Login failed for user 'certify_svc'. Reason: The password of the account has expired.The timestamps descend through the layers, so causal order is visible without opening a
file. This matters more than it looks: taking samples file-by-file instead put three
near-identical 503 lines from the top layer in every cross-finding, and the backend
line naming the cause appeared nowhere.
What it costs, and what it will not do
Section titled “What it costs, and what it will not do”- The pass runs once, after the last file — never on a per-file publish. At the 1000-file ceiling, correlating on every publish would be O(F³) rather than O(F²).
- Pairing is bounded: findings group by
(type, category)first, and only the 50 loudest files per category are paired. - Reported cross-findings are capped at 20. When the cap drops any,
warnings[]says so. Silent truncation is the one outcome this route must not produce. - A failure in this stage degrades to
crossFindings: []and a warning. It cannot fail the batch — losing 999 good per-file analyses to a defect in the second pass would be the opposite of what it is for. - A batch of one file produces none, by construction. Every cross-finding names at least two files.
?shape=reportandverbosity=conciseare separate projections and do not carrycrossFindings. Use the default envelope for cross-file results.
What Cortex shows from this response
Section titled “What Cortex shows from this response”The chat surface renders the report rather than summarising it, because everything below is already in the body and re-deriving any of it would be a second source of truth that can disagree with the first.
It is prose, not a widget. The report used to render as a collapsible card sitting beside the answer. A card is the right shape for something you consult and the wrong shape for the answer itself: the reader asked a question and got a control to open, while the chat’s own paragraphs — the part that reads like an answer — sat above it saying less. Every section below is now emitted as markdown into the assistant’s own message, so it is laid out by the same renderer as the rest of the turn and reads as one continuous answer. Measured on a two-index turn against live data: 1,541 words, against ~490 before.
Nothing is summarised in the process. Every number is copied from the body; the renderer chooses order and wording and computes nothing, so it cannot disagree with the server.
Working first, conclusion last. In order, top to bottom:
| # | Shown | From |
|---|---|---|
| 0 | What was read, before anything is filtered | the turn, not the report — see More than one index |
| 1 | ## Log analysis — <name> — <handle> | the source, named — see Naming the source |
| 1 | The headline | summary.headline — one sentence, so a reader can stop at line one |
| 2 | What YOU asked about | view.asked — the severity the question named, and whether it matched |
| 3 | Why this is happening, and what to do | description, recommendation and detectedBy on the leading findings |
| 4 | Findings, worst first, each with its evidence | the severity sections |
| 5 | What repeats, with a real line quoted | summary.topPatterns[] — label is the template, example is verbatim |
| 6 | What the service did | summary.activity |
| 7 | The numbers | eventsAnalyzed, errors, warnings, errorRate, levels, findings.bySeverity, findings.byType, timeRange, and the file facts |
| 8 | What the analysis could not do | warnings[] |
| 9 | The summary | overview and summary.highlights[] |
| 10 | Where to go next | derived from 3, 5 and 7 — see below |
The summary used to open the report. That is the wrong way round for anything longer than a paragraph: a reader given the conclusion first has no reason to read the evidence, and a reader who wanted only the conclusion had to scroll past it to confirm the rest was there.
Every severity section is rendered, info included. A healthy index’s only
finding is the INFO note the analyser writes about what the service does all day
— with a description, a recommendation and evidence. It is the whole answer for an
index with nothing wrong in it.
Every finding is written out in full. There is nothing to open, because there is no longer anything folded — the depth control decides what the RESPONSE carries, not what the page hides.
activity is omitted when the log records no request outcomes. The key is still
sent on such files, with requests: 0 — a panel headed “What the service did”
whose only row is “Requests 0” says the service served nothing, when what happened
is that this is not a request log. The busiest hour is reported with the counted
facts instead, because it is true of any log.
warnings[] is rendered even when every other section is empty — that is the case
where it matters most.
The cause, under its own heading
Section titled “The cause, under its own heading”The response has always carried a causal reading and a fix — description says why
the finding is there, recommendation says what to do, detectedBy says how it was
established. All three were rendered inside each finding row, where a reader
looking for “so what is actually wrong” had to open every finding and assemble it.
They now also lead, under Why this is happening, taken from the findings ranked
medium or worse — up to three, because a fourth cause is a list, not a diagnosis.
Each says how it was determined, and the two are not equivalent:
detectedBy | Rendered as | What it means |
|---|---|---|
rules | deterministic detector over the counted events | a threshold on numbers the parser counted |
model | correlated across the evidence, with every quantity taken from the logs | a reading over the evidence pack, forbidden from stating its own numbers |
On a healthy index the heading changes rather than the section disappearing. It reads Why it looks like this, followed by “Nothing in this window failed, so there is no fault to trace” — because a root-cause section that invents a fault to justify its own heading is worse than one that says there is none.
Naming the source
Section titled “Naming the source”Each analysed source gets its own heading, and the heading is not the bare handle.
aggregations.json is what a file happened to be saved as; log-analytics-2905-payments
is a storage key with a tenant id in the middle of it. Neither is a name, and a reader
looking at three headings in one turn had to decode all three before they could tell
which finding belonged to which service.
A readable name is derived from the handle itself — nothing is invented — and the handle is kept beside it, because it is the only string that identifies the source unambiguously and the only thing anyone can paste into a search box.
| Handle | Heading |
|---|---|
aggregations.json | Aggregations — aggregations.json |
authenticationservice_133-2905 | Authenticationservice 133 — authenticationservice_133-2905 |
log-analytics-2905-payments | Payments — log-analytics-2905-payments |
2905 | 2905 — stripping the id would leave nothing to show |
Three parameters that were overruling the question
Section titled “Three parameters that were overruling the question”The composer used to send shape=report&summary=brief&depth=full on every call. Read
the precedence rule above and the consequence follows: naming any of the three tells
the server the caller has been explicit, and an explicit parameter outranks the
question. So a reader who typed “give me an in-depth root-cause analysis” had that
sentence discarded in favour of three controls sitting at values they had never
touched.
The three controls now start on Auto, which sends nothing. resolveSectionIntent
reads the sentence; moving any one of them still wins. narrow=false is still sent,
because it is not a shaping parameter — it is the instruction not to delete anything.
This means the endpoint’s own default shape is what a reading surface receives, and that shape differs from the report shape in ways a renderer has to handle:
| Fact | Report shape (shape=report) | Default shape |
|---|---|---|
| findings | severity sections (critical … info) | findings[] |
| a finding’s event count | events | evidence.eventCount |
| a finding’s quoted lines | evidence[] | evidence.samples[] |
| file facts | summary.files[] | file |
| the id to re-read the analysis | absent | analysisId |
Both are normalised to one shape before anything renders. A surface that reads only the first form silently loses every event count and every quoted line the moment no shaping parameter is sent.
errorRate is a ratio
Section titled “errorRate is a ratio”summary.errorRate is errors / eventsAnalyzed in 0..1, not a percentage. A file
whose overview reads “1,509 errors out of 81,596 events (1.9%)” carries
errorRate: 0.0185. Rendering it straight into a % sign prints 0.0% — wrong by
two orders of magnitude on the single figure a reader uses to judge how bad a file
is. Multiply before you print.
The question, answered rather than applied
Section titled “The question, answered rather than applied”With narrow=false the body is complete, so the question has to be visible some
other way or the reader cannot tell it was read. You asked about critical leads
the report with the findings matching view.asked, or — when none match — says so
in one line and points at what was found instead:
No critical findings were raised for this index. Everything the analysis did findis below.Where to go next
Section titled “Where to go next”The turn closes with one row of up to four follow-up questions, each quoting a number or a string the server counted: the worst finding by name, the pattern that repeated and how often, the busiest hour, and either a breakdown of the error-level events or — when there were none — whether any exist outside this window. A suggestion whose source fact is missing is not produced: a plausible follow-up about data that was never measured is worse than none, because it will be asked and answer nothing.
One row, not two. The chat answer suggests its own follow-ups and the analysis suggests these; rendered as separate rows that is eight questions at the foot of one turn, which is a menu rather than a suggestion. They merge into one list capped at four: the chat’s lead, because they were composed against the prose just read, and these top the row up. Duplicates are dropped case-insensitively, and a turn that already has four gets nothing added.
A question is built from the pattern’s label, cleaned of the level and the masked
timestamp the analyser groups by — those are how lines are CLUSTERED, not what the
line says. Why does "Logout PURL - https://…" repeat 154 times? rather than a
question with DEBUG 2026-08-17 08:43:32.738 -05:00 [DBG] pasted into the middle
of it.
More than one index
Section titled “More than one index”Selecting more than one index runs one call per index, concurrently. The turn then reads: one unconditioned opening line, a titled section per index, a combined reading, and one shared row of follow-up questions drawn from across them.
The opening line is the only sentence in the turn that is true regardless of what was asked — how many indexes, how many events, over what window, and whether anything in any of them was error-level:
Read 2 indexes: 1,351 events, 2026-08-14 19:53 → 2026-08-18 18:12 UTC (3d 22h).No error-level events in any of them.Every section beneath it is scoped by those facts and none of them states them. An index whose call failed is named here as unread and excluded from the totals — counting it as zero events would make a failure indistinguishable from an empty index.
The window is written one way everywhere it appears — the opening line, the
Window fact and the server’s own summary.headline. It was three renderings of
one range on a single screen, two of them raw ISO strings.
The combined reading is a rollup of the sections above it — every number in it is a sum or a copy — so it cannot contradict them, and an index whose call FAILED contributes no zero, because it was never read. One index gets no rollup: that would be the report above it, printed twice.
It concludes rather than repeats. It carries the merged overview and the combined total, and folds its findings to their titles: it states no cause of its own and prints no evidence, because every one of those was read in full in the section directly above. Restating them cost around 300 words, none of them new.
Finding ids are minted per analysis, so two indexes both return F-1001. The rollup
re-keys each finding by its index (idx-a:F-1001) — merged under one id they are
two rows sharing one identity, and expanding either expands both.
When some events are not retained
Section titled “When some events are not retained”Findings are computed from the parsed events in memory, so they are complete whatever the store does. Retention is separate, and the response is explicit about which of the two failed.
An ECS log is the case worth knowing about. Datasets in this family carry a shared
template that maps host, source, container and os as strings, and ECS makes every
one of them an object — so those documents are refused on the first attempt.
They are not lost. The store names the field it refused and the type it wanted, which
is enough to convert the offending field and re-send the document. eventsStored counts
them and a warning says what was converted. The cost is honest: host becomes the JSON
text {"ip":"10.0.0.4","name":"web-01"} rather than an object, so a full-text search for
10.0.0.4 still finds the row while host.ip: "10.0.0.4" does not. Send an ECS log to
a dataset of its own if you need per-sub-field queries.
eventsRejected above zero with a warning naming fields means the repair did not work
either. The finding half of the response is unaffected; status is PARTIAL, never a
silent COMPLETED.
A store OUTAGE reads differently, deliberately: status stays COMPLETED — you asked
for an analysis and got a whole one — and the warning says how much landed before the
store went away.
Operator settings
Section titled “Operator settings”| Variable | Default | Purpose |
|---|---|---|
LOG_ANALYSIS_API_ENABLED | true | Master switch; off makes every path 404 |
LOG_ANALYSIS_API_KEYS | (empty) | secret:workspaceId:label, comma-separated |
LOG_ANALYSIS_PUBLIC_WORKSPACES | (empty) | name:workspaceId, comma-separated — the registered segments of the open route |
LOG_ANALYSIS_MAX_BYTES | 262144000 | Per-file ceiling on the keyed route |
LOG_ANALYSIS_MAX_EVENTS | 3000000 | Events analysed per file |
LOG_ANALYSIS_MAX_CONCURRENT | 3 | Analyses in flight at once; past it an upload is refused 503 |
LOG_ANALYSIS_MAX_INFLIGHT_BYTES | 201326592 | Uploaded bytes in flight at once. The ceiling that decides whether the process survives — measured at ~7–11MB of process memory per MB of log |
LOG_ANALYSIS_MAX_EVENT_FIELDS | 200 | Distinct fields one record may add to a dataset before the rest are folded into additionalFields |
LOG_ANALYSIS_PUBLIC_RATE_LIMIT | 5 | Uploads a minute per address on the open route |
LOG_ANALYSIS_PUBLIC_READ_RATE_LIMIT | 120 | Reads a minute per address; polling must not spend the upload allowance |
LOG_ANALYSIS_SHARED_STATE | true | Share idempotency reservations and async results between replicas |
LOG_ANALYSIS_SHARED_STATE_INDEX | logrite-log-analysis-state | Where that bookkeeping lives — deliberately outside the log-analytics-* family |
LOG_ANALYSIS_AI_ENABLED | true | Off leaves the deterministic findings, which are reproducible |
LOG_ANALYSIS_RETENTION_MS | 604800000 (7 days; was 24 h) | How long a result stays re-readable by id, listed, and eligible for reuse |
LOG_ANALYSIS_MAX_STORED | 2000 | Upload records the state index answers for per workspace — the ceiling on GET /api/v1/log-analyses?limit= |
LOG_ANALYSIS_MEMORY_ENTRIES | 200 | Entries in each process-local map (re-read cache, idempotency keys). A miss falls through to the shared state index, which is authoritative — this bounds memory, not retention |
PROXY_HOPS | 1 | Proxies we own in front of the service. Decides what req.ip reports, and therefore whether the IP allowlist and the per-caller rate limit work at all. A count, never true |
Both route ceilings are enforced against a real measurement rather than a chosen number:
scripts/load-log-analysis.js in the service repo produced them, and it should be re-run
before either is raised.
With no keys configured the endpoint still serves session tokens — a partner integration simply has no way in until a key is issued.
Removing datasets you created
Section titled “Removing datasets you created”Listing and deleting your uploads (2026-10-07)
Section titled “Listing and deleting your uploads (2026-10-07)”Every finished upload leaves a record, so a client can see and remove what it sent
without touching indices. Both routes take the same credentials as
GET /api/v1/log-analyses/{id} and only ever see the credential’s own workspace.
# newest first; default 50, capped at LOG_ANALYSIS_MAX_STORED (2000)curl -s "https://aiservice.logrite.ai/api/v1/log-analyses?limit=50" \ -H "Authorization: Bearer $LOGRITE_API_KEY"{ "analyses": [ { "id": "ANL-8f31ab27", "datasetId": "log-analytics-…", "fileName": "app.log", "bytes": 48213, "contentSha256": "…", "createdAt": "2026-10-07T09:12:44.120Z", "findings": 4, "bySeverity": { "HIGH": 1, "MEDIUM": 3 }, "status": "COMPLETED", "partial": false } ] }A limit that is not a positive integer is 400 BAD_PARAM.
curl -s -o /dev/null -w '%{http_code}\n' -X DELETE \ "https://aiservice.logrite.ai/api/v1/log-analyses/ANL-8f31ab27" \ -H "Authorization: Bearer $LOGRITE_API_KEY" # 204DELETE answers 204 and removes, for that one analysis:
- its dataset index — unless another live analysis shares it (a batch, window or
named
dataset=index holds other uploads’ events too), and only ever an index in this workspace’s ownlog-analytics-family; - its raw file in
log-uploads-<workspace>(the register record and its byte chunks); - its stored result in the state index and its upload record.
An unknown id, an expired one, and another workspace’s id all answer the same 404 NOT_FOUND. A retried delete tolerates the parts already gone.
Operator clean-up of older datasets
Section titled “Operator clean-up of older datasets”Every upload creates or extends a dataset and nothing expires it. Measured on dev on
1 September 2026: 254 of the 303 log-analytics-* indices were written by this
endpoint, holding 744MB, against a cluster with no lifecycle policy at all.
There is deliberately no blanket retention rule. The family is shared with collector agents, demo fixtures and eval data, and expiring somebody else’s records is a product commitment rather than a setting. What exists instead is a targeted operator tool that reports by default and deletes only what it can prove this endpoint owns:
node scripts/log-analysis-retention.js # report, 30 daysnode scripts/log-analysis-retention.js --older-than 7 # report, 7 daysnode scripts/log-analysis-retention.js --older-than 30 --applyAn index is a candidate only when every document in it was written by this endpoint. A dataset shared with another writer is listed and never touched.
Verifying a deployment
Section titled “Verifying a deployment”Two scripts in the service repo check a running deployment rather than a build:
BASE_URL=https://dev-aiservice.logrite.ai scripts/test-log-analysis-api.shnode scripts/log-analysis-corpus-live.js # labelled corpus, model ONThe first is the acceptance suite, organised by the defect each case would have caught rather than by the happy path. The second runs a labelled corpus — including adversarial cases that must NOT produce a finding — through a deployed build with the model enabled, and is the only thing measuring precision rather than recall.
scripts/log-analysis-replica-proof.sh covers what neither can see, because both talk to
one base URL: run it with PRIMARY and PEER pointing at two processes sharing a
cluster to check that a retry and a poll cross a replica boundary.