API Validation

The contract

Every Logrite API answers a missing mandatory field the same way, in one sentence that names the field — using the wire key the caller has to type — and names all of them at once:

POST /api/v1/analyzer    {}
400 {
  "projectId":  "projectId is mandatory",
  "templateId": "templateId is mandatory",
  "branchName": "branchName is mandatory",
  "message":    "projectId, templateId and branchName are mandatory",
  "errorCode":  "missing_fields"
}

Three wordings, and only three:

absent fields	message
one	projectId is mandatory
two	projectId and templateId are mandatory
three or more	projectId, templateId and branchName are mandatory

Order is the order the endpoint declares its checks in, so the sentence reads like the request body rather than like a hash iteration.

Where the wording lives

Two implementations, one contract, pinned on both sides.

Service	Utility	Test
Backend (Java)	com.logwrite.validation.RequiredFields	RequiredFieldsWordingTest, OneWordingForAbsenceTest
Analyzer (Node)	src/api/validation/requiredFields.ts	tests/requiredFields.test.ts (npm run test:requiredfields)

The two test files carry the same fixtures. Reword one implementation on its own and the other service’s test goes red — that is what stops POST /api/v1/analyzer and POST /analyze drifting into two vocabularies again.

Backend

RequiredFields.check()
        .text("email", input.getEmail())          // null, "" and "   " are the same thing
        .text("password", input.getPassword())
        .positive("workspaceId", workspaceId)     // 0 and negatives identify nothing
        .notEmpty("files", files)
        .entry("title", body)                     // a key in a Map-shaped body
        .either("repoUrl or repoFullName", repoUrl, repoFullName)
        .validate();                              // throws MissingFieldsException, or returns

MissingFieldsException extends ResponseStatusException deliberately. This codebase is full of catch (ResponseStatusException e) { throw e; } catch (Exception e) { … } rethrow ladders — RetentionController.requireJwt and InventoryController.requireJwt among them. A brand-new unrelated type would fall into the second arm of every one of those and surface as a 401 or a 500, which is a worse answer than the generic 400 this work replaced.

GlobalApiExceptionHandler#handleMissingFields still wins over the ResponseStatusException handler, because Spring picks the most specific @ExceptionHandler — that is what adds errorCode: "missing_fields" and the details map.

Analyzer

if (rejectMissing(res, requireFields()
  .text('userPrompt', userPrompt, 'describe the issue to fix')
  .text('content', content, 'the source to fix')
  .text('language', language, 'e.g. java, javascript, python')
  .text('fileName', fileName))) {
  return;
}

The third argument is a per-field hint. It travels in details beside the field it describes, never in the shared sentence:

400 {
  "error":        "language and fileName are mandatory",
  "message":      "language and fileName are mandatory",
  "errorCode":    "missing_fields",
  "missingFields": ["language", "fileName"],
  "details": {
    "language": "language is mandatory — e.g. java, javascript, python",
    "fileName": "fileName is mandatory"
  },
  "timestamp": "2026-10-06T08:00:00.000Z"
}

The order is part of the contract

The sentence names the fields in the order the DTO declares them, and it names them in that order every time:

POST /api/v1/gatekeeper/check {}
400  repoUrl, branch, commit and changedFiles are mandatory      ← every request, same order

It did not always. Hibernate Validator answers validate() with a Set<ConstraintViolation> and ConstraintViolationImpl inherits identity hashing, so the set iterated in allocation order — a fresh order on every request. Measured on dev against main-41556bec35a9:

POST /api/v1/gatekeeper/check {}  → commit, changedFiles, repoUrl and branch are mandatory
POST /api/v1/gatekeeper/check {}  → changedFiles, repoUrl, commit and branch are mandatory
POST /api/v1/gatekeeper/check {}  → repoUrl, commit, changedFiles and branch are mandatory
POST /api/v1/gatekeeper/check {}  → branch, repoUrl, changedFiles and commit are mandatory
POST /api/v1/gatekeeper/check {}  → branch, commit, changedFiles and repoUrl are mandatory

Five requests, five orders. Validating one four-field DTO 200 times produced 13 distinct sentences. The details key order moved with it, so no part of the body was stable: a client cannot pin the message, a contract test on it flakes, and two Postman runs of one request disagree about what is wrong.

Three sources were unordered, and all three now sort through one helper:

Path	Unordered source	Handler
@Valid request body	MethodArgumentNotValidException#getBindingResult().getFieldErrors()	AnalyzeExceptionHandler#handleValidationExceptions
@Validated parameters	ConstraintViolationException#getConstraintViolations() (a Set)	GlobalApiExceptionHandler#handleConstraintViolation
Query-string binding	BindException#getFieldErrors()	ApiErrors#firstBindingFailure — “first” has to mean something

Class#getDeclaredFields() is documented as unordered, but it is read from the class file and is constant for a given class within a JVM. Constant is the property that matters; the alphabetical tiebreak makes the order total even for names that are not fields at all (method parameters, nested paths like scanResult.total, which rank with their root).

com.logwrite.validation.FieldOrder

/** Orders by declaration on {@code target}, then alphabetically; never returns ties. */
public static Comparator<String> of(Object target) {
    Map<String, Integer> index = indexOf(target);
    return Comparator.<String>comparingInt(name -> rank(index, name))
            .thenComparing(Comparator.naturalOrder());
}

/** The same order, applied to whatever carries the names (a FieldError, a violation). */
public static <T> List<T> sort(Object target, Iterable<T> items, Function<T, String> name) {
    Comparator<String> order = of(target);
    List<T> sorted = new ArrayList<>();
    items.forEach(sorted::add);
    sorted.sort((a, b) -> order.compare(name.apply(a), name.apply(b)));
    return sorted;
}

private static Map<String, Integer> declared(Class<?> type) {
    Deque<Class<?>> chain = new ArrayDeque<>();
    for (Class<?> c = type; c != null && c != Object.class; c = c.getSuperclass()) {
        chain.addFirst(c);                      // inherited fields first, as the form asks for them
    }
    Map<String, Integer> index = new LinkedHashMap<>();
    for (Class<?> c : chain) {
        for (Field f : c.getDeclaredFields()) {
            if (!f.isSynthetic() && !Modifier.isStatic(f.getModifiers())) {
                index.putIfAbsent(f.getName(), index.size());
            }
        }
    }
    return Map.copyOf(index);                   // cached per class in a ConcurrentHashMap
}

Call sites:

// AnalyzeExceptionHandler — @Valid request bodies
FieldOrder.sort(ex.getBindingResult().getTarget(),
        ex.getBindingResult().getFieldErrors(), FieldError::getField).forEach(error -> { … });

// GlobalApiExceptionHandler — @Validated parameters
FieldOrder.sort(FieldOrder.rootBeanOf(ex.getConstraintViolations()), ex.getConstraintViolations(),
        v -> ApiErrors.leafPath(String.valueOf(v.getPropertyPath()))).forEach(v -> { … });

The test that holds it is FieldOrderIsDeterministicTest, and the assertion that matters is not the order — it is that 200 validations of the same object produce one sentence. An order assertion alone would have passed on any single run of the broken code.

Bean validation

The backend’s @Valid path is the busiest validation surface on the platform, and it is covered without touching 154 DTOs. AnalyzeExceptionHandler rewrites the message of every absence constraint through RequiredFields:

constraint	treated as	message
@NotNull, @NotBlank, @NotEmpty	absent	<field> is mandatory
@Size, @Pattern, @Email, everything else	present but wrong	whatever the DTO declares

OneWordingForAbsenceTest fails the build on an absence constraint that declares its own message, and on five literal unnamed-refusal sentences; the analyzer’s sweep fails on any '… is required' under src/api/routes.

Absent is not the same as wrong

A field that was never sent and a field whose value is unacceptable are different answers, and conflating them sends the caller looking for a supported spelling of a key they never typed. Four endpoints did exactly that before this work:

endpoint	said	meant
POST /api/v1/ai-model/credentials	unknown provider: null	provider is mandatory
POST /api/v1/warden/credentials	unknown provider: null	provider is mandatory
POST /api/v1/agents/test-connection	Unsupported database engine 'null'. Supported: postgres, mysql, sqlite.	dbConnection.engine is mandatory
PUT /api/v1/workspace/{id}/members/{userId}	Invalid role. Must be one of: Viewer, Developer, Maintainer, Admin	role is mandatory

Deliberate exceptions

Not every endpoint that accepts {} is a defect, and these were checked rather than assumed.

• POST /api/v1/demo-requests stays mute (Invalid request.). It is unauthenticated, and naming the field that failed is an aid to whoever is probing it. A silently discarded bot submission returns the accepted envelope byte for byte, for the same reason.
• PUT /api/v1/user/settings, PUT /api/v1/user/avatar, PUT /api/v1/workspace/{id}, PUT /api/v1/retention/policy answer 200 to {} because they are documented partial updates: absent means “leave unchanged”, and RetentionPolicyInput says so in as many words. Making any field mandatory there would break a request shape that works today.
• POST /api/v1/api-keys and POST /api/v1/warden/relay-keys have no mandatory field. The UI sends name only when the user typed one (useConnectProviderWizard.js: if (nm) body.name = nm;), so name is optional by design, not by omission.
• POST /analyze/stream over NDJSON still reports in band. The body is the input stream, so whether a header line exists is only knowable after reading it. The JSON-body branch is validated before the 200 is committed.

Implementing it: two traps

Use getReason(), never getMessage(). MissingFieldsException extends ResponseStatusException, whose getMessage() is <status> "<reason>". The first deployed version of this work used getMessage and 14 endpoints answered "message": "400 BAD_REQUEST \"name and key are mandatory\"". A unit test on the utility cannot see that — the leak is in the handler, so the test has to assert on the ApiError body the handler returns.

Declare absence constraints bare. A message = … on @NotNull/@NotBlank/@NotEmpty is overridden at the edge, so it is dead text that reads as live.

Path and query parameters

Same sentence, different suffix, because the caller needs to know where the value goes:

400  workspaceId is mandatory — it is a Integer request parameter.
400  avatar is mandatory — it is an uploaded file/part.

errorCode stays missing_parameter / missing_part rather than missing_fields: ApiSweepArgumentFiller keys its retry logic on the slug, and the slug is the contract while the prose is not.

A refusal has to arrive on the channel the caller is reading

POST /api/v1/analyzer answers 202 and does its work on a pool. Anything it discovers after that can only be reported over the WebSocket — and a REST or CI client is not listening to one. So where a check runs is part of whether it is a check at all.

Three things were already refused in the right way and one was not:

check	where it runs	caller sees
repository has an owner and a repo name	ScanPreflight.requireScannableRepo — controller	400, before the accept
commit absent	ScanPreflight.defaultCommitHash — controller	filled in, never refused
template belongs to this project	AnalyzerService — the pool	202, then silence
project access / role floor	OwnershipService — controller	403 / 404

AnalyzerService did reject a foreign templateId, with a good message. It simply could not reach the caller. ScanPreflight.requireProjectTemplates now runs the same check in the controller:

POST /api/v1/analyzer  {"projectId":7,"templateId":4242,...}
400  template 4242 is not configured for project 7 — pick a template that belongs
     to this project and scan again.

The list was the worse half

templateIds (multi-template merge) resolved each id with findByIdAndProjectId(id, projectId) and, on a miss, logged a warning and skipped it:

if (cfg == null || cfg.getTemplateRules() == null) {
    log.warn("[analyzer] mergeTemplateRules: config {} for project {} unresolvable — skipping", …);
    continue;                       // ← fewer rules than asked for, and a successful scan
}

A body naming one of its own templates and one of somebody else’s therefore produced a scan that ran fewer rules than asked for and reported success. The coverage figure it published was wrong and nothing in the result said so — the same failure shape as “never truncate silently”. Every id is checked now, primary and secondary, before the 202.

The message is deliberately not an oracle

template 4242 is not configured for project 7 is the same sentence whether 4242 belongs to another workspace or is not a template at all, so the refusal cannot be walked to enumerate ids. ScanTemplateOwnershipTest asserts the two messages are identical once the id is masked — an assertion that only means something because the ids differ.

The service-side check stays as well. It is the one that covers callers arriving through the orchestrator rather than the controller.

Every field, not just the mandatory ones (2026-10-08)

The contract above covers a field that is absent. A public API also has to refuse a field that is present and wrong — a string where an id belongs, a branch the repository does not have, an unknown role, a 300-character name — and to do it in the same single response. Before this round, measured on dev:

request	answered
POST /api/v1/agents-flow/ensure-project {"workspaceId":"abc",…}	500 quoting java.lang.ClassCastException
POST /api/v1/analyzer {"projectId":"abc",…}	The request body is missing or malformed — never named projectId
POST /api/v1/analyzer {"templateId":1.5,…}	read as template 1
POST /api/v1/analyzer {"branchName":"no-such-branch",…}	202, then a clone failure only on the WebSocket
POST /analyze {"files":[null]} (analyzer)	500 (V8 TypeError)

A combination probe across 344 backend endpoints and 29 analyzer routes found 61 crashes and 354 invalid requests accepted with a 2xx in the first 2,631 cases alone.

The sentences

x is mandatory                                   absent (unchanged)
x must be a positive integer — got "abc"         wrong type or range
x must be one of: Viewer, Admin — got "boss"     closed set
x must be at most 255 characters — got 300       length

They compose: absent fields first, in the shared wording, then each wrong field as its own sentence — one round trip names everything.

POST /api/v1/agents-flow/ensure-project {"workspaceId": -4, "source": "boss"}
400 {
  "message":   "repoFullName is mandatory; workspaceId must be a positive integer — got -4; source must be one of: app, oauth, db, database — got \"boss\"",
  "errorCode": "invalid_fields",
  "details": {
    "repoFullName": "repoFullName is mandatory",
    "workspaceId":  "workspaceId must be a positive integer — got -4",
    "source":       "source must be one of: app, oauth, db, database — got \"boss\""
  }
}

errorCode is missing_fields when everything wrong was absent — so the absent-only response is byte-for-byte what it was — and invalid_fields as soon as anything present was wrong. Secrets are never echoed: a field whose name contains password, token, secret, apiKey, otp, credential, attestation or assertion gets the sentence without the — got … part.

Service	Sentences	Typed reader
Backend	com.logwrite.validation.FieldProblems	Body (map-shaped bodies), RequiredFields.check() (typed values)
Analyzer	src/api/validation/fieldProblems.ts	src/api/validation/body.ts

BodyTest and tests/fieldProblems.test.ts carry the same fixtures.

Map-shaped bodies: Body

About sixty handlers took @RequestBody Map<String, Object> and read it with ((Number) body.get("workspaceId")).intValue(). That cast was the validation. Body reads the same keys typed and collecting:

Body in = Body.of(body);
Integer workspaceId = in.id("workspaceId");                 // number or numeric string, > 0
String repo = in.repoFullName("repoFullName", true);        // owner/repository
String source = in.oneOf("source", false, List.of("app", "oauth", "db", "database"));
Long installationId = in.optionalLongId("installationId");
in.validate();                                              // one 400, or carry on

Numeric strings stay valid ids ("3103"), because Jackson already accepts them on every DTO-bound endpoint and the UI sends both.

DTO bodies: Jackson errors name the field

BodyReadProblems reads the path, target type and rejected value out of Jackson’s exception:

condition	message	errorCode
no body	The request body is empty — send a JSON object with the fields this endpoint needs.	malformed_request
not JSON	The request body is not valid JSON — … (line 1, column 14).	malformed_request
wrong shape	The request body must be a JSON object — got a list.	malformed_request
wrong field type	projectId must be an integer — got "abc" (+ details)	type_mismatch

spring.jackson.deserialization.accept-float-as-int=false, so 1.5 is no longer read as 1. A query or path parameter of the wrong type answers the same sentence and slug.

Authenticate before describing the body

@Valid runs during argument binding, before the first line of the handler — so an anonymous {} used to receive the endpoint’s field list. Where that mattered (/analyzer, /generator, /config/analyzerrules (that POST was removed on 2026-10-15 with the policy model), the profile and password routes) the DTO’s own constraints are now checked after authentication (DtoConstraints, AccountFieldChecks.validateDto) in the same pass as the hand-written rules. Field problems on those routes sit in details.<field>; message is unchanged.

Scans and generation: the response says what to do next

A caller holding only the 202 used to need this page to know what came next.

POST /api/v1/analyzer {"projectId":5054,"templateId":156,"branchName":"main"}
202 {
  "accepted": true, "state": "QUEUED", "acceptedAt": "2026-10-08T06:10:00Z",
  "statusUrl": "/api/v1/scan-runs/latest?projectId=5054&since=1791446400000",
  "nextSteps": [
    {"step":1,"method":"GET","url":"/api/v1/scan-runs/latest?projectId=5054&since=…","description":"Poll every 3–5 s …"},
    {"step":2,"method":"GET","url":"/api/v1/scan-results/{scanResultId}","description":"…"},
    {"step":3,"method":"GET","url":"/api/v1/scan-results/{scanResultId}/summary","description":"…"},
    {"step":4,"method":"POST","url":"/api/v1/generator","description":"Optional: … {\"scanId\": <scanResultId>}"}
  ],
  "websocket": {"url":"/ws/stream","finalFrame":"terminal=true; …"}
}

• since — without it, a poll that lands before the pool opens the new run reads the previous run, often COMPLETED with a scanResultId. With it, /scan-runs/latest answers {"state":"QUEUED"} until the new run exists.

• The branch is checked before the 202. The project’s cached branch list first; if the branch is not in it, ls-remote with the token the clone will use. Only a list the remote actually returned can refuse — an unreachable remote lets the scan through, so a branch pushed a minute ago is never refused for a stale cache:

  400  branchName "no-such-branch" does not exist in Deployd-io/demo-flask — it has 12 branches,
       e.g. main, stable, workflow. List them all with
       GET /api/v1/agents-flow/branches/3103?repoFullName=Deployd-io/demo-flask

• The final WebSocket frame states the outcome. It was sent before the run closed, so it arrived at progress 98 with nothing but "END Scan ID:1337". The run is now closed first:

  {"message":"END Scan ID:1338","progress":100.0,"terminal":true,"state":"COMPLETED",
   "scanId":1338,"resultsUrl":"/api/v1/scan-results/1338",
   "resultsSummaryUrl":"/api/v1/scan-results/1338/summary",
   "nextStep":"Scan 1338 is complete. GET /api/v1/scan-results/1338 for the findings, or POST /api/v1/generator {\"scanId\": 1338} to generate the missing logs."}

  The message text is unchanged because the UI matches it. A failure’s final frame carries terminal: true, state: "FAILED".

• POST /api/v1/generator refuses a scan with no checkout or no repository with a 400 before its 202, instead of a 202 that died on the pool.

The API chain and the public-route list

ApiPublicSurface (the default-deny filter’s exemptions) and ApiChainPolicy.PUBLIC_API_PATHS (the Spring chain behind SECURITY_API_CHAIN_ENABLED) were maintained apart and drifted: nine declared exemptions answered 401 on dev, where the chain is on — anonymous password reset, the invitee’s invitation preview, the /experience lead form, AWS Marketplace registration and the webhook service’s PR events among them. Prod has the chain off and was unaffected. ApiChainPolicyTest#everyDeclaredExemptionIsPermitted now holds the two lists together.

Proving it: the combination probe

ops/testing/api_validation/probe.py (MasterControl) reads one spec entry per endpoint and sends every invalid combination it implies: each mandatory field absent / null / "" / whitespace, every 2–4-field combination of absent fields, wrong types for every field, zero, negative and overflowing ids, unknown enum values, malformed emails / URLs / owner/repo / shas / dates, over-long strings and lists, malformed JSON, a JSON array or string body, and malformed path ids. A case fails on a 5xx, on a 2xx for invalid input, or on a 400 that does not name the field at fault. render.py turns the spec and the measured results into paste-ready curls and a Postman collection.

PROBE_JWT=<token> python3 ops/testing/api_validation/probe.py --spec 'ops/testing/api_validation/spec/*.json' --out after.json
python3 ops/testing/api_validation/render.py --spec '…/spec/*.json' --after after.json --outdir out/

It never sends the demo user’s email to /auth/** (the attempt ceilings are per address), never sends a valid body to an endpoint with side effects unless its spec says probe: full, and reads internal secrets from the dev containers without printing them.