Skip to content
Logrite

Startup Sequence

Why Order Matters

Section titled “Why Order Matters”

Several services depend on OpenSearch being healthy before they can start:

  • Analyzer queries OpenSearch for log context
  • Semantic Search uses OpenSearch’s ML model and k-NN indexes

Starting these before OpenSearch is ready will cause failed connections and degraded functionality.

Section titled “Recommended Startup Order”
1. OpenSearch (wait for green/yellow health)
2. Frontend, Backend, Webhook (independent of OpenSearch)
3. Analyzer, Semantic Search (OpenSearch-dependent)
4. AI Service (depends on Semantic Search + Analyzer)
5. MCP Server (depends on OpenSearch + Backend)
6. Agent Runner (depends on Backend + AI Service)

How services start now

Section titled “How services start now”

There is no start-all and no logrite-manager.js — both drove PM2 on the host destroyed on 2026-08-03. Every service is a docker compose container (project logrite) with a restart policy, so after a host reboot Docker brings them all back at once; nothing orders them. The order above still matters, and it is handled by each dependent service retrying until its dependency answers.

If a service looks wedged after a reboot:

  1. Check OpenSearch first: curl -sk -u "$OS_USER:$OS_PASS" https://127.0.0.1:9200/_cluster/health on the VM (credentials in CREDENTIALS.md).
  2. Then redeploy the wedged service from the console at https://management.logrite.ai, or from the dev VM: curl -sX POST http://127.0.0.1:8090/api/deploy -H 'content-type: application/json' -d '{"service":"<svc>","host":"dev","ref":"main"}'.

The index-template pre-flight that start-all ran is not run at boot any more.

Checking Startup Status

Section titled “Checking Startup Status”
Terminal window
# What is running, and on which host port
docker ps --format '{{.Names}}\t{{.Status}}\t{{.Ports}}'


# Every service: owner, state, health, image (dev VM)
curl -s 'http://127.0.0.1:8090/api/services?host=dev' | python3 -m json.tool


# Health checks — host ports on the VM (dev shown; the analyzer is 9000 on prod)
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:9800/health   # Backend: no open /health — a 302 to /login means it is up
curl http://127.0.0.1:9001/health   # Analyzer
curl http://127.0.0.1:9100/health   # Semantic Search
curl http://127.0.0.1:9300/health   # AI Service
curl http://127.0.0.1:9500/health   # MCP Server
curl -sk -u "$OS_USER:$OS_PASS" https://127.0.0.1:9200/_cluster/health  # OpenSearch (https, self-signed, authenticated)

Reboot-Time Considerations

Section titled “Reboot-Time Considerations”

On a full host reboot, docker.service itself dominates wall-clock time (default systemd config has no DNS shortcut, no IO priority for dockerd, and systemd-tmpfiles-clean competes for the rotational disk). The 2026-05-19 tuning brings the full 21-container stack from ~27 min to ~6:30 (clean reboot) / ~7:45 (after a power outage) on the pre-2026-08-03 host. See Troubleshooting → Slow Reboot / Boot Hangs for the full procedure: daemon.json + extended drop-in + tmpfiles ordering + Coolify healthchecks/mem-limits, the post-reboot verification script (verify-boot.sh), and the per-service timeline table.

Semantic Search Model Load

Section titled “Semantic Search Model Load”

After starting, the ML embedding model may need time to load:

Semantic Search deploys and recovers its own model; opensearch-mgr and the model-health-check.js cron are gone. Check it with:

Terminal window
curl -s https://dev-search.logrite.ai/health    # checks.model: true once deployed