Data Flow

Log Ingestion Flow

sequenceDiagram
  participant App as Application
  participant DP as Data Prepper<br/>:21890/:21891
  participant OS as OpenSearch<br/>:9200
  participant ANA as Analyzer<br/>:9000
  participant SEM as Semantic Search<br/>:9100
  participant AIS as AI Service<br/>:9300
  participant UI as Frontend<br/>:3000

  App->>DP: OTEL logs/traces (gRPC)
  DP->>OS: Bulk index (pipeline processed)
  Note over OS: Grok processing<br/>tenant-suffix indexing

  UI->>AIS: Chat message (SSE)
  AIS->>SEM: semantic_search tool
  SEM->>OS: Neural k-NN query
  OS-->>SEM: Top-K results
  SEM-->>AIS: Ranked results

  AIS->>OS: opensearch_query tool
  OS-->>AIS: Aggregation results

  AIS->>ANA: analyze_logs tool
  ANA->>OS: Query for logs
  OS-->>ANA: Log entries
  ANA-->>AIS: Key findings (Groq analysis)

  AIS-->>UI: SSE stream (content blocks)

Chat Request Flow (AI Service)

sequenceDiagram
  participant UI as Frontend
  participant AIS as AI Service
  participant GR as Groq LLM
  participant TOOL as Tool Executor
  participant DEP as Downstream Services

  UI->>AIS: POST /api/v1/chat {message, conversationId}
  AIS->>AIS: Load conversation history (SQLite)
  AIS->>GR: System prompt + history + user message

  loop Agent Loop (max 10 turns)
    GR-->>AIS: Tool call request
    AIS->>TOOL: Execute tool
    TOOL->>DEP: Call service (OS/Semantic/Backend/Analyzer)
    DEP-->>TOOL: Result
    TOOL-->>AIS: Tool result
    AIS->>GR: Tool result + continue
    AIS-->>UI: SSE: tool_use_start, tool_use_result events
  end

  GR-->>AIS: Final response (text + content blocks)
  AIS-->>UI: SSE: content_delta, content_block, message_end
  AIS->>AIS: Persist message to SQLite

Compliance Scan Flow

sequenceDiagram
  participant UI as Frontend
  participant BE as Backend :8080
  participant ANA as Analyzer :9000
  participant OS as OpenSearch

  UI->>BE: POST /api/v1/scan/initiate {code, workspace}
  BE->>BE: Create scan record (DB)
  BE->>ANA: POST /analyze {scanId, code}

  ANA->>ANA: Apply 27 analysis strategies
  ANA->>OS: Query related log patterns
  OS-->>ANA: Historical context

  ANA-->>BE: Key findings + recommendations
  BE->>BE: Store scan results
  BE->>OS: Index scan summary
  BE-->>UI: WebSocket: scan_complete event

  UI->>BE: GET /api/v1/scan-results/:id
  BE-->>UI: Full scan results

Agent Execution Flow (Proposed)

sequenceDiagram
  participant AR as Agent Runner :9400
  participant BE as Backend :8080
  participant AIS as AI Service :9300
  participant DC as Discord Webhook

  AR->>BE: GET /api/v1/agents?status=active&scheduled=true
  BE-->>AR: List of scheduled agents + configs
  AR->>AR: Register cron jobs

  Note over AR: Cron fires for agent

  AR->>BE: GET /api/v1/agents/:id/context
  BE-->>AR: Tools, permissions, system prompt

  AR->>AIS: POST /api/v1/chat {message, systemPrompt, toolsAllowed}
  AIS-->>AR: SSE stream (tool calls + final response)

  AR->>BE: POST /api/v1/agents/:id/executions {result, logs}
  BE-->>AR: Execution record created

  AR->>AR: Evaluate alert conditions
  AR->>DC: POST Discord webhook (if alert triggered)

OpenSearch Index Strategy

Logrite uses tenant-suffixed indexes for data isolation:

Pattern	Example	Purpose
payment_queries	payment_queries-logrite	Payment performance logs
external_service_calls	—	External API call logs
database_queries	—	DB query logs
exceptions	—	Application exceptions
*_semantic	payment_queries_semantic	k-NN enabled variants
trace-analytics-raw-*	trace-analytics-raw-logrite	OTEL traces
log-analytics-*	log-analytics-logrite	OTEL logs
ctm_test_results	—	Worksoft CTM results
ctm_events	—	Worksoft CTM events