AiHummer Observability
AiHummer’s observability model is OTLP push only. The gateway exports telemetry to an OpenTelemetry endpoint you configure — it does not expose a scrape endpoint. Ship the data to your collector and visualise it with the bundled Grafana dashboards.
[!WARNING] There is no Prometheus
/metricsendpoint and no pprof (/debug/pprof). Observability is OTLP push only — do not configure a scrape job against the gateway; point AiHummer at your OTLP collector instead.
OTLP push
Set a single variable to turn telemetry on:
# gateway.env — export telemetry to your OTLP collector
AIHUMMER_OTEL_ENDPOINT=http://otel-collector:4317With AIHUMMER_OTEL_ENDPOINT set, the gateway pushes telemetry to that
collector. From there, route it to your backend (Tempo, a metrics store, logs)
and into Grafana.
Grafana dashboards
Ready-made Grafana dashboards ship with the release. Import them into your Grafana instance to get the operational views without building panels from scratch.
What to watch
These are the signals that tell you the system is healthy and turns are flowing:
| Signal | Why it matters |
|---|---|
| Turn latency | End-to-end responsiveness of agent turns |
| Error rate | Failing turns / requests — the first sign of trouble |
| Delivery dispositions | Whether replies are actually reaching channels |
| Outbox / DLQ depth | Backlog of undelivered messages; a growing DLQ means delivery is stuck |
A rising outbox or dead-letter-queue (DLQ) depth is the clearest early warning that delivery is backing up — watch it during rollouts and incidents.
System endpoints
Alongside OTLP, the gateway exposes small HTTP endpoints useful for probes, clocks and client diagnostics:
| Method | Endpoint | Purpose |
|---|---|---|
GET | /healthz | Liveness + version |
GET | /readyz | Readiness (checks Postgres; 503 if down) |
GET | /v1/ping | Lightweight reachability check |
GET | /v1/time | Server time |
POST | /v1/client-log | Ingest client-side log events |
The health and readiness probes are covered in detail under systemd & health checks.
Where to next
- Probes and the production pre-flight checklist: systemd & health checks.
- What to watch during a rolling upgrade: Upgrade policy.