Observability

ObjectOS operations need both infrastructure signals and application signals. The framework provides request-id, metrics, error reporter, and audit primitives; the deployment decides where to export them.

Request identity

Every production deployment should propagate or generate a request id:

X-Request-Id

Use it to correlate:

• ingress logs;
• ObjectOS logs;
• database slow queries;
• error reports;
• customer support tickets.

Metrics and errors

The runtime keeps metrics and error reports in-process, and exports them only when you choose an exporter. Track at least:

Request count and duration are emitted as http_requests_total and http_request_duration_ms, labelled by method, route pattern and status, for every inbound request — sign-in and the REST data API included.

Export with OpenTelemetry

Metrics leave the runtime over OTLP. Set OS_OBS_EXPORTER=otlp and OS_OTLP_ENDPOINT (e.g. https://<collector>/otlp). The exporter defaults to noop (zero runtime cost), so export is opt-in — setting an endpoint without OS_OBS_EXPORTER=otlp emits nothing, and otlp without an endpoint warns and falls back to noop. Use OS_OBS_EXPORTER=console or json for local debugging, and OS_OBS_DEPLOYMENT_ENV to tag the deployment (defaults to production). Environment Variables lists the rest.

The runtime does not serve a scrape endpoint of its own. To use Prometheus, point the exporter at an OpenTelemetry Collector and let the collector expose the metrics to Prometheus — or send them to any backend that accepts OTLP, such as Grafana Cloud or Datadog.

A starter alert, assuming your pipeline keeps the runtime's metric names:

groups:
  - name: objectos
    rules:
      - alert: ObjectOS5xxSpike
        expr: |
          sum(rate(http_requests_total{status=~"5.."}[5m]))
            / sum(rate(http_requests_total[5m])) > 0.02
        for: 5m
        annotations:
          summary: "ObjectOS 5xx rate above 2% for 5 minutes"

Alert on http_requests_total{status=~"5.."}, not on http_request_errors_total: that series was retired in ObjectStack 17.2 and reads a flat zero, which looks like a healthy server.

Cold starts are not on that list. The kernel is built once, at startup, and the process reports ready only afterwards — there is no steady-state cache-miss signal to alert on, so alert on readiness instead.

Auth failures are not on it either. The runtime's metric registry covers HTTP, storage, cache, background jobs and the package registry; none of those counts a rejected credential, so there is no series to alert on. Counting 401/403 out of http_requests_total is not a substitute for one: a rejected API key is indistinguishable from a request that carried no credential at all, and both statuses are dominated by ordinary permission denials against users who authenticated perfectly well. Review sign-in activity in Setup's Auth audit view instead, as a holder of view_all_audit_log — platform administrators hold it by default. Without it, sign-outs and sign-ins whose session has since been removed are not served; see Audit Logs.

Audit logs

When the audit capability is enabled, ObjectOS writes audit records to sys_audit_log.

Use audit logs for:

• permission-sensitive changes;
• settings changes;
• user/session investigation;
• integration activity;
• denied access analysis.

For regulated environments, make the audit table append-only at the database or storage layer and define a retention policy.

For the full field reference, preset views, and the sibling diagnostics logs, see Audit Logs.

Setup diagnostics

Setup exposes operational surfaces such as:

• Sessions;
• Audit Logs;
• Notifications;
• Webhook Deliveries when webhooks are enabled;
• System and Security overview dashboards.

These are intended for operators and support engineers, not only developers.