Enterprise Public REST API v2

OpsKnight Public REST API v2 (/api/v2/...) provides a versioned, secure, high-performance platform for programmatic incident response, on-call orchestration, infrastructure automation, runbook remediation, and identity management.

The v2 platform runs alongside legacy endpoints (/api/..., /api/v1/..., Events API v2) with zero regressions, sharing core certified domain workflows without duplicate business logic.

1. Architectural Highlights

• Standard Envelopes:
  • Success responses return { "data": T, "pagination"?: { ... }, "requestId": "req_...", "timestamp": "..." }.
  • Error responses strictly adhere to RFC 9457 Problem Details with content type application/problem+json.
• OpenAPI 3.1:
  • Machine-readable specification served at GET /api/v2/openapi.json as the bare OpenAPI document (openapi, info, paths at the root, no response envelope), so validators and SDK generators can consume the URL directly. Every operation has a stable operationId, typed responses, and its required scope in x-required-scope.
• High Performance & Security:
  • Instantaneous Token Revocation: Direct database credential verification without stale token caching, ensuring credential revokes take immediate effect across all replicas.
  • Distributed Fixed-Window Rate Limiting: Distributed transactional counters in PostgreSQL enforce 60-second fixed-window limits with atomic upserts and automatic TTL cleanup.
  • Tamper-Evident Keyset Cursor Pagination: Cryptographically signed HMAC-SHA256 cursor tokens protecting pagination state from client tampering while avoiding heavy OFFSET table scans.
• Transactional Idempotency:
  • Safe mutation retries via the Idempotency-Key header; see the retry guarantee for each operation in section 9. Replayed responses include Idempotency-Replayed: true.
• Optimistic Concurrency:
  • Detail reads of teams, services, schedules, integrations and escalation policies return a strong ETag ("<revision>"), backed by a revision counter that the database increments on every update from any path (REST, UI or workers). The same value is in the response body as revision. Send it back as If-Match and a stale write fails with 412 Precondition Failed instead of overwriting a newer change. A weak validator (W/"...") never matches and is also answered with 412. If-Match is optional for those existing configuration endpoints; without it the last write wins. New runbook draft edits and publication require it: a missing precondition returns 428, and a stale revision returns 412. Read the draft to obtain its ETag.
• Granular RBAC:
  • Fine-grained API scopes enforce least-privilege access; the full list is in section 7. Scopes that grant nothing yet are marked as not issuable.

2. Authentication & Credentials

All endpoints accept authentication via HTTP Bearer token or X-API-Key header:

Authorization: Bearer ok_sa_live_4a8f9b2c...

or

X-API-Key: ok_live_3d7e1a...

Credential Types

1. User API Keys: Personal access tokens associated with a human responder or administrator. Inherits the user's role-based access control and assigned teams.
2. Service Accounts: Dedicated machine identities created specifically for CI/CD, Terraform, CLI, or ChatOps bots (identified by a reserved @serviceaccount.opsknight.local address; they cannot sign in to the web UI). Service accounts own scoped tokens. Every token needs an explicit lifetime: expiresInDays, or neverExpires: true (which a deployment can forbid with API_V2_ALLOW_NON_EXPIRING_TOKENS=false, and cap with API_V2_TOKEN_MAX_DAYS).

Service-account token lists are bounded: list and detail responses return the newest tokens (25 in the list, 100 in the detail view) with a per-token status (ACTIVE, EXPIRED, REVOKED), plus activeTokensCount, tokensTotal and tokensTruncated computed in the database. A token cannot be issued for a disabled service account (409 SERVICE_ACCOUNT_DISABLED).

Keys created before 2.0, and the other API surfaces

Keys issued by OpsKnight 1.x keep working and are transparently re-hashed on first use. Every path that accepts an API key (/api/v2, /api/v1/response-policy, the older /api/* routes and status-page API tokens) applies the same protections before any expensive work: malformed tokens are rejected without a database lookup, and the CPU- and memory-hard 1.x verification runs under a shared per-address budget and a per-process concurrency cap. On /api/v1/response-policy a throttled attempt is answered with 429 and Retry-After; the older /api/* routes keep their historical 401. An invalid key on those routes now returns the accurate message "The API key is missing or invalid." (code API_KEY_INVALID) instead of a permission message.

/api/v1/service-objectives is session-authenticated (signed-in users), not an API-key API. GET .../{id}/versions is keyset-paginated: limit (default 100, max 500) and beforeVersion, with the next value in the X-Next-Before-Version header; the history endpoint returns the newest 200 versions and sets versionsTruncated when there are more. /api/v1/response-policy requires an API key with response-policy:read or response-policy:write held by an administrator.

Service Account Token Prefixes

• Production: ok_sa_live_<base64url>
• Development/Test: ok_sa_test_<base64url>

3. Rate Limiting Tiers

Requests are rate limited per API key across three tiers per 60-second fixed window:

Rate Limit Response Headers

• X-RateLimit-Limit: Maximum requests permitted in current window.
• X-RateLimit-Remaining: Remaining request quota.
• X-RateLimit-Reset: Unix timestamp (seconds) when quota resets.
• Retry-After: Seconds to wait before retrying (sent on every 429 Too Many Requests, including authentication throttling).

Repeated failed authentication from one client address is throttled with 429 as well. This limit is shared across all replicas through PostgreSQL. Requests whose client address cannot be determined (a direct deployment with no trusted proxy) are not subject to a per-address quota, because that would let one client lock out all others; they are instead bounded by a limit on concurrent token verifications (API_V2_UNIDENTIFIED_AUTH_CONCURRENCY, default 16; a brief burst waits up to API_V2_UNIDENTIFIED_AUTH_QUEUE_MS, default 250 ms, before being answered with 429) and a stricter budget for legacy 1.x token verification (API_V2_LEGACY_UNIDENTIFIED_LIMIT, default 30 per minute).

An address that exceeds its failed-authentication budget is not refused outright, because several integrations can share one NAT address: it keeps a small concurrent-verification allowance (API_V2_OVER_BUDGET_IP_CONCURRENCY, default 4). A credential that verified successfully on a replica recently is admitted through a separate priority lane (API_V2_PRIORITY_AUTH_CONCURRENCY, default 64), so a valid integration keeps working while another client floods invalid tokens. The lane is only a scheduling hint: every request is still checked against the database, so revocation stays immediate, and an unknown token can never use it. These limits are per process; size them together with your database connection pool and replica count.

4. Keyset Cursor Pagination

List endpoints support deterministic keyset cursor navigation sorted by createdAt desc, id desc:

GET /api/v2/incidents?limit=25&cursor=ZXlKaWRP...

Every collection that can grow is paginated on the server, including incident notes, schedule overrides, status-page announcements and service accounts. Schedule overrides keep their established order, latest start first, and page on (start, id).

Query Parameters

• limit (integer, default: 20, max: 100): Page size.
• cursor (string, optional): Base64url opaque cursor returned from the previous page's pagination.nextCursor.

Response Structure

{
  "data": [ ... ],
  "pagination": {
    "limit": 25,
    "nextCursor": "ZXlKaWRP...",
    "hasMore": true,
    "total": 142
  },
  "requestId": "req_1728822000_abc",
  "timestamp": "2026-10-13T12:00:00.000Z"
}

5. RFC 9457 Error Contract

When a request fails, /api/v2 returns standard application/problem+json:

{
  "type": "https://opsknight.com/errors/api-scope-required",
  "title": "API Scope Required",
  "status": 403,
  "detail": "This operation requires the API scope 'incidents:write'.",
  "instance": "/api/v2/incidents",
  "code": "API_SCOPE_REQUIRED",
  "requestId": "req_1728822000_abc",
  "timestamp": "2026-10-13T12:00:00.000Z",
  "retryable": false,
  "fields": []
}

Validation and unsupported input

• Invalid enum filters (for example GET /users?role=SUPERUSER, GET /notifications?channel=FAX) return 400 VALIDATION_FAILED, never a server error.
• A field the API cannot store is rejected rather than accepted and dropped: description on status pages and impact on postmortems (postmortem impact is structured data managed in the product UI) return 400 VALIDATION_FAILED naming the field.
• Internal errors never expose diagnostic meta; only client-actionable errors (rate limits, validation, conflicts) include it.

6. Supported API v2 Endpoints

This list is generated from the same registry that produces the OpenAPI document, so it describes the registered operations in this release. The capability catalog and implementation tracker separately record remaining work and certification gates. If an operation is not listed here, it is not available. See section 8.

Settings

• GET /api/v2/settings/chatops (getChatOpsConfiguration, scope chatops:read) — Read global ChatOps and incident collaboration policy.
• PUT /api/v2/settings/chatops (updateChatOpsConfiguration, scope chatops:write) — Replace global ChatOps policy. Retries: required.
• GET /api/v2/settings/runbooks (getRunbookSettings, scope runbook-settings:read) — getRunbookSettings.
• PATCH /api/v2/settings/runbooks (updateRunbookSettings, scope runbook-settings:write) — updateRunbookSettings. Retries: required.
• GET /api/v2/settings/automation (getAutomationSettings, scope automation-settings:read) — Read global automation settings.
• PATCH /api/v2/settings/automation (updateAutomationSettings, scope automation-settings:write) — Update global automation settings. Retries: required.

Incidents

• GET /api/v2/custom-fields (listCustomFieldDefinitions, scope custom-fields:read) — Page through incident custom-field definitions and usage counts.
• POST /api/v2/custom-fields (createCustomFieldDefinition, scope custom-fields:write) — Create a validated incident custom-field definition. Retries: required.
• GET /api/v2/custom-fields/{id} (getCustomFieldDefinition, scope custom-fields:read) — Read one custom-field definition and exact revision.
• PATCH /api/v2/custom-fields/{id} (updateCustomFieldDefinition, scope custom-fields:write) — Update a custom-field definition with exact revision. Retries: required.
• DELETE /api/v2/custom-fields/{id} (deleteCustomFieldDefinition, scope custom-fields:write) — Delete an unused custom-field definition. Retries: required.
• POST /api/v2/audit-exports (createAuditExport, scope audit:export) — Request a bounded redacted audit evidence export. Retries: required.
• GET /api/v2/audit-exports/{id} (getAuditExport, scope audit:export) — Read owned audit export job status and checksum.
• GET /api/v2/audit-exports/{id}/download (downloadAuditExport, scope audit:export) — Download and verify owned audit evidence.
• GET /api/v2/notification-providers (listNotificationProviders, scope notification-providers:read) — List safe notification provider configuration metadata.
• GET /api/v2/notification-providers/{provider} (getNotificationProvider, scope notification-providers:read) — Read one provider configuration and its revision.
• PUT /api/v2/notification-providers/{provider} (configureNotificationProvider, scope notification-providers:write) — Create or replace a notification provider configuration. Retries: required.
• GET /api/v2/notification-providers/{provider}/diagnostics (getNotificationProviderDiagnostics, scope notification-providers:read) — Inspect safe provider readiness and recent delivery outcomes.
• POST /api/v2/notification-providers/{provider}/test (testNotificationProvider, scope notification-providers:test) — Queue an idempotent provider test notification. Retries: required.
• POST /api/v2/notification-providers/web-push/rotate (rotateWebPushKeys, scope notification-providers:write) — Rotate Web Push VAPID keys. Retries: required.
• GET /api/v2/users/me/notification-preferences (getMyNotificationPreferences, scope notification-preferences:read) — Read the credential owner’s notification preferences.
• PATCH /api/v2/users/me/notification-preferences (updateMyNotificationPreferences, scope notification-preferences:write) — Update the credential owner’s notification preferences. Retries: required.
• GET /api/v2/users/invitation-policy (getUserInvitationPolicy, scope users:read) — Inspect safe invitation policy and active provider versions.
• POST /api/v2/users/invitations (createUserInvitation, scope users:write) — Create an invited user and deliver or disclose its invitation. Retries: required.
• GET /api/v2/users/{id}/invitation (getUserInvitation, scope users:read) — Inspect invitation generation, expiry and safe delivery status.
• POST /api/v2/users/{id}/invitation (renewUserInvitation, scope users:write) — Renew a reviewed pending invitation and invalidate its older generation. Retries: required.
• GET /api/v2/users/{id}/administration (getUserAdministration, scope users:read) — Read administrator-only user lifecycle state.
• GET /api/v2/users/{id}/dependencies (getUserDependencies, scope users:read) — Inspect bounded user lifecycle dependency counts.
• PATCH /api/v2/users/{id}/administration (administerUser, scope users:write) — Change user role or lifecycle, or revoke sessions. Retries: required.
• GET /api/v2/incidents/{id}/watchers (listIncidentWatchers, scope incidents:read) — List incident watchers.
• GET /api/v2/incidents/{id}/tags (listIncidentTags, scope incidents:read) — List incident tags.
• GET /api/v2/incidents/{id}/custom-fields (listIncidentCustomFields, scope incidents:read) — List incident custom-field values.
• GET /api/v2/incidents/templates (listIncidentTemplates, scope incidents:read) — List accessible incident templates.
• POST /api/v2/incidents/templates (createIncidentTemplate, scope incidents:write) — Create an incident template. Retries: required.
• GET /api/v2/incidents/templates/{templateId} (getIncidentTemplate, scope incidents:read) — Read an accessible incident template.
• PATCH /api/v2/incidents/templates/{templateId} (updateIncidentTemplate, scope incidents:write) — Update an owned incident template. Retries: required.
• DELETE /api/v2/incidents/templates/{templateId} (deleteIncidentTemplate, scope incidents:write) — Delete an owned incident template. Retries: required.
• GET /api/v2/incidents/{id}/custom-fields/{fieldId} (getIncidentCustomField, scope incidents:read) — Read an incident custom-field value.
• PUT /api/v2/incidents/{id}/custom-fields/{fieldId} (setIncidentCustomField, scope incidents:write) — Set an incident custom-field value. Retries: required.
• POST /api/v2/incidents/{id}/tags (addIncidentTag, scope incidents:write) — Add a tag to an incident. Retries: required.
• DELETE /api/v2/incidents/{id}/tags/{tagId} (removeIncidentTag, scope incidents:write) — Remove a tag from an incident. Retries: required.
• GET /api/v2/incidents/{id}/routing-decision (getIncidentRoutingDecision, scope automation:read) — Read the redacted routing decision for an accessible incident.
• POST /api/v2/incidents/{id}/watchers (setIncidentWatcher, scope incidents:write) — Add a watcher or change a watcher role. Retries: required.
• DELETE /api/v2/incidents/{id}/watchers/{watcherId} (removeIncidentWatcher, scope incidents:write) — Remove a watcher from this incident. Retries: required.
• POST /api/v2/incidents/{id}/unsuppress (unsuppressIncident, scope incidents:write) — unsuppress an incident. Retries: required.
• POST /api/v2/incidents/{id}/suppress (suppressIncident, scope incidents:write) — suppress an incident. Retries: required.
• POST /api/v2/incidents/{id}/unsnooze (unsnoozeIncident, scope incidents:write) — unsnooze an incident. Retries: required.
• POST /api/v2/incidents/{id}/snooze (snoozeIncident, scope incidents:write) — snooze an incident. Retries: required.
• POST /api/v2/incidents/{id}/unacknowledge (unacknowledgeIncident, scope incidents:write) — unacknowledge an incident. Retries: required.
• POST /api/v2/incidents/{id}/reopen (reopenIncident, scope incidents:write) — reopen an incident. Retries: required.
• POST /api/v2/incidents/{id}/acknowledge (acknowledgeIncident, scope incidents:write) — Acknowledge incident via certified domain lifecycle. Retries: supported.
• POST /api/v2/incidents/{id}/assign (assignIncident, scope incidents:write) — Assign incident to a user or team. Retries: supported.
• GET /api/v2/incidents/{id}/notes (listIncidentNotes, scope incidents:read) — List incident notes.
• POST /api/v2/incidents/{id}/notes (addIncidentNote, scope incidents:write) — Add note to incident. Retries: supported.
• POST /api/v2/incidents/{id}/resolve (resolveIncident, scope incidents:write) — Resolve incident via certified domain lifecycle. Retries: supported.
• GET /api/v2/incidents/{id} (getIncident, scope incidents:read) — Get incident details.
• PATCH /api/v2/incidents/{id} (updateIncident, scope incidents:write) — Update incident metadata atomically. Retries: supported.
• GET /api/v2/incidents/{id}/timeline (getIncidentTimeline, scope incidents:read) — List incident timeline events with cursor pagination.
• GET /api/v2/incidents (listIncidents, scope incidents:read) — List incidents with cursor pagination.
• POST /api/v2/incidents (createIncident, scope incidents:write) — Create incident with idempotency. Retries: supported.
• GET /api/v2/integrations/types (listIntegrationTypes, scope integrations:read) — List safe supported inbound provider types and configuration capabilities.
• GET /api/v2/integrations/{id}/credentials (getIntegrationCredentialState, scope integrations:read) — Read signing credential state without secret material.
• POST /api/v2/integrations/{id}/credentials (rotateIntegrationCredentials, scope integrations:credentials:write) — Rotate an integration signing secret with bounded overlap. Retries: required.
• POST /api/v2/integrations/{id}/test (validateIntegrationPayload, scope integrations:test) — Validate a bounded sample payload without ingesting an event. Retries: safe to repeat.
• GET /api/v2/integrations/{id}/diagnostics (getIntegrationDiagnostics, scope integrations:read) — Read bounded integration delivery health and safe diagnostics.
• POST /api/v2/integrations/{id}/validate (validateIntegrationConfiguration, scope integrations:test) — Validate stored integration configuration without activation. Retries: safe to repeat.
• GET /api/v2/integrations/{id}/usage (getIntegrationUsage, scope integrations:read) — Inspect service, delivery, observation and enabled runbook usage.
• DELETE /api/v2/integrations/{id} (deleteIntegration, scope integrations:delete) — Disable and safely delete an integration when no deliveries are in flight. Retries: required.
• GET /api/v2/notifications/{id}/attempts (listNotificationAttempts, scope notifications:read) — List redacted delivery attempts for a notification.
• GET /api/v2/notifications/{id}/feedback (getNotificationFeedback, scope notifications:read) — Read bounded provider feedback correlated with a notification.
• GET /api/v2/incidents/{id}/response-activity (getIncidentResponseActivity, scope incidents:read) — Read bounded incident response activity across alerting, routing, paging and remediation.
• GET /api/v2/incidents/{id}/war-rooms (listIncidentWarRooms, scope incidents:read) — List persisted War Room state for an accessible incident.
• GET /api/v2/incidents/{id}/alerts (listIncidentAlerts, scope incidents:read) — List redacted alerts linked to an incident.
• GET /api/v2/incidents/{id}/runbook-executions (listIncidentRunbookExecutions, scope incidents:read) — List incident Runbook execution metadata.
• GET /api/v2/incidents/{id}/notification-attempts (listIncidentNotificationAttempts, scope incidents:read) — List incident notification delivery attempts.
• GET /api/v2/incidents/{id}/available-actions (getIncidentAvailableActions, scope incidents:read) — Discover incident actions permitted for this credential and current state.
• GET /api/v2/incidents/{id}/related-incidents (listRelatedIncidents, scope incidents:read) — Find accessible incidents linked by recorded alert correlation evidence.
• GET /api/v2/operational-events (listOperationalEvents, scope incidents:read) — List incident operational events visible to this credential.
• POST /api/v2/escalation-policies/{id}/preview (previewEscalationPolicy, scope escalation-policies:read) — Preview escalation conditions and recipients without paging. Retries: safe to repeat.
• GET /api/v2/postmortems/{id}/versions (listPostmortemVersions, scope postmortems:read) — List retained postmortem revisions.
• GET /api/v2/postmortems/{id}/action-items (listPostmortemActionItems, scope postmortems:read) — Read corrective actions from the same records as the postmortem UI.
• PUT /api/v2/postmortems/{id}/action-items (replacePostmortemActionItems, scope postmortems:write) — Replace all corrective actions using an exact postmortem revision. Retries: required.
• PATCH /api/v2/postmortems/{id}/action-items/{itemId} (updatePostmortemActionItem, scope postmortems:write) — Update one corrective action. Retries: required.
• DELETE /api/v2/postmortems/{id}/action-items/{itemId} (deletePostmortemActionItem, scope postmortems:write) — Delete one corrective action. Retries: required.
• POST /api/v2/postmortems/{id}/publish (publishPostmortem, scope postmortems:write) — Publish a reviewed postmortem with explicit public visibility. Retries: required.
• GET /api/v2/postmortems/{id}/contributors (listPostmortemContributors, scope postmortems:read) — List postmortem collaborators and roles.
• PUT /api/v2/postmortems/{id}/contributors (replacePostmortemContributors, scope postmortems:write) — Replace the reviewed collaborator roster. Retries: required.

Teams

• GET /api/v2/teams/{id}/activity (listTeamActivity, scope teams:read) — Read bounded team administrative activity.
• GET /api/v2/teams/{id}/services (listTeamServices, scope teams:read) — List services currently managed by the team.
• PUT /api/v2/teams/{id}/services/{serviceId} (assignTeamService, scope teams:write) — Assign a service to a team. Retries: required.
• GET /api/v2/teams/{id}/members (listTeamMembers, scope teams:read) — List team memberships.
• PUT /api/v2/teams/{id}/members/{userId} (setTeamMember, scope teams:write) — Add or update a team member. Retries: required.
• DELETE /api/v2/teams/{id}/members/{userId} (removeTeamMember, scope teams:write) — Remove a team member and revoke sessions. Retries: required.
• GET /api/v2/teams/{id}/lead (getTeamLead, scope teams:read) — Read the team lead and revision.
• PUT /api/v2/teams/{id}/lead (setTeamLead, scope teams:write) — Designate or clear the team lead. Retries: required.
• GET /api/v2/teams/{id} (getTeam, scope teams:read) — Get team details with member and service relations.
• PATCH /api/v2/teams/{id} (updateTeam, scope teams:write) — Update team name or description. Retries: safe to repeat.
• DELETE /api/v2/teams/{id} (deleteTeam, scope teams:write) — Delete team (safely checking associated services and members). Retries: safe to repeat.
• GET /api/v2/teams (listTeams, scope teams:read) — List teams with cursor pagination.
• POST /api/v2/teams (createTeam, scope teams:write) — Create a new team. Retries: supported.

Runbooks

• GET /api/v2/runbooks/signing-identities (listRunbookSigningIdentities, scope runbook-signing:read) — listRunbookSigningIdentities.
• POST /api/v2/runbooks/signing-identities (stageRunbookSigningIdentity, scope runbook-signing:write) — stageRunbookSigningIdentity. Retries: required.
• POST /api/v2/runbooks/signing-identities/{id}/activate (activateRunbookSigningIdentity, scope runbook-signing:write) — activateRunbookSigningIdentity. Retries: required.
• POST /api/v2/runbooks/signing-identities/{id}/retire (retireRunbookSigningIdentity, scope runbook-signing:write) — retireRunbookSigningIdentity. Retries: required.
• GET /api/v2/runbooks/secrets/{id}/usage (listRunbookSecretUsage, scope runbook-secrets:read) — List secret-reference dependencies without values or executable configuration.
• POST /api/v2/runbooks/agent-pools/{id}/retire (retireRunbookAgentPool, scope runbook-agent-pools:write) — Retire a pool while preserving execution history. Retries: required.
• GET /api/v2/runbooks/agent-pools (listRunbookAgentPools, scope runbook-agents:read) — listRunbookAgentPools.
• POST /api/v2/runbooks/agent-pools (createRunbookAgentPool, scope runbook-agent-pools:write) — createRunbookAgentPool. Retries: required.
• GET /api/v2/runbooks/agent-pools/{id} (getRunbookAgentPool, scope runbook-agents:read) — getRunbookAgentPool.
• PATCH /api/v2/runbooks/agent-pools/{id} (updateRunbookAgentPool, scope runbook-agent-pools:write) — updateRunbookAgentPool. Retries: required.
• DELETE /api/v2/runbooks/agent-pools/{id} (deleteRunbookAgentPool, scope runbook-agent-pools:write) — deleteRunbookAgentPool. Retries: required.
• GET /api/v2/runbooks/agent-pools/{id}/deletion-check (checkRunbookAgentPoolDeletion, scope runbook-agents:read) — checkRunbookAgentPoolDeletion.
• GET /api/v2/runbooks/agent-pools/{id}/members (listRunbookAgentPoolMembers, scope runbook-agents:read) — listRunbookAgentPoolMembers.
• PUT /api/v2/runbooks/agent-pools/{id}/members/{agentId} (setRunbookAgentPoolMember, scope runbook-agent-pools:write) — setRunbookAgentPoolMember. Retries: required.
• DELETE /api/v2/runbooks/agent-pools/{id}/members/{agentId} (removeRunbookAgentPoolMember, scope runbook-agent-pools:write) — removeRunbookAgentPoolMember. Retries: required.
• GET /api/v2/runbooks/agents/{id}/configuration (getRunbookAgentConfiguration, scope runbook-agents:read) — getRunbookAgentConfiguration.
• PATCH /api/v2/runbooks/agents/{id}/configuration (updateRunbookAgentConfiguration, scope runbook-agents:write) — updateRunbookAgentConfiguration. Retries: required.
• GET /api/v2/runbooks/secrets (listRunbookSecrets, scope runbook-secrets:read) — listRunbookSecrets.
• POST /api/v2/runbooks/secrets (createRunbookSecret, scope runbook-secrets:write) — createRunbookSecret. Retries: required.
• GET /api/v2/runbooks/secrets/{id} (getRunbookSecret, scope runbook-secrets:read) — getRunbookSecret.
• POST /api/v2/runbooks/secrets/{id}/rotate (rotateRunbookSecret, scope runbook-secrets:write) — rotateRunbookSecret. Retries: required.
• GET /api/v2/runbooks/secrets/{id}/grants (listRunbookSecretGrants, scope runbook-secrets:read) — listRunbookSecretGrants.
• POST /api/v2/runbooks/secrets/{id}/grants (grantRunbookSecret, scope runbook-secrets:grant) — grantRunbookSecret. Retries: required.
• DELETE /api/v2/runbooks/secrets/{id}/grants/{grantId} (revokeRunbookSecretGrant, scope runbook-secrets:grant) — revokeRunbookSecretGrant. Retries: required.
• GET /api/v2/runbooks/agents (listRunbookAgents, scope runbook-agents:read) — List privileged Agent fleet health.
• GET /api/v2/runbooks/agents/{id} (getRunbookAgent, scope runbook-agents:read) — Read safe Agent fleet details.
• GET /api/v2/runbooks/agents/{id}/policy-status (getRunbookAgentPolicyStatus, scope runbook-agents:read) — Read reported Agent policy and capability status.
• POST /api/v2/runbooks/agent-enrollments (createRunbookAgentEnrollment, scope runbook-agents:enroll) — Issue one-time Agent enrollment. Retries: required.
• GET /api/v2/runbooks/agent-enrollments/{id} (getRunbookAgentEnrollment, scope runbook-agents:read) — Read enrollment status without token disclosure.
• POST /api/v2/runbooks/agents/{id}/revoke (revokeRunbookAgent, scope runbook-agents:revoke) — Revoke an Agent identity. Retries: required.
• GET /api/v2/runbooks/executions/{id}/steps (listRunbookExecutionSteps, scope runbooks:read) — Read bounded execution steps and eligible pending approval plans.
• GET /api/v2/runbooks/executions (listRunbookExecutions, scope runbooks:read) — List bounded Runbook executions for an accessible incident.
• GET /api/v2/runbooks/executions/{id}/evidence (listRunbookExecutionEvidence, scope runbooks:read) — Read redacted bounded execution attempt evidence.
• GET /api/v2/runbooks/executions/{id}/events (listRunbookExecutionEvents, scope runbooks:read) — Read bounded execution business-event metadata.
• POST /api/v2/runbooks/executions/{id}/steps/{stepId}/approve (approveRunbookExecutionStep, scope runbooks:approve) — Approve the exact reviewed waiting step. Retries: required.
• POST /api/v2/runbooks/executions/{id}/steps/{stepId}/reject (rejectRunbookExecutionStep, scope runbooks:approve) — Reject the exact reviewed waiting step and request cancellation. Retries: required.
• POST /api/v2/runbooks/executions/{id}/rerun-preview (previewRunbookExecutionRerun, scope runbooks:execute) — Preview a finished execution against current rules and published version. Retries: safe to repeat.
• POST /api/v2/runbooks/executions/{id}/rerun (rerunRunbookExecution, scope runbooks:execute) — Start a reviewed rerun of a finished execution. Retries: required.
• GET /api/v2/runbooks/{id}/versions/{versionId}/diff (compareRunbookVersions, scope runbooks:write) — Compare two versions with redacted step summaries and bounded service-rule impact.
• GET /api/v2/runbooks/{id}/metadata (getRunbookMetadata, scope runbooks:write) — Read privileged metadata and revision, including unpublished or archived runbooks.
• PATCH /api/v2/runbooks/{id} (updateRunbookMetadata, scope runbooks:write) — Update runbook metadata under its concrete revision. Retries: required.
• POST /api/v2/runbooks/{id}/archive (archiveRunbook, scope runbooks:write) — Archive a runbook, disable bindings and dismiss outstanding suggestions. Retries: required.
• POST /api/v2/runbooks/{id}/restore (restoreRunbook, scope runbooks:write) — Restore an archived runbook while retaining disabled bindings. Retries: required.
• GET /api/v2/runbooks/{id}/delete-check (checkRunbookDeletion, scope runbooks:write) — Inspect references preventing permanent deletion.
• DELETE /api/v2/runbooks/{id} (deleteRunbook, scope runbooks:write) — Delete a never-published draft without executions or bindings. Retries: required.
• POST /api/v2/runbooks/{id}/duplicate (duplicateRunbook, scope runbooks:write) — Duplicate the reviewed runbook into an independent editable draft. Retries: required.
• POST /api/v2/runbooks/{id}/executions/preview (previewRunbookExecution, scope runbooks:execute) — Preview an authorized execution without starting work. Retries: safe to repeat.
• POST /api/v2/runbooks/executions/{id}/cancel (cancelRunbookExecution, scope runbooks:execute) — Request cancellation; agent stop is asynchronous. Retries: required.
• POST /api/v2/runbooks (createRunbook, scope runbooks:write) — Create a runbook with an initial draft. Retries: required.
• GET /api/v2/runbooks/{id}/versions (listRunbookVersions, scope runbooks:write) — List redacted version summaries.
• POST /api/v2/runbooks/{id}/drafts (createRunbookDraft, scope runbooks:write) — Clone a version to a draft. Retries: required.
• GET /api/v2/runbooks/{id}/drafts/{versionId} (getRunbookDraft, scope runbooks:write) — Read an editable draft with hidden configuration and its revision.
• PATCH /api/v2/runbooks/{id}/drafts/{versionId} (updateRunbookDraft, scope runbooks:write) — Replace a draft definition with revision protection. Retries: required.
• POST /api/v2/runbooks/{id}/drafts/{versionId}/validate (validateRunbookDraft, scope runbooks:write) — Validate a saved draft without executing it. Retries: safe to repeat.
• POST /api/v2/runbooks/{id}/drafts/{versionId}/publish (publishRunbookDraft, scope runbooks:publish) — Publish the validated saved draft. Retries: required.
• POST /api/v2/runbooks/{id}/execute (executeRunbook, scope runbooks:execute) — Trigger safe runbook execution with idempotency. Retries: required.
• GET /api/v2/runbooks/{id} (getRunbook, scope runbooks:read) — Get runbook summary and step summaries (step configuration is never returned).
• GET /api/v2/runbooks/executions/{id} (getRunbookExecution, scope runbooks:read) — Get runbook execution status and step results.
• GET /api/v2/runbooks (listRunbooks, scope runbooks:read) — List available runbooks.

Service Objectives

• GET /api/v2/service-objectives/catalog (getServiceObjectiveCatalog, scope service-objectives:read) — Inspect the supported measurement, window and immutable-history contract.
• GET /api/v2/service-objectives (listServiceObjectives, scope service-objectives:read) — List active authorized objective definitions with bounded keyset pagination.
• POST /api/v2/service-objectives (createServiceObjective, scope service-objectives:write) — Create an objective after explicitly reviewing the active service/metric slot. Retries: required.
• GET /api/v2/service-objectives/{id} (getServiceObjective, scope service-objectives:read) — Read the latest objective version, including an archived lineage, with a strong ETag.
• PATCH /api/v2/service-objectives/{id} (updateServiceObjective, scope service-objectives:write) — Create a new immutable objective version using a concrete revision. Retries: required.
• POST /api/v2/service-objectives/{id}/archive (archiveServiceObjective, scope service-objectives:write) — Retire the active version while retaining all definitions and snapshots. Retries: required.
• GET /api/v2/service-objectives/{id}/versions (listServiceObjectiveVersions, scope service-objectives:read) — Page through every immutable definition version.
• GET /api/v2/service-objectives/{id}/snapshots (listServiceObjectiveSnapshots, scope service-objectives:read) — Read authorized daily scheduler snapshots without precision loss.
• GET /api/v2/service-objectives/{id}/history (getServiceObjectiveHistory, scope service-objectives:read) — Read separately typed objective or legacy incident-SLA history.
• POST /api/v2/service-objectives/{id}/evaluate (evaluateServiceObjective, scope service-objectives:evaluate) — Queue a bounded evaluation of an explicitly reviewed immutable version. Retries: required.
• GET /api/v2/service-objectives/{id}/evaluations/{evaluationId} (getServiceObjectiveEvaluation, scope service-objectives:read) — Inspect an authorized evaluation job and its recorded outcome.

Status Pages

• GET /api/v2/status-pages/{id}/publication (getStatusPagePublication, scope status-pages:read) — Inspect current persisted publication state separately from configuration.
• POST /api/v2/status-pages/{id}/publication/retry (retryStatusPagePublication, scope status-pages:write) — Queue durable publication reconciliation without loosening privacy withdrawal. Retries: required.
• GET /api/v2/status-pages/{id}/publication/jobs/{jobId} (getStatusPagePublicationJob, scope status-pages:read) — Inspect a page-owned publication retry job without internal queue payloads.
• GET /api/v2/status-pages/{id}/services (listStatusPageServices, scope status-pages:read) — List configured service components with cursor pagination and a page configuration ETag.
• PUT /api/v2/status-pages/{id}/services (setStatusPageServices, scope status-pages:write) — Replace reviewed service component membership and presentation. Retries: required.
• PATCH /api/v2/status-pages/{id}/services/{serviceId} (updateStatusPageService, scope status-pages:write) — Update a selected service component without changing other memberships. Retries: required.
• GET /api/v2/status-pages/{id}/appearance (getStatusPageAppearance, scope status-pages:read) — Read status-page appearance with a strong configuration revision.
• PATCH /api/v2/status-pages/{id}/appearance (updateStatusPageAppearance, scope status-pages:write) — Update reviewed status-page appearance through the canonical publication command. Retries: required.
• GET /api/v2/status-pages/{id}/privacy (getStatusPagePrivacy, scope status-pages:read) — Read status-page privacy with a strong configuration revision.
• PATCH /api/v2/status-pages/{id}/privacy (updateStatusPagePrivacy, scope status-pages:write) — Update reviewed status-page privacy through the canonical publication command. Retries: required.
• GET /api/v2/status-pages/{id}/content (getStatusPageContent, scope status-pages:read) — Read status-page content with a strong configuration revision.
• PATCH /api/v2/status-pages/{id}/content (updateStatusPageContent, scope status-pages:write) — Update reviewed status-page content through the canonical publication command. Retries: required.
• GET /api/v2/status-pages/{id}/settings (getStatusPageSettings, scope status-pages:read) — Read status-page settings with a strong configuration revision.
• POST /api/v2/status-pages/{id}/assets (uploadStatusPageAsset, scope status-pages:write) — Upload a validated content-addressed branding image. Retries: safe to repeat.
• GET /api/v2/status-pages/{id}/assets (listStatusPageAssets, scope status-pages:read) — List uploaded status-page branding assets without image bytes.
• DELETE /api/v2/status-pages/{id}/assets/{assetId} (deleteStatusPageAsset, scope status-pages:write) — Delete an unused status-page branding asset. Retries: required.
• PATCH /api/v2/status-pages/{id}/settings (updateStatusPageSettings, scope status-pages:write) — Update reviewed status-page settings through the canonical publication command. Retries: required.
• GET /api/v2/status-pages/{id}/subscribers (listStatusPageSubscribers, scope status-page-subscribers:read) — List subscriber contacts with explicit administrator privacy scope.
• GET /api/v2/status-pages/{id}/subscribers/{subscriberId} (getStatusPageSubscriber, scope status-page-subscribers:read) — Read a page-scoped subscriber with explicit privacy permission.
• DELETE /api/v2/status-pages/{id}/subscribers/{subscriberId} (removeStatusPageSubscriber, scope status-page-subscribers:write) — Revoke a subscription and suppress pending notifications. Retries: required.
• GET /api/v2/status-pages/{id}/subscriber-stats (getStatusPageSubscriberStats, scope status-page-subscribers:read) — Read consistent subscriber state counts for one page.
• GET /api/v2/status-pages/{id}/subscription-settings (getStatusPageSubscriptionSettings, scope status-pages:read) — Read supported public subscription availability and verification behavior.
• GET /api/v2/status-pages/{id}/preview (previewStatusPage, scope status-pages:read) — Preview the current unpublished public projection without publishing it.
• GET /api/v2/status-pages/{id}/announcements (listStatusPageAnnouncements, scope status-pages:read) — List status page incident announcements.
• POST /api/v2/status-pages/{id}/announcements (createStatusPageAnnouncement, scope status-pages:write) — Publish announcement on status page. Retries: supported.
• GET /api/v2/status-pages/{id} (getStatusPage, scope status-pages:read) — Get status page configuration.
• PATCH /api/v2/status-pages/{id} (updateStatusPage, scope status-pages:write) — Update status page settings with certified publication workflows. Retries: safe to repeat.
• GET /api/v2/status-pages (listStatusPages, scope status-pages:read) — List status pages with cursor pagination.
• POST /api/v2/status-pages (createStatusPage, scope status-pages:write) — Create a new status page. Retries: safe to repeat.

Services

• GET /api/v2/services/{id}/runbook-rules (listServiceRunbookRules, scope runbook-rules:read) — List bounded service runbook rule summaries.
• POST /api/v2/services/{id}/runbook-rules (createServiceRunbookRule, scope runbook-rules:write) — Create a service runbook rule with a reviewed version. Retries: required.
• GET /api/v2/services/{id}/runbook-rules/{ruleId} (getServiceRunbookRule, scope runbook-rules:read) — Read privileged rule configuration and its revision.
• PATCH /api/v2/services/{id}/runbook-rules/{ruleId} (updateServiceRunbookRule, scope runbook-rules:write) — Update rule configuration with revision protection. Retries: required.
• DELETE /api/v2/services/{id}/runbook-rules/{ruleId} (detachServiceRunbookRule, scope runbook-rules:write) — Detach a rule while preserving execution and suggestion evidence. Retries: required.
• POST /api/v2/services/{id}/runbook-rules/{ruleId}/validate (validateServiceRunbookRule, scope runbook-rules:read) — Review saved rule readiness without starting work. Retries: safe to repeat.
• GET /api/v2/services/{id}/runbook-rules/{ruleId}/policy-checks (listRunbookRulePolicyChecks, scope runbook-rules:read) — List safe Agent policy-check status metadata.
• POST /api/v2/services/{id}/runbook-rules/{ruleId}/policy-checks (requestRunbookRulePolicyChecks, scope runbook-rules:activate) — Request durable Agent policy checks for the reviewed rule. Retries: required.
• POST /api/v2/services/{id}/runbook-rules/{ruleId}/activate (activateServiceRunbookRule, scope runbook-rules:activate) — Activate an Automatic rule using current readiness and policy evidence. Retries: required.
• POST /api/v2/services/{id}/runbook-rules/{ruleId}/pause (pauseServiceRunbookRule, scope runbook-rules:write) — Pause new rule-triggered work. Retries: required.
• POST /api/v2/services/{id}/runbook-rules/{ruleId}/duplicate (duplicateServiceRunbookRule, scope runbook-rules:write) — Duplicate a rule into an independent paused configuration. Retries: required.
• PUT /api/v2/services/{id}/runbook-rules/{ruleId}/triggers (replaceRunbookRuleTriggers, scope runbook-rules:write) — Replace the reviewed incident trigger and pause Automatic execution. Retries: required.
• POST /api/v2/services/{id}/automation/validate (validateAutomation, scope automation:read) — Validate a draft without saving or publishing. Retries: safe to repeat.
• POST /api/v2/services/{id}/automation/simulate (simulateAutomation, scope automation:read) — Simulate automation without operational side effects. Retries: safe to repeat.
• POST /api/v2/services/{id}/automation/restore (restoreAutomationDraft, scope automation:write) — Restore a historical version as an editable draft. Retries: required.
• GET /api/v2/services/{id}/automation/decisions (listAutomationDecisions, scope automation:read) — List pinned incident routing decisions.
• GET /api/v2/services/{id}/automation/observations (listAutomationObservations, scope automation:read) — List safe field-discovery statistics.
• GET /api/v2/services/{id}/automation/versions (listAutomationVersions, scope automation:read) — List immutable service automation versions.
• GET /api/v2/services/{id}/automation/versions/{versionId} (getAutomationVersion, scope automation:read) — Inspect an immutable automation version.
• GET /api/v2/services/{id}/automation (getServiceAutomation, scope automation:read) — Read effective service automation configuration.
• PATCH /api/v2/services/{id}/automation/mode (updateServiceAutomationMode, scope automation:publish) — Change the service automation mode. Retries: required.
• POST /api/v2/services/{id}/automation/publish (publishAutomationVersion, scope automation:publish) — Publish a validated immutable service automation version. Retries: required.
• GET /api/v2/services/{id}/automation/draft (getAutomationDraft, scope automation:read) — Read a service automation draft.
• PUT /api/v2/services/{id}/automation/draft (saveAutomationDraft, scope automation:write) — Save a revision-protected automation draft. Retries: required.
• GET /api/v2/services/{id}/automation/candidate (getAutomationCandidate, scope automation:read) — Read a published candidate automation version and shadow metrics.
• DELETE /api/v2/services/{id}/automation/candidate (discardAutomationCandidate, scope automation:publish) — Discard a published candidate automation version. Retries: required.
• POST /api/v2/services/{id}/automation/candidates/activate (activateAutomationCandidate, scope automation:publish) — Atomically activate a candidate automation version as LIVE. Retries: required.
• POST /api/v2/services/{id}/integrations (createServiceIntegration, scope integrations:create) — Create a service inbound integration and disclose its API key once. Retries: required.
• GET /api/v2/services/{id}/change-events (listServiceChangeEvents, scope services:read) — List service configuration changes recorded in the canonical audit log.
• GET /api/v2/services/{id}/health-summary (getServiceHealthSummary, scope services:read) — Read bounded service health, visible incidents, paging failures and latest objectives.
• GET /api/v2/services/{id} (getService, scope services:read) — Get service details.
• GET /api/v2/services/{id}/notification-settings (getServiceNotificationSettings, scope services:read) — Read service notification policy.
• PATCH /api/v2/services/{id}/notification-settings (updateServiceNotificationSettings, scope services:write) — Update service notification policy. Retries: required.
• GET /api/v2/services/{id}/support-hours (getServiceSupportHours, scope response-policy:read) — Read versioned service support-hours policy.
• PUT /api/v2/services/{id}/support-hours (updateServiceSupportHours, scope response-policy:write) — Create the next immutable service support-hours policy version. Retries: required.
• GET /api/v2/services/{id}/response-policy/classification (getServiceClassificationPolicy, scope response-policy:read) — Read versioned service incident classification policy.
• PUT /api/v2/services/{id}/response-policy/classification (updateServiceClassificationPolicy, scope response-policy:write) — Create the next immutable classification policy version. Retries: required.
• GET /api/v2/services/{id}/response-policy/sla (getServiceSlaPolicy, scope response-policy:read) — Read versioned service SLA policy.
• PUT /api/v2/services/{id}/response-policy/sla (updateServiceSlaPolicy, scope response-policy:write) — Create the next immutable service SLA policy version. Retries: required.
• PATCH /api/v2/services/{id} (updateService, scope services:write) — Update service details. Retries: safe to repeat.
• DELETE /api/v2/services/{id} (deleteService, scope services:write) — Delete service (safely verifying no active incidents or objective history). Retries: safe to repeat.
• GET /api/v2/services (listServices, scope services:read) — List services with cursor pagination.
• POST /api/v2/services (createService, scope services:write) — Create a new service. Retries: supported.

Schedules

• POST /api/v2/schedules/{id}/layers/{layerId}/participants (addScheduleParticipant, scope schedules:write) — Add an active rotation participant. Retries: required.
• PATCH /api/v2/schedules/{id}/layers/{layerId}/participants/{userId} (moveScheduleParticipant, scope schedules:write) — Move a participant one position. Retries: required.
• DELETE /api/v2/schedules/{id}/layers/{layerId}/participants/{userId} (removeScheduleParticipant, scope schedules:write) — Remove a rotation participant. Retries: required.
• GET /api/v2/schedules/{id}/layers (listScheduleLayers, scope schedules:read) — List bounded rotation-layer configuration.
• POST /api/v2/schedules/{id}/layers (createScheduleLayer, scope schedules:write) — Create a rotation layer with active, unassigned responders. Retries: required.
• GET /api/v2/schedules/{id}/layers/{layerId} (getScheduleLayer, scope schedules:read) — Read a rotation layer and its current revision.
• PATCH /api/v2/schedules/{id}/layers/{layerId} (updateScheduleLayer, scope schedules:write) — Replace rotation-layer settings using an exact revision. Retries: required.
• DELETE /api/v2/schedules/{id}/layers/{layerId} (deleteScheduleLayer, scope schedules:write) — Delete a rotation layer and its participant assignments using an exact revision. Retries: required.
• PUT /api/v2/schedules/{id}/layers/order (reorderScheduleLayers, scope schedules:write) — Atomically replace rotation-layer precedence using the schedule revision. Retries: required.
• POST /api/v2/schedules/{id}/coverage/preview (previewScheduleCoverage, scope schedules:write) — Preview seven days of coverage after an unsaved layer change. Retries: safe to repeat.
• GET /api/v2/schedules/{id}/dependencies (getScheduleDependencies, scope schedules:write) — Inspect escalation-policy steps referencing a schedule.
• GET /api/v2/schedules/{id}/changes (listScheduleChanges, scope schedules:write) — Read a redacted, cursor-paginated schedule change history.
• GET /api/v2/schedules/{id}/on-call (getScheduleOnCall, scope schedules:read) — Calculate the effective on-call users at a point in time.
• GET /api/v2/schedules/{id}/overrides (listScheduleOverrides, scope schedules:read) — List overrides for a schedule.
• POST /api/v2/schedules/{id}/overrides (createScheduleOverride, scope schedules:write) — Create schedule override with domain validation. Retries: safe to repeat.
• DELETE /api/v2/schedules/{id}/overrides/{overrideId} (cancelScheduleOverride, scope schedules:write) — Cancel a schedule override. Retries: required.
• GET /api/v2/schedules/{id}/coverage (getScheduleCoverage, scope schedules:read) — Calculate bounded effective coverage and gaps.
• GET /api/v2/schedules/{id} (getSchedule, scope schedules:read) — Get schedule details with rotation layers.
• PATCH /api/v2/schedules/{id} (updateSchedule, scope schedules:write) — Update schedule name or timezone. Retries: safe to repeat.
• DELETE /api/v2/schedules/{id} (deleteSchedule, scope schedules:write) — Delete schedule (safely checking escalation policy references). Retries: safe to repeat.
• GET /api/v2/schedules (listSchedules, scope schedules:read) — List on-call schedules.
• POST /api/v2/schedules (createSchedule, scope schedules:write) — Create a new on-call schedule. Retries: supported.

Audit Logs

• GET /api/v2/audit-logs (listAuditLogs, scope audit:read) — List tamper-evident audit records with cursor pagination.

Escalation Policies

• GET /api/v2/escalation-policies/{id} (getEscalationPolicy, scope escalation-policies:read) — Get escalation policy details.
• GET /api/v2/escalation-policies/{id}/dependencies (getEscalationPolicyDependencies, scope escalation-policies:read) — Inspect service usage and live step target references.
• GET /api/v2/escalation-policies/{id}/validation (validateEscalationPolicy, scope escalation-policies:read) — Validate current policy steps and live target availability.
• PATCH /api/v2/escalation-policies/{id} (updateEscalationPolicy, scope escalation-policies:write) — Update escalation policy rules or metadata. Retries: safe to repeat.
• DELETE /api/v2/escalation-policies/{id} (deleteEscalationPolicy, scope escalation-policies:write) — Delete escalation policy (safely checking service usage). Retries: safe to repeat.
• GET /api/v2/escalation-policies (listEscalationPolicies, scope escalation-policies:read) — List escalation policies.
• POST /api/v2/escalation-policies (createEscalationPolicy, scope escalation-policies:write) — Create escalation policy. Retries: safe to repeat.

Integrations

• GET /api/v2/integrations/{id} (getIntegration, scope integrations:read) — Get integration details.
• PATCH /api/v2/integrations/{id} (updateIntegration, scope integrations:write) — Update integration name or enabled status. Retries: safe to repeat.
• GET /api/v2/integrations (listIntegrations, scope integrations:read) — List incoming integrations scoped by accessible services.

Notifications

• POST /api/v2/notifications/{id}/retry (retryNotification, scope notifications:write) — Retry a failed notification delivery safely through central control plane. Retries: safe to repeat.
• GET /api/v2/notifications (listNotifications, scope notifications:read) — List notification delivery audit records.

Postmortems

• GET /api/v2/postmortems/{id} (getPostmortem, scope postmortems:read) — Get postmortem details.
• PATCH /api/v2/postmortems/{id} (updatePostmortem, scope postmortems:write) — Update postmortem content or status. Retries: supported.
• DELETE /api/v2/postmortems/{id} (deletePostmortem, scope postmortems:write) — Delete postmortem. Retries: safe to repeat.
• GET /api/v2/postmortems (listPostmortems, scope postmortems:read) — List incident postmortems with cursor pagination.
• POST /api/v2/postmortems (createPostmortem, scope postmortems:write) — Create incident postmortem. Retries: supported.

Reports

• GET /api/v2/reports/metrics (getMetricsReport, scope reports:read) — Calculate actor-scoped SLA compliance and MTTA/MTTR metrics.
• GET /api/v2/reports/catalog (getReportCatalog, scope reports:read) — List supported report metrics and dimensions.
• POST /api/v2/reports/query (queryReportMetrics, scope reports:read) — Run a bounded actor-scoped analytical query. Retries: safe to repeat.
• GET /api/v2/reports/saved (listSavedReports, scope reports:read) — List owned and accessible saved reports.
• POST /api/v2/reports/saved (createSavedReport, scope reports:write) — Save a bounded report definition. Retries: required.
• GET /api/v2/reports/saved/{id} (getSavedReport, scope reports:read) — Read an accessible report definition.
• PATCH /api/v2/reports/saved/{id} (updateSavedReport, scope reports:write) — Update an owned report with an exact revision precondition. Retries: required.
• DELETE /api/v2/reports/saved/{id} (deleteSavedReport, scope reports:write) — Delete an owned saved report with an exact revision precondition. Retries: required.
• POST /api/v2/reports/exports (createReportExport, scope reports:read) — Queue a bounded, encrypted report export. Retries: required.
• GET /api/v2/reports/exports/{id} (getReportExport, scope reports:read) — Read owned report export job state and integrity metadata.
• GET /api/v2/reports/exports/{id}/download (downloadReportExport, scope reports:read) — Download an authorized report export.
• POST /api/v2/reports/exports/{id}/cancel (cancelReportExport, scope reports:read) — Cancel a pending owned report export. Retries: required.
• GET /api/v2/dashboards (listReportDashboards, scope reports:read) — List accessible saved dashboards and templates.
• POST /api/v2/dashboards (createReportDashboard, scope reports:write) — Create a saved dashboard. Retries: required.
• GET /api/v2/dashboards/{id} (getReportDashboard, scope reports:read) — Read an accessible saved dashboard.
• GET /api/v2/dashboards/{id}/data (getReportDashboardData, scope reports:read) — Calculate actor-scoped data for an accessible dashboard.
• GET /api/v2/dashboards/{id}/export (downloadReportDashboardData, scope reports:read) — Download a bounded JSON snapshot of accessible dashboard data.
• POST /api/v2/dashboards/{id}/duplicate (duplicateReportDashboard, scope reports:write) — Duplicate an accessible dashboard into an owned dashboard. Retries: required.
• GET /api/v2/dashboards/{id}/widgets (listReportDashboardWidgets, scope reports:read) — List widgets of an accessible dashboard.
• PUT /api/v2/dashboards/{id}/widgets (replaceReportDashboardWidgets, scope reports:write) — Replace owned dashboard widgets with an exact revision precondition. Retries: required.
• PATCH /api/v2/dashboards/{id} (updateReportDashboard, scope reports:write) — Update an owned dashboard with a strong revision precondition. Retries: required.
• DELETE /api/v2/dashboards/{id} (deleteReportDashboard, scope reports:write) — Delete an owned dashboard with a strong revision precondition. Retries: required.

Service Accounts

• GET /api/v2/service-accounts/{id} (getServiceAccount, scope service-accounts:read) — Get service account details.
• DELETE /api/v2/service-accounts/{id} (deleteServiceAccount, scope service-accounts:write) — Delete service account and revoke its tokens. Retries: safe to repeat.
• DELETE /api/v2/service-accounts/{id}/tokens/{tokenId} (revokeServiceAccountToken, scope service-accounts:write) — Revoke a specific service account token immediately. Retries: safe to repeat.
• POST /api/v2/service-accounts/{id}/tokens (issueServiceAccountToken, scope service-accounts:write) — Issue a new fine-grained token for a service account. Retries: not replayable.
• GET /api/v2/service-accounts (listServiceAccounts, scope service-accounts:read) — List automation service accounts.
• POST /api/v2/service-accounts (createServiceAccount, scope service-accounts:write) — Create automation service account. Retries: supported.

Administration

• GET /api/v2/settings/incident-sla (getWorkspaceSlaPolicy, scope response-policy:read) — Read the current workspace incident SLA policy.
• PUT /api/v2/settings/incident-sla (updateWorkspaceSlaPolicy, scope response-policy:write) — Create the next immutable workspace incident SLA policy version. Retries: required.

Users

• GET /api/v2/users/{id} (getUser, scope users:read) — Get user details with safe profile fields.
• GET /api/v2/users (listUsers, scope users:read) — List users with cursor pagination and role filtering.

Select a Runbook service rule

When executing a Runbook attached through multiple enabled service rules, supply bindingId to select the reviewed rule. Ambiguous requests fail with a validation error. Detached rules cannot be selected. The selected rule supplies its pinned version and execution target. Non-empty parameters are accepted only for unbound executions. When a service rule is selected explicitly or inferred from an incident/service, omit parameters: the rule's saved inputs are authoritative. Attempts to supply per-execution parameters return HTTP 400 before creating an execution. Update and review the rule to change its saved inputs. Unbound parameters are validated against the published version and frozen into its preview and execution plan.

7. Granular Scope Reference

Scopes are defined once in the scope registry and used for issuance, the settings UI and this table. admin:manage satisfies ordinary v2 scopes. Runbook authoring, publishing and execution each require an explicit scope.

8. Coverage and remaining REST gaps

The following list distinguishes delivered workflows from remaining gaps in /api/v2. No scope grants an unavailable operation. Registration and local checks do not establish complete enterprise or production certification.

• Runbooks: published reads, execution, creation, version summaries, draft cloning, full draft replacement, validation and publication are available. Version lists and mutation responses contain summaries. The privileged draft detail also returns editable, a PATCH body with configuration and input defaults omitted for safe preservation. Execution previews and cancellation requests are also available. Plan-bound approval/rejection, reviewed terminal reruns, execution step/evidence/event reads and service-rule administration are available with explicit scopes and resource checks. Privileged Agent fleet reads, policy status, one-time enrollment and terminal revocation are available with dedicated administrator scopes. Enrollment retry receipts contain metadata with token: null; plaintext enrollment material is returned only in the first committed response. Pools, operational settings and secret-grant administration remain in progress. Authoring, publishing and execution each require their explicit scope, even on an admin:manage credential. Existing administrator-only execution credentials must add runbooks:execute before adopting this release.
• Schedules: creating, renaming and deleting schedules, creating and cancelling overrides, bounded effective coverage reads, and adding, removing and moving rotation participants are available. Layer configuration editing and unsaved-change preview remain unavailable.
• Escalation policies: full create/update/delete of policies and steps, with target validation.
• Status pages: creating pages, changing name, slug, enabled and default state, and posting announcements. Revision-fenced configuration and component mapping, subscriber administration, branding settings and durable publication inspection/retry are available. Embedded logos must be canonical base64 PNG, JPEG, GIF or WebP with matching file signatures; legacy embedded SVG logos require replacement with a supported raster image. Domain ownership, the full branding asset upload lifecycle and publication preview remain unavailable.
• Integrations: inbound provider creation, safe metadata/configuration updates, local payload validation, encrypted routing keys, bounded key/signature-secret rotation, delivery diagnostics, and safe deletion are available. OAuth consent and external outbound connectors remain browser/provider flows and are not represented as REST-complete.
• Users: read-only directory. No user creation, role changes or deactivation.
• Notifications: delivery history and retry, self-service channel preferences, service channel/event settings, provider configuration, readiness diagnostics, asynchronous idempotent test intents and Web Push VAPID rotation are available. Provider operations require dedicated explicit administrator scopes; broad admin:manage does not imply them. Test recipients are the active administrator’s configured contact endpoints, never request-supplied addresses or provider sender numbers. A dedicated-PostgreSQL integration test verifies REST queueing, worker delivery, durable receipt and completed diagnostic visibility; final deployment certification remains pending. ADMIN and RESPONDER credentials see delivery detail; AUDITOR credentials see delivery metadata only (no message body, recipient or email).
• Runbook visibility: the published library exposes only published, non-archived runbooks; an unpublished runbook is 404 through the library detail endpoint. Administrators with explicit authoring scope can inspect draft editing views and version summaries through the authoring endpoints. Step error messages in execution status are redacted and bounded.
• Status-page publication: PATCH /status-pages/{id} commits the change first and then republishes the public page; the response includes publication (LIVE, PUBLISHING, FAILED or DISABLED) so a client can tell "saved and live" from "saved, republish pending".

9. Mutation Safety

Every write operation has a declared retry behaviour. Keys are scoped to the credential and the operation and are kept for 7 days; after that a reused key executes again. Reusing a key with a different body returns 409 IDEMPOTENCY_KEY_CONFLICT. If a request times out, retry it with the same Idempotency-Key where the operation supports it.

Every successful write commits its audit record in the same database transaction; if the audit record cannot be written, the change is rolled back. Audit details returned by GET /api/v2/audit-logs are redacted and bounded.

10. Deployment Notes

• Client address and proxies. Set TRUST_PROXY_HEADERS=true only when the application can be reached solely through your ingress and the ingress overwrites forwarded headers. By default the address is read from X-Forwarded-For, counting TRUSTED_PROXY_HOPS (default 1) hops from the right. CF-Connecting-IP and X-Real-IP are honoured only when selected with TRUSTED_PROXY_HEADER=cf-connecting-ip (or x-real-ip), because behind any other ingress a client can send them. When TRUSTED_PROXIES (addresses or CIDRs) is set and the peer address is known, the peer must match it.
• Developer workspace. Settings → API Keys has a quickstart whose Run Live Test sends real requests from the page. Paste a key into API key for live tests (or generate one); it is kept only in page memory, never saved, never put in a URL and not shown in the code samples. Scenarios that act on a resource (ingest needs a service, acknowledge needs an incident) ask for its real ID; the samples show a YOUR_…_ID placeholder, never an invented one. Requests that change data ask for confirmation. A result is green only for a 2xx that actually authenticated (GET /api/sample answers 200 with authenticated: false for a bad key); network failures are reported as failures.
• No positive caching. Credentials are verified against PostgreSQL on every request, so revocation, expiry and role changes take effect immediately on every replica.

Runbook preview and cancellation

Use POST /runbooks/{id}/executions/preview with the same incident, service, binding and parameters intended for execution. The preview applies the execution boundary, saved-rule input rules and target preflight without creating an execution or agent job. Responses omit executable configuration, input values and the internal preview authorization token. The control plane stores an expiring actor-bound preview capability.

To ensure the reviewed plan is still current, pass the returned digest as expectedPlanDigest to POST /runbooks/{id}/execute. A changed version, input, rule or target produces 412 before execution starts. Execution still requires an Idempotency-Key.

POST /runbooks/executions/{id}/cancel requires runbooks:execute, an Idempotency-Key and a nonempty reason. HTTP 202 means CANCEL_REQUESTED. Poll the execution endpoint for the final state; a running external command has not necessarily stopped when the request is accepted.

The watcher collection supports bounded cursor pagination, adding active users with FOLLOWER, STAKEHOLDER or EXEC roles, and parent-scoped removal. Public watcher mutations require operational privileges and incidents:write; existing UI self-subscription remains available to users who can see the incident.

The incident tag collection uses bounded cursor pagination. Adding an already-associated tag or removing an absent association succeeds without producing another timeline or audit event. Routing-decision reads omit stored evaluation summaries and source values.

Generated clients

The repository includes a TypeScript ESM package in sdk/typescript (Node.js 20+) and a typed Python package in sdk/python (Python 3.11+). These are source packages awaiting a distribution release. Generate their request/response types and operation manifests with npm run api:clients; npm run api:clients:check fails when the committed clients or OpenAPI export drift from the runtime registries. Build the TypeScript package with npm run build --prefix sdk/typescript and install the Python package from sdk/python.

Both clients take the API root, including /api/v2, and an in-memory token. They send the token only in the Authorization header, reject redirects and require an explicit option for HTTP. They retry at most twice by default: reads and replay-protected writes carrying an idempotency key may retry selected transient HTTP failures. One-time token issuance, writes without replay protection, transport uncertainty and precondition failures are not automatically retried. Long Retry-After values are returned to the caller instead of being ignored. Cursor traversal is bounded and detects repeated cursors.

import { Client } from '@opsknight/api-v2';

const client = new Client({
  baseUrl: 'https://ops.example.com/api/v2',
  token: process.env.OPSKNIGHT_API_KEY!,
});
const page = await client.call('listIncidents', { query: { limit: 25 } });
for (const incident of page.data) console.log(incident.id);

import os
from opsknight_v2 import Client

client = Client('https://ops.example.com/api/v2', os.environ['OPSKNIGHT_API_KEY'])
for incident in client.paginate('listIncidents', {'query': {'limit': 25}}):
    print(incident['id'])

ApiError preserves HTTP status, the stable error code and request ID without including raw error details. Validation, role checks and resource visibility remain authoritative on the server. The clients do not imply additional permissions or unsupported product coverage.

To repeat the live client check, build the application and TypeScript SDK, point DATABASE_URL and DIRECT_URL at a dedicated database whose name ends in _test, and run npm run certify:api:clients:live. It starts temporary production web and general-worker processes, creates separate short-lived credentials for TypeScript and Python, exercises both clients, revokes both credentials and stops both processes. Docker with Python 3.12 is required; set DOCKER_BIN if Docker is not on PATH. The default local test ports are 3159 and 3160 (API_SDK_TEST_PORT selects the web port; the worker uses the next port). This check is isolated from ordinary application databases by its database-name guard.

Runbook credential migration checklist

Before deploying this release:

1. Inventory integrations and service accounts that call runbook endpoints. Credentials holding only admin:manage will receive 403 for execution, authoring and publishing; ordinary incident and service operations retain their existing scope behavior.
2. Grant only the scopes each integration uses: runbooks:execute for preview, execute, cancel and rerun; runbooks:approve for reviewed approval/rejection; runbooks:write for create, draft read/edit/validate and version lists; runbooks:publish for publication. Add runbooks:read for published library and execution reads. Authoring and publishing require an ADMIN identity; execution and approval also permit RESPONDER. Approval/rejection and rerun additionally require runbooks:read and execution ownership/team access (ADMIN excepted). Scopes do not elevate the identity's role.
3. Issue a replacement token with the selected scopes, use a secret manager, and test its normal workflow in staging. Verify that authoring-only credentials cannot publish and broad administrator-only credentials cannot execute. Preserve idempotency keys across transport retries.
4. Switch the integration to the replacement credential, verify successful calls, then revoke the previous credential. Do not grant every scope to restore one operation. If rollout is paused, retain the previous application version until credentials are migrated.

Safe draft editing

GET /runbooks/{id}/drafts/{versionId} requires explicit runbooks:write and ADMIN. It returns an ETag and an editable body containing the complete definition structure, inputs and preserveConfiguration: true. Every step configuration is {} and input defaults are omitted; commands, headers, environment values and secret references inside configuration are never disclosed. Responses use Cache-Control: no-store. Ordinary library reads and version lists remain redacted summaries.

Send the edited editable body to PATCH with the exact ETag and an idempotency key. With preserveConfiguration: true, empty configurations retain existing configurations by step key within the same step container; omitted defaults retain existing defaults by input key and type. Reordering steps is safe. New steps require valid explicit configuration; changing a preserved step's type is rejected. Nonempty configurations and supplied defaults replace those fields. To deliberately clear configuration or defaults, set preserveConfiguration: false and supply the full intended replacement. Step deletion removes that step. Preservation is resolved only from the same parent draft under lock after revision comparison, never from another version. A stale revision returns 412; missing If-Match returns 428; wildcard If-Match is rejected with 400 and weak validators with 412.

Execution cancellation policy

Cancellation requires explicit runbooks:execute and an execution-capable identity. ADMIN may cancel any execution. A RESPONDER may cancel an execution they triggered or one owned by a service whose team they belong to. An execution without a service can only be cancelled by its triggering user or an ADMIN. An attached incident must also pass incident visibility checks with incidents:read. Inaccessible executions return 404, with no cancellation, queued work or audit mutation.

Watcher policy compatibility

REST watcher writes intentionally remain operational administration: ADMIN or RESPONDER plus incidents:write. Ordinary USER credentials, including a user trying to follow themselves, receive 403; no watcher or timeline event is created. The UI continues to support self-follow for eligible users who can see the incident. The API does not grant incident mutation privileges through subscription. Self-service REST subscription is deferred to a dedicated scoped contract.

Execution and cancellation permission policy

ADMIN and RESPONDER identities with explicit runbooks:execute may start published runbooks across service teams. This preserves the global responder operations capability; team membership does not restrict starts. Incident-bound starts additionally require incidents:read, incident visibility and an enabled matching service rule. USER and AUDITOR identities cannot execute even when a token contains that scope; admin:manage alone does not authorize execution.

Cancellation intentionally has a narrower interruption policy: ADMIN, the triggering operator, or a responder on the owning service team may stop a run. Global start permission does not authorize stopping another team's execution. Incident visibility still applies to attached incidents. Negative PostgreSQL tests cover lower roles, missing scopes and unrelated cancellation callers.

Start and cancellation each emit one canonical domain business audit event on RUNBOOK_EXECUTION: runbook.execution.started and runbook.execution.cancel.requested. REST supplies source: API and the non-secret API credential identifier (apiClientId); UI callers retain UI provenance. REST does not emit a second business event. An already CANCEL_REQUESTED execution returns accepted without changing the original reason/time, enqueueing another job or adding an audit, even with a different idempotency key. Terminal executions continue to reject cancellation.

Agent pools, secret grants and Runbook settings

Agent fleet and pool administration require ADMIN capability and their dedicated scopes. Pool and Agent configuration writes require a concrete If-Match and Idempotency-Key. Read /runbooks/agents/{id}/configuration for the Agent configuration ETag: ordinary heartbeats do not invalidate it. Pool membership changes advance the pool revision. Dynamic membership is derived from labels; remove it by changing the selector or Agent labels. Removing an explicit member that still matches the selector returns 409 rather than reporting success and silently restoring dynamic access. Historical executions and attempts prevent pool deletion. Use the deletion-check endpoint before attempting deletion; concurrent references are checked again by the deletion command.

Secret reads return metadata and secret://name references only. Create and rotate accept write-only values, encrypted at rest. Rotation requires the ETag from GET /runbooks/secrets/{id}. Grant creation and deletion also use that secret's ETag; the grant response returns its updated parent ETag. A grant targets exactly one Agent or one pool. The grant ID must belong to the secret in the URL. Expanding a secret-granted pool through explicit membership, selectors or Agent labels additionally requires runbook-secrets:grant. Revocation fences subsequent claims; it cannot recall material already delivered to an accepted claim. Secret values never appear in metadata responses, canonical audit details or persisted retry responses.

GET/PATCH /settings/runbooks exposes only artifact retention and per-incident remediation budgets, with dedicated settings scopes and a configuration ETag. Retention updates require confirmArtifactCleanup: true because they can make existing artifacts eligible for cleanup. Budgets are integers from 0 to 100; retention is 1 to 3650 days. Broader system configuration and signing keys are not exposed by these endpoints.

Pool retirement and secret usage

POST /runbooks/agent-pools/{id}/retire requires the pool ETag and a required idempotency key. Retirement is terminal. It disables service rules targeting the pool, prevents new membership, grants and execution assignments, and fences subsequent Agent claims and pre-execution checks. Historical execution/attempt references remain readable. Already running commands may complete and report evidence. Deletion still requires that all references be absent. Database guards also reject new assignments from older writers; this fail-closed behavior is not a mixed-version certification.

Selector changes that only remove dynamic membership do not require secret-grant scope. Expanding secret access through labels, selectors or membership requires explicit runbook-secrets:grant for REST. UI callers must pass an explicitly checked runbook.secret.manage capability; their actor ID representation grants no authority. Claims share the fleet authority fence while administration holds it exclusively. A removal cannot commit ahead of a claim that already holds the fence, and a claim accepted after removal observes the new authority. Previously delivered secrets cannot be recalled.

GET /runbooks/secrets/{id}/usage requires administrator secret-read permission and uses the search/export rate tier. It returns paginated references in service-rule inputs, execution input snapshots and SECRET_REF defaults, without their input values or executable definitions. Matching follows the resolver's exact top-level reference semantics. A stored reference indicates a dependency, not proof that an execution consumed a particular version of a secret. Indexed queries have a two-second database statement budget.

Execution-signing identity rotation

GET/POST /runbooks/signing-identities require dedicated runbook-signing:read or runbook-signing:write scopes plus administrator capability. Reads and rotation responses contain public keys and lifecycle metadata only. Private keys remain encrypted and are never copied into API responses or retry receipts.

Staging requires a reviewed expectedActiveKeyId (null only before initial setup). Activate /runbooks/signing-identities/{id}/activate with the expected current active key ID after every enrolled Agent, including offline Agents, acknowledges both old and NEXT pins. The old identity enters RETIRING for at least 24 hours. Retirement through .../{id}/retire requires expectedState: "RETIRING" after that grace period. All writes require idempotency keys; state checks, canonical API-attributed audit and public-metadata replay commit together under the signing rotation lock. A failed pin check does not invalidate a working identity. This API does not bypass provider or Agent trust establishment and does not offer emergency private-key export.

Service account identity format

New service accounts keep the supplied display name and receive a stable identity address in the form sa-<name-slug>-<24-hex-random-id>@serviceaccount.opsknight.local. The ASCII slug is bounded so the local part never exceeds 64 bytes; non-Latin-only names use account as the slug while retaining the original display name. A cryptographically random 96-bit suffix separates accounts with identical names without exposing their creation timestamp. This is an internal machine identity, not a deliverable email address. The reserved domain continues to determine machine-account classification and login restrictions.

Existing timestamp-based identities remain recognized and tokens continue to work. Display-name edits do not change identity addresses. Renaming existing stored identities requires a separately audited account update; this creation-format change does not silently rewrite deployed identities.

Service account settings show the chosen display name and stable account ID. The internal reserved-domain address is omitted from the settings client payload and is not used as a machine token's owner label. Inventory search accepts display names and account IDs. Existing REST identity fields and stored addresses remain compatible; the internal address is not an email destination or a production domain to configure.

Global automation settings

GET /api/v2/settings/automation requires an administrator with the explicit automation-settings:read scope. It returns the global enable switch, trace retention days and settings revision, with a strong ETag and Cache-Control: no-store.

PATCH /api/v2/settings/automation requires an administrator with the explicit automation-settings:write scope, a concrete If-Match ETag and an Idempotency-Key. Supply both automationEnabled and automationTraceRetentionDays (integer, 1–3650). Unknown fields are rejected. The UI and API use the same command. Disabling resets all service automation modes to DISABLED atomically with settings and the canonical audit event. Re-enabling does not reactivate services automatically. A stale revision returns 412; a missing revision returns 428; wildcard revisions are rejected. Broad admin:manage alone does not grant either settings scope.

Seventeen automation operations are implemented, including authoring and redacted diagnostics. Their combined certification and broader enterprise release gates remain separately tracked; availability is not a production certification claim.

Status-page configuration sections

Administrators with explicit status-pages:read/status-pages:write scopes can read and update appearance, privacy, content and general settings through the corresponding /status-pages/{id}/{section} endpoints. Component membership uses GET/PUT /status-pages/{id}/services; component presentation uses PATCH /status-pages/{id}/services/{serviceId}. These operations preserve the one-page product limit.

Section reads return a common configuration revision and strong ETag. Every new configuration write requires both concrete If-Match and Idempotency-Key headers. The replay record, canonical credential-attributed audit and configuration commit together. Reusing the same key and request returns the committed configuration metadata even after later edits, together with the current publication state. A new key with a stale revision returns 412. UI and nested component edits also advance the revision.

Public publication happens after the transaction commits. Inspect publication.status to distinguish saved configuration from a live projection; privacy changes retain the shared fail-closed withdrawal behavior. Unchanged assignments do not emit duplicate business audits. These section endpoints reject custom-domain/subdomain changes and inline asset uploads; domain proof and validated asset lifecycle remain unfinished. Subscriber contacts require their separate subscriber scopes.

Status-page publication reconciliation

GET /status-pages/{id}/publication reports current configuration revision and publication state. An administrator can request POST /status-pages/{id}/publication/retry with concrete If-Match and required Idempotency-Key; it returns 202 Accepted, a durable job and its Location. Inspect /status-pages/{id}/publication/jobs/{jobId} for completion, then read current publication state. Job completion can report publication FAILED or PUBLISHING; it does not guarantee that the page is live.

Acceptance stores the job, request audit and replay record in one transaction. The general worker reuses the UI publication reconciler, preserves fail-closed privacy/disable decisions, and stores the canonical outcome audit with its receipt. Request and outcome audits describe distinct lifecycle events. Queue retries after receipt do not repeat the outcome audit. Job inspection omits internal payloads and raw queue errors, and enforces page ownership. The existing status-page read role policy applies to inspection; requesting retry requires administrator management permission. Mixed-version worker deployment and production HA certification remain unverified.

Service automation drafts and publication

GET /api/v2/services/{id}/automation/draft requires automation:read and an ADMIN, RESPONDER or AUDITOR role. A service with no draft returns the empty domain snapshot and revision zero. The response is no-store and includes its strong ETag.

PUT on the same path requires automation:write, If-Match and Idempotency-Key. Supply { "snapshot": { "schemaVersion": 1, "fields": [], "rules": [] } } with the complete desired configuration. The canonical engine schema bounds and validates fields, rules, conditions and actions. Administrators can edit any service; responders must belong to its current team. Draft saving does not activate routing.

POST /api/v2/services/{id}/automation/publish requires an administrator and the explicit automation:publish scope. Supply expectedActiveVersionId (null for the first publication), the draft's If-Match and an Idempotency-Key. Publication uses the shared compiler and dependency validation, creates immutable history and records one credential-attributed publication audit. It does not enable automation. A stale draft or changed active version returns 412. Invalid draft configuration returns a structured validation error. No executable configuration is included in the publication result.

Effective automation configuration and history

GET /api/v2/services/{id}/automation returns the stored and effective modes, active/candidate version summaries, draft revision and seven-day health metrics. Errors and fallbacks are separate counters. Candidate evidence includes its latest daily bucket and a bounded pending-job sample; a truncated sample is a lower bound, not a complete backlog count. The configuration ETag fences configuration writes; embedded health and global settings are read-only related data.

PATCH /api/v2/services/{id}/automation/mode requires administrator privilege, automation:publish, the configuration ETag and an Idempotency-Key. Supply mode and expectedActiveVersionId. LIVE retains the domain's dependency and shadow-review gates. Global OFF prevents enabling SHADOW or LIVE. Publication and draft restoration do not implicitly change the mode.

GET .../automation/versions provides bounded, signed cursor pagination in immutable version-number order. GET .../automation/versions/{versionId} returns a service-owned snapshot without its internal compiled representation.

POST .../automation/restore copies { "sourceVersionId": "..." } into an editable draft, using the current draft ETag and an Idempotency-Key. It requires automation:write and the same administrator/team-edit permission as draft saving. It does not publish, activate, discard a candidate or change the mode. Further edits retain the origin in the audit/publication lineage.

Automation validation and redacted diagnostics

POST .../automation/validate accepts { "snapshot": ... } and performs shared compilation and dependency checks. POST .../automation/simulate also accepts an Events API event, integration type, optional service-owned integration ID, optional provider payload and evaluation time. Both require automation:read, apply a bounded request size, and create no incidents, notifications or observation jobs. Simulation returns rule/condition truth states and outcomes without received values, enrichment values or provider payloads. Successful validation does not waive the separate publication and activation safety checks.

GET .../automation/decisions requires both automation:read and incidents:read and applies incident visibility permissions. It returns pinned version, route, policy/rule IDs and priority metadata; stored incident summaries are omitted. GET .../automation/observations returns discovery metadata and frequencies, optionally filtered by unmapped=true or unmapped=false. It never selects raw value previews or value hashes. Observation cursors use stable first-seen timestamps and IDs so counter refreshes do not reorder existing entries. Both lists are bounded to 100 items per request and return Cache-Control: no-store.

Discovery privacy and upgrade behavior

automationStoreObservationPreviews is an optional administrator setting accepted by PATCH /api/v2/settings/automation and the same system-settings UI. Its default is false; omitting it preserves the current setting. With previews disabled, observation jobs and discovery rows retain keyed, service-scoped value identities and counts, with empty preview scalars. Alias authoring from stored values is available only when a readable preview was explicitly retained. Enabling previews permits bounded diagnostic examples; recognizable contact fields, email addresses, phone-shaped values and credential patterns remain excluded. This classifier is not an assurance that arbitrary free text contains no personal information.

Disabling previews immediately hides retained values at the UI response boundary and queues bounded, resumable physical cleanup of observation rows and queued job scalars. The settings transaction does not scan diagnostic history. Each cleanup batch scrubs at most 200 observation rows and 200 queued payloads, committing its receipt and continuation together. Worker/enqueue reads lock settings so in-flight writes cannot reintroduce a preview after disablement commits. Administrator cleanup status and retry remain available. Successfully processed jobs are minimized even when bounded preview retention is enabled. Observations remain subject to the configured trace/observation retention period.

The privacy upgrade resets legacy discovery sample rows because their unkeyed hashes cannot be safely re-keyed without retaining source values. Legacy queued job observation arrays are cleared, while shadow counters and execution metadata remain intact. New traffic repopulates discovery statistics. Incident decisions, published automation, active routing and shadow aggregate evidence are preserved. Keyed identities derive from the configured API/session signing secret with a separate domain and service identifier; key rotation starts a new identity space.

Deploy the privacy migration before the new application, and replace every web replica and worker before treating the new minimization setting as effective. Older binaries do not know this setting and can still enqueue/store legacy samples during mixed-version operation. Rolling-version privacy qualification remains a release gate; these changes do not certify mixed-version operation as safe.

Service-objective credentials and measurement contract

Use explicit service-objectives:read, service-objectives:write, and service-objectives:evaluate scopes. Broad admin:manage does not substitute for these scopes. Write access requires an administrator; read and evaluation retain the identity's service and incident visibility. Aggregate evaluation access does not require granting unrelated raw incident or service API scopes.

The catalog describes the four supported measurements. UPTIME and AVAILABILITY both measure service uptime; AVAILABILITY is not request-success telemetry. MTTA and MTTR use canonical SLA elapsed time in minutes, including paused-clock and retention semantics. Targets are compared against exact additive elapsed-time state before display rounding. No samples yield NO_DATA with null compliance; unsupported legacy telemetry and retention-clipped windows yield UNAVAILABLE. Quarterly and yearly windows mean 90 and 365 elapsed days, respectively.

Create requires an idempotency key and expectedActiveVersionId: null asserts an empty service/metric slot, while a version ID explicitly replaces that active slot. Updates create immutable versions. Edit and archive require the concrete ETag returned by GET plus an idempotency key. Archive retains historical versions and snapshots and changes the ETag. The settings management page uses the same commands and rejects stale forms.

Evaluation is asynchronous. Request an explicit immutable expectedVersionId, retain the accepted job ID, and poll its evaluation resource. At most two jobs per credential and sixteen jobs globally may be pending or processing. A 429 response indicates capacity; wait for existing work before trying again. A transport retry must reuse its original idempotency key. Ad-hoc evaluations do not overwrite scheduled snapshots. The worker checks credential validity, scopes, role and team context before computation and before persisting its result. Scoped readers cannot retrieve results recorded under a different role/team context after a permission change.

Scheduled snapshots and legacy incident-SLA history may include private incident contributions and therefore require global metrics visibility. Scoped users should request their own filtered evaluation. Legacy incident-SLA history remains a separately typed source and is never represented as a new objective sample. Large integer fields in history are decimal strings to preserve precision.

These endpoints do not certify full enterprise coverage, complete SLO history and evaluation UI parity, or production load/HA behavior.

Revision-bound replay and upgrade compatibility

For revision-protected mutations using the shared route factory, idempotency identity includes the validated concrete If-Match revision and the request body. Retry with the original key, body and revision, even after the successful write advances the resource. Reusing that key with a different revision returns 409 IDEMPOTENCY_KEY_CONFLICT. Missing, weak and wildcard validators still fail before replay. Status-page configuration uses its existing revision-bound domain receipt contract.

Generic revision-protected receipts created by older binaries did not retain proof of the original revision. They fail closed with 409 under the corrected fingerprint, including an otherwise legitimate old retry. Reconcile the resource and audit history before starting a newly reviewed operation with a new key; do not clear receipts or blindly resubmit an uncertain operation. This requires old/new binary rollout certification before release.

Publication retries coalesce different keys into pending page work, which reads the latest configuration when claimed. New requests wait while work is processing. Admission allows at most four outstanding jobs per credential and sixteen globally. Capacity responses return 429 with Retry-After. Publication lastError fields in ordinary REST state, configuration responses and job results contain a safe diagnostic code, PUBLICATION_RECONCILIATION_FAILED, rather than detailed stored error text. Detailed errors remain in privileged control-plane diagnostics.

Container builds share one digest-pinned Node/Alpine base and one verified package installation step. Transient package-fetch failures have a bounded retry budget; TLS and package-signature verification stay enabled. The image records resolved packages at /usr/local/share/opsknight/alpine-packages.txt. Repository upgrades remain mutable: byte-for-byte reproducible OS rebuilds require an immutable package snapshot/mirror and remain a release gate.

Reviewed execution decisions and reruns

Execution step, evidence and event lists require runbooks:read and enforce the existing execution visibility policy. Lists are cursor-paginated. Evidence is a bounded, redacted diagnostic string; raw configuration, artifact storage keys, lease tokens and raw audit details are omitted. Responses use no-store.

Grant the new explicit runbooks:approve scope only to intended ADMIN or RESPONDER reviewers. admin:manage and runbooks:execute do not imply it. Reviewers also need runbooks:read. Pending review plans are returned only to callers who may control the execution: ADMIN, its triggering user, or a member of its service team. Other visible readers receive pendingApproval: null. This REST control boundary is stricter than the existing browser approval capability; it deliberately follows the ownership/team restriction used for REST cancellation.

Read /runbooks/executions/{id}/steps, inspect the safe resolved plan, then POST its exact planDigest to /steps/{stepId}/approve or /steps/{stepId}/reject. Rejection also requires a nonempty reason. Both require an Idempotency-Key. A changed plan, non-waiting step or concurrent decision fails with 409; a foreign parent/step fails with 404. Approval records one canonical approval business event. Rejection records a rejection decision and a distinct parent cancellation request. HTTP 202 for rejection requests cancellation; poll the execution for the final state. The decision, queue work, audits and replay receipt commit atomically. Cached retries recheck current role, scope and execution access.

To rerun a terminal execution, POST {} to /runbooks/executions/{id}/rerun-preview. This requires runbooks:read and runbooks:execute, execution ownership/team access, and current execution policy. Inspect the safe preview, then POST its digest as expectedPlanDigest and its authorizationToken as previewAuthorizationToken to /rerun, with a fresh Idempotency-Key. The token is short-lived and actor-bound; treat it as sensitive. The rerun uses current published definitions and current rule/target policy. A changed or expired preview cannot authorize a start. The response identifies a new execution; it does not claim external work has completed. Exact retries return the same execution. Ordinary starts retain the separately documented global responder policy; rerun applies the stricter source execution control boundary.

Approval plans withhold submitted input values before interpolation. Their digest still binds the actual inputs and execution target. Evidence redacts known string input values before JSON escaping, in addition to credential-field and output-pattern redaction. This is a safe review representation, not disclosure of raw execution inputs.

Team membership and lead administration

GET /teams/{id}/members returns a bounded membership list and the parent Team ETag. GET /teams/{id}/lead returns the lead and the same revision. Reads require teams:read and either organization directory access or membership in that team. Membership responses contain IDs, role, notification preference and creation time; they do not contain email addresses or user authentication fields.

PUT /teams/{id}/members/{userId} accepts a required role (OWNER, ADMIN, MEMBER) and optional receiveTeamNotifications. DELETE on that path removes the membership. PUT /teams/{id}/lead accepts { "userId": "..." } or { "userId": null }. These operations require teams:write, administrator or current team-owner authority, Idempotency-Key, and a concrete parent Team If-Match validator. Wildcards are rejected. SCIM-managed team writes stay with the identity provider. A team lead must be an active member; the last active owner cannot be demoted or removed.

The UI and REST use the same Serializable commands. Membership/role changes increment the user's session generation; the initiating replica evicts its local projection after commit. Canonical audits, addition/removal notifications and REST receipts commit with the mutation. Replays recheck current ownership and role. No-op writes produce no new audit or revision. Removing the lead also clears the lead designation. Database triggers invalidate the Team revision for membership changes from UI, SCIM and older application writers.

GET /teams/{id}/services lists current managed services with bounded team-bound cursors. PUT /teams/{id}/services/{serviceId} assigns a service using teams:write, current administrator/team-owner authority, an empty JSON body, required Idempotency-Key and a concrete Service If-Match revision. Obtain the Service ETag from /services/{serviceId}, or quote its revision from the team service list. Team owners can claim unassigned services; only administrators can move a service between teams. The UI bulk action uses the same transactional command, limited to 100 IDs. Automation configuration locks serialize assignment. A no-op assignment keeps the Service revision and audit unchanged.

GET /teams/{id}/activity requires teams:read plus administrator or current team-owner authority. It shares the UI event query and redacted projection. Ordinary members and auditors cannot read this administrative history. Signed keyset cursors must belong to the same team; responses omit actor emails, IP addresses and the raw audit envelope. SCIM-managed team history remains readable.

These operations do not complete the wider user identity lifecycle plan.

User role and lifecycle administration

GET /users/{id}/administration requires users:read and a current active administrator. It returns safe lifecycle metadata and a User ETag. It excludes credentials, invitation capabilities, email addresses and session security versions.

PATCH /users/{id}/administration requires an explicit users:write scope and a current active administrator. Existing admin:manage credentials do not acquire this new destructive capability. Administrators must deliberately issue or rotate a credential with users:write. Send both a unique Idempotency-Key and the concrete If-Match validator from the administration read. Any User update, including a legacy UI, SCIM or OIDC update, invalidates that validator. Re-read after a 412; do not blindly overwrite a concurrent change.

The strict command body is one of:

{ "operation": "set-role", "role": "RESPONDER" }

{ "operation": "deactivate" }

{ "operation": "reactivate" }

{ "operation": "revoke-sessions" }

Role changes share the UI's explicit manual override policy and set roleSource to MANUAL. Configured identity-provider mappings may reassert a role at the next synchronization; this operation does not modify provider mappings or SCIM identity ownership. Administrators cannot change their own role or deactivate themselves. Concurrent changes preserve at least one active administrator.

Deactivation revokes the user's API keys, unused invitation/reset tokens, OIDC linking approvals and devices. Reactivation only accepts disabled users; users with a password become active, while passwordless users return to invited status and need the normal invitation/linking workflow. It restores no revoked credential or linking approval. Session revocation increments the session security version and marks registered sessions revoked while retaining API credentials.

The user change, canonical attributed audit and retry receipt commit together. Session caches are evicted after commit. Replaying a completed request rechecks current administrator authority and does not repeat its effects. Setting an already-manual role to its existing value or deactivating an already-disabled user does not increment revisions or duplicate audits.

This is partial identity coverage. Dependency inspection, bulk/profile parity and final combined certification remain tracked separately in the enterprise workflow inventory.

Reviewed user invitations

Administrators first read GET /users/invitation-policy with users:read and explicitly select PASSWORD or SSO, and EMAIL or MANUAL delivery. POST /users/invitations requires explicit users:write, an idempotency key, and the reviewed authentication-policy and OIDC configuration versions. SSO Required policies reject password invitations. Existing active or disabled identities cannot be taken over through invitation creation.

GET /users/{id}/invitation returns safe invitation state and a User ETag. Renewal through POST /users/{id}/invitation requires that ETag, a fresh idempotency key and the reviewed linking-approval revision. Renewal invalidates earlier password capabilities and fences concurrent SSO approval changes. Ordinary active administrators can issue invitations; emergency recovery credentials cannot use these operations.

Manual delivery returns a one-time fragment URL in the initial response. Store it securely: idempotent replays return inviteUrl: null and ALREADY_DISCLOSED, and receipt storage never contains the capability. Email delivery atomically queues an encrypted notification; QUEUED does not mean delivered. The worker checks invitation generation, status and authentication trust before sending. Superseded or revoked invitations are skipped. Pending email invitation admission is bounded at 64; capacity rejection returns HTTP 429. Provider failures remain subject to the durable notification retry policy.