Enterprise MCP architecture and threat model
How the MCP server authenticates, authorizes, isolates and audits assistant requests, and what is still outstanding.
Historical foundation baseline: f9f772a1fb43798bf45400c8e2ec7a0a823fa43f. Generate the current source-labelled inventory with npm run --silent mcp:coverage; inventory counts and local test passes do not certify a release.
Status: shipped in OpsKnight 2.2.0, off by default (an administrator turns it on). The formal release-certification manifest (the 15 gates in release status) has not been signed for 2.2.0, so treat MCP as a supported but not release-certified feature. What has and has not been verified is listed in release status. This document records engineering decisions and remaining work.
Execution boundary
src/app/mcp/route.ts validates host/origin and the PostgreSQL-backed OAuth grant before reading a request body, validates bounded UTF-8 JSON and duplicate keys, charges weighted quota once, and delegates protocol negotiation to official @modelcontextprotocol/server 2.4.0. Revision 2026-07-28 is served natively. Revision 2025-06-18 and 2025-11-25 clients, and 2025-03-26 clients on a best-effort basis (no structured tool output for them; a POST without MCP-Protocol-Version is treated as 2025-03-26), are served statelessly (legacy: 'stateless': one fresh server per request, no session); their notifications return 202 before any quota is charged. Sessions are refused, and Tasks and subscriptions/listen are 2026-07-28 only. SDK request metadata and header/body checks remain authoritative.
The canonical REST operation registry, finalized OpenAPI document and mutation contract supply IDs, scopes and schemas. src/lib/mcp/catalog/review.json contains explicit per-operation disposition, risk, justification and route/contract fingerprints. The route-policy AST check derives mandatory authorization actions and their complete scopes, including joined projections. Registration, discovery, execution and transactional mutation checks require the complete scope intersection. New IDs, removed IDs, action/scope/schema changes and route changes require reclassification; coverage is not a certification of executable parity. The current inventory (2026-11-09) classifies 325 operations, including 147 generic executable candidates. Approval-gated adapters, equivalent mappings, pending operations and reviewed exclusions are tracked separately in the generated parity report; these counts are not full parity certification.
src/lib/mcp/core/execution.ts invokes static allowlisted REST handlers in process. The exact NextRequest is bound to an authenticated actor using AsyncLocalStorage; no user-controlled header or network route can manufacture this capability. Nested REST calls reuse top-level credential admission, while retaining their own domain schemas, role checks, resource filters, transaction, audit and replay behavior. There is no raw HTTP, filesystem, SQL, shell or JavaScript execution tool. This is a transitional contract adapter, not completed extraction of all domain commands. Joined incident context currently performs up to four bounded domain calls (the incident, then its timeline, related incidents and recent notes) and requires measurement against a direct projection.
Mutation transactions use the existing idempotency journal and require a current grant, client, principal, token, policy and resource decision before replay or commit. Grant/client/user rows are locked through commit; incident writes lock their target and recheck resource visibility. Current membership changes invalidate a previously projected actor. Requests require an explicit idempotency key for R1 operations; multiple operations never constitute one atomic transaction.
Independently reviewed runbook execution
runbook_prepare_execution and runbook_execute_approved add one controlled sensitive workflow alongside the generic operation candidates. Direct call_operation(executeRunbook) remains unavailable. Both tools require writes enabled and all four scopes: runbooks:execute, runbooks:read, incidents:read, services:read, intersected with current native permissions. No MCP tool can approve (runbook_get_approval only reports status), and the commit input carries no approval text or caller-supplied action parameters.
Preparation calls the existing native resolver and previewRunbookExecution, creating a RunbookPreviewAuthorization capability extended with independent-review provenance. MCP previews live for the policy's review window (approvalTtlMinutes, default 240) instead of the native five minutes, and approval sets a commit deadline (approvalCommitWindowMinutes, default 30, capped at 24 hours after the request). It starts no execution or advance job. Its raw preview token and canonical request are AES-256-GCM encrypted using the existing keyring; MCP receives only the preview handle, digest, expiry and native review URL. The existing runbook plan digest binds the actual inputs, resolved version/checksum, target and response context. The native review summary uses the existing resolvedApprovalPlan masking policy; for assistant-prepared requests the review page also lists each submitted input (redacted like run output, cut at 200 characters and marked when shortened or partly hidden), while a secret reference, a SECRET_REF input and an input whose name looks like a credential are shown only as withheld. The native preview keeps withholding every input value.
An active user with native runbook-read and runbook-approve capabilities (Responders and Administrators by default), plus current resource visibility, reviews through /settings/integrations/mcp/approvals/:id. Eligible approvers receive an in-app notification inside the prepare transaction (at most 50, never the requester, no email). Next Server Actions supply the authenticated session identity; form data cannot choose an approver. Decisions compare the frozen digest and expiry under a row lock, persist the approver's session version, and emit one transactional audit event. A rejected or already-reviewed proposal cannot receive another decision. A rejection may store a reason (at most 500 characters), returned to the assistant labelled untrusted. Normally a second person decides; an administrator may approve their own request only with an explicit confirmation, recorded as selfApproved. Anyone else needs a second person.
Execution locks current MCP security state and revalidates requester scope/resource access. The existing native execution command checks the frozen plan again, requires the stored approval and server-only grant provenance, and rechecks the reviewer's current capabilities, session version and resource visibility. Current tokens must belong to the same grant and remain unrotated/unexpired; legitimate token refresh can continue a pending approval using its new valid credential, while the originating credential is retained only as provenance. Ordinary UI/REST execution cannot consume an independently reviewed preview without that internal provenance.
The canonical execution, preview consumption, advance job, audit and durable idempotency result commit together. Pre-commit failure rolls all of them back; a lost response can be retried with the identical key and handle. A second key cannot consume the same preview again. Serializable retries handle competing decisions/commits. Request envelopes are cleared on rejection or successful execution; unused encrypted envelopes follow the existing native preview retention window (30 days after expiry). The receipt identifies the durable native execution and its UI URL. Task-capable clients can request the negotiated Tasks wrapper for approved runbook execution; it does not waive native per-step approval, Agent policy, signing or runbook recovery controls.
Other reviewed domains use the separate operation prepare/review/commit adapters listed in the canonical catalog. Unsupported sensitive operations remain blocked. These implementations do not establish full workflow parity or production crash/HA certification.
Identity and consent
The self-hosted authorization-code flow uses @node-oauth/oauth2-server 5.3.1, PKCE S256 and exact preregistered redirect URIs. Consent requests are owned by the signed-in user, expire after five minutes and are consumed by a compare-and-set inside a serializable transaction. Native Server Actions provide the browser mutation boundary. Authorization responses carry the issuer identifier.
Access tokens live for five minutes. Grants slide by the policy idle period (default 30 days) and end 90 days after sign-in at the latest. PostgreSQL stores keyed HMAC digests, not bearer strings. Each token is bound to a grant, canonical MCP resource, issuer installation and registered client. Refresh rotation and code consumption commit with issuance. Reuse revokes the grant, except a replay of the just-rotated refresh token within the reuse grace period (the refreshReuseGraceSeconds policy column, default 30; it has no setting in the administration UI) while its successor is unused, which withdraws that successor and reissues. Grants pin User.mcpRevocationVersion, which moves only on security events (password change or reset, email change, demotion from ADMIN, deactivation, sign-out everywhere, admin session revocation, SCIM disable); a mismatch returns MCP_ACCESS_CHANGED. Ordinary role and team changes narrow access live without revoking. User disabling, client revocation and grant revocation also deny subsequent protected operations. Scopes are intersected across client, grant, token and current role/resource authority; admin:manage is prohibited as an MCP wildcard.
Each operation resolves grant/client state, current user role/status/session version and team memberships in one parameterized PostgreSQL statement. The optional credential check is bound to the same grant and requires an unrotated, unexpired token. This replaces multiple ORM round trips with a single current-statement snapshot; it does not cache authority. Top-level admission still verifies the presented bearer digest and charges quota, and sensitive mutations retain their transaction-lock rechecks. Ordinary read authorization is evaluated again when the next operation begins, rather than holding security locks for an entire investigation.
Required configuration needs no MCP environment variables (optional ones are MCP_FORCE_DISABLED, MCP_PUBLIC_ORIGIN, MCP_ALLOWED_HOSTS, MCP_DATABASE_URL, MCP_DATABASE_POOL_SIZE and MCP_LOG_SAMPLE_SERVED): the canonical public origin is the App URL from Settings → System, and token digests use an HKDF-SHA-256 key derived, with MCP-specific domain separation, from the active application encryption key. Rollout, mutation enablement and workload budgets live in PostgreSQL and are administered through Settings → Integrations & ChatOps → MCP (AI Assistants). Native user consent, grant/client revocation and administration are audited. Expired grant/code/token/consent metadata is removed in bounded batches by existing scheduler maintenance after a seven-day replay-detection retention window.
Hostile inputs and privacy
Threat actors include malicious remote clients, an authorized user targeting inaccessible IDs, compromised model instructions in business text, leaked or replayed bearer credentials, stale cached tool catalogs and malicious redirect/metadata endpoints. Host and Origin checks use the configured origin, not forwarded headers. Redirects are preregistered; there is no token passthrough or arbitrary egress endpoint. The only outbound fetch is the bounded, DNS-pinned, redirect-free client metadata document fetch to public HTTPS addresses (see the OAuth sections below); dynamic client registration is an administrator-enabled policy.
MCP business text is returned as untrusted evidence and never interpolated into privileged instructions. Recursive output projection removes credential/PII fields and bearer or query-bearing URLs. Resources invoke the same operation authorization as tools; resources/list reads one bounded page per domain through the domain's list operation and skips a domain it cannot read. The public Server Card (/mcp/server-card) and AI Catalog (/.well-known/ai-catalog.json) are static metadata with no credentials and are advisory only: clients verify them against server/discover. Safe failures omit raw REST problem details. Prompt templates do not confer authority and cannot approve actions. Static schemas may be cached; authorization and business data are not cached.
Residual risks requiring release tests include secrets embedded in arbitrary free text, resource ownership races across all domains, OAuth consent/browser parity, the complete role/team/scope matrix, schema abuse across every adapter, query amplification and slow/oversized projections. Field-name redaction alone does not prove absence of secret disclosure.
Workload isolation
Request bodies are capped at 64 KiB and 24 nesting levels; the complete serialized response, including the JSON-RPC envelope and text/structured duplicates, is capped at 256 KiB. Plans allow eight steps, two concurrent independent reads, a ten-second cooperative deadline and a per-operation default page (20 when none is advertised, commonly 25; maximum 50). Plans cannot call plans, mutate, execute arbitrary code or fan out on wildcard IDs. Shared PostgreSQL quota is charged by declared operation cost. Nested REST READ admission is reused; native WRITE and SEARCH_EXPORT distributed quotas remain enforced. A process-local admission semaphore sheds authentication work and admitted MCP requests; it never decides authorization or shared quota. Cancellation retains the admitted slot until actual outstanding domain work settles, including non-cancellable database operations.
OAuth token/revoke endpoints apply 60 requests/minute per known trusted peer and a separate installation-wide 2,048 requests/minute safety ceiling per endpoint. When Next.js does not expose the socket address and proxy headers are untrusted, only the global ceiling applies; arbitrary forwarded addresses cannot establish peer identity.
MCP HTTP/OAuth handlers and operation execution select a separate, lazy Prisma client through an in-process async context. Shared domain commands retain their existing implementation and explicit transaction clients; ordinary web traffic, native emergency revocation and readiness checks use the primary client. Next.js chunks share the same context/client globals. The MCP pool defaults to two connections per process, MCP_DATABASE_POOL_SIZE sets it between 1 and 4, waits at most one second for a pool slot and has a three-second connection timeout. PostgreSQL startup settings enforce a five-second statement timeout, one-second lock timeout and five-second idle-in-transaction timeout, overriding relaxed options in an inherited URL. Shutdown disconnects both pools.
The MCP datasource uses DIRECT_DATABASE_URL, then DATABASE_URL, so no MCP-specific variable is required; an optional MCP_DATABASE_URL override can point it at a dedicated login role. It does not inherit WEB_DATABASE_URL. It must address the same logical database through a direct PostgreSQL endpoint. URLs marked pgbouncer=true are rejected: PgBouncer can ignore configured startup parameters, and that MCP topology remains uncertified. The existing web tier can continue to use its PgBouncer route. A dedicated MCP login role is optional infrastructure configuration and must have the privileges required by the same domain commands; this does not create a machine grant or a new application principal.
Deployment connection preflight reserves the MCP pool for every web/integrated replica even while rollout is disabled, separately from pooled web demand and direct worker demand. At defaults, split mode reserves 35 connections (previously 33), integrated mode reserves 13 (previously 11), and two-replica split HA with two web PgBouncer replicas reserves 80 (previously 76). These are arithmetic connection budgets, not measured capacity. Operators must reserve PostgreSQL/system headroom beyond the configured application limit.
Real PostgreSQL tests verify independent backends and transaction settings, pool exhaustion while primary reads continue, SQL cancellation/recovery and lock contention. These limits do not create PostgreSQL CPU/I/O capacity or prove worker latency isolation. Database-wide saturation shedding, expensive-operation limits, dedicated-role/PgBouncer topology verification and matched mixed-ingestion/worker load remain release blockers. No baseline latency, throughput cliff or production p95/p99 capacity is claimed.
tests/integration/mcp-query-profile.test.ts measures the actual MCP route for discovery, incident search, joined context, a three-step read plan and four concurrent reads sharing a grant. The owned local fixture has one incident and a two-connection direct PostgreSQL MCP pool. It counts SQL events, authority/policy/domain reads and quota writes without retaining SQL, parameters or identifiers. The profile explicitly includes top-level authentication and quota admission. Joined context and the three-step plan each retain two writes: top-level weighted MCP admission and the native related-incidents SEARCH_EXPORT quota. ORM batching can reduce statement counts for identical concurrent reads; aggregate concurrency samples are not serial per-request costs.
The query-profile suite enforces small-fixture statement ceilings (3 discovery, 8 search, 20 joined context/three-step plan, 32 for four concurrent searches) and checks quota accounting. These are regression guards for this fixture, not service-level capacity promises. Set MCP_QUERY_PROFILE_OUTPUT to an absolute path outside the repository for the source-labelled JSON artifact. Local setup writes it automatically; CI uploads it with its evidence. Summed engine SQL duration is not a measurement of pool wait, CPU, WAL or paging contention.
Required implementation still outstanding
- Vetted external issuer configuration and scoped machine-identity issuance. Public-client CIMD already uses DNS-pinned SSRF defenses; signed private-key client assertions are not implemented.
- Independent prepare/approve/commit for the remaining sensitive domains. Runbook execution now reuses native preview/approval controls with local authority, concurrent-decision/commit and pre-commit rollback tests; production recovery certification remains outstanding.
- Production recovery/HA certification of the negotiated durable Tasks wrapper for
runbook_execute_approved. Task ownership, polling, cancellation and unsupported-client handling are implemented; arbitrary operations do not become durable tasks. - Remaining appropriate domain commands, full UI/REST/MCP mutation parity and semantic capability review; classification is not execution coverage.
- Optional MCP Apps and stdio bridge, real supported-client consent/read/write/task verification.
- Database-wide saturation shedding and reproducible performance/fault/HA/deployment certification, backup/restore and production-scale upgrade/rollback evidence. Independent local pool/SQL budgets are implemented; they do not certify shared-database capacity.
Sensitive operations stay unavailable while these boundaries are incomplete. The separate manual MCP release workflow requires reviewed final-SHA evidence attached to the tagged GitHub Release and a protected environment approval. Passing PR automation or local unit tests cannot release this feature.
Automatic assistant onboarding
The optional RFC 7591 registration endpoint /oauth/mcp/register lets clients add the server URL and start native OAuth without administrators copying per-installation client IDs or callback ports. Discovery advertises it only while MCP and automatic registration are enabled with a nonempty scope ceiling. Registration creates an opaque public-client identity, never a grant, authorization code, token or client secret. PKCE S256, exact redirect matching, explicit per-user native consent, current roles/resources, and independent mutation approval remain mandatory.
The administrator's scope ceiling applies at registration, consent, code/refresh exchange and every protected operation. Existing stored client scopes cannot override a later policy reduction. Disabling registration blocks existing dynamic clients immediately. Dynamically registered names are explicitly identified as unverified in native consent, and the return address is displayed. Metadata-document (CIMD) clients share the same switch and ceiling; manual clients keep their own stored scopes.
Registration accepts at most 8 KiB, five exact redirects (HTTPS, loopback HTTP or a native app scheme), and only public none token endpoint authentication. It rejects credential-bearing metadata, arbitrary grant types, unknown/duplicate/prototype keys and scope escalation. Informational client URLs are never fetched or displayed as trusted identity. Separate per-process registration admission permits three in-flight requests; it does not consume ordinary MCP authentication slots or hold a database transaction while reading a request body. Shared PostgreSQL registration limits are ten requests/minute per trusted peer and sixty/minute installation-wide. Unknown peers share the global ceiling. The active dynamic-client registry is capped at 5,000 entries under a policy-row transaction lock.
The existing maintenance worker removes up to 500 never-authorized registrations older than 30 days per pass (when the registry is full, registration also replaces the oldest never-used registration older than one hour), skipping locked rows and live consent forms. Authorization creates a consent form under a shared client lock; cleanup checks pending consent under an exclusive lock. Once a client has received native authorization, its identity is retained independently of grant/token retention so cached client IDs remain reusable. No Redis, new worker role, provider key or MCP environment variable is introduced.
The migration is additive and default-off for existing installations. Enable registration only after all replicas run code that enforces dynamic-client policy. An older application interprets such rows as ordinary clients, so rollback requires disabling MCP and revoking dynamic client rows before returning to older code. Existing revokedAt checks then continue to reject those identities on both versions. Retain the columns; do not use a destructive down migration.
Authentication availability and metadata hardening
Anonymous /mcp requests must prove current bearer authority before the application reads their body or occupies a work slot. Authentication admission is released before body reading. The complete grant/credential/user/policy authority is refreshed after body completion, including for discovery, so revocation or permission changes during an upload cannot reuse a stale actor. Each authentication reads token, grant, client, user and policy in one statement; the query-profile budgets account for this security check. The weighted PostgreSQL quota is then charged once for the validated operation cost before execution. Token and revocation endpoints each have eight admission slots with a 250 ms wait before 503 temporarily_unavailable with Retry-After, separate from the four MCP authentication slots and the three registration slots. A peer that presents 30 bad bearer tokens within a minute receives 429 without a database lookup (failing open when the peer address is unknown). All request bodies retain their byte and total-time limits. These process-local bounds complement shared quotas; they do not replace ingress connection limits or production load certification.
CIMD follows the Automatic connections switch and its scope ceiling; an optional trusted-hosts list narrows it. DNS resolution and the HTTPS exchange each have a three-second absolute deadline. HTTP failure, oversized content or timeout destroys the response and request instead of draining an unbounded body. Every resolved address must remain public; the TLS connection stays pinned to the validated address, follows no redirects, and carries no application cookies or credentials. Metadata parsing rejects duplicate/prototype keys and excess nesting.
This server advertises only public none token authentication. When a metadata document publishes several authentication methods, the intersection must explicitly include none; a private-key-only document is rejected. A same-origin HTTPS jwks_uri may appear as informational metadata for a client that also supports the public flow, but it is never fetched, persisted or used to authenticate a client. Public keys are not client secrets. This supports modern ChatGPT metadata without claiming private-key assertion verification or downgrading private-only clients.
MCP OAuth endpoints and native MCP settings/consent/review pages carry Cache-Control: private, no-store and Referrer-Policy: no-referrer. Existing TLS, CSP, frame restrictions, native-session checks and permission/approval controls continue to apply. This is defense in depth, not a statement that all production security gates have passed.
Remaining low-severity dependency advisory
After upgrading Zod and the compatible qs dependency, the production audit contains no critical, high or moderate advisory. Eleven low package rows trace to the same upstream deepmerge advisory, GHSA-gcqx-w48m-w2h4. No patched upstream version was published when checked. It is not added to the blocking-advisory baseline, and its presence remains visible in audit counts.
The affected dependency branches include PWA/Workbox/Rollup build tooling and email SDK/rendering dependencies. Review found fixed internal build defaults in the Rollup use and no user-controlled MCP path to SendGrid dynamic-template merging; the OpsKnight adapters send HTML/text rather than React or dynamic-template data. This does not establish whole-application absence of reachability. A scoped future Resend SDK migration can remove one rendering branch; remaining branches need an upstream fix or a maintained, tested patch. Downgrading packages to evade the advisory range, forced transitive major overrides and removing product functionality are not accepted remediation.
Last updated for v2.2.0
Edit this page on GitHub