Skip to main content
Early Preview: Rootly Private Agent is under active development and available only to approved customers. Features, configuration, limits, and APIs may change before general availability. Confirm the approved agent and backend versions with your Rootly representative before production use.
The combined Private Agent can connect to customer-managed Model Context Protocol servers that are reachable only inside your network. The agent calls each server locally over Streamable HTTP and carries bounded tool results to Rootly over its existing outbound gRPC connection. You do not expose the MCP server to the public internet.
Internal MCP support requires matching early-access agent and Rootly backend builds. Confirm availability with your Rootly representative. This guide does not announce a public image or Helm release.
This is different from Custom MCP. Custom MCP is a cloud connector for publicly reachable OAuth MCP servers. Private Agent MCP is a customer-network adapter for fixed internal endpoints using non-interactive service credentials, secret headers, or mTLS.

​
Configure an MCP server

Add one entry for each endpoint and identity. Provider IDs must be stable and unique across every provider registered by your Private Agents.
The URL must include the exact Streamable HTTP endpoint path, normally /mcp. HTTPS is required by default. allow_insecure_http: true is an explicit operator trust decision intended only for deliberately trusted local or private in-cluster testing; the agent does not prove that an HTTP host is network-local. Do not attach credentials to plain HTTP unless the entire network path is trusted. URLs cannot include credentials, query parameters, or fragments, and redirects are rejected. Secret values do not belong in YAML. bearer_token_file, every header_files value, private CA bundles, and mTLS files must be absolute paths to mounted regular files. Bearer tokens and secret header values are reread for requests so projected credential rotation does not require embedding a secret in the configuration. Header names must be valid HTTP field names, use canonical spelling, and cannot override authorization, connection, content, host, or MCP protocol headers. Invalid names fail startup validation instead of failing on the first request. HTTPS verifies hostnames and certificates with TLS 1.2 or newer; there is no skip-verification option. A ca_bundle_file extends system trust and requires an agent restart after trust-root changes. The mTLS certificate and key are validated at startup and reread for new TLS handshakes; restart the agent for an immediate cutover of established sessions. The first version does not run an interactive MCP OAuth authorization-code flow inside the agent. Use a dedicated non-interactive bearer credential, mTLS identity, supported secret header, or a customer-local authenticated gateway. Rootly control-plane credentials are never forwarded to the MCP server.
The agent image runs as UID/GID 65532. For Kubernetes Secret volumes using mode 0440, set Pod-level securityContext.fsGroup: 65532. Give only the agent identity equivalent read access on other platforms; do not make secret files world-readable.

​
Select tools deliberately

allowed_tools is mandatory and uses the MCP server’s exact tool names. Discovery does not automatically grant access. A configured tool is registered only when the server also advertises annotations.readOnlyHint: true. MCP annotations are hints, not a security boundary. Before allowlisting a tool, verify its implementation, accepted commands or query languages, downstream credentials, and provider-side permissions. Keep the MCP server itself in read-only mode when available. A misleading annotation or tool name does not make a mutating operation safe. Every accepted MCP tool is a sensitive read in Rootly. Only an attended owner or admin can use it through AI SRE. Unattended system investigations, ordinary users, observers, and no-access users cannot execute these capabilities. The agent converts each accepted tool into a stable Rootly capability ID. The original provider ID selects the exact MCP server instance; the model cannot supply another endpoint, credential, header, or tool name at call time. Configure a second provider entry when another server, tenant, or identity needs separate routing.

​
Protocol boundary

The MCP adapter exposes tools only. It does not expose MCP prompts or resources, and the client advertises no roots, sampling, or elicitation capability. Automatic multi-round-trip requests are disabled. A tool that requests interactive follow-up input fails instead of asking the MCP server to drive another model, filesystem, or user interaction. Tool schemas are treated as input that is not trusted before they reach AI SRE:
  • the input must be a JSON object schema no larger than 64 KiB;
  • remote and recursive references are rejected;
  • descriptions, titles, comments, examples, defaults, regular-expression constraints, and x-mcp-header transport annotations are removed from the model-facing schema;
  • composition keywords, dynamic additional-property schemas, excessive nesting, and excessive node counts are rejected;
  • nested schemas must declare concrete types, boolean child schemas are rejected, object schemas reject undeclared fields, and array schemas declare typed item schemas;
  • the agent validates every call against the accepted schema before contacting the MCP server;
  • the Rootly backend independently validates the reduced schema grammar before creating an AI tool.
The MCP server still validates the original arguments, including constraints removed from the model-facing schema. Use ordinary typed fields and explicit bounds for the most portable tool contract. Each result is normalized into one object containing an array-valued content, optional structured_content, and is_error. A server that omits content produces an empty array rather than null, so every result matches the capability schema registered with Rootly. MCP tool errors remain visible so AI SRE can correct an argument. Protocol, connection, or authentication failures return a redacted provider error. Oversized results fail instead of being silently truncated; narrow the tool input and retry. Successful MCP output is sensitive evidence and is not generically redacted. It can enter AI context, evaluation traces, and investigation or conversation history under the Private Agent retention model. Registration excludes endpoint URLs, credential paths and values, TLS material, secret headers, and the local tool allowlist.

​
Health, discovery, and changes

Health connects to the endpoint and refreshes the tool list. Protocol version 2026-07-28 uses negotiated discovery and tool listing as its health probe; legacy revisions additionally use MCP ping. A configured tool that is missing, renamed, malformed, no longer explicitly read-only, or outside local schema limits makes that provider unhealthy and removes the unsafe capability set. The same fail-closed behavior applies when the normalized capability inventory exceeds the per-instance registration budget; one oversized MCP server cannot block other providers from registering. Tool-list change notifications and lost sessions revoke call-time approval immediately. The last advertised schema remains available only so an already leased invocation can reach the provider and trigger rediscovery. The agent does not call the MCP server until it rediscovers the tool, rechecks its read-only annotation and schema, and validates the invocation again. The agent’s periodic provider snapshot then publishes the next accepted set. An unavailable MCP server does not stop Kubernetes, Prometheus, Loki, or another MCP instance from starting and reporting health. Overall /readyz returns not_ready whenever any configured provider is not healthy, including a degraded provider; readiness is stricter than dispatch eligibility. In Settings → Private agents, inspect the instance’s health, last check, and registered capability names. Refreshing that page reads the last stored snapshot; it does not force an immediate MCP probe.

​
Limits and compatibility

See Private Agent Limits for concurrency, tool-count, timeout, schema, and result ceilings. Admission is bounded per MCP instance and also consumes the shared runtime capacity. Requests honor the Rootly invocation deadline and local timeout. There is no unbounded local queue, offline result cache, or on-disk MCP result store. The adapter uses the official Go MCP SDK and Streamable HTTP. It negotiates the current 2026-07-28 protocol and supports the legacy Streamable HTTP revisions 2025-11-25, 2025-06-18, and 2025-03-26. You do not configure a protocol version manually. Local stdio and the deprecated HTTP+SSE transport are not supported. The SDK validates and sends parameter headers declared with x-mcp-header in the server’s original schema, while the annotation itself is omitted from the reduced schema sent to Rootly. CI verifies a typed call against the official TypeScript Everything reference server and runs a real pods_list call through a digest-pinned, read-only containers/kubernetes-mcp-server into a disposable Kind cluster. This matrix checks two independent server implementations; it is not certification of every MCP SDK, extension, authentication gateway, or tool schema. For a new server, first test a small, deterministic read-only tool with bounded output. Confirm discovery, health, authorization, cancellation, and result handling before adding broader query tools.