# GraphCanon MCP inspection methodology

Method revision: 2026-10-15. These are dated observations of specific endpoints and access profiles. Services are grouped by use case; the list is not a performance ranking.

## What the current reports measure

The October 15 batch covers 13 external endpoints selected from their official documentation. Eleven returned complete catalogs containing 39 advertised tools in total. Sentry and Linear returned HTTP 401 to anonymous initialization. Their catalogs and authenticated behavior remain unobserved. GraphCanon's own server is a separate reference, excluded from this external sample.

Each metadata run checks protocol negotiation, acknowledgement of the initialized notification, bounded tool-catalog enumeration and SDK metadata shape. Passing the shape check does not validate every JSON Schema keyword or tool behavior. The runner records endpoint, source, access profile, UTC timestamps, protocol and runner version. It supplies no credentials and never invokes tools.

Metadata runner 1.1.0 supports MCP 2025-03-26, 2025-06-18 and 2025-11-25. It is limited to 5 seconds per request, 30 seconds per run, 1 MiB per response, seven requests, five catalog pages and 200 tools. Redirects are not followed. A limit, unsupported protocol or access restriction is recorded explicitly. Unknown and not tested are distinct from failed.

These external snapshots used metadata runner 1.1.0. The subsequent 1.1.1 patch accepts valid SSE line endings and empty pagination cursors, distinguishes a cursor-size limit from server failure, and rejects malformed JSON-RPC error payloads. It does not rewrite the saved observations; each report identifies the runner that produced it.

## Separate read-only smoke calls

Five services also have one recorded tool call each: Microsoft Learn, Cloudflare Docs, Astro, Nuxt and DeepWiki. The tool, public input and expected text were fixed before execution. The report exposes those inputs, assertions, response excerpt and result hash. A pass means the tool returned nonempty text without an MCP tool-error flag and contained the listed case-insensitive text matches. It does not establish complete factual accuracy or success on other tasks.

Smoke runner 1.0.0 uses the MCP SDK with a reviewed endpoint and scenario allowlist, no authentication and no background notification stream. It permits at most one tool call, 12 seconds per request, 35 seconds per run, 1 MiB per response and nine requests. It checks that the selected tool appears in the first catalog page; otherwise it records the scenario as not tested. It does not follow redirects or accept arbitrary caller-supplied tools, arguments or URLs.

All five recorded smoke scenarios passed their narrow assertions. They are different tasks with one observation each. Their timings are not a speed ranking, reliability estimate or cross-service benchmark. These runs do not use an AI model or the ChatGPT or Claude applications. OAuth, authenticated workflows, writes, approval behavior and host compatibility remain untested.

The recorded runs used smoke runner 1.0.0. The subsequent 1.0.1 patch classifies recognized invalid tool results and RPC failures as failed while retaining access, transport and budget uncertainty as unknown. It does not alter these saved results or their success criteria.

## Evidence and storage

The endpoint registry, scenario definitions and runner code are versioned with the application. Every run receives a unique ID and UTC timestamp and is saved as JSON plus readable Markdown. Public pages select a reviewed snapshot through a manifest; older artifacts remain available when a later snapshot replaces the displayed one. Opening a report does not rerun a test.

The metadata artifact contains derived tool/schema facts, a catalog fingerprint and request status, size and duration. Raw tool descriptions and complete schemas are not retained, so the fingerprint alone cannot support full offline re-evaluation. The catalog hash definition and whether it covers a complete catalog or an observed prefix are recorded in the artifact.

Smoke artifacts retain the fixed inputs, assertion results, a bounded response excerpt and a SHA-256 hash of the JSON-serialized, SDK-parsed result. They do not retain the complete response. Neither artifact stores authorization headers or session identifiers. Internal planning fields are omitted from public exports. JSON and Markdown downloads are available on each report.

## What a workflow benchmark would add

A scoped benchmark starts with one user task and roughly three to five meaningful scenarios. Comparable services need the same task, fixtures, expected outcome, access scope and budget. Authenticated testing needs a scoped sandbox identity. Actual client testing needs evidence from that client, with the environment and relevant versions recorded; model API results alone cannot establish ChatGPT or Claude compatibility.

Repeated trials would measure task completion, argument correctness, recovery, latency and cost where available. Separate control tasks would check whether fixes generalize beyond the remediation examples. Every finding should link to an attempt and explain its limits. More test cases do not automatically produce better evidence.

Public URL inspection and automatic release monitoring are not available in this version. They require durable jobs and quotas, controlled outbound networking, private credential handling, tenant access and retention controls. A future MCP tool, API and website form could submit jobs to the same engine and return the same versioned report.

## Free evidence and paid work

Published inspection reports and selected smoke examples are free to read. A proposed free inspection service would expose a bounded subset of the same method, rather than a paid listing score.

Paid engagements can cover a company-specific workflow benchmark, prioritized diagnosis, an improvement sprint and a comparable rerun. Recurring checks become useful once the workflow suite is stable and needs repeat execution. Scope, access, deliverables and price must be agreed before work; this site does not currently sell an automated scan or subscription.

Payment buys the work and its evidence. It cannot change findings, buy a favorable result or imply approval by an AI platform. Customer runs would remain private unless publication is explicitly agreed.