All 20 Criteria
P1-A Structured Data — 1/5
No schema.org or JSON-LD markup found on portkey.ai homepage or pricing page. Web search for "Portkey schema.org JSON-LD" returned only generic schema.org guides with no Portkey-specific results. robots.txt is fully open (Allow: /) but there is no structured markup layer for agents to parse.
P1-B Machine-Readable Pricing — 3/5
Pricing tiers at portkey.ai/pricing are clearly structured: Free (0, 10K logs/month), Production ($49/month, 100K logs, $9/100K overage up to 3M), Enterprise (custom). Feature comparison table is detailed and HTML-rendered. Not tagged as schema.org/Offer or PriceSpecification, but the data is parseable with moderate effort.
P1-C llms.txt / Agent Layer — 4/5
docs/llms.txt exists and is referenced in the MCP Gateway documentation. The documentation index covers AI Gateway, Guardrails, Observability, Prompts, Governance, API Reference (Admin API, Inference API, Data Plane), SDKs (Python, Node.js, C#/.NET), 100+ LLM providers, agent framework integrations (CrewAI, LangGraph, OpenAI Agents), and a changelog spanning 2024–2026. Structured for LLM consumption. Minor deduction: lightweight llms.txt entry point at portkey.ai/llms.txt itself redirects rather than serving a clean file.
P1-D API / MCP Availability — 5/5
Full REST API (OpenAI-compatible endpoint at api.portkey.ai/v1/chat/completions), Python SDK, Node.js SDK, C#/.NET SDK, Admin API. Official MCP Gateway at mcp.portkey.ai/{server-slug}/mcp with OAuth 2.1, API token, and JWT authentication. "HTTP Streamable transport for all connections." Agent framework integrations (CrewAI, LangGraph, OpenAI Agents SDK, Claude Desktop, Cursor, VS Code). Open-source gateway with 10.2K GitHub stars processing 1T+ tokens/day.
P1-E Discoverability (GEO) — 4/5
Listed on AWS Marketplace, G2 top-rated, 3,000+ GenAI teams using platform, cited in enterprise AI infrastructure comparisons. Documentation indexed by llms.txt. Open-source GitHub presence provides additional discoverability signal.
P2-A Offer Completeness — 4/5
Single pricing page with numeric tier breakpoints: Free (0), Pro ($49/mo), Enterprise (custom). Feature comparison table maps every feature to tier. Overage pricing explicitly stated ($9/100K). What, who, and how much are all findable. Not fully machine-parseable due to HTML rendering, but very close.
P2-B Scope & Limits — 4/5
Log limits per tier explicitly stated (10K/Free, 100K/Pro, 10M+/Enterprise). Overage policy documented. Budget limit policies configurable per team/workspace. Rate limiting per user available. Log retention periods stated: 3 days (Dev), 30 days (Pro), configurable (Enterprise). Clear, structured documentation.
P2-C Substitution Rules — 3/5
Auto-fallback and retry logic are core documented features of the AI Gateway — described as built-in load balancing, fallbacks, and retries. These are agent-native substitution behaviors. However, they're framed as product features rather than explicit "if unavailable, do X" rules. Partial credit for native fallback infrastructure.
P2-D Conditional Logic Transparency — 3/5
Tier boundaries clearly disclosed for Free/Pro. Enterprise requires "custom pricing" contact. SSO, VPC hosting, advanced compliance (SOC2, HIPAA) are Enterprise-only without thresholds disclosed. Conditions are documented across the feature comparison page but Enterprise gates are opaque.
P2-E Semantic Precision — 3/5
Mix of precise and imprecise: "99.99% uptime SLA," "1T+ tokens/day," "120M+ requests/day," "3-line code integration," "250+ LLMs," "$180M+ annualized AI spend managed" are precise. Claims like "blazing fast" and "no-brainer" appear in marketing copy. Overall better than average but not fully precise.
P3-A Verifiable Performance — 4/5
Public status page at status.portkey.ai (307+ outage events tracked by StatusGator over past year, showing transparency). G2 top-rated with developer reviews. 99.99% uptime SLA published. SOC2 Type 2, ISO27001, GDPR, HIPAA compliance certifications. AWS Marketplace listing adds third-party validation. Deduction: status page shows real outage history including incidents, which is honest but shows service has had reliability events.
P3-B Scoped Permissions — 4/5
Explicit multi-level RBAC: organization, workspace, team, and individual user levels. OAuth 2.1 with JWT validation and identity forwarding. Integration with Okta and Entra (enterprise identity providers). Rate limiting and budget policies per team. Fine-grained "MCP Invoke permissions" at key level. Close to 5 but no documented time-bounded or amount-bounded per-agent key scoping.
P3-C Audit Trail — 3/5
Audit logs available at Enterprise tier. MCP Gateway logs every tool invocation (server, tool, inputs, outputs, latency, authorization status) in unified traces. OpenTelemetry export for traces. "Configurable exports to data lakes" at Enterprise. Deduction: audit logs not available below Enterprise tier; Pro users have 30-day log retention but it's described as observability, not a formal audit trail API.
P3-D Behavioral Consistency — 3/5
Changelog spanning 2024–2026 documented in llms.txt index. Open-source gateway provides version control transparency (GitHub). API versioning present. No explicitly published notice period for breaking changes or version deprecation policy found.
P4-A Friction-Free Activation — 5/5
Free developer tier (no credit card required per typical SaaS convention, though not explicitly confirmed), API key from dashboard, 3-line code integration ("baseURL, api_key, x-portkey-provider header"). Self-serve from free to Pro ($49/mo) without human gate. Open-source self-hosted option requires no account.
P4-B Agent Decision Signals — 3/5
Free tier for trial. Clear price/log-count breakpoints signal upgrade triggers. Budget alerts configurable (agents can be set to notify at thresholds). No explicit "agent-legible" programmatic signal for when to act, upgrade, or initiate. Deduction for absence of a /status or /readiness endpoint agents can query to confirm service state before high-stakes requests.
P5-A Integration Depth — 4/5
1,600+ LLM integrations, 250+ providers, 100+ guardrail partners, native integrations with CrewAI, LangGraph, OpenAI Agents SDK, LangChain, LlamaIndex, Langfuse, LangSmith. Prompt management with version history. Virtual keys for credential management. Switching means rebuilding routing logic, guardrail configs, prompt libraries, and virtual key mappings across all integrated providers. High real switching cost.
P5-B Agent Memory Layer — 3/5
Account-level activity logs with custom metadata and filtering. Prompt version history accessible via API. Semantic caching retains and reuses prior responses. Feedback mechanisms for tracking agent output quality. Not a purpose-built "agent memory" layer but provides richer history than most competitors.
P5-C Programmatic Renewal — 2/5
Auto-billing in place for Pro tier. No documented agent-accessible billing management API (check usage vs limit, trigger tier upgrade, adjust budget programmatically). Budget limit policies are configurable but require human setup in the dashboard initially.
P5-D Compounding Value Signal — 3/5
Semantic caching compounds value over time as cached responses accumulate. FinOps + Executive Dashboard (Enterprise) shows cost optimization trends. Cost savings data surfaced in product ("significant annual cost optimization"). Not explicitly exposed via API metadata so agents can't read their own compounding value signal directly.
Rubric v1 (April 2026). Scores reflect the company's state on the audit date and may have improved since.