Federation-Compatible Gateway Implementations

The following GraphQL Gateways support acting as an Apollo Federation Gateway.
Their support is tracked in The Guilds's gateway compatibility repository.

📖 Read more about our methodology and why we created this audit

Gateway Compatibility Test Cases Test Suites Full report
GraphQL Mesh 100.00% ✓ 163 ✓ 40 View report
Apollo Router 99.39% ✓ 162 ✗ 1 ✓ 39 ✗ 1 View report
Apollo Router (Rust-based QP) 95.71% ✓ 156 ✗ 7 ✓ 37 ✗ 3 View report
Cosmo Router 63.80% ✓ 104 ✗ 59 ✓ 19 ✗ 21 View report
Grafbase Gateway 46.63% ✓ 76 ✗ 87 ✓ 13 ✗ 27 View report
✓ Passed tests
✗ Failed
Perfect compatibility
75% and higher
Less than 75%

Methodology and why this audit exists

Advancing the GraphQL Ecosystem by introducing an open-source audit for Apollo Federation Gateways.

At The Guild, we are committed to advancing the GraphQL ecosystem by collaboration and open-sourcing knowledge and solutions. Recently, we decided to audit various GraphQL Gateways that support acting as an Apollo Federation Gateway. Unlike existing benchmarks that primarily focus on performance aspects, our audit measures the correctness and compliance of these gateways with the Apollo Federation specification. This audit is now fully open-source, and we are excited to share our methodology, findings, and detailed test results with the broader community.

Why We Built the Benchmark

We built this audit for two main reasons. First, as maintainers of GraphQL Mesh (our Gateway for GraphQL Federation), in order to be fully compatible with the Apollo Federation specification, we needed a set of tests covering a wide range of aspects of Apollo Federation. Instead of building something solely for ourselves, we wanted to create a solution that is generic enough to be used by everyone in the community.

Second, we noticed that many GraphQL gateways lacked comprehensive testing and weren't fully compliant with the Apollo Federation spec. Existing benchmarks focused mainly on performance, leaving a gap in assessing correctness and compliance. By openly sharing our audit, we aim to help the entire community improve their implementations and achieve higher compatibility.

Audit Methodology

Our benchmark consists of a thorough examination of each gateway's compatibility with the Apollo Federation specification. We designed 160+ test cases, grouped into 40 test suites, to cover a wide range of scenarios and edge cases. These test cases were crafted based on our experience with Apollo Federation and the feedback received from users of GraphQL Mesh and GraphQL Hive.

Audited GraphQL Gateways:

The testing process involves:

  • Setup: Each gateway is set up with the necessary configurations to support Apollo Federation.
  • Subgraphs: Each test suite contains a set of subgraphs that are started alongside a gateway. These subgraphs are GraphQL APIs serving data over HTTP.
  • Execution of Test Cases: The test cases are run sequentially against each gateway. These tests cover various aspects of the Apollo Federation specification, including error handling.
  • Result Analysis: After running the tests, we analyze the results to determine the compatibility level of each gateway. The compatibility score is a percentage of the total number of test cases passed.

The audit is conducted in a controlled environment to ensure consistency and fairness. We run the tests on the latest version of each gateway, and we update the audit regularly to reflect changes and improvements.

The audit is fully open-source, and we encourage the community to contribute by adding new test cases, improving existing ones, or suggesting new features.

Our tests cover a comprehensive range of directives, including @interfaceObject, @key, @external, @provides, @requires, @extends, @inaccessible, @shareable, @composeDirective, and @override.

However, due to limitations of the Enterprise license of Apollo Router, we are currently unable to test the @authenticated, @policy, @requiresScopes, and @override(label:) directives. We plan to include these directives in our tests as soon as we have access to the Enterprise license.