MCP Gateway
With the growing popularity of MCP, numerous MCP servers are now available. These MCP servers can be started locally using stdio or connected to remote servers using HTTP. To use them, we can simply configure the MCP server's connection method based on the MCP client's requirements. The configuration methods are generally similar.
The following JSON content demonstrates common MCP server configurations. It includes a server using stdio and one using HTTP SSE.
{
"mcpServers": {
"local-stdio-server": {
"command": "/path/to/my-mcp-server",
"args": ["--option", "value"],
"env": {
"MY_ENV_VAR": "some_value"
}
},
"remote-http-sse": {
"url": "https://my-mcp-server.example.com/sse",
"additionalHeaders": {
"Authorization": "Bearer YOUR_API_TOKEN"
}
}
}
}
The application's MCP client establishes one-to-one connections with multiple MCP servers. The actual connection method is shown in the figure below.
This connection method has some issues. The configuration of these MCP servers is maintained by the applications themselves and cannot be centrally managed. For example, new versions of the stdio server may add new startup parameters, or the HTTP server URL may add new query parameters. Clients need to manually update the MCP server configuration to apply these changes.
The typical solution to this decentralized management problem is to add a central management center. Similar issues arise with open APIs. API gateways address these issues, providing a single access point and numerous additional features. In MCP scenarios, MCP gateways can also solve the same problem. In fact, for MCP servers using HTTP transport, MCP gateways are essentially API gateways. However, MCP handles specific entities such as prompts, resources, and tools in a unique way.
With the MCP gateway, MCP clients connect to the MCP gateway, which in turn connects to the actual MCP server. Client configuration is greatly simplified, requiring only connection configuration to the MCP gateway. The gateway provides centralized management of MCP servers, including adding and removing MCP servers, filtering tools, and controlling server and tool permissions.
In other words, clients connect to a virtual MCP server. This server contains prompt templates, resources, and tools aggregated from multiple MCP servers by a gateway. The gateway can employ different aggregation strategies, ranging from simply aggregating tools from all MCP servers to controlling tool visibility based on user permissions.
The following diagram illustrates how to connect to the MCP gateway.
The MCP gateway has many practical use cases. For example, an MCP server may contain tools that pose security risks. After connecting to the gateway, these tools can be disabled on the gateway, preventing them from appearing in the tool list and becoming unavailable to clients. Another example is the following scenario: a file server stores shared files in different directories. On the file server, you can start multiple MCP file servers using stdio, then use the MCP proxy to convert them to HTTP, and finally connect them to the MCP gateway. At the gateway level, you can restrict access to these MCP servers based on user permissions, thereby limiting the file directories that users can access.
The following diagram illustrates the connection method for this scenario.
If you simply aggregate multiple MCP servers, as the number of MCP servers increases, the number of tools included will also increase. Simply providing all these tools to a large model, which then makes the selection, will affect the effectiveness of the large model's tool selection. Finding the most appropriate tool for a specific task becomes a critical consideration.
Here are several solutions for reference. The Nacos MCP router provides a routing mode. In this mode, the MCP router acts as an MCP server and provides three tools. The search_mcp_server tool searches for MCP servers based on task descriptions and keywords. Server information is stored in the vector database Chroma. Searches can be performed based on keywords and semantics. For the MCP servers found, use the add_mcp_server tool to install them. The use_tool tool invokes tools on the MCP server.
Another solution is to create a unique virtual MCP server for each agent or application scenario. This virtual MCP server aggregates different MCP servers and the tools within them. The aggregation method can be selected manually or based on rules. MCP servers and tools can be dynamic, but they do not change based on prompts sent to the master model. For example, an agent writing code and an agent performing code review use different tools, but they may also share some similarities. On the MCP gateway, virtual MCP servers corresponding to these two agents can be created and configured for use.
In the figure below, the MCP gateway manages three MCP servers: GitHub, Code Execution, and Playwright.
- Code writing agent uses two MCP servers: Code Execution and Playwright.
- Code review agent uses two MCP servers: Code Execution and GitHub.
On the MCP gateway, virtual MCP servers are created for each agent. Agents only need to connect to the corresponding MCP server. The actual MCP servers aggregated by these two virtual MCP servers can be adjusted as needed. The Agent configuration does not need to be modified.
Many MCP gateway implementations exist. Here are a few open-source options.
Docker MCP Gateway offers the greatest advantage of deep integration with Docker. Using Docker to run the MCP server solves the MCP server deployment problem. The gateway provides a directory for MCP servers, allowing you to install MCP servers into the gateway. The Docker MCP Gateway uses a simple aggregation strategy. Adding or removing MCP servers from the gateway affects the tools it provides externally.
IBM's ContextForge is powerful and provides many enterprise-grade features, including a management interface, deployment, and observability support.
Octelium is a zero-trust resource access tool designed to replace VPNs. The MCP gateway is just one of its sub-features. The server is licensed under the AGPLv3 license and may not be suitable for internal enterprise use.
In addition to these newly created projects, some API gateway providers are also moving towards AI and MCP. For example, the open-source API gateway Kong offers new AI gateway functionality, including an MCP gateway. Kong leverages its existing plugin system to offer plugins related to large models and MCP.