Integrate MCP Tools with AI Agents – Remote MCP server

By benperk · · , , , , , ,

I recently wrote this article Integrate custom tools into an AI Agent which illustrates how to configure tools and an AI Agent to call those tools.  In that article there was no Model Context Protocol (MCP) capability utilized.  MCP provides the following benefits when compared to integrating custom tools directly from code.

  • Dynamic tool discovery which enables an AI Agent to discovery a list of tools dynamically, improving adaptability and reduces maintenance.
  • Interoperability across LLMs occurs when the model is not statically bound to any tool or MCP server.  Instead, the model can be loaded at runtime and easily changed to evaluate inference across different models for performance and output comparison.
  • Standard security eliminates the need multiple authentication protocols or the management of separate keys per MCP server.

To learn about integrating MCP tools with an AI Agent using a Local MCP server read this article: Integrate MCP Tools with AI Agents – Local MCP server

To integrate MCP tools with an AI Agent continue reading through these steps.

  • Create an AI Agent that utilizes the Microsoft Learn MCP server
  • Send initial request to the LLM endpoint
  • Render the Natural Language Processing (NLP) response

In this and other articles the Azure AI Projects client library for Python is utilized.  All references made towards “How to” are in that context unless otherwise stated.

Create an AI Agent that utilizes the Microsoft Learn MCP server

Firstly, a short recommendation concerning Microsoft Learn.  This is the site where you get access to all of Microsoft product, training, troubleshooting, and code samples, it is a goldmine of knowledge.

Creating an AI Agent utilizes the azure.ai.projects.AIProjectClient class using the agents.create_version() method.  Here is a brief snippet of that.

agent = project_client.agents.create_version(
    agent_name="LearnAgent",
    definition=PromptAgentDefinition(
        model=model_deployment,
        instructions="""You are a helpful agent that can use MCP tools
        to assist users. ...""",
        tools=[mcp_tool],
    ),
)
Listing 1: remote MPC server, agent

This instantiates the the agent with the associated model, the instructions which I like to refer to as the system prompt, and the tools.

To utilize an MCP server this first action you must do is register it in your code, like the following.  Notice that the name of this MCP server, mcp_tool is the same which is being passed to the tools parameter when instantiating the agent, as seen in Listing 1.

mcp_tool = MCPTool(
        server_label="api-specs",
        server_url="https://learn.microsoft.com/api/mcp",
        require_approval="always",
    )
Listing 2: remote MCP server, register MCP tools

It is really that simple because the work is being handled by the MCPTool class. Take a look at the azure.ai.projects.models.MCPTool source code here.  Looking at the code helps gain better understanding how all this fits together.  The require_approval parameter is the ‘List of allowed tools for MCP server.’  That value enables the option for you to allow or deny the tools which can be used during the LLM inference.  Some of the tools available on the MCP server may not add any value for this give context of this AI Agent and therefore can be filtered out to remove ambiguity and improve NLP response.

Send initial request to the LLM endpoint

You can certainly capture the user prompt from a bot, but perhaps in some scenarios you already know what the prompt will be, either way it is sent using the input parameter.   This one asks what the conversation is and how it is related to the integration with an MCP server.

response = openai_client.responses.create(
        conversation=conversation.id,
        input="""Explain what the azure.ai.projects.models conversation is  
        in the Azure AI Projects client library for Python SDK.
        What role does it play for accessing remote MCP servers?""",
        extra_body={"agent_reference": 
          {"name": agent.name, "type": "agent_reference"}},
    )
Listing 3: remote MCP server, send inference request

This code will return a JSON response with a list of tools available at the remote hosted MCP server.  As mentioned before, this gives you the opportunity for filtering out any tool which is not applicable in your given context.  Checking for a property named mcp_approval_request and using the McpApprovalResponse class will result in a list of tools that will be used to complete the response.

Here is a list of the selected tools from the Microsoft Learn MCP server that were used to infer the prompted instructions.

  • mcp_mslearn-documentation.microsoft_docs_search – to locate authoritative Microsoft documentation about the Azure AI Projects client library, Conversations API, and MCP integration
  • mcp_mslearn-documentation.microsoft_docs_fetch – to retrieve the full documentation pages (API reference and the Azure AI Projects README) so I could extract exact fields, endpoints, and types for an accurate explanation.

The code required to execute the document search and fetch on the Microsoft Learn MCP server are written by Microsoft.  For any public or protected remotely hosted MCP server tools, this is the case.  Unlike when you implement a custom AI Agent or a locally hosted MCP server, the code running on these remote MCP server is written for you.  This is a nice benefit to get logic and features for free, just make certain that you do not pass PII, customer information, or security tokens to them.

TIP: to get the tools which were used in the inference of the prompt I added an instruction in the system prompt to do so.  I also instructed it to explain why it chose the tool.

Before answering:
   1. State which tool you selected.
   2. State why you selected that tool.

Instructions like these are helpful in learning the internals of the AI model by explaining the reasons for the decisions that are made.

Render the Natural Language Processing (NLP) response

After receiving the list of tools which should be used to complete the prompt instructions, you need to make another call to the LLM endpoint.  Remember that it is very innovative that given the system prompt, a user prompt, and a remotely hosted (HTTPS) MCP server that it was able to infer what functions were needed to be successful.  To understand the awesomeness of this, visualize for a few moments how you would of coded this, you might try but such a dynamic solution that could mimic this behavior consistently would be very unlikely. 

response = openai_client.responses.create(
            input=input_list,
            previous_response_id=response.id,
            extra_body={"agent_reference": 
               {"name": agent.name, "type": "agent_reference"}},
        )
Listing 4: remote MPC server, send second inference request for NLP

And here is a short snippet of the response from gpt-5-mini in response to the user prompt.

1) What a "conversation" is
- A Conversation is a first-class resource in the Azure OpenAI / 
Foundry Models API (the Conversations API). It represents a 
sequence of conversation items (messages, tool calls, tool 
outputs, reasoning items, etc.) and stores metadata and 
timestamps for that session. The service exposes REST endpoints
 to create, retrieve, update, delete a conversation, and to 
list/create conversation items (see create/retrieve/update/delete 
conversation and list/create items endpoints).
- Conversation items are typed objects (OpenAI.ConversationItem). 
Typical item types include message, function_call (tool calls), 
function_call_output, reasoning, and many tool-specific types. 
The API’s type list explicitly includes MCP-related item types: 
mcp_list_tools, mcp_approval_request, mcp_approval_response, and 
mcp_call (MCP tool call).

2) How Conversations are used to access remote MCP servers
- Role: The conversation is the protocol-level message stream 
through which the model (assistant) requests and reports tool 
use. When an assistant decides to call an MCP-hosted tool, it 
emits a conversation item of type mcp_call (or requests a 
tools list with mcp_list_tools, or asks for human approval 
via mcp_approval_request). Those items contain the fields the 
client needs to execute the tool call against a remote MCP 
server (for example: server_label, tool name, arguments JSON, 
call id, and later output/status).
- MCP tool definitions: The OpenAI/Foundry tool schema has a 
Tool type for MCP (OpenAI.MCPTool). MCPTool includes properties 
such as server_label, server_url (or connector_id), 
headers/authorization, and filters (allowed_tools, 
require_approval). These definitions tell the client which 
remote MCP server to contact and how.
- Execution flow (typical):
  1. The model emits a conversation item indicating it wants to 
call a tool on a remote MCP server (type = mcp_call). The item 
includes: server_label, name (tool), arguments (JSON), and 
call id.
  2. Your client/agent (the host app using the Azure AI Projects 
library or the OpenAI client obtained via 
project_client.get_openai_client()) inspects the conversation 
item, resolves the server_label to a configured MCP server 
(server_url or connector), obtains any required authorization 
token, and invokes the MCP tool using the MCP protocol or via 
a connector.
  3. When the remote MCP server returns a result (or when human 
approval is provided), the client posts the result back into 
the conversation as the tool output item (mcp_call output / 
tool-call-output item), updating status and output fields. The 
model then consumes that output in a subsequent turn.
- Approval and safety: If the MCPTool asks for approval, the 
model can place an mcp_approval_request item in the 
conversation. The client or a human operator responds with an 
mcp_approval_response item to approve or deny. The conversation 
is the mechanism for that handshake.
- Discovery: The model (or client) can request mcp_list_tools 
items to discover what tools are available on a labeled MCP 
server; that item contains the tool catalog returned by the 
server and is carried in the conversation.