Introduction

These days Artificial Intelligence (AI) systems are widely used, and autonomous AI agents are beaming as the technology becomes more self-sufficient and makes more complex decisions across various business sectors. Each autonomous system has its own requirements. Microsoft Azure AI Foundry is a sophisticated autonomous AI development platform aimed at creating and deploying systems efficiently. This blog focuses on Azure AI Foundry, its support on autonomous AI agents’ development, the principles of AI agents, and providing a walkthrough example of developing an AI agent that summarizes given documents.

What is Azure AI Foundry?

Microsoft offers a comprehensive ecosystem within Azure AI, anchored by Azure AI Foundry—a development platform that provides an integrated set of tools for building, deploying, and managing AI solutions. By leveraging the Azure ecosystem, it simplifies AI workflows, scales them in the cloud, and eases infrastructure management—removing barriers that slow innovation.

Azure AI Foundry also integrates Azure’s machine learning and cognitive services, supports the deployment of complex AI models, and includes additional features that make the platform highly AI-driven. This makes it an ideal foundation for developing autonomous AI systems that can operate with minimal human intervention.

Azure AI Foundry App: Autonomous, Agentic Development

Developers are supported by the following distinctive tools and frameworks within Azure AI Foundry that help them develop autonomous AI agents:

• Cognitive Services: This encompasses a broad range of AI services within Azure, from pre-built cognitive services such as Natural Language Processing (NLP), Speech, and Computer Vision, to more advanced capabilities.
• For Curriculum-based Machine Learning Pipelines: These workflows facilitate model training and evaluation, automating processes to ensure high efficiency and self-sufficiency.
• Azure DevOps Integration: Allows AI models to be served via continuous integration and continuous delivery pipelines.
• AI Agent Custom Frameworks: Systems capable of perceptive, reasoning, and autonomous actions are intelligent agents.
• Complex Intelligent Automation Systems: These allow agents to perform complex workflows and interact with other tools. This is what makes them perfect for real-world usage.

These capabilities provided by the Foundry simplify and speed up development processes for AI agents.

What are AI Agents?

AI agents are designed to function like software systems that can perceive their surroundings, process data, think, and act independently to achieve objectives. Unlike older software, AI agents are capable of learning and adapting. This makes them exceptionally suited for applications in real-world scenarios.

AI Agents Do Tasks Such As:

• Stream data in real-time
• Enable task automation, both basic and compound
• Automate your smallest tasks or most complex interactions
• Natural language user interface in AI
• Context based decisions that could be human-less decisions
• Today, AI applications are more advanced and rely on autonomous agents to power things like virtual assistants, recommendation systems, and chatbots of a far more complex nature

Benefits of AI agents

AI agents offer notable advantages, such as:

• Better Solution: Generative AI agents provide more economical and scalable solutions because they analyze massive datasets and conduct transactions at speeds and precisions that far exceed human norms, all while keeping the demand for additional human resources flat.
• Enhanced Decision Making: AI agents utilize different types of data and leverage machine learning to construct more sophisticated decisions.
• Automated Processes: AI agents do not require breaks and can perform elaborate tasks continuously, reducing the cost of operations as compared to employing humans.

The benefits of this operation are to create more responsive, efficient yet cost-effective operations.

Building Autonomous AI Agents with Azure AI Foundry – Follow these Steps

It is a complex process to create an autonomous AI agent to summarize the uploaded documents using Azure AI Foundry. The following are some points that indicate a proper step-by-step process for creating the AI agent from scratch. Let’s follow these steps to cover all the major areas.

1. Define the Agent’s Purpose and Scope

Defining the AI agent’s purpose and scope is critical to achieve the objectives that you have in mind for it.

Key Decisions:

Purpose: The agent is designed to summarize documents, but you need to determine whether it is more focused on the document categories such as legal, medical, or business.

Scope: What is the maximum length of documents that the agent is summarizing? For instance, is the agent promoting whitepapers or books? It is also important to define the level of detail required in the summaries.

2. Utilizing Cognitive Services in Azure

Azure provides one of the most helpful cognitive services that enable you to achieve the goal of summarization and document processing.

Key Tools:

• Azure Text Analytics API:
• Text Summarization: This API does not summarize the text directly, but it helps in generating a summarizer by extracting key phrases and performing sentiment and entity analysis.
• Sentiment and Key Phrase Analysis: Using indicated phrases or entities; this function can generate summaries of documents by extracting the most pertinent parts within the document.

Azure OpenAI (GPT-4):

GPT-4 can generate human-like abstractions and summaries.

• GPT-4 is available through the Azure OpenAI Service
• Azure Form Recognizer (if works with forms or scanned documents): This service extracts the relevant information from scanned documents, forms, and even pictures that contain texts such as tables. Hence, if a summarization service is being utilized for forms, invoices, or any other structured text, this service is useful along with the summarization service

3. Construct the AI Model Utilizing Machine Learning Technologies

In this step, you build and train an AI model on your own if the pre-trained models offered in the cognitive services are not suitable to your requirements.

Steps to Build the AI Model:

• Prepare Your Dataset: Gather a collection of documents that are relevant to the scope you set in Step 1. This is in the form of training data for the custom model you are intending to build. Ensure that documents are tagged properly to indicate a summary or key phrases exist for the training set.
• Microsoft Azure: Microsoft also offers a plethora of cloud-based environments designed to nurture and grow your ML model (e.g., Azure Machine Learning [AML]). Create a document-based summarization model using BERT or T5 through Hugging Face on Azure. Train the model on user authored documents and summaries.
• Example with T5 (Text-to-Text Transformer):
  1. Do pre-processing: Convert your documents into a format suitable for fine-tuning T5 (Text-to-Text Transformer).
  2. Training: It is important to properly train the summarization model.
  3. Evaluation: Summarization models can be assessed using various metrics such as the ROUGE scores.
• Custom Models vs. Pre-Trained Models: For documents that are specialized, such as legal ones, a custom-tuned model is likely to provide better results.

4. Join Azure AI Foundry Frameworks

Azure AI Foundry is a complete framework to give shape to your AI system, from its early days as an incubating project to production. Azure Foundry’s services like Azure ML, Azure Logic Apps, and Azure Functions can significantly improve your workflow.

Integration Approach:

• An AI-based Solution Can Be Created in Azure AI Foundry:
  • Start a new project in Azure AI Foundry, ensuring you select appropriate tools for document processing and summarization.
  • Through Azure AI Foundry, you can invite other users to collaborate in the building, testing, and deployment of AI-based systems, increasing the organizational efficiency of AI units.
• Build the Pipeline:
  • Azure Logic Apps or Azure Functions can be used to automate document processing. For instance, an event in Blob Storage could trigger an Azure Function that processes the document. Write the scripts for document fetching, processing, summarization, and save the processed documents.
• Connect Cognitive Services:
  • Use the Text Analytics and Open AI services to add text extraction and summarization capabilities to the AI agent.
• Test and Validate:
  • Refine and fine-tune the agent’s summarization capabilities by testing them on different documents before scaling up the deployment.
  • You can host the application in Azure App Services or Azure Kubernetes Service (AKS) for scaling it.

5. Deploy the Chatbot

The next action with respect to making the AI agent viable for use is to deploy it. Scaling the bot is possible depending on the volume of documents anticipated. For deployment, you may use Azure App Services / AKS.

6. Monitor and Iterate

Strategic oversight of the agent’s operations, algorithmic adjustments, and repetitive testing—all influence the AI systems capability to function optimally over time.

How to build autonomous AI agents with Azure AI Foundry Best Practices

• Transparency in Design, Give the User a Reason to Trust You: AI agents should justify their decision making.
• Leverage Automated Intelligence: Mix AI agents with automated intelligence.
• Emphasize security: Secure the information and AI models by utilizing out of the box security provided with Azure.
• Scalability: The capacity to scale up and down easily in the cloud is not a problem as Azure allows vast reserves of cloud resources.

Example

1. Configure your Azure AI Foundry Environment

• Prepare Resources
  • Establish a “hub-based” Azure AI Foundry project, e.g., in West US.
  • In Foundry portal, go to Models + endpoints → click Deploy model → select gpt4omini (or another suitable model).
• Create Your Agent
  • Navigate to Build and Customize → Agents → + Create agent.
  • Choose your deployed model, enter a name (e.g., Summary Agent), and provide directions such as:
  • You summarize uploaded documents and provide concise summaries.
  • Provide summaries, but also provide key insights, tables, or charts outlining important information, if provided.

2. Upload Files and Build a Vector Store

To allow the agent to retrieve and search through uploaded data, do the following:

• Upload Your Documents
  • Use the Python SDK with purpose= “assistants” or standard agent setup with Blob + AI Search.
• Create a Vector Store
  • • This processes, chunks, and indexes your documents with Azure AI Search.
      from azure.ai.agents import AgentsClient, FileSearchTool
      from azure.identity import DefaultAzureCredentialclient = AgentsClient(endpoint, credential=DefaultAzureCredential())
      file = client.files.upload_and_poll(file_path=”data.xlsx”, purpose=”assistants”)
      vs = client.agents.create_vector_store_and_poll(file_ids=[file.id], name=”data_store”

3. Set Up the Summarization Agent and Enable File Search

• Textbox 2, Textbox Utilize File Search tool to enact agent-driven search on the provided datasets.
• This enables the agent to respond with extracted text.
  from azure.ai.agents import FileSearchTool
  file_search = FileSearchTool(vector_store_ids=[vs.id])
  agent = client.agents.create_agent(
  model=”gpt-4o-mini”,
  name=”SummaryAgent”,
  instructions=”You summarize the contents of uploaded files on request.”,
  tools=file_search.definitions,
  tool_resources=file_search.resources
  )

4. Engage in an Agent-driven Conversation and Apply Summarization, Chart, or Analysis as Desired

• Query the vector index
• Retrieve relevant text chunks
• Summarize given text, extract trends, or offer insights

Questions related to chart summaries may also be requested:

“Generate a bar chart of monthly sales by region.”

The agent can also generate image files in .png format using the Code Interpreter tool.

Conduct your Experiments in the Playground

• In Foundry portal → Agents → select your agent → click Try in playground.
• Use the interface under Knowledge → Add Files to upload sample documents that are automatically indexed.
• Try the following prompts:
  • “Summarize key metrics from the uploaded report.”
  • “Create a pie chart showing expense categories.”

Conclusion

Azure AI Foundry is more than a repackaging of the latest text summarization technology into an integrated platform. Such sophisticated and autonomous AI agents have the potential to transform entire sectors of the economy. With Azure AI Foundry, these AI agents can offer powerful capabilities for automation, adaptively perform complex, real-time tasks, and learn through experience to improve productivity and drive innovation. Azure AI Foundry’s NLP, machine learning, and data integration capabilities strengthen this claim.

Azure AI Foundry helps companies to develop and deploy self-sufficient AI agents that address real-world challenges. With capabilities from flow automation to personalized engagement and optimized business processes, Azure AI Foundry fully harnesses the potential of autonomous AI agents as strategic assets and extends their utility beyond conventional use.