AI-300T00: Operationalize machine learning and generative AI solutions

Course structure

Module 1: Get started with machine learning in Azure

• 1.1 Introduction 
• 1.2 Define the problem 
• 1.3 Get and prepare data 
• 1.4 Train the model 
• 1.5 Use Azure Machine Learning studio 
• 1.6 Integrate a model 
• 1.7 Summary 

Module 2: Experiment with Azure Machine Learning

• 2.1 Introduction 
• 2.2 Preprocess data and configure featurization 
• 2.3 Run an automated machine learning experiment 
• 2.4 Evaluate and compare models 
• 2.5 Configure MLflow for model tracking in notebooks 
• 2.6 Train and track models in notebooks 
• 2.7 Evaluate models with the Responsible AI dashboard 
• 2.8 Exercise – Find the best classification model with Azure Machine Learning 
• 2.9 Summary 

Module 3: Run training scripts and track models with MLflow in Azure Machine Learning

• 3.1 Introduction 
• 3.2 Convert a notebook to a script 
• 3.3 Run a script as a command job 
• 3.4 Use parameters in a command job 
• 3.5 Track metrics with MLflow 
• 3.6 View metrics and evaluate models 
• 3.7 Exercise – Optimize model training in Azure Machine Learning 
• 3.8 Summary 

Module 4: Perform hyperparameter tuning with Azure Machine Learning

• 4.1 Introduction 
• 4.2 Define a search space 
• 4.3 Configure a sampling method 
• 4.4 Configure early termination 
• 4.5 Use a sweep job for hyperparameter tuning 
• 4.6 Exercise – Run a sweep job 
• 4.7 Summary 

Module 5: Run pipelines in Azure Machine Learning

• 5.1 Introduction 
• 5.2 Create components 
• 5.3 Create a pipeline 
• 5.4 Run a pipeline job 
• 5.5 Exercise – Run a pipeline job 
• 5.6 Summary 

Module 6: Automate model training with GitHub Actions

• 6.1 Introduction 
• 6.2 Use source control for machine learning assets 
• 6.3 Apply trunk-based development 
• 6.4 Validate changes with GitHub Actions 
• 6.5 Connect GitHub Actions to Azure Machine Learning securely 
• 6.6 Trigger Azure Machine Learning jobs and pipelines 
• 6.7 Exercise 
• 6.8 Summary 

Module 7: Deploy and monitor a model in Azure Machine Learning

• 7.1 Introduction 
• 7.2 Prepare a model for deployment 
• 7.3 Explore the deployment and monitoring architecture 
• 7.4 Control deployments with GitHub environments 
• 7.5 Deploy, promote, and roll back a model with GitHub Actions 
• 7.6 Monitor the deployed model 
• 7.7 Exercise 
• 7.8 Summary 

Module 8: Plan and prepare a GenAIOps solution

• 8.1 Introduction 
• 8.2 Define agent specifications 
• 8.3 Design and optimize your agent 
• 8.4 Explore the GenAIOps lifecycle 
• 8.5 Explore available tools and frameworks to implement GenAIOps 
• 8.6 Exercise – Design agent specifications from team input 
• 8.7 Summary 

Module 9: Manage prompts for agents in Microsoft Foundry with GitHub

• 9.1 Introduction 
• 9.2 Apply version control to prompts 
• 9.3 Understand Microsoft Foundry agents and prompt versioning 
• 9.4 Organize prompts in GitHub repositories 
• 9.5 Develop safe prompt deployment workflows 
• 9.6 Exercise – Develop prompt and agent versions 
• 9.7 Summary 

Module 10: Evaluate and optimize AI agents through structured experiments

• 10.1 Introduction 
• 10.2 Design evaluation experiments 
• 10.3 Apply Git-based workflows to optimization experiments 
• 10.4 Apply evaluation rubrics for consistent scoring 
• 10.5 Exercise – Evaluate and compare AI agent versions 
• 10.6 Summary 

Module 11: Automate AI evaluations with Microsoft Foundry and GitHub Actions

• 11.1 Introduction 
• 11.2 Understand why automated evaluations matter 
• 11.3 Align evaluators with human criteria 
• 11.4 Create evaluation datasets 
• 11.5 Implement batch evaluations with Python 
• 11.6 Integrate evaluations into GitHub Actions 
• 11.7 Exercise – Set up automated evaluations 
• 11.8 Summary 

Module 12: Monitor your generative AI application

• 12.1 Introduction 
• 12.2 Why do you need to monitor? 
• 12.3 Understand key metrics to monitor 
• 12.4 Explore how to monitor with Azure 
• 12.5 Integrate monitoring into your app 
• 12.6 Interpret monitoring results 
• 12.7 Exercise – Enable monitoring for a generative AI application 
• 12.8 Summary 

Module 13: Analyze and debug your generative AI app with tracing

• 13.1 Introduction 
• 13.2 Why do you need to use tracing? 
• 13.3 Identify what to trace in generative AI applications 
• 13.4 Implement tracing in generative AI applications 
• 13.5 Debug complex workflows with advanced tracing patterns 
• 13.6 Make informed decisions with trace data analysis 
• 13.7 Exercise – Enable tracing for a generative AI application 
• 13.8 Summary