DP-750T00: Implement data engineering solutions using Azure Databricks

Course structure

Module 1: Explore Azure Databricks

• 1.1 Introduction 
• 1.2 Get started with Azure Databricks 
• 1.3 Identify Azure Databricks workloads 
• 1.4 Understand key concepts 
• 1.5 Data governance using Unity Catalog and Microsoft Purview 
• 1.6 Exercise – Explore Azure Databricks 
• 1.7 Summary 

Module 2: Understand Azure Databricks architecture

• 2.1 Introduction 
• 2.2 Understand Azure Databricks architecture 
• 2.3 Understand Unity Catalog managed storage 
• 2.4 Understand external storage 
• 2.5 Understand default storage 
• 2.6 Summary 

Module 3: Understand Azure Databricks Integrations

• 3.1 Introduction 
• 3.2 Understand integration with Microsoft Fabric 
• 3.3 Understand integration with Power BI 
• 3.4 Understand integration with VS Code 
• 3.5 Understand integration with Power Platform 
• 3.6 Understand integration with Copilot Studio 
• 3.7 Understand integration with Microsoft Purview 
• 3.8 Understand integration with Microsoft Foundry 
• 3.9 Summary 

Module 4: Select and Configure Compute in Azure Databricks

• 4.1 Introduction 
• 4.2 Choose an appropriate compute type 
• 4.3 Configure compute performance 
• 4.4 Configure compute features 
• 4.5 Install libraries for compute 
• 4.6 Configure compute access 
• 4.7 Exercise – Select and Configure Compute in Azure Databricks 
• 4.8 Summary 

Module 5: Create and organize objects in Unity Catalog

• 5.1 Introduction 
• 5.2 Apply naming conventions 
• 5.3 Create catalog 
• 5.4 Create schema 
• 5.5 Create tables and views 
• 5.6 Create volumes 
• 5.7 Implement DDL operations 
• 5.8 Implement foreign catalog 
• 55.9 Configure AI/BI Genie instructions 
• 5.10 Exercise – Create and Organize Objects in Unity Catalog 
• 5.11 Summary 

Module 6: Secure Unity Catalog objects

• 6.1 Introduction 
• 6.2 Understand query lifecycle 
• 6.3 Implement access control strategies 
• 6.4 Understand fine-grained access control 
• 6.5 Implement row filtering and column masking 
• 6.6 Access Azure Key Vault secrets 
• 6.7 Authenticate data access with service principals 
• 6.8 Authenticate resource access with managed identities 
• 6.9 Exercise – Secure Unity Catalog Objects 
• 6.10 Summary 

Module 7: Govern Unity Catalog objects

• 7.1 Introduction 
• 7.2 Create and preserve table definitions 
• 7.3 Configure ABAC with tags and policies 
• 7.4 Apply data retention policies 
• 7.5 Set up and manage data lineage 
• 7.6 Configure audit logging 
• 7.7 Design secure Delta Sharing strategy 
• 7.8 Exercise – Govern Unity Catalog Objects 
• 7.9 Summary 

Module 8: Design and implement data modeling with Azure Databricks

• 8.1 Introduction 
• 8.2 Design ingestion logic and data source configuration 
• 8.3 Choose a data ingestion tool 
• 8.4 Choose a data table format 
• 8.5 Design and implement a data partitioning scheme 
• 8.6 Choose a slowly changing dimension (SCD) type 
• 8.7 Implement a slowly changing dimension (SCD) type 
• 8.8 Design and implement a temporal (history) table to record changes over time 
• 8.9 Choose granularity on a column or table based on requirements 
• 8.10 Choose managed vs external tables 
• 8.11 Design and implement a clustering strategy 
• 8.12 Exercise – Design and Implement Data Modeling with Azure Databricks 
• 8.13 Summary 

Module 9: Ingest data into Unity Catalog

• 9.1 Introduction 
• 9.2 Ingest data with Lakeflow Connect 
• 9.3 Ingest data with notebooks 
• 9.4 Ingest data with SQL methods 
• 9.5 Ingest data with CDC feed 
• 9.6 Ingest data with Spark Structured Streaming 
• 9.7 Ingest data with Auto Loader 
• 9.8 Ingest data with Lakeflow Spark Declarative Pipelines 
• 9.9 Exercise – Ingest Data into Unity Catalog 
• 9.10 Summary 

Module 10: Cleanse, transform, and load data into Unity Catalog

• 10.1 Introduction 
• 10.2 Profile data 
• 10.3 Choose column data types 
• 10.4 Resolve duplicates and nulls 
• 10.5 Transform data with filters and aggregations 
• 10.6 Transform data with joins and set operators 
• 10.7 Transform data with denormalization and pivots 
• 10.8 Load data with merge, insert, and append 
• 10.9 Exercise – Cleanse, Transform, and Load Data into Unity Catalog 
• 10.10 Summary 

Module 11: Implement and manage data quality constraints with Azure Databricks

• 11.1 Introduction 
• 11.2 Implement validation checks 
• 11.3 Implement data type checks 
• 11.4 Detect and manage schema drift 
• 11.5 Manage data quality with pipeline expectations 
• 11.6 Exercise – Implement and Manage Data Quality Constraints with Azure Databricks 
• 11.7 Summary 

Module 12: Design and implement data pipelines with Azure Databricks

• 12.1 Introduction 
• 12.2 Design order of operations for a pipeline 
• 12.3 Choose notebook vs Lakeflow Pipelines 
• 12.4 Design Lakeflow job logic 
• 12.5 Design error handling in pipelines and jobs 
• 12.6 Create pipeline with notebook 
• 12.7 Create pipeline with Lakeflow Spark Declarative Pipelines 
• 12.8 Exercise – Design and Implement Data Pipelines with Azure Databricks 
• 12.9 Summary 

Module 13: Implement Lakeflow Jobs with Azure Databricks

• 13.1 Introduction 
• 13.2 Create job setup and configuration 
• 13.3 Configure job triggers 
• 13.4 Schedule a job 
• 13.5 Configure job alerts 
• 13.6 Configure automatic restarts 
• 13.7 Exercise – Implement Lakeflow Jobs with Azure Databricks 
• 13.8 Summary 

Module 14: Implement development lifecycle processes in Azure Databricks

• 14.1 Introduction 
• 14.2 Apply Git version control best practices 
• 14.3 Manage branching and pull requests 
• 14.4 Implement testing strategy 
• 14.5 Configure and package Declarative Automation Bundles 
• 14.6 Deploy bundle with Databricks CLI 
• 14.7 Exercise – Implement Development Lifecycle Processes in Azure Databricks 
• Summary 

Module 15: Monitor, troubleshoot and optimize workloads in Azure Databricks

• 15.1 Introduction 
• 15.2 Monitor and manage cluster consumption 
• 15.3 Troubleshoot and repair Lakeflow Jobs 
• 15.4 Troubleshoot Spark jobs and notebooks 
• 15.5 Investigate caching, skewing, spilling, shuffle 
• 15.6 Implement log streaming with Azure Log Analytics 
• 15.7 Exercise – Monitor, Troubleshoot and Optimize Workloads in Azure Databricks 
• 15.8 Summary