End-to-End Data Analysis and Machine Learning in the Cloud
Explore a comprehensive example of working with data in the cloud using Databricks, Spark, Azure Synapse Analytics, and machine learning. Dive into a practical guide covering data analysis, data lake setup, ML model creation, deployment, and integration with Power BI. Join the discussion on leveraging tools like Azure ML, Python, and Databricks for predictive analysis in Azure. Discover how to tackle the challenge of predicting offenders based on limited victim data using machine learning techniques.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Databricks, Spark, Machine Learning and Azure Synapse Analytics AN END-TO-END EXAMPLE OF DATA IN THE CLOUD We re Hiring a Big Data Engineer! Talk to me about the opportunities Simon Kingaby Manager, Global Data and Analytics Deloitte Touche Tohmatsu Limited | Global Shared Services skingaby@deloitte.com Blog: omwtm.blog linkedin.com/in/skingaby/
Agenda Tools Setup 1: Getting Some Data to Analyze 2: Loading the Data Lake 3: Processing the data in Databricks 4: Creating the Machine Learning Model 5: Detour! Create a Custom Docker Base Image 6: Configure the Model for Deployment 7: Build and Deploy the Docker image 8: Testing the Webservice 9: Loading the Data Warehouse 10: Creating a Power BI Report Breathe
Tools Data Studio Storage Explorer Resource Group Data Factory Blob Storage Data Lake Key Vault Application Insights SQL Server Databricks Container Registry Machine Learning Synapse Analytics (SQL DW) Power BI
What is the Question? We re looking at 2009 to 2015 Crime Data from the FBI s Uniform Crime Reporting Program What we want to know: Given the limited information we have about the victim, can we use Machine Learning to predict who the offender was? In other words: Whodunnit?
Why Bother? 1. To solve the question of Whodunnit? We need to use Machine Learning 2. To solve it in Azure, we need to set up an ML Model 3. To do that, you can use the Azure ML tools, or Databricks and Python (we ll be doing the latter as this seems more automatable ) 4. To expose your Azure ML to Power BI and the Web you need to deploy it as a Webservice (and there are some issues with doing that currently, which we will cover in this session)
