Streamlining Dynamics Integration in the Cloud: A User-Centric Approach
Integrating With
Dynamics by
Leveraging Azure
John, Software Engineer, Root16
•
Our client required an integration between
CRM and on-prem services
•
Previously, dynamics instance hosted on-
prem sever
•
Plugins could make direct access to on-prem
SQL server
•
With Dataverse, no longer direct access
•
Objective: hoist communication into the cloud
to act as ‘middle-man’
Intro
•
Discuss the problem we were brought in to
solve
•
What other architectures were considered
•
Top-down architecture
•
Cloud architecture component deep dive
•
Issues we encountered and solved
The Problem to
Solve
The Problem
•
As they migrated their Dynamics instance from on-
prem to the cloud, some integrations had to be
reconfigured
•
They had plugins which would reach out to on-
prem SQL servers to perform data
validation/integration
•
On-prem services inaccessible from Dynamics
cloud instance
•
Develop a system to act as a ‘middle-man’ to pass
messages from Dynamics to on-prem
•
Needed to be fast and scalable for high-throughput
•
Easily repeatable to build
•
Needed to reuse a lot of preexisting code
Possible Solutions
Possible Solutions
•
Azure Functions
•
Using the Azure Service Bus integration with Dynamics to send messages
•
Have an Azure function as the listener
•
Scrapping the integration
•
Lots of pushback on this one :)
Issues With Azure Function
•
Asynchronous by design
•
Needed a way to then get the data back to D365
•
Key insight: one of these integrations needed to be real time
•
Http web request is a more appropriate choice
Advantages to AKS
•
Client already had Kubernetes infrastructure
•
Scalable horizontally during times of peak load
•
Easy ASP .NET REST API project creation
•
Realtime communication via HTTP
High Level
Architecture
High Level Architecture
•
Dynamics makes a synchronous web request via plugin
•
Hits
Azure Application Gateway
•
Forwards the connection to
Azure Kubernetes Service
(AKS)
•
AKS Custer has two components:
•
Load balancer for pods (virtual machines)
•
Pods do the business logic and connect to on-prem services
•
Business logic copied right out of the plugin
•
AKS’s subnet has connection to on-prem network via a Microsoft
Azure virtual network
•
Pods connect to on-prem services
•
Forward the request back up the chain
High Level Architecture
Components Deep
Dive
AKS Cluster
•
Managed Kubernetes Cluster
•
Client already had an AKS service in their
Azure subscription
•
D365 team assigned our own AKS
namespace in each environment
•
Pods run on shared virtual machines
AKS Logs
•
Set your app to log output to
‘Console’ logs are streamed
straight to AKS
•
Querable in Azure dashboard
•
Kusto Query Language (KQL)
•
Read-only syntax focused on
transformations of rows
•
Readable and writeable syntax
Azure Dev Ops Pipeline Deploys
•
ADO pipelines help automate and control our
deploys
•
Can choose which environment to deploy to
•
QA and PROD require admin approval
.NET 7 API
•
Standard REST application, containerized via a Docker file
•
Visual Studio has a containerized application template
•
Can run via docker on dev machine
•
Make deployment to Kubernetes painless
.NET 7 API
App Configuration
•
Configuration parameters and secret values are
stored in
Azure Key Vault
•
In order to access secrets in KV, the Integration API
makes use of
App Configuration
in Azure,
•
When run locally, the .NET runtime consumes the
auth information store in
`appsettings.Development.json` to access the Dev
Key Vault
•
When deployed on Kubernetes, the cluster handles
all authentication to Azure KV
App Configuration
•
From the perspective of the
.NET program, nothing has
changed
•
Access the `Configuration`
object like normal
Lens
•
Lens: ‘The Kubernetes IDE‘
is a program that
facilitates Kubernetes deployments and
services
•
View deployments, view logs, kill/restart pods
App Insights
•
Service that your applications can write logs and
other metrics to
•
Remarkably easy to setup
•
Only required step is to add nuget package
reference:
Microsoft.ApplicationInsights.AspNetCore
•
Then include instrumentation key:
•
Instrumentation key is already in the
‘Configuration’ object
Issues and Solutions
Open Ports
•
Port 1433 between AKS subnet and on-prem subnet needed to be open
•
Required coordinating with network team
•
To confirm ports open, could remote into pods via lens
•
Run ‘nslookup’ and ‘nmap’ to validate and debug
503 Gateway
•
Endpoints returning ‘503 Service Unavailable server error response’
•
Checking app insights, no non-200 response ever returned to client
•
Turned out the
Web Application Firewall
was returning 503 to the App Gateway
•
WAF wasn’t able to keep up with load, as payload sizes too large
Explore how a software engineer tackled the challenge of integrating Dynamics CRM with on-prem services in the cloud, focusing on decreasing complexities and delivering results. The project involved migrating Dynamics from on-prem to the cloud, requiring the development of a scalable system to act as a middle-man for message passing. Discover the solutions considered, such as Azure Functions and Azure Service Bus integration, in this insightful journey shared at the 2022 Dynamic Communities event.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
The Largest IndependentGathering of the Microsoft User Ecosystem For Users, By Users Integrating With Dynamics by Leveraging Azure John, Software Engineer, Root16 Decrease Complexities, Deliver Results. @ 2022 Dynamic Communities
Intro For Users, By Users Our client required an integration between CRM and on-prem services Previously, dynamics instance hosted on- prem sever Plugins could make direct access to on-prem SQL server With Dataverse, no longer direct access Objective: hoist communication into the cloud to act as middle-man The Largest Independent Gathering of the Microsoft User Ecosystem @ 2022 Dynamic Communities
Agenda For Users, By Users Discuss the problem we were brought in to solve What other architectures were considered Top-down architecture Cloud architecture component deep dive Issues we encountered and solved The Largest Independent Gathering of the Microsoft User Ecosystem @ 2022 Dynamic Communities
Decrease Complexities, Deliver Results. The Problem to Solve For Users, By Users @ 2022 Dynamic Communities
The Problem For Users, By Users As they migrated their Dynamics instance from on- prem to the cloud, some integrations had to be reconfigured They had plugins which would reach out to on- prem SQL servers to perform data validation/integration On-prem services inaccessible from Dynamics cloud instance Develop a system to act as a middle-man to pass messages from Dynamics to on-prem Needed to be fast and scalable for high-throughput Easily repeatable to build Needed to reuse a lot of preexisting code Decrease Complexities, Deliver Results. @ 2022 Dynamic Communities
Decrease Complexities, Deliver Results. Possible Solutions For Users, By Users @ 2022 Dynamic Communities
Possible Solutions For Users, By Users Azure Functions Using the Azure Service Bus integration with Dynamics to send messages Have an Azure function as the listener Scrapping the integration Lots of pushback on this one :) Decrease Complexities, Deliver Results. @ 2022 Dynamic Communities
Issues With Azure Function For Users, By Users Asynchronous by design Needed a way to then get the data back to D365 Key insight: one of these integrations needed to be real time Http web request is a more appropriate choice Decrease Complexities, Deliver Results. @ 2022 Dynamic Communities
Advantages to AKS For Users, By Users Client already had Kubernetes infrastructure Scalable horizontally during times of peak load Easy ASP .NET REST API project creation Realtime communication via HTTP Decrease Complexities, Deliver Results. @ 2022 Dynamic Communities
Decrease Complexities, Deliver Results. High Level Architecture For Users, By Users @ 2022 Dynamic Communities
High Level Architecture For Users, By Users Dynamics makes a synchronous web request via plugin Hits Azure Application Gateway Forwards the connection to Azure Kubernetes Service (AKS) AKS Custer has two components: Load balancer for pods (virtual machines) Pods do the business logic and connect to on-prem services Business logic copied right out of the plugin AKS s subnet has connection to on-prem network via a Microsoft Azure virtual network Pods connect to on-prem services Forward the request back up the chain Decrease Complexities, Deliver Results. @ 2022 Dynamic Communities
High Level Architecture For Users, By Users Decrease Complexities, Deliver Results. @ 2022 Dynamic Communities
Decrease Complexities, Deliver Results. Components Deep Dive For Users, By Users @ 2022 Dynamic Communities
AKS Cluster For Users, By Users Managed Kubernetes Cluster Client already had an AKS service in their Azure subscription D365 team assigned our own AKS namespace in each environment Pods run on shared virtual machines Decrease Complexities, Deliver Results. @ 2022 Dynamic Communities
AKS Logs For Users, By Users Set your app to log output to Console logs are streamed straight to AKS Querable in Azure dashboard Kusto Query Language (KQL) Read-only syntax focused on transformations of rows Readable and writeable syntax Decrease Complexities, Deliver Results. @ 2022 Dynamic Communities
Azure Dev Ops Pipeline Deploys For Users, By Users ADO pipelines help automate and control our deploys Can choose which environment to deploy to QA and PROD require admin approval Decrease Complexities, Deliver Results. @ 2022 Dynamic Communities
.NET 7 API For Users, By Users Standard REST application, containerized via a Docker file Visual Studio has a containerized application template Can run via docker on dev machine Make deployment to Kubernetes painless Decrease Complexities, Deliver Results. @ 2022 Dynamic Communities
.NET 7 API For Users, By Users Decrease Complexities, Deliver Results. @ 2022 Dynamic Communities
App Configuration For Users, By Users Configuration parameters and secret values are stored in Azure Key Vault In order to access secrets in KV, the Integration API makes use of App Configuration in Azure, When run locally, the .NET runtime consumes the auth information store in `appsettings.Development.json` to access the Dev Key Vault When deployed on Kubernetes, the cluster handles all authentication to Azure KV Decrease Complexities, Deliver Results. @ 2022 Dynamic Communities
App Configuration For Users, By Users From the perspective of the .NET program, nothing has changed Access the `Configuration` object like normal Decrease Complexities, Deliver Results. @ 2022 Dynamic Communities
Lens For Users, By Users Lens: The Kubernetes IDE is a program that facilitates Kubernetes deployments and services View deployments, view logs, kill/restart pods Decrease Complexities, Deliver Results. @ 2022 Dynamic Communities
App Insights For Users, By Users Service that your applications can write logs and other metrics to Remarkably easy to setup Only required step is to add nuget package reference: Microsoft.ApplicationInsights.AspNetCore Then include instrumentation key: Instrumentation key is already in the Configuration object Decrease Complexities, Deliver Results. @ 2022 Dynamic Communities
Decrease Complexities, Deliver Results. Issues and Solutions For Users, By Users @ 2022 Dynamic Communities
Open Ports For Users, By Users Port 1433 between AKS subnet and on-prem subnet needed to be open Required coordinating with network team To confirm ports open, could remote into pods via lens Run nslookup and nmap to validate and debug Decrease Complexities, Deliver Results. @ 2022 Dynamic Communities
503 Gateway For Users, By Users Endpoints returning 503 Service Unavailable server error response Checking app insights, no non-200 response ever returned to client Turned out the Web Application Firewall was returning 503 to the App Gateway WAF wasn t able to keep up with load, as payload sizes too large Decrease Complexities, Deliver Results. @ 2022 Dynamic Communities
For Users, By Users Thank you for attending! The Largest Independent Gathering of the Microsoft User Ecosystem Decrease Complexities, Deliver Results. @ 2022 Dynamic Communities
