Understanding Complex Event Processing (CEP) in SQL Server 2008 R2 StreamInsight
Explore the world of Complex Event Processing (CEP) in SQL Server 2008 R2 StreamInsight, a powerful tool for real-time event stream processing. Learn about its applications, benefits, and use cases in various industries.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
SQL Server 2008 R2 StreamInsight Complex Event Processing Event Stream Processing
Who Am I? SQL Server MVP SQL Server Consultant Joint author on Wrox Professional SSIS book www.SQLDTS.com and www.SQLIS.com Specialise in Moving Data @allanSQLIS (twitter)
Agenda Use Cases & Challenges Formulating Declarative Queries Windows in Time Event Flow Debugging Demos
What is CEP? Complex Event Processing (CEP) is the continuous and incremental processing of event streams from multiple sources based on declarative query and pattern specifications with near-zero latency.
What is CEP? Complex Event Processing (CEP) is the continuous and incremental processing of event streams from multiple sources based on declarative query and pattern specifications with near-zero latency. Database Applications Event-driven Applications Query Paradigm Ad-hoc queries or requests Continuous standing queries Event request output stream input stream response
What is CEP? Complex Event Processing (CEP) is the continuous and incremental processing of event streams from multiple sources based on declarative query and pattern specifications with near-zero latency. Database Applications Event-driven Applications Query Paradigm Ad-hoc queries or requests Continuous standing queries Latency Seconds, hours, days Milliseconds or less Event request output stream input stream response
What is CEP? Complex Event Processing (CEP) is the continuous and incremental processing of event streams from multiple sources based on declarative query and pattern specifications with near-zero latency. Database Applications Event-driven Applications Query Paradigm Ad-hoc queries or requests Continuous standing queries Latency Seconds, hours, days Milliseconds or less Data Rate Hundreds of events/sec > Tens of thousands of events/sec Event request output stream input stream response
Who might need CEP Fraud Detection Real-Time Trade Risking Algorithmic Trading/Betting Meter throughputs Oil, Gas, Water, Electricity Use to drive alarms, alerts etc
StreamInsight Platform .NET C# LINQ Learn More About Visual Studio 2008 StreamInsight Application Development StreamInsight Application at Runtime Event sources Event targets Input Adapters Output Adapters StreamInsight Engine Devices, Sensors Pagers & Monitoring devices Standing Queries KPI Dashboards, SharePoint UI ` 1 Web servers Query Logic Query Logic 4 2 3 Trading stations Event stores & Databases Query Logic Event stores & Databases Stock ticker, news feeds
Query Expressiveness Selection of events (filter) Calculations on the payload (project) Correlation of streams (join) Stream partitioning (group and apply) Aggregation (sum, count, ) over event windows Ranking over event windows (topK)
Query Expressiveness Projection var result = from e in inputStream select new { id = e.id, W = (double)e.intW / 10 };
Query Expressiveness Projection Filter var result = from e in inputStream where e.id > 3 select new { id = e.id, W = (double)e.intW / 10 };
Query Expressiveness Projection Filter Correlation (Join) var result = from eLeft in inputStream1 join eRight in inputStream2 on eLeft.id equals eRight.id select new { id = eLeft.id, diff = eLeft.W - eRight.w };
Query Expressiveness Projection Filter Correlation (Join) Aggregation over windows var result = from win in inputStream.TumblingWindow( select new { avg = win.Avg(e => e.W) }; TimeSpan.FromSeconds(10))
Query Expressiveness Projection Filter Correlation (Join) Aggregation over windows Group and Aggregate var result = from e in inputStream group e by e.id into eachGroup from win in eachGroup.TumblingWindow( TimeSpan.FromSeconds(10)) select new { eachGroup.Key, avg = win.Avg(e => e.W) };
Windowing Hopping Tumbling Snapshot Count
Time Windows Hopping Window Time
Time Windows Tumbling Window Time
Debugger Has its own debugger You cannot live without it (believe me) Set breakpoints Filter A wonderful tool
DEMOS Tour Trace Reader Live Trace File Reader Query Demo (Joining) Debugger Tool (How it will save you)
