Enhancing Network Security Using Snort Virtual Network Function with DPI Service
Deep Packet Inspection (DPI) as a service is explored in this work, aiming to improve performance, innovation, and security in network operations. By extracting DPI from middleboxes and offering it as a shared service, the paper suggests benefits such as optimized packet scanning, enhanced functionality, and cost reduction in network security implementations.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Snort Virtual Network Function with DPI Service Asher Gruber | January 2017 This work was carried out under the supervision of Prof. Anat Bremler-Bar and Mr. Yotam Harhol
Agenda Introduction DPI as a Service paper Project Goals Background Deep Packet Inspection Snort Overview Service Chaining Network Service Header (NSH) Implementation NSH Support DPI Service Enhancement Snort Enhancement Experimental Results Conclusions Snort VNF with DPI Service 2
Introduction Snort VNF with DPI Service 3
DPI as a Service Problem Deep Packet Inspection (DPI) is a widespread functionality among middlebox applications In the common architecture, each packet is inspected from scratch by multiple middleboxes on its path until reaching its final destination For numerous middleboxes the IDS task is the most time-consuming and can take most of the processing time Snort VNF with DPI Service 4
DPI as a Service (Cont.) Solution Extract DPI out of the middleboxes Provide DPI as a service to the middleboxes Service inspects the packets once, against the rules of all the middleboxes Service reports the results Middleboxes consume the results, while avoiding the highly expensive pattern search phase, and focus on performing more complex processing based on to the pattern match results Snort VNF with DPI Service 5
DPI as a Service (Cont.) Solution Advantages Performance each packet is scanned once Shared network services create room for innovation Advanced DPI functionality Consolidated DPI allows to focus more on functionality Security Focus on securing a single DPI implementation Deploy more instances across the network to mitigate attacks Reduced cost of middleboxes Snort VNF with DPI Service 6
DPI as a Service (Cont.) The paper outlines a framework for deploying the service and provide a reference implementation on a simulation environment The DPI as a Service framework implementation supports the deployment of multiple DPI services across the network which are all controlled by a centralized DPI Controller The controller is responsible for managing the overall DPI process such as middlebox registration, pattern set management, and DPI service initialization Snort VNF with DPI Service 7
Project Goals Integrate the DPI as a Service framework with the complex "real world" Snort NIDS Integrate the DPI Service with Snort using the Network Service Header (NSH) protocol The integrations will allow to evaluate if the suggested framework can operate in a more realistic environment setup Snort VNF with DPI Service 8
Background Snort VNF with DPI Service 9
Deep Packet Inspection (DPI) Inspection of packet payloads to identify predefined sets of patterns Patterns Strings matching Regular expressions matching String matching is a core component in most DPI engines Used for pre-filtering Constitutes most of the work performed by the engine Network Intrusion Detection Systems (NIDS) perform DPI to detect malicious content in packet payloads routed through the network Snort VNF with DPI Service 10
Snort overview Open source NIDS Widely adopted by the security community Used by numerous enterprises Modes Sniffer Packet Logger NIDS Snort VNF with DPI Service 11
Packet Processing Flow Network packets processed by Snort follow a similar flow Snort VNF with DPI Service 12
Packet Acquisition Performed via Data AcQuisition library (DAQ) Snort supports packet capturing Network interface Input file Acquire Decode Preprocess Detect Output Snort VNF with DPI Service 13
Packet Decoding Packet are decoded according to network protocol stack Acquire Packet Decode DecodeEthPkt (Ethernet) Preprocess DecodeIP (IPv4) Detect Output DecodeICMP (ICMP) DecodeTCP (TCP) DecodeUDP (UDP) Snort VNF with DPI Service 14
Packet Preprocessing Preprocessors perform variety of operation Packet checks and alerting Packet data modification Normalization Reassembly Decompression More Acquire Decode Preprocess Detect Output Snort VNF with DPI Service 15
Detection Detection engine is focused around identifying predefined sets of attack patterns Snort rules allow to define NIDS policies DPI options Content match content in packet payload PCRE search packet payload for RegEx pattern Acquire Decode Preprocess Detect Output Snort VNF with DPI Service 16
Detection (Cont.) Rule Engine Common Snort setup includes vast number of rules The rule engine performance is extremely important Rule are grouped by protocol, direction and port Groups are stored in special data structures Each rule group has a dedicated MPSE instance Multi-Pattern Search Engine Pattern matching on real-time network traffic Interface for pattern matching algorithms Wrapper of the Snort implementation of the Aho-Corasick (AC) algorithm Acquire Decode Preprocess Detect Output Snort VNF with DPI Service 17
Detection (Cont.) Aho-Corasick (AC) String matching algorithm which is commonly used by NIDS Matches multiple strings simultaneously Constructs a DFA from the pattern set (defined in rules) Accepting state = pattern match O(n) search time; n = payload size Implementation is critical, since it has a significant impact on the overall performance of Snort Acquire Decode Preprocess Detect Output Snort VNF with DPI Service 18
Snort Output Packets are matched during detection and queued Actions (log, alert) are fetched from the event queue Actions are filtered (rate, threshold) and triggered Acquire Decode Preprocess Detect Output Snort VNF with DPI Service 19
Snort Configuration Snort configuration are defined in snort.conf Rule operations (e.g. path to rule file) Plug-ins activation Much more Distribution contains sensible defaults Usually customized for specific environment Snort VNF with DPI Service 20
Service Chaining Modern way for deploying and delivering composite services Logical group of Service Functions (SF) (Firewall, DPI, etc.) a packet needs to go through Elastic, simple and modern service deployment model by treating SF as resources which can be scheduled and consumed Linked based off policy Web NAT FW DPI FW LB Classifier Classifier Classifier Web Snort VNF with DPI Service 21
Service Chaining (Cont.) Services are assembled using Software Defined Network (SDN) Service Path is required for implementation Packets are "steered" to the next SF in the path With metadata (the killer app) Using Network overlay (VxLAN, GRE, MPLS) Possible today, but suffers from tight coupling to the transport layer Web NAT FW DPI FW LB Classifier Classifier Classifier Web Snort VNF with DPI Service 22
Network Service Header (NSH) Data-plan protocol which defines a service plane IETF adopted protocol (NSH) Two main components: path info and metadata Topology Independence De-coupling of service topology and the actual network topology Service forwarding is within the service plane Enables Service Chaining Has path ID needed to realize service path Provides ability to monitor and troubleshoot service chain end-to-end Transport Agnostic appropriate transport to encapsulate traffic NSH-aware control plane is required (e.g. OpenDaylight) Snort VNF with DPI Service 23
SDN Controller platform 1 1 SF1 SF2 SF3 SF4 NSH: SPI ID 10, SI: 1 4 6 NSH: SPI ID 10, SI: 2 Metadata Packet Packet Service Classifier Classification Policy SFF SFF 2 3 5 7 Packet Packet VXLAN: D-IP VXLAN: D-IP NSH: SPI ID 10, SI: 2 NSH: SPI ID 10, SI: 1 Packet Metadata Packet Snort VNF with DPI Service 24
Implementation Snort VNF with DPI Service 25
Implementation DPI Service was enhanced to support the NSH protocol and can now pass the pattern match results with the inspected packet Snort was enhanced in order to apply its rules without the need to re- scan the packets from scratch, while leveraging the match results reported by the DPI Service Snort was furthermore enhanced in order to allows it's registration to the DPI controller Snort VNF with DPI Service 26
NSH Support DPI Service match results are reported using NSH NSH supports metadata exchange along the service path The metadata is used to transfer the pattern match results Snort VNF with DPI Service 27
NSH Support (Cont.) We use VxLAN-gpe as the encapsulation protocol VxLAN is encapsulated as specified in the RFC UDP (port = 4789) IP Ethernet Snort VNF with DPI Service 28
DPI Service Enhancements Original Implementation used dedicated packets to report pattern match results DPI Service was extended to report pattern match results within the inspected packet using the NSH protocol Reporting result using dedicated packets is still supported Packets without pattern matches are forwarded as received Snort VNF with DPI Service 29
DPI Service (Cont.) DPI Controller Initialize 1 Network Services Header NSH base header 4 Service path (24 bit) / Index DPI Service Optional Metadata 2 3 5 Packet Packet Packet Eth Eth IP UDP VxLAN IP Payload NSH Packet Scanned Results Aggregated Port 4790 Snort VNF with DPI Service 30
Snort Enhancements Snort configurations were added to support the DPI Service functionality (e.g. On\Off, settings, etc.) Snort was extended to communicate with the DPI Controller Registration of Snort instance Registration of instance rule patterns Snort was modified to leverage the DPI Service pattern match results NSH Support (decode) VxLAN Support (decode) Bypass the Snort DPI (decode, detect) More Snort VNF with DPI Service 31
Snort DPI Service Configuration DPI Service functionality can be controlled via the Snort configuration file (i.e. snort.conf) Parsing logic was added to the Snort initialization phase Snort VNF with DPI Service 32
Snort Controller Communication Snort registration to the DPI Controller was added JSON message Middlebox s ID (i.e. Snort ID) Rule patterns required for DPI Allows Controller to register Snort to DPI Service Allows DPI Service to report matches to Snort Message is sent according to snort.config Snort VNF with DPI Service 33
Snort Controller Communication (Cont.) Message is constructed during Snort initialization . . . SnortConf prmUdpRTNX . . . . . . prmIpRTNX prmIcmpRTNX prmTcpRTNX PORT_RULE_MAP PORT_GROUP (Content) MPSE 88, 8080 prmSrcPort[] MPSE . . . 9000 prmDstPort[] . . . 53 prmGeneric MPSE Snort VNF with DPI Service 34
Snort Controller Communication (Cont.) Rule patterns are fetched from the AC DFA of the various rule groups The patterns are taken from the DFA accepting states Fetching rules via the DFA is essential We want the DPI Service to search only for patterns that are searched by the DFA We want to bypass the AC DFA execution by Snort and use the rule match results We later use the same code base to associate rules to their DFA accepting state Snort VNF with DPI Service 35
Snort DPI Service Integration The goal is to prevent Snort from re-scanning the packets using the MPSE (AC DFA) which has a significate impact on the overall performance Enabling Snort to use the DPI Service pattern match results required multiple medications to the code base Snort Initialization Snort Packet Processing Decoding Detection Snort still needs to have the ability to perform other DPI operations using the MPSE and AC DFA, since finding a matched pattern using the AC DFA does not guarantee a rule match Snort VNF with DPI Service 36
Snort Initialization To take advantage of the DPI Service results, while skipping the AC DFA search, a mapping between a Snort rule and its associated AC DFA accepting state is required The mapping will allow Snort to use the accepting state in order to perform additional DPI rule operations which are required for every matched rule that is reported by the DPI Service The creation of the mapping is performed during the Snort initialization phase while constructing the registration massaged, since it involves traversing the same data structures Snort VNF with DPI Service 37
Snort Initialization (Cont.) The registration message is built while visiting the AC DFA accepting states associated to each of the rule groups (one AC DFA per rule group) DFA accepting states represents a pattern of at least one or more rules Obtaining an accepting state provides access to the associated rules and patterns During the registration message construction, whenever a rule-pattern pair is added to the message the rule-accepting state pair is added to the mapping Every rule group has a dedicated AC DFA, therefore a rule-accepting state map will be created per rule group (or rule group AC DFA) Snort VNF with DPI Service 38
Snort Initialization (Cont.) The Rule Group => (rid => acc. state) mapping is essential, since it is possible that the DPI Service will report a rule match within a packet that does not meet the packets rule group Having a rule-accepting state map which is shared amount all rule groups will potentially cause false positive rule match alerts rid acc. state 56 C1 Rule Group DFA 71 C7 DFA 34 C9 DFA DFA rid acc. state DFA 9 C3 12 C5 86 C14 Snort VNF with DPI Service 39
Snort Packet Processing - Decoding Traditionally when a packet is captured by Snort it is decoded according to it s network protocol stack We have extended Snort to support the NSH protocol Supporting NSH required supporting VxLAN (UDP, port = 4789) The support for the protocols was added to the UDP decoding stack Once the NSH Base Header is decoded we extract the pattern match results from the metadata Snort VNF with DPI Service 40
Packet DecodeEthPkt (Ethernet) DecodeIP (IPv4) DecodeTCP (TCP) DecodeUDP (UDP) DecodeICMP (ICMP) Snort VNF with DPI Service 41
Packet DecodeEthPkt (Ethernet) DecodeIP (IPv4) DecodeTCP (TCP) DecodeUDP (UDP) DecodeICMP (ICMP) DecodeVxLAN (VxLAN) DecodeNSH (NSH) Snort VNF with DPI Service 42
Snort Packet Processing Decoding (Cont.) The decoded match reports are added to the packet structure The list of match reports is used to bypass the AC DFA search during the packet detection phase When the decoding of the NSH is completed we continue to decode the original inner packet which follows the standard Snort decoding stack Snort VNF with DPI Service 43
Snort Packet Processing Detection No packet re-scan If DPI Service active go to alternative func Bypass AC DFA Once func completed packet returns to standard processing Snort VNF with DPI Service 44
Snort Packet Processing Detection (Cont.) Snort VNF with DPI Service 45
Snort Packet Processing Detection (Cont.) The Match operation is also called in the standard Snort analysis functionality whenever an accepting state is reached in the AC DFA search By calling the Match operation we assure that all the DPI content options which were not included in the DPI Service search are met before the rule is considered as matched Once the Match operation concludes that all the options of a given accepting state are met it registrars a match event to the queue. The analysis functionality ends when all the rule match results are evaluated. Then the packet continues through the standard Snort packet processing flow to the output phase Snort VNF with DPI Service 46
Experimental Results Snort VNF with DPI Service 47
Experimental Results We analyze the integration of Snort with the DPI Service by repeating a subset of the original paper experiments We compare the results, to those of the original paper Repeating the original experiments will allow us to evaluated if the promising results presented in the paper, can be reproduced in a more realistic environment setup Snort VNF with DPI Service 48
Experimental Results - Environment Machine Intel Xeon E3-1270 v3 CPU Quad-Core, quad-core, each core having two hardware threads Cache: 32 KB L1 (per core), 256 KB L2(per core), 8 MB L3 (shared) Linux Ubuntu 14.04 LTS (Trusty) Setup Input traffic - 148MB HTTP trace crawled from most popular websites Rules Snort - 3498 original Snort rules DPI Service Rules sent by Snort to the Controller Exact match patterns of length 4 or more Experiments were conducted using the DPI Service and Snort Snort VNF with DPI Service 49
Pipelined Middlebox Scenario At least 38% faster Without a DPI Service Latency traditional: 32.73 s/p 16% improvement With a DPI Service Latency DPI Service: 27.57 s/p Snort VNF with DPI Service 50
