Understanding Data Science Ecosystem Functional Layer
Explore the functional layer of the data science ecosystem, essential for programming and modeling. Learn about data structures, processing steps, and infrastructure provisioning in this key layer.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
FUNCTIONAL LAYER The functional layer of the data science ecosystem is the largest and most essential layer for Programming and modeling. Any data science project must have processing elements in this layer. The layer performs all the data processing chains for the practical data science. The functional layer is part of ecosystem that that runs the comprehensive data science eco system.
Data structures in the functional layer of the ecosystem are: Data schemas and data formats: Functional data schemas and data formats deploy onto the data lake s raw data, to perform the required schema-on-query via the functional layer. Data models: These form the basis for future processing to enhance the processing capabilities of the data lake, by storing already processed data sources for future use by other processes against the data lake. Processing algorithms: The functional processing is performed via a series of well-designed algorithms across the processing chain. Provisioning of infrastructure: The functional infrastructure provision enables the framework to add processing capability to the ecosystem, using technology such as Apache Mesos, which enables the dynamic previsioning of processing work cells.
Six Six Supersteps Supersteps for processing the data: for processing the data: 1. Retrieve: This super step contains all the processing chains for retrieving data from the raw data lake into a more structured format. 2. Assess: This super step contains all the processing chains for quality assurance and additional data enhancements. 3. Process: This super step contains all the processing chains for building the data vault. 4. Transform: This super step contains all the processing chains for building the data warehouse from the core data vault. 5. Organize: This super step contains all the processing chains for building the data marts from the core data warehouse. 6. Report: This super step contains all the processing chains for building virtualization and reporting of the actionable knowledge.
DATA SCIENCE PROCESS Following are the five fundamental data science process steps. Begin process by asking a What if question Attempt to guess at a probably potential pattern Create a hypothesis by putting together observations Verify the hypothesis using real-world evidence Promptly and regularly collaborate with subject matter experts and customers as and when you gain insights
