The definitive guide to Azure data engineering : modern ELT, DevOps, and analytics on the Azure Cloud Platform / Ron C. L'Esteve.
L'Esteve, Ron C., author.
2021
QA76.9.D3 L47 2021
Available Online

Linked e-resources

Linked Resource
Online Access
Concurrent users
Unlimited
Authorized users
Authorized users
Document Delivery Supplied
Can lend chapters, not whole ebooks

Details

Title
The definitive guide to Azure data engineering : modern ELT, DevOps, and analytics on the Azure Cloud Platform / Ron C. L'Esteve.
Author
L'Esteve, Ron C., author.
ISBN
9781484271827 (electronic bk.)
1484271823 (electronic bk.)
9781484271810
1484271815
DOI
https://doi.org/10.1007/978-1-4842-7182-7
Published
[Berkeley] : Apress, [2021]
Copyright
©2021
Language
English
Description
1 online resource : illustrations
Item Number
10.1007/978-1-4842-7182-7 doi
Call Number
QA76.9.D3 L47 2021
Dewey Decimal Classification
005.75/65
Summary
Build efficient and scalable batch and real-time data ingestion pipelines, DevOps continuous integration and deployment pipelines, and advanced analytics solutions on the Azure Data Platform. This book teaches you to design and implement robust data engineering solutions using Data Factory, Databricks, Synapse Analytics, Snowflake, Azure SQL database, Stream Analytics, Cosmos database, and Data Lake Storage Gen2. You will learn how to engineer your use of these Azure Data Platform components for optimal performance and scalability. You will also learn to design self-service capabilities to maintain and drive the pipelines and your workloads. The approach in this book is to guide you through a hands-on, scenario-based learning process that will empower you to promote digital innovation best practices while you work through your organization's projects, challenges, and needs. The clear examples enable you to use this book as a reference and guide for building data engineering solutions in Azure. After reading this book, you will have a far stronger skill set and confidence level in getting hands on with the Azure Data Platform. You will learn to: Build dynamic, parameterized ELT data ingestion orchestration pipelines in Azure Data Factory Create data ingestion pipelines that integrate control tables for self-service ELT Implement a reusable logging framework that can be applied to multiple pipelines Integrate Azure Data Factory pipelines with a variety of Azure data sources and tools Transform data with Mapping Data Flows in Azure Data Factory Apply Azure DevOps continuous integration and deployment practices to your Azure Data Factory pipelines and development SQL databases Design and implement real-time streaming and advanced analytics solutions using Databricks, Stream Analytics, and Synapse Analytics Get started with a variety of Azure data services through hands-on examples.
Access Note
Access limited to authorized users.
Source of Description
Online resource; title from PDF title page (SpringerLink, viewed August 24, 2021).
Available in Other Form
Print version: 9781484271810
Linked Resources
Online Access
Record Appears in
Online Resources > Ebooks
All Resources
Introduction
Part I. Getting Started
1. The Tools and Pre-Requisites
2. Data Factory vs SSIS vs Databricks
3. Design a Data Lake Storage Gen2 Account
Part II. Azure Data Factory for ELT
4. Dynamically Load SQL Database to Data Lake Storage Gen 2
5. Use COPY INTO to Load Synapse Analytics Dedicated SQL Pool
6. Load Data Lake Storage Gen2 Files into Synapse Analytics Dedicated SQL Pool
7. Create and Load Synapse Analytics Dedicated SQL Pool Tables Dynamically
8. Build Custom Logs in SQL Database for Pipeline Activity Metrics
9. Capture Pipeline Error Logs in SQL Database.-10. Dynamically Load Snowflake Data Warehouse.-11. Mapping Data Flows for Data Warehouse ETL
12. Aggregate and Transform Big Data Using Mapping Data Flows
13. Incrementally Upsert Data.-14. Loading Excel Sheets into Azure SQL Database Tables.-15. Delta Lake
Part III. Real-Time Analytics in Azure
16. Stream Analytics Anomaly Detection
17. Real-time IoT Analytics Using Apache Spark
18. Azure Synapse Link for Cosmos DB
Part IV. DevOps for Continuous Integration and Deployment
19. Deploy Data Factory Changes
20. Deploy SQL Database
Part V. Advanced Analytics
21. Graph Analytics Using Apache Spark's GraphFrame API
22. Synapse Analytics Workspaces
23. Machine Learning in Databricks
Part VI. Data Governance
24. Purview for Data Governance.

Browse Subjects

Show more subjects...

Statistics

from
to
Export