Data Lake Architectures & Data Management Principles

A key component to wrangling data is the data lake framework. In this 9-video Skillsoft Aspire course learners discover how to implement data lakes for real-time management. Explore data ingestion data processing and data lifecycle management with Amazon Web Services (AWS) and other open-source ecosystem products. Begin by examining real-time big data architectures and how to implement Lambda and Kappa architectures to manage real-time big data. View benefits of adopting Zaloni data lake reference architecture. Examine the essential approach of data ingestion and comparative benefits provided by file formats Avro and Parquet. Explore data ingestion with Sqoop and various data processing strategies provided by MapReduce V2 Hive Pig and Yam for processing data with data lakes. Learn how to derive value from data lakes and describe benefits of critical roles. Learners will explore steps involved in the data lifecycle and the significance of archival policies. Finally learn how to implement an archival policy to transition between S3 and Glacier depending on adopted policies. Close the course with an exercise on ingesting data and archival policy.