SRE Data Pipelines & Integrity: Pipeline Design

Site reliability engineers (SREs) encounter numerous and varied pipeline technologies and frameworks in their work. When building a pipeline SREs need to invest considerable time during the design phase to ensure the results work best for the specific case. In this course youll explore the numerous features of a pipeline such as latency high availability development and operations. Youll also examine the two different pipeline mutations: idempotent and two-phase as well as the checkpointing technique and various code patterns. Youll then investigate the five core characteristics of the pipeline maturity matrix and outline how they should be used to design the pipeline technology. Youll then identify potential failure modes outage causes and different prevention and response techniques. Finally youll outline event delivery system design and operations and how to plan for customer integration and support.