Continuous Data: Automating Data Ingestion from Cloud Storage into Snowflake

The Snowpipe feature allows Snowflake to input micro-batches of data as it becomes available generally within minutes of the data being added to a stage and submitted for ingestion. In this course you will implement the auto-ingestion of CSV files from external Snowflake stages located on the AWS and Azure cloud platforms. You will begin by setting up a continuous data ingestion pipeline where the data source is located in an Azure Storage Container. This pipeline will include several components such as queues enterprise applications and storage integrations as well as the permissions required to get these pieces to talk to one another. You will then implement something similar with an Amazon S3 bucket as the source of data. This set-up will involve AWS services such as IAM roles SNS topics as well as Snowflake objects such as notification integrations and pipes.