top of page

Data Lake in AWS

Roman Echeverria
Jul 6, 2021
2 min read

Updated: Jul 8, 2021

Start Date: 4/Oct/2021

Seats: 20

Language: Spanish

Location: Remote (Zoom)

Duration: 12 hours


Description:


In this course, we will start by understanding when a data lake is the right solution as opposed to a data warehouse, you will learn all the components of a data lake.


You will start by building a Glue Data catalog and using Athena to query.

Then, we will work on Glue ETL, a powerful Apache Spark-based solution for data transformation.


You will learn finer-points on Glue Catalog Management and Schema Evolution


Finally, we will build a serverless application using Kinesis Firehose, Lambda, Comprehend AI, Glue, Athena and S3 that can process unlimited customer reviews, perform sentiment analysis, and store it in the data lake for querying.

Course content:


1.Introduction to Cloud Computing


AWS Cloud Computing and Global Infrastructure

Benefits of Cloud Computing

AWS Global Infrastructure


2.Querying and Analytics


In-place Querying and Analytics Tools

Streaming vs. Batch Analysis

Kinesis - Process and Analyze Streaming Data


3.Lab - AWS Account and User Setup


Account Setup

Lab - AWS Account Setup, Free Tier Offers, Billing, Support

Lab - Billing Alerts, Delegate Access

Lab - Configure IAM User


4.Monitoring, Optimization and Security


Monitoring and Optimization

S3 Labs

Lab - S3 Storage Classes

Data Encryption

Security and Protection

Lab - S3 Versioning

Lab - Age Based Retention (Lifecycle Management)

Lab - Storage Tiering (Lifecycle Management)

Lab - Replication (for disaster recovery)

Lab - Encryption with SSE-S3 and KMS


5.Data Catalog and In-place Querying


Glue and Athena Labs

Article - Source Code Download

Lab - Glue Data Catalog

Set up a query result location in S3

Lab - Athena In-place Querying

Lab - Glue ETL with Apache Spark - Convert Format to Parquet

Lab - Amazon Customer Review In-place Querying


6.Glue Catalog Management


Downloadable Resources

Download Latest Code and Files from GitHub

How to create PARQUET files?

Handling Schema Changes with CSV Format

Handling Schema Changes with PARQUET Format

Lab - Schema changes with CSV

Lab - Schema changes with PARQUET

Data Organization Best Practices - Folder Structure, Partitions, Classification

Lab - Partitioned Table Creation and Maintenance

Lab - Manual Catalog Maintenance

Lab - Hive Compatible Partitioning


7.Customer Review Sentiment Analysis


Customer Review Sentiment Analysis

Lab - Comprehend AI Service for Text Analysis

Lab - SageMaker Notebook Instance

Downloadable Resources

Lab - Source Code Setup

Lab - Python Introduction

Lab - Sentiment of Customer Review

Lab - Query Review By Sentiment


8.Serverless Application with Kinesis Firehose, Lambda, Comprehend AI, Glue


Streaming Data Consumption

Serverless Application for Customer Review Sentiment Analysis

Lambda UI Changes

Lab Part 1 - Serverless Customer Review Application

Lab Part 2 - Serverless Customer Review Application

 
 
 

Comments


Data Creed - Black Background.png

©2027 Data Creed

bottom of page