Building Data Lakes on AWS
Please not that this is not a prep course for the “AWS Certified Data Engineer - Associate” (or any other) certification!
This document contains links mentioned in the “Building Data Lakes on AWS” course.
Module 0: Course overview
Full name: Building Data Lakes on AWS
Product code: 200-ANBDLK-15
These links are based on the course version 1.5.3.
The official course material: eVantage Bookshelf (VitalSource)
Additional material: AWS Skill Builder
Module 1: Introduction to data lakes
Websites
- AWS serverless data analytics pipeline reference architecture
- Reference architecture
- AWS serverless data analytics pipeline reference architecture
- Analytics on AWS
Customer Stories
Module 2: Data ingestion, cataloging, and preparation
Websites
- Data Lakes on AWS
- Zero-ETL Integrations
- Using Amazon S3 Storage Lens to optimize your storage costs
- Storage Best Practices for Data and Analytics Applications
- Data ingestion methods
- Autodesk Builds Unified Log Analytics Solution on AWS to Gain New Insights
- Sending data to a Firehose stream
- Real-Time Insights on AWS Account Activity
- Catalog and search
- AWS Glue
- Built-in classifiers in AWS Glue
- Working with classifiers on the AWS Glue console
- Populating the AWS Glue Data Catalog
- Data format options for inputs and outputs in AWS Glue for Spark
- Data protection in AWS Glue
- Amazon Athena
- Partitioning data in Athena
- Top 10 Performance Tuning Tips for Amazon Athena
- Using Amazon Athena Federated Query
- Understanding columnar storage formats
Module 3: Building a data lake with AWS Lake Formation
Websites
- AWS Lake Formation
- Getting started with Lake Formation
- Changing the default settings for your data lake
- Upgrading AWS Glue data permissions to the AWS Lake Formation model
Module 4: Data processing and analytics
Websites
- AWS Glue announces AWS Glue Data Quality (Preview)
- Recipe step and function reference
- Redacting PII from application log output with Amazon Comprehend
- AWS Glue PySpark transforms reference
- Recipe step and function reference
- Editing AWS Glue managed data transform nodes
- AWS Marketplace / Masking/Tokenization
- Discover sensitive data by using custom data identifiers with Amazon Macie
- Use Macie to discover sensitive data as part of automated data pipelines
- Working with Spark jobs in AWS Glue
- Streaming ETL jobs in AWS Glue
- Matching patient records with the AWS Lake Formation FindMatches transform
- Developing and testing AWS Glue job scripts locally
- Working with Ray jobs in AWS Glue
- Tutorial: Writing an AWS Glue for Spark script
- What is AWS Glue?
- Getting started with AWS Glue interactive sessions
- Resource-based policies within AWS Glue
- Overview of workflows in AWS Glue
- Running and monitoring a workflow in AWS Glue
- Amazon CloudWatch
- Amazon EventBridge events
- What is Amazon CloudWatch Logs?
- What Is AWS CloudTrail?
Videos
- Build Your Data Lake on Amazon S3 - AWS Online Tech Talks
- Getting Started with AWS Glue ETL
- Build ETL Processes for Data Lakes with AWS Glue - AWS Online Tech Talks
Module 5: AWS Lake Formation Additional Configurations
Websites
- Blueprints and workflows in Lake Formation
- Creating a workflow
- How Athena accesses data registered with Lake Formation
- Managing Lake Formation permissions
- Underlying data access control
- Metadata access control
- How Lake Formation application integration works
- Lake Formation best practices, considerations, and limitations
- Data filters in Lake Formation
- Overview of data filtering
- Lake Formation tag-based access control (TBAC)
- Granting LF-Tag permissions using the console
- Cross-account data sharing in Lake Formation
- Cross-account data sharing best practices and considerations
- Lake Formation personas and IAM permissions reference
- Amazon QuickSight
- Managing SPICE memory capacity
- Publishing dashboards
- Working with embedded analytics
- Embed interactive dashboards in your application with Amazon QuickSight
Courses
Module 6: Modern data architecture
Websites
- Analytics on AWS
- Learn how to build Modern Data Analytics Solutions on AWS
- Modern Data Analytics Reference Architecture on AWS
- OpenSearch
- Apache Spark on Amazon EMR
- Apache Hive on Amazon EMR
- Presto on Amazon EMR
- Modern Data Reference Architecture
- Derive Insights from AWS Modern Data
- Using Redshift Spectrum with AWS Lake Formation
- Use an event-driven architecture to build a data mesh on AWS
- Data Mesh Reference Architecture
- How JPMorgan Chase built a data mesh architecture to drive significant value to enhance their enterprise data platform
- How GoDaddy built a data mesh to decentralize data ownership using AWS Lake Formation
- AWS Analytics Reference Architecture
- Build a Data Mesh on AWS
Courses
- Building Batch Data Analytics Solutions on AWS
- Building Data Analytics Solutions Using Amazon Redshift
- Building Streaming Data Analytics Solutions on AWS
- Authoring Visual Analytics using Amazon QuickSight
Module 7: Course review and wrap-up
Websites
- Data Analytics Lens
- Modern Data Reference Architecture
- Configuration notes
- Derive Insights from AWS Modern Data
- Monitoring and optimizing the data lake environment
- AWS Glue Best Practices: Building a Performant and Cost Optimized Data Pipeline
- AWS Pricing Calculator
- AWS Certification exam preparation
- AWS Certification
- AWS Recertification
- All AWS Certification exams
- Schedule an Exam
- AWS Skill Builder subscription
- AWS Training and Certification
- AWS Online Tech Talks
- AWS Ramp-Up Guides