Skip to main content
Interactive Learning Path

Data Engineering Curriculum.

A beginner-to-advanced curriculum for building reliable data pipelines with batches and streams.

This path is designed to be stronger than generic catalogs:

  • Beginner theory + safe mental models
  • Intermediate design patterns (partitioning, idempotency, retries)
  • Advanced correctness for streaming systems (replay + sink safety)

Suggested workflow

  1. Start with fundamentals
  2. Learn batch pipeline safety contracts
  3. Finish with streaming exactly-once design

Roadmap Curriculum

1
data-engineering Step 1 of 3

Data Engineering Fundamentals

Build a complete ingest-transform-serve pipeline in fifty lines, then break it the three ways real pipelines break.

2
data-engineering Step 2 of 3

Batch Pipelines and Time Windows

Logical time versus wall-clock time, high-water marks that lose rows, and backfills that finish — the three things that decide whether a nightly job is correct.

3
data-engineering Step 3 of 3

Streaming: Exactly-Once Semantics

Simulate a consumer crash and watch at-least-once produce duplicates, then fix it three ways — atomic offset commits, idempotent keys, and deduplication on read.