Data Engineer Professional Certificate

Build the pipelines that feed analytics and machine learning: SQL, data modelling, orchestration, batch and stream processing, and data quality.

Start with SQL and the foundations of data engineering, then model a data warehouse, orchestrate pipelines with Airflow and transform data with dbt. Process data at scale with Spark and as it arrives with Kafka, make it trustworthy with tests and data contracts, and learn how the cloud runs it all. Every course has hands-on labs on your own machine and a capstone project your instructors review.

Prepares you for

Data engineer, Analytics engineer, ETL developer, Big data engineer

9 courses in this Professional Certificate

  • SQL for Data Analysis with DataGrip: Answer real business questions with SQL: filter, summarise, join and rank a shop's sales data in DataGrip, then export the results to share. (Beginner, about 10 hours)
  • Data Engineering Foundations: Batch and Streaming Pipelines: Build reliable data pipelines in Python and DuckDB: file formats, idempotent and incremental loads, partitioned Parquet, quality checks and a first stream. (Beginner, about 15 hours)
  • Data Warehousing and Dimensional Modelling: Design star schemas the Kimball way, keep history with slowly changing dimensions, and build and load a working warehouse in DuckDB with SQL you run. (Intermediate, about 18 hours)
  • Orchestrating Data Pipelines with Apache Airflow: Write, schedule, test and operate Airflow 2 pipelines in Python, with TaskFlow, data intervals, backfills, sensors and retries, and know what changes in Airflow 3. (Intermediate, about 15 hours)
  • Analytics Engineering with dbt and DuckDB: Turn raw data into tested, documented models with dbt Core and DuckDB: sources, staging, marts, tests, snapshots, incremental models and CI. (Intermediate, about 16 hours)
  • Big Data Processing with Apache Spark: Process large datasets with PySpark DataFrames and Spark SQL, and make jobs fast: partitions, shuffles, joins, windows, caching, Parquet, the Spark UI and tuning. (Intermediate, about 14 hours)
  • Real-Time Data Streaming with Apache Kafka: Run Kafka 4 in KRaft mode, write reliable Python producers and consumers, use transactions and schemas, and build windowed stream processing that copes with late data. (Advanced, about 18 hours)
  • Data Quality, Testing and Data Contracts: Measure data quality, test data in pipelines with pandera, agree data contracts between producers and consumers, and catch and handle bad data before it does harm. (Intermediate, about 16 hours)
  • Cloud Computing Fundamentals: Understand how the cloud works, from service models and regions to compute, storage, networking, access and cost, with AWS, Azure and Google Cloud side by side and labs that need no account. (Beginner, about 15 hours)