publish date
Oct 12, 2022
duration
29
min
Difficulty
Case details
Sometimes creating a Data Pipeline can seem like a trivial problem can it just be solved with a small script and a CRON job. You would be surprised at how many ways this could go wrong. During this talk we will talk about creating data pipelines using Apache Airflow and what it brings to the table. The talk will be structured in the following way: - introduction about data pipelines and problems around poor implementations - brief introduction to airflow - running airflow on docker - live coding a DAG - analysis of a real-word scenario [h2][b][url=https://docs.google.com/presentation/d/1F7aUyLgxcn88mB449QMWiThba-Tc2I2ADmyeuiVwCfc/edit?usp=sharing]Slides[/url][/b][/h2] [color=#111111][size=2][font=Verdana, Arial, Helvetica, sans-serif][h2][b][url=https://github.com/david1983/pygeekle22-airflow]Repo[/url][/b][/h2][/font][/size][/color]
Share case: