[ENG] Michal Karzynski: “Developing elegant workflows with Apache Airflow“ / #PiterPy

Developing elegant workflows with Apache Airflow Every time a new batch of data comes in, you start a set of tasks. Some tasks can run in parallel, some must run in a sequence, perhaps on a number of different machines. That’s a workflow. Did you ever draw a block diagram of your workflow? Imagine you could bring that diagram to life and actually run it as it looks on the whiteboard. With Airflow you can just about do that. Apache Airflow is an open-source Python tool for orchestrating data processing pipe