Luigi Goes Brrr: Simplifying Task Orchestration Without the Headaches

Presented at Montréal-Python MP-108 at the NAD School of Digital Arts, Animation and Design.

This presentation explores Luigi as a lightweight solution for managing complex workflows without the overhead of tools like Airflow. Designed for batch processing, ETL, and ML pipelines, Luigi offers scalable task orchestration with dependency management, parallel execution, and built-in error handling. Key topics include DAG-based sequencing, resilient processing, and use cases that highlight Luigi’s fit for small to medium-sized data workflows. This talk demonstrates how Luigi enables efficient task orchestration, allowing Python developers to manage dependencies and streamline operations in a straightforward, maintainable way.