All workshops

Details

Apache Airflow is a powerful tool for orchestrating and automating complex tasks and data workflows, but it can be overwhelming to navigate without guidance. This comprehensive workshop will take you from the basics to advanced concepts, helping you master Airflow for building, scheduling, and managing workflows at any scale. By the end, you’ll know how to use Airflow to create reliable, maintainable data pipelines, automate tasks, and troubleshoot complex workflows effectively.

What You'll Learn

This workshop is structured to build your expertise step-by-step, covering everything from foundational skills to advanced techniques:

• Introduction to Workflow Orchestration and Apache Airflow: Understand the role of Airflow in data engineering and why it’s a popular choice for building and automating complex workflows.
• Installing and Configuring Airflow: Get started with Airflow by setting up a development environment and learning the essentials of configuring the platform.
• Understanding DAGs (Directed Acyclic Graphs): Learn how to define workflows using DAGs and understand their structure to create clear, maintainable workflows.
• Creating and Scheduling Tasks: Discover the basics of creating tasks with Python operators, dependencies, and setting up scheduling rules.
• Managing Data Pipelines: Build modular and reusable pipelines by breaking workflows into smaller tasks and organizing code efficiently.
• Error Handling and Retries: Learn to manage failures and set up retry logic to make workflows more resilient and fault-tolerant.
• Working with Airflow Operators: Explore the range of built-in operators (Python, Bash, Email, SQL, and more) and use custom operators to tailor workflows to your needs.
• Managing Dependencies and Task Concurrency: Set up task dependencies and manage parallel execution to optimize workflow performance.
• Advanced Scheduling with CRON and Timetables: Deepen your understanding of scheduling and configure custom schedules to fit specific data needs.
• Data Transfer and Integration: Use Airflow’s integrations with popular data storage and processing tools (e.g., AWS, Google Cloud, Spark) for smooth data transfers.
• Managing Secrets and Credentials: Securely handle credentials and API keys using Airflow’s connection management and environment variables.
• Workflow Monitoring and Logging: Track and troubleshoot workflows with Airflow’s logging and monitoring tools, helping you spot issues before they become problems.
• Optimizing Performance and Scaling: Dive into performance tuning, resource management, and scaling Airflow to handle large, complex pipelines.
• Best Practices for Airflow in Production: Learn deployment strategies, backup and recovery, DAG version control, and ongoing maintenance for production environments. Dynamic DAGs and Custom Plugins: Implement dynamic DAG generation for flexible workflows and use custom plugins to extend Airflow’s functionality.

Who Should Attend

This workshop is ideal for data engineers, analysts, and anyone who needs to automate and manage data workflows. Beginners with basic Python knowledge are welcome, while more advanced participants will gain new techniques to optimize and scale their Airflow setups.

Details

Deploying your Django application for the first time can feel daunting. From selecting a hosting provider to configuring your server, handling secrets, and ensuring security, there's a lot to tackle. This workshop will help you cut through the confusion and walk you through each step of getting your Django project live on FlyIO, a flexible and powerful hosting platform. By the end, you’ll have your own deployed Django application, complete with database setup, secure static file handling, and optimized performance.

What You'll Learn

In this tutorial, we’ll cover a complete deployment workflow, ensuring your application is production-ready. Topics include:

• Fundamentals of Application Deployment: Learn the basics of deploying a Django application, from initial setup to going live.
• Static File Hosting: Configure static file handling to serve CSS, JavaScript, and other resources efficiently.
• Setting Up a PostgreSQL Database: Configure a cloud-based PostgreSQL database to handle your application’s data.
• Performance Optimization: Explore caching strategies and performance tweaks that can improve load times and responsiveness.
• Managing Secrets and Environment Variables: Securely manage sensitive information like API keys and database credentials.
• Domain Setup and HTTPS Configuration: Set up a custom domain name for your application and configure HTTPS to ensure secure communication.
• Monitoring and Logging: Implement monitoring and logging to gain visibility into your app’s performance and troubleshoot any issues that arise in production.

Who Should Attend

This workshop is designed for developers with intermediate experience in Django who are ready to take the next step and deploy their applications. Basic knowledge of Django is required, but prior experience with cloud deployment is not necessary.

Join us to turn your Django project into a live, production-ready application!

Details

Once upon a time, if you wanted a truly modern web frontend experience for your Python web app you would have limited options. Many people would reach for heavy frontend tools and frameworks such as React, Vue or one of their many competitors. And those tools generally have their own recommendations around testing.

Times have changed - tools like HTMX and AlpineJS are taking Python Web dev by storm and simplifying the creation of modern frontends

These tools make full-stack web dev fast and fun, they simplify development, shrink code bases, and give backend devs the ability to do modern frontend work. In my personal experience, I've seen the adoption of these tools lead to massive productivity gains for junior and senior devs alike.

This workshop will walk you through the process of building and testing your own unique CRUD application with a rich and modern frontend.

Social impact

Profits raised through this workshop will go towards funding teacher and technical training for those who would otherwise not be able to afford it.

The stack

We'll be using the following tools:

• Django
• HTMX
• TailwindCSS
• AlpineJS
• Playwright

Prerequisite knowledge

It would be useful if you are comfortable with the following:

• Python
• Virtual environments
• HTML
• Basic SQL database familiarity

Why Django?

Django is brilliant. It's the web framework for perfectionists with deadlines. It's worth knowing about. And, for newer web developers, Django is an amazing teacher.

That said, if you are not into Django then that is fine! The focus of this tutorial is on building the frontend. The vast majority of what we cover in this tutorial can be used as is with other web frameworks. For example FastAPI, expressJS or similar.

Why CRUD?

90% of the web is CRUD. CRUD stands for Create, Read, Update and Delete. Once you know how to implement simple CRUD apps, you will be able to do quite a lot.

My own unique CRUD app?

The workshop will guide you through the process of building a specific application. You will be encouraged to choose your own adventure and make use of the skills covered to build your own project.

In previous workshops learners have worked on projects like:

• A music store for independent creators
• A household chore scheduler
• A conference talk tracker
• A personal learning pathway creator
• Yet another todo list app
• much more

You can do quite a lot with a CRUD app.

Small group - high support

We are keeping the number of tickets on sale low so that each course participant can get the attention they need. We believe in the power of engaged teachers.

Self-paced

The workshop is designed to empower learners to move at their own pace. We wont push anyone to keep up when they can't, or make anyone slow down when they are on a roll.

If you struggle with earlier parts of the workshop then you'll be supported. The goal here is not to rush through things and tick boxes. The goal here is to empower learners with real skills.

Filling the gaps

Different people will find this workshop challenging in different ways. Some will need extra support while setting up their projects, some will be challenges by the database interactions.

We're here to support you with your unique challenges.

About the instructor

Sheena O'Connell is the instructor for this workshop.

Sheena says:

"My early career saw me working as a software engineer and technical leader across multiple startups. But it was my passion for education that led me to devote the last 5+ years to reimagining how we teach people to code professionally.

Over the last half decade I have had the opportunity to work in the NGO space and build alternative education systems from the ground up. Along the way I have learned a lot about how to teach well, how to build systems that teach well, and how traditional education systems fall short.

I've always had a passion for education and had the opportunity to work directly in tech education for the last half decade. The way I think of my work is: I take the science of learning and turn it into the engineering of learning."

Want to know more about Sheena? Here are some links:

• Sheena's blog
• LinkedIn
• Mastodon

Details

Data science projects often involve multiple data sources, evolving codebases, and a need for reproducibility and collaboration. Version control is essential for managing these complexities, yet it’s often an overlooked skill among data scientists. In this hands-on workshop, we’ll introduce you to Git and GitHub, helping you effectively manage your code, track changes, and collaborate smoothly on data-driven projects. By the end of the day, you’ll feel confident using version control to organize your projects and work efficiently, whether you’re collaborating with a team or working solo.

What You'll Learn

This workshop will cover everything a data scientist needs to know about version control, including:

• Introduction to Version Control and Git: Understand the core principles of version control, why it matters for data science, and how Git can streamline your workflow.
• Setting Up and Managing Repositories: Learn to create, clone, and manage repositories for your projects, setting a solid foundation for any project.
• Tracking Code and Data Changes: Discover how to track changes in your code, datasets, and notebooks to preserve and revert to previous versions.
• Branching and Merging for Experimentation: Use branches to manage and document different versions of your models and experiments, and learn best practices for merging updates.
• Collaborating on GitHub: Master the basics of GitHub for sharing code, managing repositories, and collaborating with other data scientists.
• Conflict Resolution: Gain practical skills in resolving merge conflicts and other common issues that arise when collaborating.
• Documentation Best Practices: Learn to document your project clearly to keep track of experiments, assumptions, and results.
• Data-Specific Version Control Tools: Explore tools and extensions like DVC (Data Version Control) for managing large datasets and keeping data in sync with your code.
• Reproducibility with Git Tags and Releases: Use tags and releases to document and reproduce results, ensuring others can follow and replicate your work.
• Git for Jupyter Notebooks: Learn special techniques for version-controlling Jupyter notebooks and effectively handling large cell outputs.

Who Should Attend

This workshop is designed for data scientists and analysts who are new to version control or want to deepen their skills with Git and GitHub. No prior experience with Git is necessary, but a basic familiarity with the command line will be helpful.