DSAN 5500: Data Structures, Objects, and Algorithms in Python
Thursday, March 13, 2025
Today’s Planned Schedule:
Start | End | Topic | |
---|---|---|---|
Lecture | 6:30pm | 6:50pm | Key Concepts → |
6:50pm | 7:00pm | Execution Graphs → | |
7:00pm | 7:15pm | Deployments (Preview) → | |
7:15pm | 8:00pm | Lab Part 1 → | |
Break! | 8:00pm | 8:10pm | |
8:10pm | 9:00pm | Lab Part 2 → |
(Underlined terms link to relevant section of docs)
Flow
: The “main thing” your pipeline is doing!
Tasks
Flow
s and Task
s alone already provide much more functionality than “basic” functions…Deployment
s: Flow
s + Task
s + Metadata about how and when you want them to run.
Deployment
s (More Next Week)Deployments
enables Triggers
:
Logging
, Notifications
(Slack, email, text messages)Results
as natural-language explanations (produced by Prefect) or custom summaries, called Artifacts
, that you define as part of your flowsSchedules
crontab.sh
Events
Webhooks
Logging
log_prints=True
as a parameters of your Flow
:Notifications
Jinja
which is VERY worth learning!Jinja
ExampleDSAN 5500 Week 9: ETL Pipelines