Resource Hub
2025 Recordings
Lecture recordings from the Fall 2025 version of the course can be watched here
Key Books By Week
Week 1: Course Overview / Week 2: Cloud Computing
- The Boar Book: Kleppmann and Riccomini (2026), Designing Data-Intensive Applications (2nd Edition)
Week 3: Parallel Concepts
- The Wolohan MapReduce Book: Wolohan (2020), Mastering Large Datasets with Python, Chapters 1-6
Week 4: SQL, DuckDB, Polars
Week 5: Data Warehousing
Two Packt Books:
Weeks 6 and 7: Hadoop → PySpark
- The Wolohan MapReduce Book: Wolohan (2020), Mastering Large Datasets with Python, Chapters 7-10
Weeks 8 and 9: Spark in Depth
- The Electric Eel Book: Damji et al. (2020), Learning Spark
In-Class Demos
- Week 4 Polars Demo (Colab)
- Week 4 DuckDB Demo (Colab)
- Week 6 Hadoop Demo (Deepnote): Map-Reduced Matrix-Vector Multiplication
- Week 6 Hadoop → PySpark Demo (Deepnote): Tennis Analytics with Hadoop and PySpark RDDs
- Week 7 PySpark DataFrames Demo (Colab)
- (Adapted from Jacob Celestine’s PySpark on Colab demo)
- Week 8 SparkNLP Demo
- Week 9 Spark ML Demo (Colab)
- Week 9 MLFlow Demo (GitHub)
PDFs of Books
The remainder of this page is auto-generated from all of the references across the slides for each week: click on the name of a reference to download the ebook version, if available!
Damji, Jules S., Brooke Wenig, Tathagata Das, and Denny Lee. 2020. Learning Spark. O’Reilly Media, Inc.
Eagar, Gareth. 2021. Data Engineering with AWS: Learn How to Design and Build Cloud-Based Data Transformation Pipelines Using AWS. 1st ed. Birmingham: Packt Publishing Limited.
Firth, John Rupert. 1957. Papers in Linguistics, 1934-1951. Oxford University Press.
Gopalan, Rukmani. 2022. The Cloud Data Lake: A Guide to Building Robust Cloud Data Architecture. O’Reilly Media, Inc.
Hennessy, John L., and David A. Patterson. 2017. Computer Architecture: A Quantitative Approach. Morgan Kaufmann.
Janssens, Jeroen, and Thijs Nieuwdorp. 2025. Python Polars: The Definitive Guide: Transforming, Analyzing, and Visualizing Data with a Fast and Expressive DataFrame API. O’Reilly Media, Inc.
Kleppmann, Martin, and Chris Riccomini. 2026. Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems. 2nd edition. Santa Rosa, CA: O’Reilly.
Leskovec, Jure, Anand Rajaraman, and Jeffrey David Ullman. 2014. Mining of Massive Datasets. Cambridge University Press.
Loukides, Mike. 2010. “What Is Data Science?” O’Reilly Media. June 2, 2010.
Mell, Peter, and Timothy Grance. 2011. “The NIST Definition of Cloud Computing.” National Institute of Standards and Technology, Special Publication 800 (2011): 145.
Needham, Mark, Michael Hunger, and Michael Simons. 2024. DuckDB in Action. Simon and Schuster.
Noble, David F. 1995. Progress Without People: New Technology, Unemployment, and the Message of Resistance. Toronto: Between the Lines.
Polanyi, Karl. 1944. The Great Transformation: The Political and Economic Origins of Our Time. Beacon Press.
Raasveldt, Mark, and Hannes Mühleisen. 2019. “DuckDB: An Embeddable Analytical Database.” In Proceedings of the 2019 International Conference on Management of Data, 1981–84. SIGMOD ’19. New York, NY, USA: Association for Computing Machinery.
Raff, Edward, Drew Farris, and Stella Biderman. 2025. How Large Language Models Work.
Reis, Joe, and Matt Housley. 2022. Fundamentals of Data Engineering: Plan and Build Robust Data Systems. O’Reilly Media, Inc.
Saussure, Ferdinand de. 1916. Course in General Linguistics. Open Court.
Tanimura, Cathy. 2021. SQL for Data Analysis: Advanced Techniques for Transforming Data into Insights. O’Reilly Media, Inc.
Topol, Matthew, and Wes McKinney. 2024. In-Memory Analytics with Apache Arrow. Packt Publishing Ltd.
Virtuoso, Anthony, Mert Turkay Hocanin, Aaron Wishnick, and Rahul Pathak. 2021. Serverless Analytics with Amazon Athena: Query Structured, Unstructured, or Semi-Structured Data in Seconds Without Setting up Any Infrastructure. Packt Publishing Ltd.
Wojtowicz, Zachary, and Simon DeDeo. 2025. “Undermining Mental Proof: How AI Can Make Cooperation Harder by Making Thinking Easier.”
Wolohan, John. 2020. Mastering Large Datasets with Python: Parallelize and Distribute Your Python Code. Simon and Schuster.
Zilliz. 2025. “Introduction to Unstructured Data.” 2025.