.jpg){kind=link}
Week 3: Data Science Workflow
DSAN 5000: Data Science and Analytics
Thursday, September 12, 2024
How the Internet Works
Intranet vs. Internet
- Crucial distinction: can set up a “mini-internet”, an intranet, within your own home
- Organizations (businesses, government agencies) with security needs often do exactly this: link a set of computers and servers together, no outside access
- Internet = basically a giant intranet, open to the whole world
HTTP(S) Requests in Action
Image from Menczer, Fortunato, and Davis (2020, 90)
How Does a Web Server Work?
- We use the term “server” metonymously1
- Sometimes we mean the hardware, the box of processors and hard drives
- But, sometimes we mean the software that runs on the hardware
- A web server, in the software sense, is a program that is always running, 24/7
- Waits for requests (via HTTPS), then serves HTML code in response (also via HTTPS)
hello_server.py
GET requests
How Does a Web Client Work?
- Once the server has responded to your request, you still only have raw HTML code
- So, the browser is the program that renders this raw HTML code as a visual, (possibly) interactive webpage
- As a data scientist, the most important thing to know is that different browsers can render the same HTML differently!
- A headache when pages are accessed through laptops
- A nightmare when pages are accessed through laptops and mobile
Connecting to Servers
- We’ve talked about the shell on your local computer, as well as the Georgetown Domains shell
- We used Georgetown Domains’ web interface to access that shell, but you can remotely connect to any other shell from your local computer using the
sshcommand!
Transferring Files to/from Servers
- Recall the copy command,
cp, for files on your local computer - There is a remote equivalent,
scp(Secure Copy Protocol), which you can use to copy files to/from remote servers to your local computer
Quarto and Reproducible Research
Why Do We Need Reproducible Research?
- Main human motivations (Max Weber): Wealth, Prestige, Power → “TED talk circuit”
Science vs. Human Fallibility
- Scientific method + replicability/pre-registration = “Tying ourselves to the mast”
John William Waterhouse, Ulysses and the Sirens, Public domain, via Wikimedia Commons
- If we aim to disprove (!) our hypotheses, and we pre-register our methodology, we are bound to discovering truth, even when it is disadvantageous to our lives…
R vs. RStudio vs. Quarto
- GUI wrapper around R (Integrated Development Environment = IDE)
- Run blocks of R code (
.qmdchunks)
- Programming language
- Runs scripts via
Rscript <script>.r
+
- GUI wrapper around Python (IDE)
- Run blocks of Python code (
.ipynbcells)
- Scripting language
- On its own, just runs scripts via
python <script>.py
Git Diagram
The Commit Log
- View the commit log using
git log:
user@hostname:~/cool-project$ git log
commit b40dc252a3b7355cc4c28397fefe7911ff3c94b9 (HEAD -> main)
Author: Jeff Jacobs <jjacobs3@cs.stanford.edu>
Date: Sun May 28 00:57:16 2023 +0000
Initial version of cool_code.pygitGraph commit id: "b40dc25"
The git log will show the new version:
user@hostname:~/cool-project$ git log
commit e3bc497acbb5a487566ff2014dcd7b83d0c75224 (HEAD -> main)
Author: Jeff Jacobs <jjacobs3@cs.stanford.edu>
Date: Sun May 28 00:38:05 2023 +0000
Added code to cool_code.py
commit b40dc25b14c0426b06c8d182184e147853f3c12e
Author: Jeff Jacobs <jjacobs3@cs.stanford.edu>
Date: Sun May 28 00:37:02 2023 +0000
Initial version of cool_code.py
Undoing a Commit I
First check the git log to find the hash for the commit you want to revert back to:
commit e3bc497acbb5a487566ff2014dcd7b83d0c75224
Author: Jeff Jacobs <jjacobs3@cs.stanford.edu>
Date: Sun May 28 00:38:05 2023 +0000
Added code to cool_code.pyOnwards and Upwards
user@hostname:~/cool-project$ echo "3 + 3" >> cool_code.py
user@hostname:~/cool-project$ git add cool_code.py
user@hostname:~/cool-project$ git status
On branch main
Changes to be committed:
(use "git restore --staged <file>..." to unstage)
modified: cool_code.py
user@hostname:~/cool-project$ git commit -m "Added different code to cool_code.py"
[main 700d955] Added different code to cool_code.py
1 file changed, 1 insertion(+)But Why These Diagrams?
Even the simplest projects can start to look like:
gitGraph
commit id: "537dd67"
commit id: "6639143"
branch nice_feature
checkout nice_feature
commit id: "937ded8"
checkout main
commit id: "9e6679c"
checkout nice_feature
branch very_nice_feature
checkout very_nice_feature
commit id: "7f4de03"
checkout main
commit id: "6df80c1"
checkout nice_feature
commit id: "bd0ebb8"
checkout main
merge nice_feature id: "9ff61cc" tag: "V 1.0.0" type: HIGHLIGHT
checkout very_nice_feature
commit id: "370613b"
checkout main
commit id: "9a07a97"
The GitHub Side: Remote
An Empty Repo
Refresh after git push
Commit History
Checking the diff
Getting Content onto the Internet
-
Step 1:
index.html - Step 2: Create GitHub repository
-
Step 3:
git init,git add -A .,git push - Step 4: Enable GitHub Pages in repo settings
-
Step 5:
<username>.github.io!
Deploying from a Branch/Folder
