Skip to content
This repository has been archived by the owner on Jan 8, 2021. It is now read-only.

Commit

Permalink
Add rudimentary COVID Dashboard parser
Browse files Browse the repository at this point in the history
Since the dashboard might change layout in the future, this does nearly no
error checking/data formatting.
  • Loading branch information
benjaminrsherman committed Sep 11, 2020
1 parent e3181b0 commit 4309777
Showing 1 changed file with 21 additions and 0 deletions.
21 changes: 21 additions & 0 deletions covid_scraper/main.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import os
import requests
from bs4 import BeautifulSoup
import json

res = requests.get("https://covid19.rpi.edu/dashboard")
soup = BeautifulSoup(res.text.encode("utf8"), "lxml")

data = {}

dashboard = soup.find("div", "dashboard-stats").find("div", "field__items")

for div in dashboard.find_all("div", "field__item", recursive=False):
field_name = div.find("div", "field--name-field-stat-description").text
field_stat = div.find("div", "field--name-field-stat").text
data[field_name] = field_stat

data["caption"] = soup.find("div", "field--name-field-stats-caption").text

with open(f"covid.json", "w") as outfile:
json.dump(data, outfile, sort_keys=False, indent=4)

0 comments on commit 4309777

Please sign in to comment.