Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🚀 [Feature] Separate Data from Tool #36

Open
ulfgebhardt opened this issue Apr 2, 2021 · 3 comments
Open

🚀 [Feature] Separate Data from Tool #36

ulfgebhardt opened this issue Apr 2, 2021 · 3 comments

Comments

@ulfgebhardt
Copy link
Member

🚀 Feature

It is common practice that scraper and data is stored separately, but here this is not the case - or at least partly.

We have a data folder containing jsons: https://github.com/bundestag/gesetze-tools/tree/master/data

But there is a repo associated with this scraper as well: https://github.com/bundestag/gesetze

It is still unclear to me how the tool produces the output stored in the gesetze repo.

Nevertheless I consider it useful to have all data separated from the tools creating them. I think it would be wise to create a new repo for the scraped data (please in English)

Design & Layout

Data in a data-repo should be stored in a data folder

image

@darkdragon-001
Copy link
Collaborator

There should be a README.md always IMHO.

I suggest separate repositories for separate data sets (bgbl, banz, ...).

@ulfgebhardt
Copy link
Member Author

The repos should have proper naming - "banz" has no meaning at all. Event tho I say have english names "Bundesanzeiger" as Entity-name is acceptable I guess and the reader understands what the repo is about

@mk-pmb
Copy link
Contributor

mk-pmb commented Nov 14, 2023

We may not even need a separate repo. Using a separate branches would probably already cover most of the way.
Then we should have a cleaned-up version of the tools branch that omits all the data commits, so that the tools themselves are quick to clone.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants