-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make salary 2021 script year agnostic #135
Comments
@AetherUnbound This line seems to be the only year-specific line in the script from what I can tell: OpenOversight/data-munging/spd_2021_salary_data.py Lines 58 to 59 in 6f913bb
Would you mind expanding on what you mean by "save the raw salary data for future ingestions/tests"? |
Ah, I should have linked the dataset - I meant the City Of Seattle Wage Data! https://data.seattle.gov/City-Business/City-of-Seattle-Wage-Data/2khk-5ukd |
Oh right, I meant where we're planning on saving it to - in the database or in S3? |
Oh that's a good point! Probably S3, we can have a separate bucket for it (and use minio locally for that). |
Fixed by #324 |
The salary 2021 script uses public salary information that's updated regularly. Therefore, it's not actually just 2021 data but data from whenever the script itself is run. We should make the script more agnostic to which year it's being run in, and also save the raw salary data for future ingestions/tests (since that raw data is likely to change).
The script in question is here: https://github.com/OrcaCollective/OpenOversight/blob/main/data-munging/spd_2021_salary_data.py
The text was updated successfully, but these errors were encountered: