I0617 rescu sensors eval

[gabrielwol]

What this pull request accomplishes:

• SQL to identify network wide outages and individual sensor VDS outages.

• Identify RESCU detectors to repair this year during annual repair blitz. Committing this outdated work to vds schema is confusing and if committed to rescu it would never be seen again so I chose to delete it. We have a new dashboard being developed by Chris which identifies sensors to repair and supersedes this piece of the PR. I kept and documented the two views I developed which may be helpful for others.

Issue(s) this solves:

• Identify RESCU VDS locations in need of repair #617

What, in particular, needs to reviewed:

What needs to be done by a sysadmin after this PR is merged

Nothing

[radumas]

can you rename the Data Validation folder to validation? The fluffer doesn't like the space, and I am allergic to capitalization.

[radumas]

Nice work.

Mostly formatting changes on volumes/rescu/rescu_sensors_eval_readme.md and the recommendation to split up the content.

I haven't super wrapped my head around how this PR interacts with Sarah's we can try to merge that one first? And then I might have further specific suggestions about where things go in documentation land.

[radumas]

I think this could this be a mat view in the rescu schema, change the end dates to be now() plus some manipulation to include the last pulled date.

[radumas]

orrrrrrrrrr... it could be a function that returns a SET/Table, with inputs of start and end date. I don't think this is necessary to implement rn, but a thought.

[radumas]

So we're going to revisit this and find the work a new home within the new volumes/vds folder after #646 gets merged

[scann0n]

Thanks for this gwolofs!

I found one smol typo in the readme (see inline comments) and there are some outstanding fluffs.

Organizationally speaking:

• on bigdata - these detector mat views are too good to be hidden in gwolofs - have they been moved somewhere more public? I snooped on bigdata.vds but I did not see them there - are they somewhere else? Have they been superceded?
• on bdit_data-sources - this work is saved to the rescu folder which is appropriate given the focus. Is there any desire to migrate these to the vds folder and purge the rescu folder?
  • If there is, I would just create a "rescu_checks" subfolder in vds and migrate your folder structure there (pending any @radumas objections).
  • I also think it would be fine to leave the rescu folder as-is

[radumas]

Realize this wasn't introduced in this PR but the below bullets didn't render properly

bdit_data-sources/volumes/vds/readme.md

Lines 146 to 148 in 4045015

	- Data quality checks have not been implemented in this new schema. For examples of past work see:
	- @scann0n did some work on identifying good date ranges for RESCU sensors based on volumes from days with all 96 15-minute bins present which is written up [here](https://github.com/CityofToronto/bdit_data-sources/blob/master/volumes/rescu/README.md#6--how-often-are-data-quality-assessment-processes-for-the-data-undertaken).
	- @gabrielwol did work to identify periods of network wide or individual sensor outages on the RESCU network which is written up [here](https://github.com/CityofToronto/bdit_data-sources/blob/3ba3af5068e96191caffab524d42ae52fe7be7b2/volumes/rescu/README.md#1--are-there-known-data-gapsincomplete-data)

[radumas]

SQL looks good. Main issue is markdown tables don't render properly.

Given the deletion of the broken sensor identification work because it is superseded, I identified some links that are likely going to rot when this PR is merged and the branch is deleted

[radumas]

:meow_mage:
:party-porg:

stale review. Gabe has made substantial changes that make the requested changes irrelephant