Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Fideslang Typing #3839

Merged
merged 31 commits into from
Sep 22, 2023
Merged

Fix Fideslang Typing #3839

merged 31 commits into from
Sep 22, 2023

Conversation

ThomasLaPiana
Copy link
Contributor

@ThomasLaPiana ThomasLaPiana commented Jul 20, 2023

Closes #N/A

Description Of Changes

I inadvertently fixed typing over in fideslang, which means we now have a bunch of things to fix here!

This uncovered a lot of legitimate issues in our type annotations.

Code Changes

  • get mypy passing
  • please see code comments for more specifics

Steps to Confirm

  • list any manual steps for reviewers to confirm the changes

Pre-Merge Checklist

  • All CI Pipelines Succeeded
  • Issue Requirements are Met
  • Relevant Follow-Up Issues Created

@ThomasLaPiana ThomasLaPiana self-assigned this Jul 20, 2023
@cypress
Copy link

cypress bot commented Jul 20, 2023

Passing run #4267 ↗︎

0 4 0 0 Flakiness 0
⚠️ You've recorded test results over your free plan limit.
Upgrade your plan to view test results.

Details:

Merge b1e20ae into 143d9a6...
Project: fides Commit: fad0cc32f4 ℹ️
Status: Passed Duration: 01:19 💡
Started: Sep 22, 2023 1:53 AM Ended: Sep 22, 2023 1:54 AM

This comment has been generated by cypress-bot as a result of this project's GitHub integration settings.

@ThomasLaPiana ThomasLaPiana added the run unsafe ci checks Runs fides-related CI checks that require sensitive credentials label Jul 31, 2023
src/fides/api/db/system.py Outdated Show resolved Hide resolved
@ThomasLaPiana
Copy link
Contributor Author

update here: looks like everything is good except for the unsafe checks. I suspect at least the ctl ones are actually related here but these are much more difficult to troubleshoot. Will start poking at them locally

@codecov
Copy link

codecov bot commented Aug 9, 2023

Codecov Report

Patch coverage: 77.06% and project coverage change: +22.54% 🎉

Comparison is base (143d9a6) 64.96% compared to head (b1e20ae) 87.51%.

Additional details and impacted files
@@             Coverage Diff             @@
##             main    #3839       +/-   ##
===========================================
+ Coverage   64.96%   87.51%   +22.54%     
===========================================
  Files         326      326               
  Lines       20250    20283       +33     
  Branches     2622     2634       +12     
===========================================
+ Hits        13155    17750     +4595     
+ Misses       6629     2076     -4553     
+ Partials      466      457        -9     
Files Changed Coverage Δ
.../service/privacy_request/request_runner_service.py 77.17% <ø> (+19.92%) ⬆️
src/fides/cli/commands/generate.py 84.33% <0.00%> (ø)
src/fides/core/pull.py 100.00% <ø> (ø)
src/fides/core/system.py 31.65% <10.00%> (-1.43%) ⬇️
src/fides/core/annotate_dataset.py 22.97% <36.36%> (+0.75%) ⬆️
src/fides/api/models/datasetconfig.py 96.80% <50.00%> (+24.00%) ⬆️
src/fides/api/util/endpoint_utils.py 79.36% <71.42%> (+20.34%) ⬆️
src/fides/api/schemas/dataset.py 94.64% <80.00%> (ø)
src/fides/core/dataset.py 71.01% <80.00%> (+0.42%) ⬆️
src/fides/core/audit.py 81.31% <86.66%> (-0.17%) ⬇️
... and 19 more

... and 144 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ThomasLaPiana ThomasLaPiana marked this pull request as ready for review September 17, 2023 06:05
@ThomasLaPiana
Copy link
Contributor Author

ThomasLaPiana commented Sep 17, 2023

@galvana can you give me confirmation that the failing tests here are expected? I looked at main and they look like the same failures but I want to be sure!

Note: The failing ctl external test is a known issue

Copy link
Contributor

@pattisdr pattisdr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great @ThomasLaPiana thanks for the work to allow us to return on automated mypy checks again! so useful

@@ -43,7 +43,6 @@ module = [
"dask.*",
"deepdiff.*",
"defusedxml.ElementTree.*",
"fideslang.*",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this do?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this section tells mypy to ignore missing type imports from imported libraries. Everything from fideslang is typed so this is redundant

at least that's my understanding

tests/lib/test_system_oauth_util.py Outdated Show resolved Hide resolved

FieldDataCategoryValidation.update_forward_refs()

class CollectionDataCategoryValidation(
DatasetCollection, DataCategoryValidationMixin
):
fields: List[FieldDataCategoryValidation] = []
fields: Sequence[FieldDataCategoryValidation] = [] # type: ignore[assignment]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this coming from?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is required due to Lists in Python being invariant, as a result of their mutability

src/fides/api/oauth/system_manager_oauth_util.py Outdated Show resolved Hide resolved
Copy link
Contributor

@adamsachs adamsachs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you for not losing sight of this and staying diligent with maintenance! nothing stands out to me as particularly problematic. i do think @pattisdr had some good questions that may be nice to get answers to before we merge.

🎊

EDIT -- just wanted to note a reminder that we need to follow up with fidesplus - both to make any needed changes based on dependencies on what we've changed here (not sure if there's anything applicable), and just generally getting mypy fixed there! i'll see if i can help push that forward once this is merged 👍

.github/workflows/backend_checks.yml Show resolved Hide resolved
src/fides/api/util/endpoint_utils.py Outdated Show resolved Hide resolved
@ThomasLaPiana ThomasLaPiana merged commit 37c32f8 into main Sep 22, 2023
41 of 45 checks passed
@ThomasLaPiana ThomasLaPiana deleted the ThomasLaPiana-fix-fideslang-types branch September 22, 2023 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
run unsafe ci checks Runs fides-related CI checks that require sensitive credentials
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants