Update GCS hook to get crc32c hash for CMEK-protected objects

[dmedora]

Closes: #34980

Updated _prepare_sync_plan function to get blobs using the Objects Get API (.get_blob(..)) so that their crc32c value is included in the returned Blob object. As discussed in #34980, the current Objects List method does not return this for CMEK-protected objects, leading to inaccurate hash comparisons.

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: [email protected]
  Slack: https://s.apache.org/airflow-slack

[heaven00]

Shouldn't this need a test against whatever that blob should be?

[dmedora]

Shouldn't this need a test against whatever that blob should be?

Not sure I understand, could you please explain further or with an example? Thanks.

[heaven00]

sure,

so the function that has been changed is here is a private function, so we can't test the logic that has been added but from what I can understand there should be some behavioral changes to the sync function https://github.com/apache/airflow/pull/34982/files#diff-82854006b5553665046db26d43a9dfa90bec78d4ba93e2d2ca7ff5bf632fa624R1179

in terms of the data it is parsing from the blob and in my opinion there should be respective tests added to https://github.com/RNHTTR/airflow/blob/78d1031b34d70784d22fc1360c6f3deeb2c66a2c/airflow/providers/google/cloud/hooks/gcs.py so that the behavior is captured.

[Taragolis]

Need to fix static checks and failed test

[dmedora]

Thanks, still figuring out how to write the test correctly.

[Taragolis]

Are you worry about that you can't test internal static method _prepare_sync_plan?
I don't see any problem here because it is internal part of Google Provider so you could also test it as part of Google Provider

[shahar1]

The test TestSyncGcsHook.test_should_overwrite_cmek_files should be fixed.
Besides - LGTM.

[eladkal]

@dmedora can you please fix the test?

[github-actions]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.