Codecov bot is posting very old updates to our repo

[jhamman]

Describe the bug
Our GitHub project (zarr-developers/zarr-python) has recently started receiving codecov bot updates literally years after PRs were closed.

Environment (please complete the following information):

• Browser [e.g. chrome, safari] (if applicable)
• System: GitHub

To Reproduce
Go to one of these PRs and observe the very late bot post:

• Release notes for 2.12.0a1 zarr-developers/zarr-python#1023
• drop py37 zarr-developers/zarr-python#1067
• Move master to main zarr-developers/zarr-python#1033
• Bump azure-storage-blob from 12.11.0 to 12.12.0 zarr-developers/zarr-python#1026
• (and many more)

Expected behavior
It seems the bot failed to make a timely update to these PRs. Expected behavior would be that 2 year old PRs don't get bot updates.

Screenshots

Additional context

Our tracking issue: zarr-developers/zarr-python#2138

[eliatcodecov]

@drazisil-codecov to investigate, we'll figure out next steps once we have more clarity on the underlying issue.

[drazisil-codecov]

@jhamman It looks like GitHub says the PR was updated. https://api.github.com/repos/zarr-developers/zarr-python/pulls/1023

Codecov responded to a status web hook from GitHub. Any idea what might have changed?

[jhamman]

Any idea what might have changed?

No. The last change was over 2 years ago. Are you sure the "updated_at" field is not coming from Codecov's comment?

[drazisil-codecov]

Any idea what might have changed?

No. The last change was over 2 years ago. Are you sure the "updated_at" field is not coming from Codecov's comment?

Hmm, that's possible. 🤦

I did see a hook in the log though... 👀

[drazisil-codecov]

Hook Delivery id: e0b53a3a-6517-11ef-94eb-86bf97e52810 was received by Codecov at 2024-08-28T08:31:10.979108533Z

We don't log the exact payloads for sanity reasons, but that is what trigged us to send the comment. Hopefully GitHub support may be able to tell you what triggered that hook? Unfortunately I have no way to know :-/

[webknjaz]

Hook Delivery id: e0b53a3a-6517-11ef-94eb-86bf97e52810 was received by Codecov at 2024-08-28T08:31:10.979108533Z

You should be able to check that comment's timestamp and see if it's within milliseconds or earlier than the webhook delivery. If it's after the webhook delivery, then it's not the trigger. But the timestamp seems to correlate, so that comment is likely not the event trigger but a reaction to some other trigger.

[drazisil-codecov]

Hook Delivery id: e0b53a3a-6517-11ef-94eb-86bf97e52810 was received by Codecov at 2024-08-28T08:31:10.979108533Z

We don't log the exact payloads for sanity reasons, but that is what trigged us to send the comment. Hopefully GitHub support may be able to tell you what triggered that hook? Unfortunately I have no way to know :-/

I forgot to tag @jhamman for this response.

[jhamman]

Thanks for looking into this. I don't have time to chase this further right now. If you hypothesis is that something else sent the webhook, I'm happy if you close this.