Coverage error on travis

[kngwyu]

E.g., see https://travis-ci.org/PyO3/pyo3/jobs/521167916

What's happening?

cc: @althonos

[konstin]

While I don't know the cause, this doesn't seem to happen on 3.5; Would it be an option to only collect coverage from 3.5, as afaik there's no real difference in coverage between the version?

[pganssle]

@konstin The datetime stuff around fold is Python 3.6 -only. I will soon be adding some Python 3.7+ features as well, so I think it would be best to find the fix.

Can anyone reproduce it locally? If not, maybe we can do a rough "git bisect" by making a PR that reverts a bunch of commits and play with the number of reverted commits until we find the one that breaks it (assuming it's not broken because of something on Travis or codecov's end).

[konstin]

According to the travis history it's broken since #432, but I doubt there's any problem with that pull request. Also travis fails to show any proper error, which makes this weird.

[pganssle]

I think maybe you have to e-mail Travis to request it* (I think that was the procedure when I did it, very early on in this feature's life), but it's possible to run Travis in debug mode if you can't reproduce locally: https://docs.travis-ci.com/user/running-build-in-debug-mode/

*By "request it" I mean "request access to the feature", not request every debug build.

[konstin]

Having spent much time debugging ci services and especially travis ci in projects has drained my motivation to put more than the minimal effort into fixing this (which would be limiting coverage to 3.5). If someone else however is willing to work on this, I'd be happy to help.

FWIW you can also pull their docker container and copy&paste the commands for reproducing, which is kinda convenient. I haven't tried debug mode yet though.

[pganssle]

If this is a Travis-specific issue, it might make sense to migrate to Azure Pipelines. Microsoft has been making a big push for them in the open source world lately, so the speed and quality of the builds is pretty high.

[konstin]

Azure pipelines look really promising and I'd be happy to try it out. However I don't think the coverage error is travis ci's fault and also azure pipelines likely still has some rough edges.

[Alexander-N]

The error seems to be gone, maybe we can close this issue?

[kngwyu]

Coverage is broken again in #503(maybe because of trybuild?).
So.. how about stopping the use of coverage?
Honestly I don't think it is very important and meaningful for us.

[Alexander-N]

Since the coverage seems to be a constant hassle, I wouldn't be opposed to removing it again.

[althonos]

Or other option, make the coverage script fail silently, so that it does not make the whole build a failure: this would allow the coverage to be reported still most of the time.

[konstin]

The build failure in #503 is a bug in trybuild: dtolnay/trybuild#14

I agree with @althonos that we should just make that script an allowed failure. Coverage can be useful sometimes, even though we might want to remove the badge from the readme.

[konstin]

Coverage has been allowed to fail for some time