-
-
Notifications
You must be signed in to change notification settings - Fork 187
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
coveralls --finish not working as expected in Circle-CI #235
Comments
My current work around is to just directly call this, rather than using
|
Hmm, it seems that using CIRCLE_BUILD_NUM is not right :( |
Shoot, happy to update this if we can figure out what the correct value is. By any chance have you figured out what value we need to be using, if According to @caleb15's PR here it might be |
Sure, here's my WIP attempting to use it facebookresearch/ParlAI#2950 Indeed, CIRCLE_WORKFLOW_ID mostly works well. I think the one problem I have is that our build system is rather large: something like 92 builds for every commit. (We heavily parallelize). Coveralls seemed to be choking on it (only realizing 62 of them: https://coveralls.io/builds/32619558) and vastly underestimating our coverage: codecov.io puts us at ~38% right now. This coveralls build think we're at ~11%. Browsing the file tree, large parts that I know are covered aren't currently being caught. |
Not sure if I'm just missing it in your CircleCI config, but it seems you might need to set
Does everything work out when you use both |
Coveralls has been having peformance issues recently so that might be related: https://status.coveralls.io/ |
I feel somewhat confident at this point that using CIRCLE_WORKFLOW_ID is the right thing, and that there is another bug further down the line I haven't caught yet. |
Alrighty -- I'll go merge and deploy #233 then, which should let you nuke the |
Alright, I'm coming back to this now, thanks for helping before. I feel like I'm very close now. You can see a report here: https://coveralls.io/builds/33679104 After #233, one discussion above is fixed, but I still had to make another change for our workflow (which consists of many different jobs, each run in parallel; see here). Before calling
This avoids the different parallel workers stomping on each other, I believe. When I do my final call to the webhook, it seems to report the correct number
I see all the jobs on the corresponding coveralls page. The only frustrating thing is the final build coverage says 21.99, when it should be closer to 42-45. When I look at the tree at the bottom, that all looks correct (the two main folders have 39.9 and 44.0 coverage), just the final total looks wrong. Comparing the tree with what codecov outputs, the per-folder totals all look right. |
I'm pretty lost on this one -- as far as I can tell, it seems like coveralls.io is getting all the right data but reporting the aggregate stats incorrectly? Unfortunately, I don't think there's anything I'll be able to help with, then; data presentation is strictly out of our hands haha! The only thing which comes to mind is the settings for what sort of coverage you're reporting (eg. line coverage vs branch coverage, that sort of thing). Maybe playing with those settings might help? Closing this issue -- sorry we couldn't help more! |
Thank you! |
Thanks for this nice client.
It looks like since the github api was introduced, the Circle-CI version seems broken. When I try putting having a run that finishes up a parallel go, the wrong build number is being used to notify the service
Here you see that the CIRCLE_BUILD_NUM value has been put into "service_job_id" field of the config. In
However, the call to finish uses the "service_number" field instead:
https://github.com/coveralls-clients/coveralls-python/blob/30e4815169b3db2616981939d55d2f4495816821/coveralls/api.py#L217-L218
As such, when you try calling
coveralls --finish
you get the following error:Note that Circle always puts None into the service_number field:
https://github.com/coveralls-clients/coveralls-python/blob/30e4815169b3db2616981939d55d2f4495816821/coveralls/api.py#L87
https://github.com/coveralls-clients/coveralls-python/blob/30e4815169b3db2616981939d55d2f4495816821/coveralls/api.py#L52-L59
The text was updated successfully, but these errors were encountered: