-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Set http caching for tests #207
base: master
Are you sure you want to change the base?
Conversation
tgrandje
commented
Aug 8, 2024
- install requests-cache on workflows
- set http caching when requests-cache present (might be absent (because of the workflow or the local environment)
- remove testing for python 3.9 to 3.11
* install requests-cache on workflows * set http caching when requests-cache present (might be absent (because of the workflow or the local environment)
The current test results should not be considered at all. It will need to be merged first to be tested against the updated workflow (with requests-cache installed). In time, we can add a |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks for that!
it looks great, could you please modify tests py files accordingly? some tests run only on some specific py versions. thanks |
Oh, so that's why the 3.8 tests were almost always concluant :-). Sorry about that, I've missed it since my machine is currently running a 3.10 python... I'll remove all python version checks, that should be fine with the refactored python versions control in the workflow. |
* Remove version control from unittests * Remove (one or two) unnecessary module imports * Black formatting
Done |
@tgrandje the tests timed out, so something might not be working as expected with caching... or it's just that doing all the tests is too slow, maybe that's why the version test was there? |
I'd say the second one. |
Well, tests on y machine seems to be unconclusive: might be something wrong with my proxy (and/or me switching back and forth on different branches and juggling with the tempdirs). Tests are not all finished, and I already have failures on some tests which were correctly ran on github. If I got the time, I'll try to re-run some of those this evening on a personnal laptop outside my office's network (ie without proxies). But I may be short in time... If you don't hear from me before tomorrow, fell free to do what you think is best! |
Ok, so I ran the tests outside of a corporate network and python 3.9. I got 3 failing tests (which were silenced previously as they only ran on 3.8) :
I've seen something about the arrow/gpd issue on the last one, but I'm not certain this triggered the failure. The 2 other failures seems to run ok outside of pytest, so maybe there is something wrong with the patching indeed. |
Forget that. I ran it again (using cache) and that 3 failed test now passed. Maybe it has to do with a failure of the source, but the tests are ok. |
* Add optional test flag --clean-cache to remove existing cache previous to tests * Add optional test flag --no-cache to deactivate cache during tests * Set 30 days timeout expiration on cache during tests
Note Personal note: tests computation times, without cache (results for tests > 30sec) on personal machine:
All tests passed in 2077.28s (0:34:37) with python 3.9. |
…com/InseeFrLab/pynsee into enhancement/tests_requests_patching
Ok, I've gone farther than I thought I'd do. So far:
I still got a problem with 3 tests ( |
It seems there is definitely something buggy with I'll transfer the 2 patches on proxies/exceptions into separate PR in case I can't resolve this bug... |
There is something very strange going on on this bug. As far as I understand, But, when I track the http response, I see that for those failures, we get perfectly valid XML results from INSEE's API instead of json. This is kind of funny as the server is not meant to serve XML results (according to the API's documentation anyway...). For instance, for
I think I've tracked every argument (proxies, stream, headers, ....) sent to the webserver and I can't see a difference between valid and unvalid results. I'm starting to think that might be a webserver failure, but I'm not sure either (why mostly those 3 tests, why only when patching, why would the temporisation have no impact...). If that's really the case, I see 2 ways out of here:
@hadrilec @tfardet do you see anything I'm missing? (If anybody is ready to test this locally, please send me an email and I'll send you the S3 credentials.) |
This reverts commit 5552528.
Ok, colleagues from INSEE put me on a lead and I finally got this after much debugging. It appears Regarding the (apparent) random results, that's entirely the results of me selecting different tests and/or the result of improper caching through that history... For memory's sake:
I still got to check different backend on different machines to determine what's fastest a priori, but I hope we can soon test this directly on github. |
Best backend seems to be SQLite (tested only between SQLite & filesystem, that took long enough 🙄 ). Other features in this PR I've forgot to describe in conversation include the following:
I'm still not so sure we should conserve the compression of the artifacts: that will have to be tested in real conditions on github. The fact is that with compression, the restauration is fast (~3min), but the storage is real slow (like 19min slow on the same machine). Further modifications to consider (not yet done, but for memory's sake):
In any case, for now I'd say someone can check that PR. If it's ok, we can move to real conditions tests and iterate from there. |
Hello, I tried to run the tests on onyxia, I got the following error, an idea on how to proceed?
|
Apparently the file is corrupt (I downloaded and checked it with other softwares). Did you ran the tests multiple times? On any machine (running the tests manually of course), you are also able to remove the previous cache by adding the flag That being said we should add a check on the 7z file's state before uploading it to sspcloud... 🤔 |
* handle bad 7z file (dumping the cache in case of corruption) * check 7z's integrity before sending it to S3
@hadrilec I've just added a quick fix to handle corrupted 7z files (I've also reverted some changes which were due to go to another PR). If that's ok with you, I'll re-run the tests on my laptop ; you should then be able to test (one more time) directly on onyxia. |
OK, that's done. |
Should we merge this to see whether that fixes the tests and iterate from there if it doesn't? EDIT: nevermind, I think there might actually be an issue with the tempfile commit EDIT 2 : I'm a bit tired, I actually don't think there is an issue with the tempfile commit but I'm running the tests locally too on python 3.12, just to check. @tgrandje with which python version did you test? was it with a clean install in a venv? EDIT 3 : local API is failing with code 500 (internal server error) at the moment, for me, so no tests) |
@tfardet I think I tried this on both 3.9 & 3.10, each time through a venv: I'm using a local Re-reading the code, I may have forgotten to handle the case where you have a valid s3fs installation but your credentials are not valid (I'll rewrite that). Is that what occured to you? I think that should trigger a (Do you have access to the CI variables? If not I'll send you the credentials by email.) |