[Feature] Delete orphan files #1575
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…letion [Feature] Delete orphan files
|
EDIT: again the problem comes from me, I did not realized the storage analytics task was needed to update the status. https://github.com/codalab/codabench/wiki/Administrator-procedures#storage-analytics I continue having 113 orphans files after the deletion: 
codabench-django-1 | Delete orphan files started
codabench-django-1 | Delete oprhan files finished
codabench-django-1 | Delete orphan files started
codabench-django-1 | Delete oprhan files finished |
This was referenced Sep 3, 2024
Didayolo
added a commit
that referenced
this pull request
Oct 11, 2024
* autorun collectstatic when django container starts, added caddy_data and caddy_config to gitignore * programs and datasets bug fixed * if condition format fixed * Downloading submissions witha stream v1 * updated toml file with python version and bpython version, deleted lock file, update dockerfile to not copy lock file and to generate lock file in the docker build * Change base image for Dockerfile.flower (#1520) * Change base image for Dockerfile.flower * Remove platform argument * Resources bundles dumps (#1505) * Add a Bundles tab in Resources view * add dump single and multiple deletion mechanism * add competition title in bundle name + add competition link in bundle resource tab * Update datasets.py (flake8) --------- Co-authored-by: OhMaley <[email protected]> Co-authored-by: Adrien Pavão <[email protected]> * Improve formatting * leaderboard api 500 error resolved (#1527) * leaderboard api 500 error resolved * flake issues resolved * Changed submission name to add ID when downloading many submissions (#1565) * changed submission name to add ID * Flake8 white space fix --------- Co-authored-by: Nicolas HOMBERG <[email protected]> * Server Status: submissions limit removed (#1562) * server status submissions limit removed, pagination added, monitor queues separated * renamed server status title to recent submissions * Email template fixes (#1567) * email logo fixed, some changes in email text * participant side email templates updated * task dataset/program removal fixed * Updated docker-compose to docker compose * whiltelist emails leaking fixed (#1560) * competition whitelist feature added * updated whitelist * Update whitelist note * whiltelist emails leaking fixed * Remove conflict --------- Co-authored-by: Adrien Pavão <[email protected]> * Updates to server_status and monitor_queues (#1568) * server_status and monitor_queues pages secured * Update monitor_queues.html * [Feature] Delete orphan files (#1575) * Add button for orphan files deletion + API routes and views * Fix flake8 formatting --------- Co-authored-by: OhMaley <[email protected]> * Fix string concat in case of no email in analytics api * Poetry error tentative fix (#1588) * Poetry error tentative fix * RabbitMQ static image in Dockerfile.rabbitmq to fix a recent bug * Reverted RabbitMQ image change * Fix Rabbit version --------- Co-authored-by: Obada Haddad <[email protected]> Co-authored-by: didayolo <[email protected]> * `Release PR#1` - Display release version in Codabench footer (#1583) * release version added to header * version moved to footer * Minor UI change --------- Co-authored-by: Adrien Pavão <[email protected]> * `Release PR#2` Release version will load dynamically from version.json (#1592) * release version added to header * version moved to footer * version on the website is now dynamic and will be loaded from the version.json file --------- Co-authored-by: Adrien Pavão <[email protected]> * `Release PR#3` created workflow for new releases (#1584) * created workflow for new releases * version json file added * typo fix * Update release-version-update.yml * Included realease url to navigate user to the url on click on the website * release json updated --------- Co-authored-by: Obada Haddad <[email protected]> Co-authored-by: Adrien Pavão <[email protected]> * Updates Sept 15-22 2024 (#1593) * jinja and pillow * jinja and pillow * Adding Versino_Bump_Instruction_README * Version_Bumps file * Version_Bumps file and req_treeFile updates * gunicorn from 20.0.4 to 22.0.0 * Bump requests from 2.20.0 to 2.32.2 * Bump django from 2.2.17 to 2.2.18 * Bump django from 2.2.18 to 2.2.19 and to 3.2.25 eventually * Bump django from 2.2.19 to 2.2.20 * Bump django from 2.2.20 to 2.2.25 * Bump django from 2.2.25 to 2.2.28 * Bump django from 2.2.25 to 2.2.28 * Version_Bumps.md text * Update Dockerfile (upgrade Python) * checking in some outstanding changes * Remove Version Bumps * removing version bumbs --------- Co-authored-by: Adrien Pavão <[email protected]> * organizers can now delete forum posts (#1613) Co-authored-by: Adrien Pavão <[email protected]> * bundles renamed to competition bundles, swapped postions of bundles and tasks (#1591) Co-authored-by: Adrien Pavão <[email protected]> --------- Co-authored-by: Ihsan Ullah <[email protected]> Co-authored-by: Nicolas HOMBERG <[email protected]> Co-authored-by: OhMaley <[email protected]> Co-authored-by: Obada Haddad <[email protected]> Co-authored-by: Obada Haddad-Soussac <[email protected]> Co-authored-by: Benjamin Bearce <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Original PR
Description
This adds a button in the storage view to trigger, after confirmation, the deletion of orphan files in the storage. The list of orphan files are not computed during the process since it is time expensive. Instead is uses the last storage analytics results and its storage_inconsistency log file.
To learn more about the storage analytics task: https://github.com/codalab/codabench/wiki/Administrator-procedures#storage-analytics
Issues this PR resolves
Related to issue #713
To do once merged (manual intervention)
Copy and paste last storage_inconsistency files from /logs folder to /var/logs/ folder. Mind the 's' at the end of 'logs'.
Checklist