Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delete expired access logs #5097

Merged
merged 9 commits into from
Oct 22, 2024
Merged

Delete expired access logs #5097

merged 9 commits into from
Oct 22, 2024

Conversation

rgraber
Copy link
Contributor

@rgraber rgraber commented Sep 5, 2024

Checklist

  1. If you've added code that should be tested, add tests
  2. If you've changed APIs, update (or create!) the documentation
  3. Ensure the tests pass
  4. Make sure that your code lints and that you've followed our coding style
  5. Write a title and, if necessary, a description of your work suitable for publishing in our release notes
  6. Mention any related issues in this repository (as #ISSUE) and in other repositories (as kobotoolbox/other#ISSUE)
  7. Open an issue in the docs if there are UI/UX changes

Description

Delete access logs older than the configured number of days (defaults to 60). The number of days is configurable in constance.

Notes

Adds a new task to run every day at midnight.

  1. Figure out the cut off date based on the configured lifespan of the access logs.
  2. Collect the ids of all access logs older than the cut off date
  3. In batches, submit these for deletion by spawning new batch_delete_audit_logs_by_id tasks

Additional changes:
Add the more-itertools library so we can easily break up the list of ids into batches of a constant size. Plain itertools has this functionality in python 3.12, but for the older versions we need the more-itertools library.

The batch size is configurable in settings. It defaults to 1000.

Replaces #5084

@rgraber rgraber marked this pull request as ready for review September 5, 2024 17:59
@rgraber rgraber force-pushed the TASK-869-group-submissions-better branch from 67579ff to 55b66ea Compare September 17, 2024 12:47
@rgraber rgraber marked this pull request as draft September 18, 2024 19:32
@rgraber
Copy link
Contributor Author

rgraber commented Sep 18, 2024

converting to draft until the questions about grouping submissions are resolved

@rgraber rgraber force-pushed the delete-access-logs-config branch 2 times, most recently from 0db95de to f3eeb98 Compare September 30, 2024 13:51
@rgraber rgraber changed the base branch from TASK-869-group-submissions-better to TASK-869-group-at-fetch September 30, 2024 14:13
@rgraber rgraber marked this pull request as ready for review September 30, 2024 15:22
Base automatically changed from TASK-869-group-at-fetch to beta-refactored October 2, 2024 13:17
@noliveleger noliveleger changed the base branch from beta-refactored to main October 7, 2024 18:30
@noliveleger noliveleger self-requested a review October 10, 2024 17:30
@noliveleger noliveleger self-assigned this Oct 10, 2024
@noliveleger noliveleger merged commit e0637af into main Oct 22, 2024
7 checks passed
@noliveleger noliveleger deleted the delete-access-logs-config branch October 22, 2024 19:14
@noliveleger noliveleger restored the delete-access-logs-config branch October 22, 2024 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants