-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SLT-17: Periodical drush SQL-Dump using gdpr-dump. #40
Conversation
b8733b0
to
40b28f4
Compare
40b28f4
to
b8d3b49
Compare
…rd drush sql-dump when the gdpr.json file can't be located; Documentation;
b8d3b49
to
0e41f76
Compare
- run: composer install -n --prefer-dist --ignore-platform-reqs --no-dev | ||
- run: | | ||
composer install -n --prefer-dist --ignore-platform-reqs --no-dev | ||
composer config repositories.gdpr-dump-mods git https://github.com/Jancis/gdpr-dump |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it work to add this to composer.json? This way we control when we get updates (otherwise things might break whenever a new version is released).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We have a cronjob by default that assumes there is a gdpr-dump installed we have to be sure developers include it in their composer.json
file. I'm doing this manually just because we can't be sure, but then again, build is slower because of that.
I can just assume that it's in there or add an additional check in cron execution.
We should provision the volume as well, but only if the current branch is the reference branch. |
761a498
to
a14d3e5
Compare
…-project-k8s into feature/SLT-17-gdpr-dump
What this PR does:
I've set up a cron rule that is invoked every day at 3:00 AM and runs a
drush sql-dump
command provided by GDPR Tools. The sql dump is only run when the current branchname equals predefinedproduction_branchname
invalues.yaml
We have a slightly modified version currently because of following issues/improvements:
GDPR replacements from JSON file machbarmacher/gdpr-dump#28Once the issues are fixed, we can go back to original.
This allows site developers to create a
/gdpr.json
file with Faker formatters that will allow replacing data as it's dumped from database usingmysqldump
/drush sql-dump
command.Available formatters:
Developers can also add extra elements and attributes, like
_cookies
,_description
or_purpose
that could bring this file closer to automatic PD documentation generation. Just have to be marked or prefixed so that it does not mess up GDPR dump when it looks for table data replacements.Testing:
It works on my machine, I swear! But you can branch it, change the cron interval to something like
*/5 * * * *
andproduction_branchname
to the name of your branch (i.e.feature/SLT-17-gdpr-dump
). Deploy and you should see updated sql dump at/var/backups/db/
every five minutes. You can search the dump file for stringusers_
and see that all user names, emails and such are changed. It also changes the name of anonymous user, which is not an ideal situation, but we can't do much about it at this time.Important note
While working on this, I ran into issue where dump couldn't rewrite existing dump.
This also happened on
sites/default/files/
, but I found the way around by removing the file first and then making the database dump. This seems to work perfectly, but there could still be a bigger issue with persistent storage so I created a bug report out of this.