Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Makes changes to dataset-related logging in audit log #664

Closed
matthew-white opened this issue Nov 11, 2022 · 9 comments · Fixed by #931
Closed

Makes changes to dataset-related logging in audit log #664

matthew-white opened this issue Nov 11, 2022 · 9 comments · Fixed by #931
Assignees
Labels
entities Multiple Encounter workflows

Comments

@matthew-white
Copy link
Member

For after v2022.3 — moving from #649 (comment):

Backend logs dataset.create and dataset.update in the audit log. However, I think that there are a couple of possibilities for confusion there:

  • The events are logged too early.
  • The events are logged too frequently.

More specifically:

  • If you upload but don't publish a form definition that would create entities for a new dataset, then dataset.create is immediately logged, even if the form draft is ultimately abandoned and the dataset never becomes visible on the project.
  • If you upload but don't publish a form definition that would add a property to an existing dataset, then dataset.update is immediately logged, even if the form draft is ultimately abandoned and the property never becomes visible on the dataset. (Maybe related to Tracking progress of Datasets & Entities #649 (comment)?)
  • dataset.update is logged even when there are no new properties. It is logged when a form definition is uploaded that would create entities for an existing dataset, not when that form definition would add a property to the dataset.

I'd expect dataset.create and dataset.update to be logged only when the form draft is published. I know that creating a draft can have an immediate effect on the datasets table, and I think it makes sense that it works that way under the hood. But I think we're exposing those mechanics over the audit log in a way that could be surprising and that ultimately makes the audit log less actionable. It also results in a mismatch between what's in the audit log and what's returned by …/datasets and …/dataset-diff.

How about we don't set .audit on Datasets.createOrMerge, but instead wait until publishing the form to log dataset.create or dataset.update? Maybe we could query the dataset diff then to determine whether and what to log.

@sadiqkhoja has made the point that it could be helpful for debugging to log events earlier / more frequently. One idea is that we could add more dataset actions, distinguishing between draft changes to a dataset and published changes to a dataset. It's not clear whether we'd return all those actions over the API or keep some just internal.

@ktuite
Copy link
Member

ktuite commented Jul 11, 2023

This issue is kind of related to this one about listing dataset audit logs via the api: #795

@sadiqkhoja sadiqkhoja moved this from 🕒 backlog to ✏️ in progress in ODK Central Jul 18, 2023
sadiqkhoja added a commit to sadiqkhoja/central-backend that referenced this issue Jul 18, 2023
sadiqkhoja added a commit that referenced this issue Aug 3, 2023
* Enhances #664: add logs only when changes are published

* updated API doc

* updated comments
@github-project-automation github-project-automation bot moved this from ✏️ in progress to ✅ done in ODK Central Aug 3, 2023
@matthew-white
Copy link
Member Author

@sadiqkhoja, now that dataset.update.publish events are no longer logged, how about we remove that action from the AuditFiltersAction component in Frontend?

@sadiqkhoja
Copy link
Contributor

@getodk/testers this is ready for testing in staging

Expected behaviour

  • When you upload a draft form that defines dataset, you won't see any logs on the Systems page
  • When you publish that form, you should see a log entry that a new dataset has been created
  • When you re-upload a draft form that adds more property in the dataset, you shouldn't see any log, but when you publish this new draft you should see a log entry that the dataset has been updated

@srujner
Copy link

srujner commented Aug 17, 2023

@sadiqkhoja Should change in the Datasets -> Entity name -> Settings -> Entity Workflow
Screenshot(153)
be also visible in the Server Audit Logs?
Screenshot(154)

@sadiqkhoja
Copy link
Contributor

yes

@dbemke
Copy link

dbemke commented Aug 18, 2023

If I move a field in a form (I don't add anything) should there be Dataset update event in audit logs?

@sadiqkhoja
Copy link
Contributor

no

@srujner
Copy link

srujner commented Aug 21, 2023

Tested with Success!

1 similar comment
@dbemke
Copy link

dbemke commented Aug 21, 2023

Tested with Success!

sadiqkhoja added a commit to sadiqkhoja/central-backend that referenced this issue Sep 1, 2023
…#931)

* Enhances getodk#664: add logs only when changes are published

* updated API doc

* updated comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
entities Multiple Encounter workflows
Projects
Status: ✅ done
Development

Successfully merging a pull request may close this issue.

5 participants