Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Have client insert functions return insert result instead of raw job row (breaking) #292

Merged
merged 1 commit into from
Apr 19, 2024

Conversation

brandur
Copy link
Contributor

@brandur brandur commented Mar 28, 2024

This one's aimed at resolving the problem described in this discussion
[1]. In short, when an inserted job with unique properties in skipped
because it was a duplicate, there's no easy way to ascertain through
River's API that this is what happened.

The cleanest resolution is what's proposed here. Instead of returning a
raw JobRow, insert functions return a JobInsertResult that contains
a job row, along with a boolean indicating whether an insertion was
skipped due to a duplicate unique. The result struct is also better for
future compatibility in that it could contain other metadata pertaining
to job insertions.

The downside of this approach is that it's a breaking change. The hope
is that it's not that bad of a breaking change because most of the
time callers don't care that much about properties on a return result
row given they just sent most of it as a parameter to Insert, so they
might not be accessing any return values and therefore not be broken.

You can get a rough idea for changes in callers by looking at what
changed in this diff. client_test.go needed quite a few tweaks because
it's doing some more comprehensive testing, but only one example test
(more representative of user code) had to change, which is a good sign.

[1] #86

@brandur brandur added the breaking change Breaking API change label Mar 28, 2024
@brandur
Copy link
Contributor Author

brandur commented Mar 28, 2024

@bgentry So I'm kind of warming up to just making this (hopefully minor) breaking change because it feels like by far the most "correct" design, and I have a theory that most users won't be accessing the insert return values anyway, so it won't be breaking for most users.

I'm thinking this: we ship this breaking change, but in the same version as other breaking changes we know of: #236, and one other small one that I'll tee up soon. Shipping all breaking changes in one release doesn't make them any less breaking, but it means that when users upgrade, they only have to fix things once, and hopefully not for a long time again. Thoughts?

I added a new "breaking change" tag to track these.

Copy link
Contributor

@bgentry bgentry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it makes sense to rip this bandaid off and I don't think the breakage will be too bad (obvious compiler errors and simple fix).

I do wonder if it'll be an issue that the InsertMany APIs don't have the ability to return this data, but then those aren't checked for uniqueness anyway. However if that API is going to continue relying on COPY FROM, then it really can't support flexible uniqueness. Actually, just realized we don't clearly document the lack of uniqueness support in InsertMany—want to take care of that too?

Finally, this will need a changelog entry.

@brandur brandur force-pushed the brandur-insert-result branch 3 times, most recently from b093d93 to 4912df5 Compare March 29, 2024 04:24
@brandur
Copy link
Contributor Author

brandur commented Mar 29, 2024

I do wonder if it'll be an issue that the InsertMany APIs don't have the ability to return this data, but then those aren't checked for uniqueness anyway. However if that API is going to continue relying on COPY FROM, then it really can't support flexible uniqueness.

Yeah, those were my thoughts. It's going to be difficult to support uniqueness ever on InsertMany given how things are today, so it seems better for now just to punt on that. The return values for Insert versus InsertMany are already significantly different today as it is, and even if InsertMany were to respect uniqueness in the future, it might be reasonable to say that it won't return a UniqueSkippedAsDuplicate flag for the same reason InsertMany doesn't currently return JobRows — presumably if you're using it, you care a lot about insert performance, and you'd rather a faster operation than more information return values (which you can easily get by just using normal Insert).

Actually, just realized we don't clearly document the lack of uniqueness support in InsertMany—want to take care of that too?

Added.

Finally, this will need a changelog entry.

Cool. Added.

@brandur
Copy link
Contributor Author

brandur commented Mar 29, 2024

And as elsewhere, will hold off merge until we're ready for all breaking changes at once.

@brandur brandur force-pushed the brandur-insert-result branch 2 times, most recently from fb9375b to 6dffc2a Compare April 16, 2024 01:46
…row (breaking)

This one's aimed at resolving the problem described in this discussion
[1]. In short, when an inserted job with unique properties in skipped
because it was a duplicate, there's no easy way to ascertain through
River's API that this is what happened.

The cleanest resolution is what's proposed here. Instead of returning a
raw `JobRow`, insert functions return a `JobInsertResult` that contains
a job row, along with a boolean indicating whether an insertion was
skipped due to a duplicate unique. The result struct is also better for
future compatibility in that it could contain other metadata pertaining
to job insertions.

The downside of this approach is that it's a breaking change. The hope
is that it's not _that_ bad of a breaking change because most of the
time callers don't care that much about properties on a return result
row given they just sent most of it as a parameter to `Insert`, so they
might not be accessing any return values and therefore not be broken.

You can get a rough idea for changes in callers by looking at what
changed in this diff. `client_test.go` needed quite a few tweaks because
it's doing some more comprehensive testing, but only one example test
(more representative of user code) had to change, which is a good sign.

[1] #86
@brandur brandur merged commit 22b69ce into master Apr 19, 2024
10 checks passed
@brandur brandur deleted the brandur-insert-result branch April 19, 2024 02:21
brandur added a commit that referenced this pull request Apr 21, 2024
A few small variable name changes that were on my TODO list:

* Rename job list result variable names from `res` to a more specific
  `listRes`. This matches up better with the `insertRes` convention used
  in #292.

* Rename `result` in job parameter functions to the more descriptive
  `paramsCopy`.
brandur added a commit that referenced this pull request May 2, 2024
A few small variable name changes that were on my TODO list:

* Rename job list result variable names from `res` to a more specific
  `listRes`. This matches up better with the `insertRes` convention used
  in #292.

* Rename `result` in job parameter functions to the more descriptive
  `paramsCopy`.
brandur added a commit that referenced this pull request May 2, 2024
A few small variable name changes that were on my TODO list:

* Rename job list result variable names from `res` to a more specific
  `listRes`. This matches up better with the `insertRes` convention used
  in #292.

* Rename `result` in job parameter functions to the more descriptive
  `paramsCopy`.
brandur added a commit that referenced this pull request May 2, 2024
…309)

A few small variable name changes that were on my TODO list:

* Rename job list result variable names from `res` to a more specific
  `listRes`. This matches up better with the `insertRes` convention used
  in #292.

* Rename `result` in job parameter functions to the more descriptive
  `paramsCopy`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking change Breaking API change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants