Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Assert fields exist in collection before saving to them #604

Merged
merged 3 commits into from
Jul 7, 2022

Conversation

AndrewSisley
Copy link
Contributor

Relevant issue(s)

Resolves #573

Description

Asserts that fields exist in collection before saving to them. Issue made a lot easier by Shahzad spotting the return 0 in collection.go. Also adds support for testing the collection abi within our test framework - should hopefully make testing it much easier going forward.

Tasks

  • I made sure the code is well commented, particularly hard-to-understand areas.
  • I made sure the repository-held documentation is changed accordingly.
  • I made sure the pull request title adheres to the conventional commit style (the subset used in the project can be found in tools/configs/chglog/config.yml).
  • I made sure to discuss its limitations such as threats to validity, vulnerability to mistake and misuse, robustness to invalidation of assumptions, resource requirements, ...

Specify the platform(s) on which this was tested:

  • Debian Linux

@AndrewSisley AndrewSisley added bug Something isn't working area/collections Related to the collections system action/no-benchmark Skips the action that runs the benchmark. labels Jul 7, 2022
@AndrewSisley AndrewSisley added this to the DefraDB v0.3.1 milestone Jul 7, 2022
@AndrewSisley AndrewSisley requested a review from a team July 7, 2022 15:50
@AndrewSisley AndrewSisley self-assigned this Jul 7, 2022
@AndrewSisley AndrewSisley force-pushed the sisley/fix/I573-collection-field-validation branch from def69ca to 9750553 Compare July 7, 2022 15:52
@codecov
Copy link

codecov bot commented Jul 7, 2022

Codecov Report

Merging #604 (9750553) into develop (49523de) will increase coverage by 0.04%.
The diff coverage is 72.72%.

❗ Current head 9750553 differs from pull request most recent head 26ec210. Consider uploading reports for the commit 26ec210 to get more accurate results

Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #604      +/-   ##
===========================================
+ Coverage    56.36%   56.41%   +0.04%     
===========================================
  Files          121      121              
  Lines        14303    14316      +13     
===========================================
+ Hits          8062     8076      +14     
+ Misses        5543     5542       -1     
  Partials       698      698              
Impacted Files Coverage Δ
db/collection_update.go 40.71% <25.00%> (-0.26%) ⬇️
db/collection.go 59.53% <83.33%> (+1.36%) ⬆️

Copy link
Collaborator

@fredcarle fredcarle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving assuming my two concerns are addressed :)

db/collection.go Outdated
return core.DataStoreKey{
CollectionId: key.CollectionId,
DocKey: key.DocKey,
FieldId: fmt.Sprint(c.getSchemaFieldID(fieldName)),
}
FieldId: fmt.Sprint(fieldId),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: I suggestion using something a little more efficient than Sprint.

Suggested change
FieldId: fmt.Sprint(fieldId),
FieldId: strconv.FormatUint(uint64(fieldId), 10),

Copy link
Contributor Author

@AndrewSisley AndrewSisley Jul 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is a surprising performance gap between the two, and a bit odd fmt.Sprint doesnt take advantage of this. Will use strconv.Iota as that reads better. Thanks for flagging - is for sure good to know

https://developpaper.com/performance-test-of-several-methods-based-on-go-int-to-string/

  • do it this way

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah Iota only takes an int, which could technically overflow on a 32 bit machine - will use strconv.FormatUint as originally suggested. Really weird that Golang doesnt have a nicer way of doing this.

@@ -78,6 +78,9 @@ type QueryTestCase struct {
// updates is a map from document index, to a list
// of changes in strinigied JSON format
Updates map[int]map[int][]string

CollectionCalls map[int]func(client.Collection) error
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: can we call this UpdateFuncs?

Copy link
Contributor Author

@AndrewSisley AndrewSisley Jul 7, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a problem, will do

  • Rename

Panic hides any expected errors in case results are still returned
@AndrewSisley AndrewSisley force-pushed the sisley/fix/I573-collection-field-validation branch from 9750553 to b591f94 Compare July 7, 2022 19:27
Should allow us to assert collection-abi functionality within the main integration test suite
@AndrewSisley AndrewSisley force-pushed the sisley/fix/I573-collection-field-validation branch from b591f94 to 26ec210 Compare July 7, 2022 19:31
@AndrewSisley AndrewSisley merged commit d5b3cbb into develop Jul 7, 2022
@AndrewSisley AndrewSisley deleted the sisley/fix/I573-collection-field-validation branch July 7, 2022 19:37
shahzadlone pushed a commit to shahzadlone/defradb that referenced this pull request Feb 23, 2024
…etwork#604)

* Check expected length before accessing index

Panic hides any expected errors in case results are still returned

* Add support for collection abi calls in tests

Should allow us to assert collection-abi functionality within the main integration test suite

* Error if field does not exist
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
action/no-benchmark Skips the action that runs the benchmark. area/collections Related to the collections system bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Document/Collection interface does not validate existence of fields when updating
2 participants