Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: Comment the benchmarking result comparison to the PR #305

Merged
merged 5 commits into from
Mar 31, 2022
Merged

ci: Comment the benchmarking result comparison to the PR #305

merged 5 commits into from
Mar 31, 2022

Conversation

shahzadlone
Copy link
Member

resolves #304

@shahzadlone shahzadlone added the ci/build This is issue is about the build or CI system, and the administration of it. label Mar 22, 2022
@shahzadlone shahzadlone added this to the DefraDB v0.3 milestone Mar 22, 2022
@shahzadlone shahzadlone self-assigned this Mar 22, 2022
@sourcenetwork sourcenetwork deleted a comment from source-devs Mar 22, 2022
@sourcenetwork sourcenetwork deleted a comment from source-devs Mar 22, 2022
@sourcenetwork sourcenetwork deleted a comment from source-devs Mar 22, 2022
@shahzadlone shahzadlone force-pushed the lone/ci/comment-benchmark-results-on-pr branch from f53c775 to 7e20d6e Compare March 22, 2022 15:29
@shahzadlone shahzadlone added the action/full-benchmark Runs the full / longer benchmark action. label Mar 22, 2022
@shahzadlone shahzadlone force-pushed the lone/ci/comment-benchmark-results-on-pr branch from fc9f56d to c387586 Compare March 22, 2022 15:35
@sourcenetwork sourcenetwork deleted a comment from source-devs Mar 22, 2022
@sourcenetwork sourcenetwork deleted a comment from source-devs Mar 22, 2022
@source-devs

This comment was marked as outdated.

Sign in to view
@shahzadlone shahzadlone removed the action/full-benchmark Runs the full / longer benchmark action. label Mar 22, 2022
@source-devs

This comment was marked as outdated.

Sign in to view
@source-devs

This comment was marked as spam.

Sign in to view
jsimnz
jsimnz approved these changes Mar 25, 2022
View reviewed changes
Copy link
Member

@jsimnz jsimnz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. We can do further comment formatting (markdown summary) in the next PR?

@shahzadlone
Copy link
Member Author

Or we can also do it in this PR, if it's a quick thing. Want to share the approach you wanted to take to output those table summaries?

@source-devs
Copy link

Benchmark Results

Summary

  • 113 Benchmarks successfully compared.
  • 2 Benchmarks were ✅ Better.
  • 111 Benchmarks were ❌ Worse .
  • 0 Benchmarks were ✨ Unchanged.
✅ See Better Results...
time/opdelta
_Collection_UserSimple_Read_Async_10_10-4280µs ± 0%278µs ± 0%−0.77%(p=1.000 n=1+1)
_Collection_UserSimple_Read_Async_1000_100-42.12ms ± 0%2.05ms ± 0%−3.09%(p=1.000 n=1+1)
 
❌ See Worse Results...
time/opdelta
_Collection_UserSimple_CreateMany_Sync_0_100-4222ms ± 0%287ms ± 0%+29.40%(p=1.000 n=1+1)
_Collection_UserSimple_Create_Sync_0_10-49.91ms ± 0%13.03ms ± 0%+31.39%(p=1.000 n=1+1)
_Collection_UserSimple_Create_Sync_0_100-496.6ms ± 0%137.9ms ± 0%+42.83%(p=1.000 n=1+1)
_Collection_UserSimple_Create_Sync_0_1000-4985ms ± 0%1180ms ± 0%+19.85%(p=1.000 n=1+1)
_Collection_UserSimple_Create_Async_0_100-441.4ms ± 0%45.1ms ± 0%+8.97%(p=1.000 n=1+1)
_Collection_UserSimple_Create_Async_0_1000-4412ms ± 0%435ms ± 0%+5.76%(p=1.000 n=1+1)
_Collection_UserSimple_Create_Async_0_10000-44.25s ± 0%4.45s ± 0%+4.73%(p=1.000 n=1+1)
_Collection_UserSimple_Read_Sync_10_10-4399µs ± 0%432µs ± 0%+8.04%(p=1.000 n=1+1)
_Collection_UserSimple_Read_Sync_100_100-44.10ms ± 0%4.92ms ± 0%+20.00%(p=1.000 n=1+1)
_Collection_UserSimple_Read_Sync_1000_1000-442.4ms ± 0%43.3ms ± 0%+2.13%(p=1.000 n=1+1)
_Collection_UserSimple_Read_Sync_1000_10-4421µs ± 0%435µs ± 0%+3.25%(p=1.000 n=1+1)
_Collection_UserSimple_Read_Sync_1000_100-44.32ms ± 0%4.35ms ± 0%+0.76%(p=1.000 n=1+1)
_Collection_UserSimple_Read_Async_100_100-42.03ms ± 0%2.06ms ± 0%+1.47%(p=1.000 n=1+1)
_Collection_UserSimple_Read_Async_1000_1000-426.3ms ± 0%26.7ms ± 0%+1.68%(p=1.000 n=1+1)
_Collection_UserSimple_Read_Async_1000_10-4274µs ± 0%294µs ± 0%+7.19%(p=1.000 n=1+1)
_Query_UserSimple_Query_Sync_100-41.89ms ± 0%2.35ms ± 0%+24.04%(p=1.000 n=1+1)
_Query_UserSimple_Query_Sync_1000-417.4ms ± 0%18.6ms ± 0%+6.70%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithFilter_Sync_10-4485µs ± 0%675µs ± 0%+39.35%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithFilter_Sync_100-41.98ms ± 0%2.66ms ± 0%+34.15%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithFilter_Sync_1000-418.2ms ± 0%20.3ms ± 0%+11.67%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithLimitOffset_Sync_10-4441µs ± 0%530µs ± 0%+20.30%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithLimitOffset_Sync_100-4534µs ± 0%647µs ± 0%+21.17%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithLimitOffset_Sync_1000-4540µs ± 0%569µs ± 0%+5.23%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithMultiLookup_Sync_10-4649µs ± 0%799µs ± 0%+23.07%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithMultiLookup_Sync_100-4652µs ± 0%896µs ± 0%+37.42%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithMultiLookup_Sync_1000-4716µs ± 0%746µs ± 0%+4.11%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithSingleLookup_Sync_10-4258µs ± 0%330µs ± 0%+28.03%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithSingleLookup_Sync_100-4258µs ± 0%307µs ± 0%+19.15%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithSingleLookup_Sync_1000-4257µs ± 0%267µs ± 0%+4.09%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithSort_Sync_10-4459µs ± 0%566µs ± 0%+23.20%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithSort_Sync_100-42.05ms ± 0%2.60ms ± 0%+26.65%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithSort_Sync_1000-419.5ms ± 0%22.6ms ± 0%+15.83%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_1_10/ValueSize:0128-413.9µs ± 0%17.6µs ± 0%+26.50%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_1_10/ValueSize:0256-414.5µs ± 0%17.6µs ± 0%+21.27%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_1_10/ValueSize:0512-416.3µs ± 0%22.2µs ± 0%+36.76%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_1_10/ValueSize:1024-419.7µs ± 0%25.5µs ± 0%+29.67%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_1_100/ValueSize:0064-4126µs ± 0%152µs ± 0%+19.96%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_1_100/ValueSize:0128-4134µs ± 0%159µs ± 0%+18.54%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_1_100/ValueSize:0256-4143µs ± 0%171µs ± 0%+19.61%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_1_100/ValueSize:0512-4161µs ± 0%202µs ± 0%+25.41%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_1_100/ValueSize:1024-4199µs ± 0%240µs ± 0%+20.29%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_100_10/ValueSize:0064-414.6µs ± 0%18.9µs ± 0%+29.03%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_100_10/ValueSize:0128-415.1µs ± 0%19.0µs ± 0%+25.68%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_100_10/ValueSize:0256-416.0µs ± 0%22.9µs ± 0%+43.35%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_100_10/ValueSize:0512-418.4µs ± 0%22.3µs ± 0%+20.98%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_100_10/ValueSize:1024-421.3µs ± 0%25.7µs ± 0%+20.65%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_100_100/ValueSize:0064-4139µs ± 0%174µs ± 0%+25.92%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_100_100/ValueSize:0128-4141µs ± 0%208µs ± 0%+47.39%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_100_100/ValueSize:0256-4150µs ± 0%194µs ± 0%+29.54%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_100_100/ValueSize:0512-4164µs ± 0%219µs ± 0%+33.52%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_100_100/ValueSize:1024-4200µs ± 0%276µs ± 0%+37.91%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_0_10/ValueSize:0064-444.2µs ± 0%62.4µs ± 0%+41.03%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_0_10/ValueSize:0128-445.1µs ± 0%59.2µs ± 0%+31.23%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_0_10/ValueSize:0256-446.3µs ± 0%62.5µs ± 0%+35.07%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_0_10/ValueSize:0512-453.0µs ± 0%63.0µs ± 0%+18.92%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_0_10/ValueSize:1024-460.6µs ± 0%75.5µs ± 0%+24.71%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_0_100/ValueSize:0064-4348µs ± 0%498µs ± 0%+42.90%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_0_100/ValueSize:0128-4356µs ± 0%413µs ± 0%+16.00%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_0_100/ValueSize:0256-4379µs ± 0%486µs ± 0%+28.22%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_0_100/ValueSize:0512-4398µs ± 0%488µs ± 0%+22.68%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_0_100/ValueSize:1024-4455µs ± 0%621µs ± 0%+36.41%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_100_10/ValueSize:0064-443.3µs ± 0%53.8µs ± 0%+24.36%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_100_10/ValueSize:0128-444.7µs ± 0%60.4µs ± 0%+35.07%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_100_10/ValueSize:0256-447.4µs ± 0%60.1µs ± 0%+26.62%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_100_10/ValueSize:0512-449.5µs ± 0%60.1µs ± 0%+21.34%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_100_10/ValueSize:1024-461.3µs ± 0%68.8µs ± 0%+12.33%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_100_100/ValueSize:0064-4368µs ± 0%457µs ± 0%+24.33%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_100_100/ValueSize:0128-4352µs ± 0%420µs ± 0%+19.32%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_100_100/ValueSize:0256-4377µs ± 0%392µs ± 0%+3.90%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_100_100/ValueSize:0512-4391µs ± 0%559µs ± 0%+42.76%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_100_100/ValueSize:1024-4497µs ± 0%650µs ± 0%+30.72%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_0_10/ValueSize:0064-4114µs ± 0%127µs ± 0%+12.06%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_0_10/ValueSize:0128-4112µs ± 0%137µs ± 0%+22.53%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_0_10/ValueSize:0256-4116µs ± 0%143µs ± 0%+24.11%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_0_10/ValueSize:0512-4121µs ± 0%152µs ± 0%+25.65%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_0_10/ValueSize:1024-4133µs ± 0%159µs ± 0%+19.36%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_0_100/ValueSize:0064-41.13ms ± 0%1.40ms ± 0%+23.64%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_0_100/ValueSize:0128-41.13ms ± 0%1.43ms ± 0%+26.48%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_0_100/ValueSize:0256-41.15ms ± 0%1.31ms ± 0%+13.50%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_0_100/ValueSize:0512-41.20ms ± 0%1.36ms ± 0%+13.94%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_0_100/ValueSize:1024-41.29ms ± 0%1.87ms ± 0%+44.84%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_100_10/ValueSize:0064-4109µs ± 0%137µs ± 0%+25.32%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_100_10/ValueSize:0128-4107µs ± 0%144µs ± 0%+35.36%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_100_10/ValueSize:0256-4110µs ± 0%138µs ± 0%+24.74%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_100_10/ValueSize:0512-4111µs ± 0%151µs ± 0%+36.21%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_100_10/ValueSize:1024-4129µs ± 0%164µs ± 0%+26.82%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_100_100/ValueSize:0064-41.08ms ± 0%1.25ms ± 0%+14.93%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_100_100/ValueSize:0128-41.09ms ± 0%1.44ms ± 0%+32.25%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_100_100/ValueSize:0256-41.10ms ± 0%1.35ms ± 0%+23.01%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_100_100/ValueSize:0512-41.19ms ± 0%1.45ms ± 0%+22.32%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_100_100/ValueSize:1024-41.27ms ± 0%1.67ms ± 0%+32.02%(p=1.000 n=1+1)
_Storage_Simple_Txn_Read_Sync_10_10/ValueSize:0064-48.41µs ± 0%12.01µs ± 0%+42.78%(p=1.000 n=1+1)
_Storage_Simple_Txn_Read_Sync_10_10/ValueSize:0128-48.87µs ± 0%13.53µs ± 0%+52.47%(p=1.000 n=1+1)
_Storage_Simple_Txn_Read_Sync_10_10/ValueSize:0256-410.3µs ± 0%15.4µs ± 0%+49.03%(p=1.000 n=1+1)
_Storage_Simple_Txn_Read_Sync_10_10/ValueSize:0512-412.1µs ± 0%14.5µs ± 0%+19.78%(p=1.000 n=1+1)
_Storage_Simple_Txn_Read_Sync_10_10/ValueSize:1024-415.5µs ± 0%17.0µs ± 0%+10.07%(p=1.000 n=1+1)
_Storage_Simple_Txn_Read_Sync_100_100/ValueSize:0064-489.7µs ± 0%118.7µs ± 0%+32.32%(p=1.000 n=1+1)
_Storage_Simple_Txn_Read_Sync_100_100/ValueSize:0128-491.1µs ± 0%163.5µs ± 0%+79.43%(p=1.000 n=1+1)
_Storage_Simple_Txn_Read_Sync_100_100/ValueSize:0256-4104µs ± 0%202µs ± 0%+93.58%(p=1.000 n=1+1)
_Storage_Simple_Txn_Read_Sync_100_100/ValueSize:0512-4121µs ± 0%144µs ± 0%+19.51%(p=1.000 n=1+1)
_Storage_Simple_Txn_Read_Sync_100_100/ValueSize:1024-4154µs ± 0%204µs ± 0%+32.07%(p=1.000 n=1+1)
_Storage_Simple_Txn_Iterator_Sync_10_1_10/ValueSize:0064-4112µs ± 0%139µs ± 0%+23.64%(p=1.000 n=1+1)
_Storage_Simple_Txn_Iterator_Sync_10_1_10/ValueSize:0128-4108µs ± 0%130µs ± 0%+19.84%(p=1.000 n=1+1)
_Storage_Simple_Txn_Iterator_Sync_10_1_10/ValueSize:0256-4114µs ± 0%130µs ± 0%+14.03%(p=1.000 n=1+1)
_Storage_Simple_Txn_Iterator_Sync_10_1_10/ValueSize:0512-4117µs ± 0%132µs ± 0%+12.74%(p=1.000 n=1+1)
_Storage_Simple_Txn_Iterator_Sync_10_1_10/ValueSize:1024-4120µs ± 0%131µs ± 0%+9.28%(p=1.000 n=1+1)
_Storage_Simple_Txn_Iterator_Sync_100_1_100/ValueSize:0064-41.08ms ± 0%1.21ms ± 0%+12.30%(p=1.000 n=1+1)
_Storage_Simple_Txn_Iterator_Sync_100_1_100/ValueSize:0128-41.09ms ± 0%1.17ms ± 0%+7.53%(p=1.000 n=1+1)
_Storage_Simple_Txn_Iterator_Sync_100_1_100/ValueSize:0256-41.15ms ± 0%1.21ms ± 0%+5.38%(p=1.000 n=1+1)
_Storage_Simple_Txn_Iterator_Sync_100_1_100/ValueSize:0512-41.13ms ± 0%1.29ms ± 0%+14.67%(p=1.000 n=1+1)
_Storage_Simple_Txn_Iterator_Sync_100_1_100/ValueSize:1024-41.23ms ± 0%1.31ms ± 0%+6.71%(p=1.000 n=1+1)
 
✨ See Unchanged Results...
time/opdelta
 
🐋 See Full Results...
develop.txtcurrent.txt
time/opdelta
pkg:github.com/sourcenetwork/defradb/bench/collection goos:linux goarch:amd64
_Collection_UserSimple_CreateMany_Sync_0_10-410.7ms ± 0%11.1ms ± 0%+4.28%(p=1.000 n=1+1)
_Collection_UserSimple_CreateMany_Sync_0_100-4222ms ± 0%287ms ± 0%+29.40%(p=1.000 n=1+1)
_Collection_UserSimple_Create_Sync_0_10-49.91ms ± 0%13.03ms ± 0%+31.39%(p=1.000 n=1+1)
_Collection_UserSimple_Create_Sync_0_100-496.6ms ± 0%137.9ms ± 0%+42.83%(p=1.000 n=1+1)
_Collection_UserSimple_Create_Sync_0_1000-4985ms ± 0%1180ms ± 0%+19.85%(p=1.000 n=1+1)
_Collection_UserSimple_Create_Async_0_100-441.4ms ± 0%45.1ms ± 0%+8.97%(p=1.000 n=1+1)
_Collection_UserSimple_Create_Async_0_1000-4412ms ± 0%435ms ± 0%+5.76%(p=1.000 n=1+1)
_Collection_UserSimple_Create_Async_0_10000-44.25s ± 0%4.45s ± 0%+4.73%(p=1.000 n=1+1)
_Collection_UserSimple_Read_Sync_10_10-4399µs ± 0%432µs ± 0%+8.04%(p=1.000 n=1+1)
_Collection_UserSimple_Read_Sync_100_100-44.10ms ± 0%4.92ms ± 0%+20.00%(p=1.000 n=1+1)
_Collection_UserSimple_Read_Sync_1000_1000-442.4ms ± 0%43.3ms ± 0%+2.13%(p=1.000 n=1+1)
_Collection_UserSimple_Read_Sync_1000_10-4421µs ± 0%435µs ± 0%+3.25%(p=1.000 n=1+1)
_Collection_UserSimple_Read_Sync_1000_100-44.32ms ± 0%4.35ms ± 0%+0.76%(p=1.000 n=1+1)
_Collection_UserSimple_Read_Async_10_10-4280µs ± 0%278µs ± 0%−0.77%(p=1.000 n=1+1)
_Collection_UserSimple_Read_Async_100_100-42.03ms ± 0%2.06ms ± 0%+1.47%(p=1.000 n=1+1)
_Collection_UserSimple_Read_Async_1000_1000-426.3ms ± 0%26.7ms ± 0%+1.68%(p=1.000 n=1+1)
_Collection_UserSimple_Read_Async_1000_10-4274µs ± 0%294µs ± 0%+7.19%(p=1.000 n=1+1)
_Collection_UserSimple_Read_Async_1000_100-42.12ms ± 0%2.05ms ± 0%−3.09%(p=1.000 n=1+1)
pkg:github.com/sourcenetwork/defradb/bench/query/simple goos:linux goarch:amd64
_Query_UserSimple_Query_Sync_10-4423µs ± 0%423µs ± 0%+0.00%(p=1.000 n=1+1)
_Query_UserSimple_Query_Sync_100-41.89ms ± 0%2.35ms ± 0%+24.04%(p=1.000 n=1+1)
_Query_UserSimple_Query_Sync_1000-417.4ms ± 0%18.6ms ± 0%+6.70%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithFilter_Sync_10-4485µs ± 0%675µs ± 0%+39.35%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithFilter_Sync_100-41.98ms ± 0%2.66ms ± 0%+34.15%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithFilter_Sync_1000-418.2ms ± 0%20.3ms ± 0%+11.67%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithLimitOffset_Sync_10-4441µs ± 0%530µs ± 0%+20.30%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithLimitOffset_Sync_100-4534µs ± 0%647µs ± 0%+21.17%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithLimitOffset_Sync_1000-4540µs ± 0%569µs ± 0%+5.23%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithMultiLookup_Sync_10-4649µs ± 0%799µs ± 0%+23.07%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithMultiLookup_Sync_100-4652µs ± 0%896µs ± 0%+37.42%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithMultiLookup_Sync_1000-4716µs ± 0%746µs ± 0%+4.11%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithSingleLookup_Sync_10-4258µs ± 0%330µs ± 0%+28.03%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithSingleLookup_Sync_100-4258µs ± 0%307µs ± 0%+19.15%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithSingleLookup_Sync_1000-4257µs ± 0%267µs ± 0%+4.09%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithSort_Sync_10-4459µs ± 0%566µs ± 0%+23.20%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithSort_Sync_100-42.05ms ± 0%2.60ms ± 0%+26.65%(p=1.000 n=1+1)
_Query_UserSimple_Query_WithSort_Sync_1000-419.5ms ± 0%22.6ms ± 0%+15.83%(p=1.000 n=1+1)
pkg:github.com/sourcenetwork/defradb/bench/storage goos:linux goarch:amd64
_Storage_Simple_Read_Sync_1_10/ValueSize:0064-412.4µs ± 0%15.6µs ± 0%+25.11%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_1_10/ValueSize:0128-413.9µs ± 0%17.6µs ± 0%+26.50%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_1_10/ValueSize:0256-414.5µs ± 0%17.6µs ± 0%+21.27%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_1_10/ValueSize:0512-416.3µs ± 0%22.2µs ± 0%+36.76%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_1_10/ValueSize:1024-419.7µs ± 0%25.5µs ± 0%+29.67%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_1_100/ValueSize:0064-4126µs ± 0%152µs ± 0%+19.96%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_1_100/ValueSize:0128-4134µs ± 0%159µs ± 0%+18.54%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_1_100/ValueSize:0256-4143µs ± 0%171µs ± 0%+19.61%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_1_100/ValueSize:0512-4161µs ± 0%202µs ± 0%+25.41%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_1_100/ValueSize:1024-4199µs ± 0%240µs ± 0%+20.29%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_100_10/ValueSize:0064-414.6µs ± 0%18.9µs ± 0%+29.03%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_100_10/ValueSize:0128-415.1µs ± 0%19.0µs ± 0%+25.68%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_100_10/ValueSize:0256-416.0µs ± 0%22.9µs ± 0%+43.35%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_100_10/ValueSize:0512-418.4µs ± 0%22.3µs ± 0%+20.98%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_100_10/ValueSize:1024-421.3µs ± 0%25.7µs ± 0%+20.65%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_100_100/ValueSize:0064-4139µs ± 0%174µs ± 0%+25.92%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_100_100/ValueSize:0128-4141µs ± 0%208µs ± 0%+47.39%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_100_100/ValueSize:0256-4150µs ± 0%194µs ± 0%+29.54%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_100_100/ValueSize:0512-4164µs ± 0%219µs ± 0%+33.52%(p=1.000 n=1+1)
_Storage_Simple_Read_Sync_100_100/ValueSize:1024-4200µs ± 0%276µs ± 0%+37.91%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_0_10/ValueSize:0064-444.2µs ± 0%62.4µs ± 0%+41.03%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_0_10/ValueSize:0128-445.1µs ± 0%59.2µs ± 0%+31.23%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_0_10/ValueSize:0256-446.3µs ± 0%62.5µs ± 0%+35.07%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_0_10/ValueSize:0512-453.0µs ± 0%63.0µs ± 0%+18.92%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_0_10/ValueSize:1024-460.6µs ± 0%75.5µs ± 0%+24.71%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_0_100/ValueSize:0064-4348µs ± 0%498µs ± 0%+42.90%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_0_100/ValueSize:0128-4356µs ± 0%413µs ± 0%+16.00%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_0_100/ValueSize:0256-4379µs ± 0%486µs ± 0%+28.22%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_0_100/ValueSize:0512-4398µs ± 0%488µs ± 0%+22.68%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_0_100/ValueSize:1024-4455µs ± 0%621µs ± 0%+36.41%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_100_10/ValueSize:0064-443.3µs ± 0%53.8µs ± 0%+24.36%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_100_10/ValueSize:0128-444.7µs ± 0%60.4µs ± 0%+35.07%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_100_10/ValueSize:0256-447.4µs ± 0%60.1µs ± 0%+26.62%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_100_10/ValueSize:0512-449.5µs ± 0%60.1µs ± 0%+21.34%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_100_10/ValueSize:1024-461.3µs ± 0%68.8µs ± 0%+12.33%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_100_100/ValueSize:0064-4368µs ± 0%457µs ± 0%+24.33%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_100_100/ValueSize:0128-4352µs ± 0%420µs ± 0%+19.32%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_100_100/ValueSize:0256-4377µs ± 0%392µs ± 0%+3.90%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_100_100/ValueSize:0512-4391µs ± 0%559µs ± 0%+42.76%(p=1.000 n=1+1)
_Storage_Simple_WriteMany_Sync_100_100/ValueSize:1024-4497µs ± 0%650µs ± 0%+30.72%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_0_10/ValueSize:0064-4114µs ± 0%127µs ± 0%+12.06%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_0_10/ValueSize:0128-4112µs ± 0%137µs ± 0%+22.53%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_0_10/ValueSize:0256-4116µs ± 0%143µs ± 0%+24.11%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_0_10/ValueSize:0512-4121µs ± 0%152µs ± 0%+25.65%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_0_10/ValueSize:1024-4133µs ± 0%159µs ± 0%+19.36%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_0_100/ValueSize:0064-41.13ms ± 0%1.40ms ± 0%+23.64%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_0_100/ValueSize:0128-41.13ms ± 0%1.43ms ± 0%+26.48%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_0_100/ValueSize:0256-41.15ms ± 0%1.31ms ± 0%+13.50%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_0_100/ValueSize:0512-41.20ms ± 0%1.36ms ± 0%+13.94%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_0_100/ValueSize:1024-41.29ms ± 0%1.87ms ± 0%+44.84%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_100_10/ValueSize:0064-4109µs ± 0%137µs ± 0%+25.32%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_100_10/ValueSize:0128-4107µs ± 0%144µs ± 0%+35.36%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_100_10/ValueSize:0256-4110µs ± 0%138µs ± 0%+24.74%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_100_10/ValueSize:0512-4111µs ± 0%151µs ± 0%+36.21%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_100_10/ValueSize:1024-4129µs ± 0%164µs ± 0%+26.82%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_100_100/ValueSize:0064-41.08ms ± 0%1.25ms ± 0%+14.93%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_100_100/ValueSize:0128-41.09ms ± 0%1.44ms ± 0%+32.25%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_100_100/ValueSize:0256-41.10ms ± 0%1.35ms ± 0%+23.01%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_100_100/ValueSize:0512-41.19ms ± 0%1.45ms ± 0%+22.32%(p=1.000 n=1+1)
_Storage_Simple_Write_Sync_100_100/ValueSize:1024-41.27ms ± 0%1.67ms ± 0%+32.02%(p=1.000 n=1+1)
_Storage_Simple_Txn_Read_Sync_10_10/ValueSize:0064-48.41µs ± 0%12.01µs ± 0%+42.78%(p=1.000 n=1+1)
_Storage_Simple_Txn_Read_Sync_10_10/ValueSize:0128-48.87µs ± 0%13.53µs ± 0%+52.47%(p=1.000 n=1+1)
_Storage_Simple_Txn_Read_Sync_10_10/ValueSize:0256-410.3µs ± 0%15.4µs ± 0%+49.03%(p=1.000 n=1+1)
_Storage_Simple_Txn_Read_Sync_10_10/ValueSize:0512-412.1µs ± 0%14.5µs ± 0%+19.78%(p=1.000 n=1+1)
_Storage_Simple_Txn_Read_Sync_10_10/ValueSize:1024-415.5µs ± 0%17.0µs ± 0%+10.07%(p=1.000 n=1+1)
_Storage_Simple_Txn_Read_Sync_100_100/ValueSize:0064-489.7µs ± 0%118.7µs ± 0%+32.32%(p=1.000 n=1+1)
_Storage_Simple_Txn_Read_Sync_100_100/ValueSize:0128-491.1µs ± 0%163.5µs ± 0%+79.43%(p=1.000 n=1+1)
_Storage_Simple_Txn_Read_Sync_100_100/ValueSize:0256-4104µs ± 0%202µs ± 0%+93.58%(p=1.000 n=1+1)
_Storage_Simple_Txn_Read_Sync_100_100/ValueSize:0512-4121µs ± 0%144µs ± 0%+19.51%(p=1.000 n=1+1)
_Storage_Simple_Txn_Read_Sync_100_100/ValueSize:1024-4154µs ± 0%204µs ± 0%+32.07%(p=1.000 n=1+1)
_Storage_Simple_Txn_Iterator_Sync_10_1_10/ValueSize:0064-4112µs ± 0%139µs ± 0%+23.64%(p=1.000 n=1+1)
_Storage_Simple_Txn_Iterator_Sync_10_1_10/ValueSize:0128-4108µs ± 0%130µs ± 0%+19.84%(p=1.000 n=1+1)
_Storage_Simple_Txn_Iterator_Sync_10_1_10/ValueSize:0256-4114µs ± 0%130µs ± 0%+14.03%(p=1.000 n=1+1)
_Storage_Simple_Txn_Iterator_Sync_10_1_10/ValueSize:0512-4117µs ± 0%132µs ± 0%+12.74%(p=1.000 n=1+1)
_Storage_Simple_Txn_Iterator_Sync_10_1_10/ValueSize:1024-4120µs ± 0%131µs ± 0%+9.28%(p=1.000 n=1+1)
_Storage_Simple_Txn_Iterator_Sync_100_1_100/ValueSize:0064-41.08ms ± 0%1.21ms ± 0%+12.30%(p=1.000 n=1+1)
_Storage_Simple_Txn_Iterator_Sync_100_1_100/ValueSize:0128-41.09ms ± 0%1.17ms ± 0%+7.53%(p=1.000 n=1+1)
_Storage_Simple_Txn_Iterator_Sync_100_1_100/ValueSize:0256-41.15ms ± 0%1.21ms ± 0%+5.38%(p=1.000 n=1+1)
_Storage_Simple_Txn_Iterator_Sync_100_1_100/ValueSize:0512-41.13ms ± 0%1.29ms ± 0%+14.67%(p=1.000 n=1+1)
_Storage_Simple_Txn_Iterator_Sync_100_1_100/ValueSize:1024-41.23ms ± 0%1.31ms ± 0%+6.71%(p=1.000 n=1+1)
 

jsimnz
jsimnz approved these changes Mar 31, 2022
View reviewed changes
Copy link
Member

@jsimnz jsimnz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@shahzadlone shahzadlone merged commit 702a966 into develop Mar 31, 2022
@shahzadlone shahzadlone deleted the lone/ci/comment-benchmark-results-on-pr branch March 31, 2022 00:08
shahzadlone added a commit to shahzadlone/defradb that referenced this pull request Feb 23, 2024
@shahzadlone
ci: Comment the benchmarking result comparison to the PR (sourcenetwo…
8f56758 
…rk#305)

Output prettier Benchmark Result Summary.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jsimnz jsimnz jsimnz approved these changes

@AndrewSisley AndrewSisley Awaiting requested review from AndrewSisley

@orpheuslummis orpheuslummis Awaiting requested review from orpheuslummis

Assignees

@shahzadlone shahzadlone

Labels
ci/build This is issue is about the build or CI system, and the administration of it.
Projects
None yet
Milestone
DefraDB v0.3
Development

Successfully merging this pull request may close these issues.

Comment on PR the results of the benchmark runs.
3 participants
@shahzadlone @source-devs @jsimnz