Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Set CurrentCodeHash before running some dispatchable benchmarks #4645

Merged
merged 5 commits into from
Jan 3, 2022
Merged

Set CurrentCodeHash before running some dispatchable benchmarks #4645

merged 5 commits into from
Jan 3, 2022

Conversation

KiChjang
Copy link
Contributor

Aims to fix #4616.

@github-actions github-actions bot added the A0-please_review Pull request needs code review. label Dec 31, 2021
@KiChjang KiChjang added B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit. labels Dec 31, 2021
@KiChjang
Copy link
Contributor Author

/benchmark runtime westend runtime_parachains::paras

@parity-benchapp
Copy link

parity-benchapp bot commented Dec 31, 2021 •
edited
Loading

Benchmark Runtime Westend Pallet for branch "kckyeung/fix-paras-benchmarks" with command cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

Results 
error: failed to parse manifest at `/home/benchbot/bench-bot/git/polkadot/Cargo.toml`

Caused by:
  feature `named-profiles` is required

  The package requires the Cargo feature called `named-profiles`, but that feature is not stabilized in this version of Cargo (1.56.0 (4ed5d137b 2021-10-04)).
  Consider trying a newer version of Cargo (this may require the nightly release).
  See https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#custom-named-profiles for more information about the status of this feature.

ERROR: Unable to commit file ./runtime/westend/src/weights/runtime_parachains_paras.rs

@shawntabrizi
Copy link
Member

/benchmark runtime westend runtime_parachains::paras

@parity-benchapp
Copy link

parity-benchapp bot commented Dec 31, 2021 •
edited
Loading

Benchmark Runtime Westend Pallet for branch "kckyeung/fix-paras-benchmarks" with command cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic="*" --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

Results 
Pallet: "runtime_parachains::paras", Extrinsic: "force_set_current_code", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras CurrentCodeHash (r:1 w:1)
Storage: Paras CodeByHashRefs (r:1 w:1)
Storage: Paras PastCodeMeta (r:1 w:1)
Storage: Paras PastCodePruning (r:1 w:1)
Storage: Paras PastCodeHash (r:0 w:1)
Storage: Paras CodeByHash (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 4 + (0 * c)
Writes = 6 + (0 * c)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    c   mean µs  sigma µs       %
    1     37.97     0.099    0.2%
62915     175.3      1.78    1.0%
125829     297.8     0.664    0.2%
188743     415.1     0.795    0.1%
251657     534.5     0.692    0.1%
314571       655     1.019    0.1%
377485     777.3     7.746    0.9%
440399     904.6     11.47    1.2%
503313      1019     12.12    1.1%
566227      1140     12.19    1.0%
629141      1269     13.46    1.0%
692055      1388     13.99    1.0%
754969      1514      14.6    0.9%
817883      1683     4.165    0.2%
880797      1776     14.97    0.8%
943711      1888     18.33    0.9%
1006625      2003     14.32    0.7%
1069539      2132     12.07    0.5%
1132453      2260     12.24    0.5%
1195367      2403     14.58    0.6%
1258281      2512     13.03    0.5%
1321195      2639     17.07    0.6%
1384109      2780     9.677    0.3%
1447023      2962     14.09    0.4%
1509937      3091     4.589    0.1%
1572851      3295     99.73    3.0%
1635765      3293     8.151    0.2%
1698679      3413     12.51    0.3%
1761593      3571     10.88    0.3%
1824507      3699     11.94    0.3%
1887421      3826     11.73    0.3%
1950335      3963     14.94    0.3%
2013249      4080     11.92    0.2%
2076163      4233     24.44    0.5%
2139077      4379     9.167    0.2%
2201991      4494     14.61    0.3%
2264905      4619     17.69    0.3%
2327819      4756     16.97    0.3%
2390733      4888     13.21    0.2%
2453647      5024     13.52    0.2%
2516561      5158     37.42    0.7%
2579475      5291     21.31    0.4%
2642389      5446      29.3    0.5%
2705303      5538     18.81    0.3%
2768217      5674     16.41    0.2%
2831131      5827     15.29    0.2%
2894045      5954     17.58    0.2%
2956959      6129     14.23    0.2%
3019873      6207     11.36    0.1%
3082787      6354     13.47    0.2%
3145701      6511     20.52    0.3%

Quality and confidence:
param     error
c             0

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 4 + (0 * c)
Writes = 6 + (0 * c)

Pallet: "runtime_parachains::paras", Extrinsic: "force_set_current_head", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras Heads (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    16.29
    + s        0
              µs

Reads = 0 + (0 * s)
Writes = 1 + (0 * s)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    s   mean µs  sigma µs       %
    1     11.74     0.077    0.6%
20972     23.54     0.205    0.8%
41943     50.67     0.171    0.3%
62914     67.49     0.292    0.4%
83885     83.33     0.748    0.8%
104856     100.1     0.307    0.3%
125827     116.1     0.247    0.2%
146798     130.1      0.42    0.3%
167769     142.7      0.16    0.1%
188740     157.5     0.235    0.1%
209711     173.2     0.254    0.1%
230682     192.5     0.194    0.1%
251653     203.3     0.314    0.1%
272624     218.7     0.404    0.1%
293595     234.3     0.589    0.2%
314566     249.4     0.383    0.1%
335537     264.8     0.783    0.2%
356508       280     0.562    0.2%
377479     295.4     1.066    0.3%
398450     310.6     0.386    0.1%
419421     327.8     2.729    0.8%
440392     341.2      0.77    0.2%
461363     356.9     1.209    0.3%
482334     371.8     0.586    0.1%
503305     391.4     4.301    1.0%
524276     407.4     9.327    2.2%
545247     417.3     1.228    0.2%
566218       436     8.795    2.0%
587189     449.2     1.513    0.3%
608160     466.2      4.96    1.0%
629131     480.4     0.912    0.1%
650102     496.5     1.843    0.3%
671073     516.5     8.147    1.5%
692044     526.3     2.357    0.4%
713015     544.7     1.349    0.2%
733986       563     8.302    1.4%
754957     576.2      4.19    0.7%
775928     590.5     1.662    0.2%
796899     612.3     8.408    1.3%
817870     622.7     1.907    0.3%
838841     641.3     5.648    0.8%
859812     669.5     15.13    2.2%
880783     684.3     11.75    1.7%
901754     690.2     8.218    1.1%
922725       712     11.61    1.6%
943696     722.4     2.139    0.2%
964667     738.5     3.192    0.4%
985638       767     14.73    1.9%
1006609     782.1     12.93    1.6%
1027580     799.6     14.91    1.8%
1048551     809.8      7.74    0.9%

Quality and confidence:
param     error
s             0

Model:
Time ~=    13.71
    + s    0.001
              µs

Reads = 0 + (0 * s)
Writes = 1 + (0 * s)

Pallet: "runtime_parachains::paras", Extrinsic: "force_schedule_code_upgrade", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras FutureCodeHash (r:1 w:1)
Storage: Paras CurrentCodeHash (r:1 w:0)
Storage: Paras UpgradeCooldowns (r:1 w:1)
Storage: Paras PvfActiveVoteMap (r:1 w:0)
Storage: Paras CodeByHash (r:1 w:1)
Storage: Paras UpcomingUpgrades (r:1 w:1)
Storage: System Digest (r:1 w:1)
Storage: Paras CodeByHashRefs (r:1 w:1)
Storage: Paras FutureCodeUpgrades (r:0 w:1)
Storage: Paras UpgradeRestrictionSignal (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 8 + (0 * c)
Writes = 8 + (0 * c)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    c   mean µs  sigma µs       %
    1      62.5      0.24    0.3%
62915     196.7     0.666    0.3%
125829     321.8     0.494    0.1%
188743     438.6     0.782    0.1%
251657     561.1     3.965    0.7%
314571     692.4     14.68    2.1%
377485     805.1      11.6    1.4%
440399     924.6     8.596    0.9%
503313      1060     14.61    1.3%
566227      1167     12.24    1.0%
629141      1287     14.66    1.1%
692055      1414     15.48    1.0%
754969      1542     16.84    1.0%
817883      1669     15.08    0.9%
880797      1786     15.69    0.8%
943711      1920     13.63    0.7%
1006625      2058     11.72    0.5%
1069539      2175     14.99    0.6%
1132453      2293     11.18    0.4%
1195367      2439     13.94    0.5%
1258281      2582     9.834    0.3%
1321195      2704     15.18    0.5%
1384109      2830     17.63    0.6%
1447023      2956     17.11    0.5%
1509937      3099     9.122    0.2%
1572851      3221     10.16    0.3%
1635765      3375     12.07    0.3%
1698679      3501     17.02    0.4%
1761593      3631     18.32    0.5%
1824507      3783     22.68    0.5%
1887421      3901     14.52    0.3%
1950335      4064     9.376    0.2%
2013249      4186     19.93    0.4%
2076163      4319     9.993    0.2%
2139077      4459     12.58    0.2%
2201991      4564      13.7    0.3%
2264905      4729     12.13    0.2%
2327819      4877     23.49    0.4%
2390733      4998     13.29    0.2%
2453647      5133     12.79    0.2%
2516561      5261     14.41    0.2%
2579475      5452     42.31    0.7%
2642389      5554     24.04    0.4%
2705303      5686     26.65    0.4%
2768217      5833     20.13    0.3%
2831131      5942     9.416    0.1%
2894045      6080     11.48    0.1%
2956959      6218     23.79    0.3%
3019873      6355     19.97    0.3%
3082787      6498     14.89    0.2%
3145701      6614     19.25    0.2%

Quality and confidence:
param     error
c             0

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 8 + (0 * c)
Writes = 8 + (0 * c)

Pallet: "runtime_parachains::paras", Extrinsic: "force_note_new_head", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras FutureCodeUpgrades (r:1 w:0)
Storage: Paras Heads (r:0 w:1)
Storage: Paras UpgradeGoAheadSignal (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    22.11
    + s        0
              µs

Reads = 1 + (0 * s)
Writes = 2 + (0 * s)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    s   mean µs  sigma µs       %
    1      16.6     0.086    0.5%
20972     29.05      0.13    0.4%
41943     56.43     0.151    0.2%
62914     72.83     0.227    0.3%
83885        90     0.149    0.1%
104856     104.9     0.991    0.9%
125827     119.7     0.198    0.1%
146798     132.8      0.42    0.3%
167769     147.5     0.183    0.1%
188740     162.9     0.201    0.1%
209711     178.4     0.372    0.2%
230682     194.7     1.478    0.7%
251653     210.5     1.848    0.8%
272624     229.8     0.924    0.4%
293595     239.8     0.443    0.1%
314566     254.9     0.214    0.0%
335537     269.9     0.133    0.0%
356508     284.8     0.146    0.0%
377479     299.9     0.267    0.0%
398450     321.3     2.903    0.9%
419421     330.7     0.383    0.1%
440392     346.1     0.452    0.1%
461363     362.5     2.108    0.5%
482334     376.4     0.415    0.1%
503305     403.2     1.207    0.2%
524276     414.6     3.623    0.8%
545247     423.2     1.324    0.3%
566218     446.2     8.313    1.8%
587189     453.9     0.534    0.1%
608160     470.7     1.542    0.3%
629131     493.6     6.123    1.2%
650102     500.5     0.848    0.1%
671073     516.7     2.004    0.3%
692044     530.6     0.795    0.1%
713015       551     8.613    1.5%
733986     561.3     0.846    0.1%
754957     597.2      1.83    0.3%
775928     596.1     4.387    0.7%
796899     616.4     7.169    1.1%
817870     642.1     14.95    2.3%
838841     643.3     1.065    0.1%
859812     659.6     0.996    0.1%
880783     681.3     7.121    1.0%
901754     710.9     13.06    1.8%
922725     722.7     11.02    1.5%
943696     732.1     8.556    1.1%
964667     760.2     17.62    2.3%
985638     762.5     5.054    0.6%
1006609     795.2     11.57    1.4%
1027580     801.8     11.11    1.3%
1048551     819.2     9.688    1.1%

Quality and confidence:
param     error
s             0

Model:
Time ~=    18.54
    + s    0.001
              µs

Reads = 1 + (0 * s)
Writes = 2 + (0 * s)

Pallet: "runtime_parachains::paras", Extrinsic: "force_queue_action", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: ParasShared CurrentSessionIndex (r:1 w:0)
Storage: Paras ActionsQueue (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    22.15
              µs

Reads = 2
Writes = 1

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    22.15
              µs

Reads = 2
Writes = 1

Pallet: "runtime_parachains::paras", Extrinsic: "add_trusted_validation_code", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras PvfActiveVoteMap (r:1 w:0)
Storage: Paras CodeByHash (r:1 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 2 + (0 * c)
Writes = 1 + (0 * c)

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    c   mean µs  sigma µs       %
    1     7.045     0.077    1.0%
62915     142.7     0.545    0.3%
125829     270.2     1.425    0.5%
188743       379     0.545    0.1%
251657     500.3      4.05    0.8%
314571     627.2      13.1    2.0%
377485     756.9     15.76    2.0%
440399     866.4     10.94    1.2%
503313     982.4     11.69    1.1%
566227      1110     13.35    1.2%
629141      1226      10.3    0.8%
692055      1343     12.96    0.9%
754969      1482     14.12    0.9%
817883      1598     11.27    0.7%
880797      1725     15.29    0.8%
943711      1846     13.22    0.7%
1006625      1983     15.06    0.7%
1069539      2103      12.2    0.5%
1132453      2228     15.51    0.6%
1195367      2342     12.18    0.5%
1258281      2494     13.54    0.5%
1321195      2620     12.47    0.4%
1384109      2750     15.56    0.5%
1447023      2881     15.62    0.5%
1509937      3033     12.42    0.4%
1572851      3151     13.29    0.4%
1635765      3309     18.63    0.5%
1698679      3419      11.6    0.3%
1761593      3545     12.86    0.3%
1824507      3693     17.48    0.4%
1887421      3817     7.391    0.1%
1950335      4027     12.42    0.3%
2013249      4088     14.56    0.3%
2076163      4219     8.912    0.2%
2139077      4365     26.07    0.5%
2201991      4504      24.8    0.5%
2264905      4652     26.07    0.5%
2327819      4785     13.95    0.2%
2390733      4914     10.54    0.2%
2453647      5047     14.41    0.2%
2516561      5193     22.74    0.4%
2579475      5334     13.36    0.2%
2642389      5474     11.91    0.2%
2705303      5570     17.88    0.3%
2768217      5734     32.18    0.5%
2831131      5861     12.76    0.2%
2894045      5991     16.77    0.2%
2956959      6116     25.46    0.4%
3019873      6278      17.3    0.2%
3082787      6416     35.39    0.5%
3145701      6542     19.15    0.2%

Quality and confidence:
param     error
c             0

Model:
Time ~=        0
    + c    0.002
              µs

Reads = 2 + (0 * c)
Writes = 1 + (0 * c)

Pallet: "runtime_parachains::paras", Extrinsic: "poke_unused_validation_code", Lowest values: [], Highest values: [], Steps: 50, Repeat: 20
Raw Storage Info
========
Storage: Paras CodeByHashRefs (r:1 w:0)
Storage: Paras CodeByHash (r:0 w:1)

Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=    4.207
              µs

Reads = 1
Writes = 1

Min Squares Analysis
========
-- Extrinsic Time --

Model:
Time ~=    4.207
              µs

Reads = 1
Writes = 1

Parity Bot added 2 commits December 31, 2021 23:33
Merge branch 'master' of https://github.com/paritytech/polkadot into …
3598bec 
…kckyeung/fix-paras-benchmarks
cargo run --quiet --release --features=runtime-benchmarks -- benchmar…
b5dd50f 
…k --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs
@shawntabrizi
Copy link
Member

/benchmark rustup update

@parity-benchapp
Copy link

parity-benchapp bot commented Jan 1, 2022 •
edited
Loading

Benchmark Rustup Update for branch "kckyeung/fix-paras-benchmarks" with command rustup update

Toolchain: stable-x86_64-unknown-linux-gnu (default)
rustc 1.57.0 (f1edd0429 2021-11-29)

Results 

   stable-x86_64-unknown-linux-gnu unchanged - rustc 1.57.0 (f1edd0429 2021-11-29)
  nightly-x86_64-unknown-linux-gnu unchanged - rustc 1.59.0-nightly (b60e32c82 2021-12-30)

@paritytech paritytech deleted a comment from parity-benchapp bot Jan 1, 2022
@paritytech paritytech deleted a comment from parity-benchapp bot Jan 1, 2022
@paritytech paritytech deleted a comment from parity-benchapp bot Jan 1, 2022
@shawntabrizi
Copy link
Member

bot merge

@paritytech-processbot paritytech-processbot bot merged commit 2f4b417 into master Jan 3, 2022
@paritytech-processbot paritytech-processbot bot deleted the kckyeung/fix-paras-benchmarks branch January 3, 2022 02:05
ordian added a commit that referenced this pull request Jan 3, 2022
@ordian
Merge branch 'master' into ao-lower-collator-inactivity-period
048e7c3 
* master:
  Set CurrentCodeHash before running some dispatchable benchmarks (#4645)
  paras: split tests (#4636)
  Bump quote from 1.0.10 to 1.0.14 (#4632)
  Bump pin-project from 1.0.8 to 1.0.9 (#4606)
  chore: fix copy&paste and tidy comments (#4646)
  derive Copy and Clone for Upgrade signals (#4637) (#4647)
  paras: fix upgrade restriction signal (#4603)
  configuration: Rename validation_upgrade_{frequency -> cooldown} (#4635)
  Bump lru from 0.7.1 to 0.7.2 (#4633)
  paras: add governance control dispatchables (#4575)
drahnr pushed a commit that referenced this pull request Jan 4, 2022
@KiChjang @drahnr
Set CurrentCodeHash before running some dispatchable benchmarks (#4645)
57397cb 
* Set CurrentCodeHash before running some dispatchable benchmarks

* Use insert instead of put

* Actually hash the ValidationCode

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

Co-authored-by: Parity Bot <[email protected]>
Wizdave97 pushed a commit to ComposableFi/polkadot that referenced this pull request Feb 3, 2022
@KiChjang @Wizdave97
Set CurrentCodeHash before running some dispatchable benchmarks (pari…
95c066a 
…tytech#4645)

* Set CurrentCodeHash before running some dispatchable benchmarks

* Use insert instead of put

* Actually hash the ValidationCode

* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras.rs

Co-authored-by: Parity Bot <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@emostov emostov emostov approved these changes

@shawntabrizi shawntabrizi shawntabrizi approved these changes

@kianenigma kianenigma Awaiting requested review from kianenigma

@gui1117 gui1117 Awaiting requested review from gui1117

Assignees
No one assigned
Labels
A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Paras benchmarks log about inconsistencies
3 participants
@KiChjang @shawntabrizi @emostov