Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make coverage success optional in CI #670

Merged
merged 1 commit into from
Oct 11, 2024
Merged

Make coverage success optional in CI #670

merged 1 commit into from
Oct 11, 2024

Conversation

bantonsson
Copy link
Contributor

What does this PR do?

Allows merging of a PR even if Coverage fails

Motivation

Strange behavior with code coverage and library paths in CI leads to failures that are not related to the PRs.

Additional Notes

Anything else we should know when reviewing?

How to test the change?

Describe here in detail how the change can be validated.

@pr-commenter
Copy link

pr-commenter bot commented Oct 10, 2024

Benchmarks

Comparison

Benchmark execution time: 2024-10-10 20:34:12

Comparing candidate commit 687b3dc in PR branch ban/make-coverage-successs-optional with baseline commit f363618 in branch main.

Found 3 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 2 unstable metrics.

scenario:normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて

  • 🟩 execution_time [-16.831µs; -16.720µs] or [-3.466%; -3.443%]
  • 🟩 throughput [+73433.244op/s; +73921.637op/s] or [+3.566%; +3.590%]

scenario:tags/replace_trace_tags

  • 🟩 execution_time [-215.654ns; -206.778ns] or [-7.247%; -6.948%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 687b3dc 1728591804 ban/make-coverage-successs-optional
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.421µs 3.238µs ± 1.422µs 3.100µs ± 0.026µs 3.121µs 3.186µs 14.007µs 15.026µs 384.70% 7.607 58.118 43.81% 0.101µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [3.041µs; 3.435µs] or [-6.087%; +6.087%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 687b3dc 1728591804 ban/make-coverage-successs-optional
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 2.013µs 2.014µs ± 0.001µs 2.014µs ± 0.001µs 2.015µs 2.016µs 2.017µs 2.024µs 0.47% 4.700 40.758 0.05% 0.000µs 1 200
credit_card/is_card_number/ throughput 494133249.973op/s 496449574.615op/s ± 241994.029op/s 496462045.487op/s ± 140423.477op/s 496615016.038op/s 496669632.550op/s 496691684.747op/s 496718953.715op/s 0.05% -4.670 40.365 0.05% 17111.562op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 123.232µs 125.566µs ± 0.515µs 125.594µs ± 0.325µs 125.918µs 126.309µs 126.540µs 126.667µs 0.85% -0.851 2.087 0.41% 0.036µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 7894698.927op/s 7964093.083op/s ± 32769.007op/s 7962172.913op/s ± 20562.214op/s 7982716.026op/s 8023290.691op/s 8067788.116op/s 8114788.553op/s 1.92% 0.892 2.240 0.41% 2317.119op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 114.717µs 116.932µs ± 0.578µs 116.974µs ± 0.342µs 117.313µs 117.757µs 118.321µs 118.398µs 1.22% -0.362 0.947 0.49% 0.041µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 8446089.446op/s 8552168.432op/s ± 42385.738op/s 8548890.044op/s ± 24914.530op/s 8575485.689op/s 8620956.492op/s 8660274.155op/s 8717085.274op/s 1.97% 0.404 1.011 0.49% 2997.124op/s 1 200
credit_card/is_card_number/37828224631 execution_time 2.013µs 2.014µs ± 0.001µs 2.014µs ± 0.000µs 2.015µs 2.016µs 2.016µs 2.023µs 0.46% 4.862 42.825 0.05% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 494219621.246op/s 496472565.617op/s ± 232326.773op/s 496501936.358op/s ± 123058.185op/s 496627484.275op/s 496678558.426op/s 496710136.704op/s 496732023.528op/s 0.05% -4.833 42.430 0.05% 16427.984op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 110.341µs 112.933µs ± 0.715µs 112.967µs ± 0.476µs 113.398µs 114.079µs 114.262µs 114.358µs 1.23% -0.611 1.107 0.63% 0.051µs 1 200
credit_card/is_card_number/378282246310005 throughput 8744439.912op/s 8855177.166op/s ± 56326.310op/s 8852168.198op/s ± 37466.934op/s 8891041.440op/s 8949878.667op/s 9040904.458op/s 9062780.221op/s 2.38% 0.664 1.247 0.63% 3982.872op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 112.180µs 113.351µs ± 0.303µs 113.326µs ± 0.201µs 113.531µs 113.861µs 114.098µs 114.426µs 0.97% 0.292 1.236 0.27% 0.021µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 8739290.564op/s 8822181.939op/s ± 23535.597op/s 8824114.382op/s ± 15655.186op/s 8839031.908op/s 8853836.937op/s 8871425.530op/s 8914264.606op/s 1.02% -0.267 1.248 0.27% 1664.218op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 23.254µs 24.011µs ± 0.333µs 24.020µs ± 0.226µs 24.228µs 24.605µs 24.811µs 24.904µs 3.68% 0.176 -0.291 1.38% 0.024µs 1 200
credit_card/is_card_number/x371413321323331 throughput 40154051.893op/s 41655105.162op/s ± 576852.503op/s 41632813.013op/s ± 388138.232op/s 42062162.953op/s 42597315.556op/s 42824572.728op/s 43003604.408op/s 3.29% -0.107 -0.333 1.38% 40789.632op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 2.013µs 2.014µs ± 0.001µs 2.014µs ± 0.001µs 2.015µs 2.015µs 2.016µs 2.017µs 0.14% 0.829 0.570 0.03% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 495761315.640op/s 496472362.641op/s ± 167306.446op/s 496474232.996op/s ± 130144.014op/s 496615065.712op/s 496667610.419op/s 496685115.039op/s 496689536.727op/s 0.04% -0.827 0.562 0.03% 11830.352op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 97.433µs 98.351µs ± 0.422µs 98.330µs ± 0.284µs 98.641µs 98.994µs 99.377µs 99.955µs 1.65% 0.548 0.610 0.43% 0.030µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 10004504.970op/s 10167799.576op/s ± 43552.421op/s 10169785.073op/s ± 29507.607op/s 10198723.885op/s 10232494.413op/s 10251408.657op/s 10263425.919op/s 0.92% -0.519 0.541 0.43% 3079.621op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 88.818µs 90.386µs ± 0.662µs 90.330µs ± 0.463µs 90.808µs 91.515µs 91.885µs 91.956µs 1.80% 0.307 -0.458 0.73% 0.047µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 10874749.146op/s 11064206.083op/s ± 80809.377op/s 11070492.713op/s ± 56770.040op/s 11124854.173op/s 11179949.194op/s 11211291.405op/s 11258918.509op/s 1.70% -0.275 -0.475 0.73% 5714.086op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 2.013µs 2.014µs ± 0.001µs 2.014µs ± 0.001µs 2.015µs 2.016µs 2.016µs 2.029µs 0.73% 7.844 87.081 0.06% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 492906710.083op/s 496450932.491op/s ± 307621.160op/s 496502954.196op/s ± 128073.580op/s 496619672.926op/s 496668974.243op/s 496713590.321op/s 496724966.737op/s 0.04% -7.792 86.275 0.06% 21752.101op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 84.824µs 86.411µs ± 0.567µs 86.393µs ± 0.369µs 86.798µs 87.346µs 87.648µs 87.954µs 1.81% -0.054 0.045 0.66% 0.040µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 11369524.949op/s 11573040.098op/s ± 76039.848op/s 11575056.344op/s ± 49374.088op/s 11619367.822op/s 11691222.656op/s 11771873.188op/s 11789136.656op/s 1.85% 0.094 0.062 0.66% 5376.829op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 112.698µs 113.424µs ± 0.304µs 113.383µs ± 0.174µs 113.560µs 114.010µs 114.175µs 114.456µs 0.95% 0.752 0.569 0.27% 0.022µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 8736994.762op/s 8816506.754op/s ± 23601.819op/s 8819644.659op/s ± 13521.399op/s 8832809.923op/s 8846731.224op/s 8855420.784op/s 8873240.967op/s 0.61% -0.736 0.537 0.27% 1668.901op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 23.357µs 24.168µs ± 0.342µs 24.159µs ± 0.244µs 24.414µs 24.736µs 24.960µs 24.992µs 3.45% 0.086 -0.544 1.41% 0.024µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 40013125.078op/s 41384730.922op/s ± 584985.980op/s 41392705.513op/s ± 418532.352op/s 41790543.514op/s 42316382.728op/s 42560760.667op/s 42814536.631op/s 3.43% -0.025 -0.556 1.41% 41364.755op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [2.014µs; 2.014µs] or [-0.007%; +0.007%] None None None
credit_card/is_card_number/ throughput [496416036.570op/s; 496483112.660op/s] or [-0.007%; +0.007%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [125.494µs; 125.637µs] or [-0.057%; +0.057%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [7959551.614op/s; 7968634.553op/s] or [-0.057%; +0.057%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [116.852µs; 117.012µs] or [-0.069%; +0.069%] None None None
credit_card/is_card_number/ 378282246310005 throughput [8546294.176op/s; 8558042.688op/s] or [-0.069%; +0.069%] None None None
credit_card/is_card_number/37828224631 execution_time [2.014µs; 2.014µs] or [-0.007%; +0.007%] None None None
credit_card/is_card_number/37828224631 throughput [496440367.361op/s; 496504763.874op/s] or [-0.006%; +0.006%] None None None
credit_card/is_card_number/378282246310005 execution_time [112.834µs; 113.032µs] or [-0.088%; +0.088%] None None None
credit_card/is_card_number/378282246310005 throughput [8847370.881op/s; 8862983.451op/s] or [-0.088%; +0.088%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [113.310µs; 113.393µs] or [-0.037%; +0.037%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [8818920.131op/s; 8825443.746op/s] or [-0.037%; +0.037%] None None None
credit_card/is_card_number/x371413321323331 execution_time [23.965µs; 24.057µs] or [-0.192%; +0.192%] None None None
credit_card/is_card_number/x371413321323331 throughput [41575158.953op/s; 41735051.371op/s] or [-0.192%; +0.192%] None None None
credit_card/is_card_number_no_luhn/ execution_time [2.014µs; 2.014µs] or [-0.005%; +0.005%] None None None
credit_card/is_card_number_no_luhn/ throughput [496449175.577op/s; 496495549.705op/s] or [-0.005%; +0.005%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [98.293µs; 98.410µs] or [-0.059%; +0.059%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [10161763.630op/s; 10173835.523op/s] or [-0.059%; +0.059%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [90.295µs; 90.478µs] or [-0.101%; +0.101%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [11053006.681op/s; 11075405.486op/s] or [-0.101%; +0.101%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [2.014µs; 2.014µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [496408299.156op/s; 496493565.825op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [86.333µs; 86.490µs] or [-0.091%; +0.091%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [11562501.706op/s; 11583578.489op/s] or [-0.091%; +0.091%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [113.382µs; 113.467µs] or [-0.037%; +0.037%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [8813235.769op/s; 8819777.739op/s] or [-0.037%; +0.037%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [24.121µs; 24.216µs] or [-0.196%; +0.196%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [41303657.491op/s; 41465804.353op/s] or [-0.196%; +0.196%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 687b3dc 1728591804 ban/make-coverage-successs-optional
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 38.232µs 38.791µs ± 0.977µs 38.340µs ± 0.048µs 38.475µs 40.870µs 40.917µs 42.201µs 10.07% 1.736 1.194 2.51% 0.069µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [38.656µs; 38.926µs] or [-0.349%; +0.349%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 687b3dc 1728591804 ban/make-coverage-successs-optional
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 506.267µs 507.506µs ± 0.641µs 507.496µs ± 0.463µs 507.941µs 508.639µs 508.880µs 509.479µs 0.39% 0.309 -0.495 0.13% 0.045µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1962788.078op/s 1970423.073op/s ± 2489.123op/s 1970459.109op/s ± 1796.339op/s 1972276.865op/s 1974246.466op/s 1974568.173op/s 1975240.477op/s 0.24% -0.303 -0.502 0.13% 176.008op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 468.282µs 468.852µs ± 0.275µs 468.853µs ± 0.180µs 468.999µs 469.346µs 469.466µs 470.307µs 0.31% 0.948 3.103 0.06% 0.019µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2126270.713op/s 2132867.872op/s ± 1250.623op/s 2132863.475op/s ± 820.232op/s 2133769.551op/s 2134632.424op/s 2135095.140op/s 2135465.622op/s 0.12% -0.941 3.066 0.06% 88.432op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 180.657µs 180.963µs ± 0.129µs 180.950µs ± 0.073µs 181.038µs 181.186µs 181.304µs 181.447µs 0.27% 0.446 0.765 0.07% 0.009µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5511262.332op/s 5525981.476op/s ± 3935.127op/s 5526388.878op/s ± 2235.454op/s 5528426.563op/s 5532516.485op/s 5534429.641op/s 5535354.512op/s 0.16% -0.441 0.756 0.07% 278.255op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 44.516µs 44.619µs ± 0.039µs 44.619µs ± 0.028µs 44.647µs 44.682µs 44.701µs 44.716µs 0.22% -0.077 -0.336 0.09% 0.003µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 22363111.563op/s 22412009.825op/s ± 19722.819op/s 22411868.072op/s ± 14306.179op/s 22426407.081op/s 22442859.083op/s 22460500.595op/s 22463967.325op/s 0.23% 0.081 -0.334 0.09% 1394.614op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 49.035µs 49.175µs ± 0.086µs 49.171µs ± 0.025µs 49.196µs 49.250µs 49.321µs 50.219µs 2.13% 8.886 105.093 0.18% 0.006µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 19912754.669op/s 20335412.382op/s ± 35163.684op/s 20337342.751op/s ± 10340.157op/s 20347428.840op/s 20366857.434op/s 20380630.144op/s 20393438.657op/s 0.28% -8.738 102.750 0.17% 2486.448op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [507.417µs; 507.595µs] or [-0.018%; +0.018%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1970078.105op/s; 1970768.042op/s] or [-0.018%; +0.018%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [468.814µs; 468.891µs] or [-0.008%; +0.008%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2132694.547op/s; 2133041.196op/s] or [-0.008%; +0.008%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [180.946µs; 180.981µs] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5525436.105op/s; 5526526.847op/s] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [44.614µs; 44.624µs] or [-0.012%; +0.012%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [22409276.432op/s; 22414743.218op/s] or [-0.012%; +0.012%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [49.163µs; 49.187µs] or [-0.024%; +0.024%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [20330539.034op/s; 20340285.730op/s] or [-0.024%; +0.024%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 687b3dc 1728591804 ban/make-coverage-successs-optional
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 8.953ms 8.989ms ± 0.027ms 8.988ms ± 0.009ms 8.996ms 9.011ms 9.031ms 9.278ms 3.23% 6.882 68.183 0.30% 0.002ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [8.985ms; 8.993ms] or [-0.041%; +0.041%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 687b3dc 1728591804 ban/make-coverage-successs-optional
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 290.439ns 302.532ns ± 15.577ns 294.778ns ± 3.387ns 306.834ns 334.927ns 345.855ns 348.421ns 18.20% 1.445 0.834 5.14% 1.101ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [300.373ns; 304.691ns] or [-0.714%; +0.714%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 687b3dc 1728591804 ban/make-coverage-successs-optional
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 1.119µs 1.183µs ± 0.025µs 1.188µs ± 0.016µs 1.203µs 1.207µs 1.209µs 1.209µs 1.74% -1.111 0.362 2.08% 0.002µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [1.180µs; 1.186µs] or [-0.289%; +0.289%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 687b3dc 1728591804 ban/make-coverage-successs-optional
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 269.772µs 271.006µs ± 0.555µs 270.943µs ± 0.293µs 271.269µs 271.896µs 272.764µs 274.057µs 1.15% 1.257 4.552 0.20% 0.039µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 3648871.082op/s 3689973.304op/s ± 7534.318op/s 3690810.700op/s ± 3993.078op/s 3694485.051op/s 3701140.504op/s 3704175.919op/s 3706829.797op/s 0.43% -1.227 4.397 0.20% 532.757op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 26.560µs 26.611µs ± 0.046µs 26.591µs ± 0.017µs 26.641µs 26.709µs 26.737µs 26.780µs 0.71% 1.260 0.710 0.17% 0.003µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 37341880.475op/s 37578025.643op/s ± 65311.394op/s 37607311.419op/s ± 24128.356op/s 37624867.529op/s 37637741.064op/s 37643134.097op/s 37650536.782op/s 0.11% -1.254 0.687 0.17% 4618.213op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 16.107µs 16.138µs ± 0.029µs 16.122µs ± 0.010µs 16.156µs 16.190µs 16.208µs 16.371µs 1.55% 2.976 18.929 0.18% 0.002µs 1 200
normalization/normalize_name/normalize_name/good throughput 61081773.350op/s 61966711.641op/s ± 111689.300op/s 62028919.834op/s ± 36716.889op/s 62045944.509op/s 62065178.286op/s 62072115.045op/s 62083788.486op/s 0.09% -2.912 18.227 0.18% 7897.626op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [270.929µs; 271.083µs] or [-0.028%; +0.028%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [3688929.120op/s; 3691017.488op/s] or [-0.028%; +0.028%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [26.605µs; 26.618µs] or [-0.024%; +0.024%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [37568974.112op/s; 37587077.174op/s] or [-0.024%; +0.024%] None None None
normalization/normalize_name/normalize_name/good execution_time [16.134µs; 16.142µs] or [-0.025%; +0.025%] None None None
normalization/normalize_name/normalize_name/good throughput [61951232.579op/s; 61982190.704op/s] or [-0.025%; +0.025%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 687b3dc 1728591804 ban/make-coverage-successs-optional
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 74.279µs 74.415µs ± 0.162µs 74.392µs ± 0.037µs 74.430µs 74.543µs 74.842µs 76.380µs 2.67% 9.267 106.771 0.22% 0.011µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [74.392µs; 74.437µs] or [-0.030%; +0.030%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 687b3dc 1728591804 ban/make-coverage-successs-optional
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.736µs 2.765µs ± 0.019µs 2.758µs ± 0.010µs 2.775µs 2.806µs 2.813µs 2.815µs 2.04% 0.939 0.113 0.70% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.762µs; 2.767µs] or [-0.097%; +0.097%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 687b3dc 1728591804 ban/make-coverage-successs-optional
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 19.775µs 25.475µs ± 10.728µs 20.305µs ± 0.175µs 30.960µs 39.665µs 41.577µs 105.692µs 420.53% 3.592 20.256 42.01% 0.759µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [23.988µs; 26.962µs] or [-5.836%; +5.836%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 687b3dc 1728591804 ban/make-coverage-successs-optional
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 137.445µs 138.248µs ± 0.321µs 138.265µs ± 0.169µs 138.417µs 138.715µs 139.157µs 140.047µs 1.29% 0.797 4.947 0.23% 0.023µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [138.203µs; 138.292µs] or [-0.032%; +0.032%] None None None

Baseline

Omitted due to size.

@bantonsson bantonsson force-pushed the ban/make-coverage-successs-optional branch from 687b3dc to a248819 Compare October 10, 2024 20:52
@bantonsson bantonsson marked this pull request as ready for review October 10, 2024 21:04
@bantonsson bantonsson requested a review from a team as a code owner October 10, 2024 21:04
@bwoebi bwoebi merged commit dab84ba into main Oct 11, 2024
29 of 30 checks passed
@bwoebi bwoebi deleted the ban/make-coverage-successs-optional branch October 11, 2024 02:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bwoebi bwoebi bwoebi approved these changes

Assignees
No one assigned
Labels
ci-build
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@bantonsson @bwoebi