Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add build system infrastructure. #586

Merged
merged 9 commits into from
Sep 5, 2024
Merged

Add build system infrastructure. #586

merged 9 commits into from
Sep 5, 2024

Conversation

hoolioh
Copy link
Contributor

@hoolioh hoolioh commented Aug 13, 2024 •
edited
Loading

What does this PR do?

Adds a new build system as described in RFC 0002.

How to test the change?

The builder crate has been excluded from the default members in the workspace so it's not built when 'build' command is issued with no specific package.

export LIBDD_OUTPUT_FOLDER=<path_to_release>
cargo build -p builder--features profiling,data-pipeline,symbolizer,crashtracker --release

@codecov-commenter
Copy link

codecov-commenter commented Aug 13, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 0% with 146 lines in your changes missing coverage. Please review.

Project coverage is 72.69%. Comparing base (c035537) to head (498e0ed).

Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #586      +/-   ##
==========================================
- Coverage   72.93%   72.69%   -0.25%     
==========================================
  Files         245      246       +1     
  Lines       34951    34940      -11     
==========================================
- Hits        25490    25398      -92     
- Misses       9461     9542      +81
Components Coverage Δ
crashtracker 20.44% <ø> (ø)
datadog-alloc 98.73% <ø> (ø)
data-pipeline 90.12% <ø> (ø)
data-pipeline-ffi 0.00% <ø> (ø)
ddcommon 82.82% <ø> (ø)
ddcommon-ffi 68.11% <ø> (ø)
ddtelemetry 59.02% <ø> (ø)
ipc 83.61% <ø> (ø)
profiling 84.26% <ø> (ø)
profiling-ffi 77.42% <ø> (ø)
serverless 0.00% <ø> (ø)
sidecar 40.13% <ø> (ø)
sidecar-ffi 0.00% <ø> (ø)
spawn-worker 50.36% <ø> (ø)
tinybytes 91.66% <ø> (ø)
trace-mini-agent 70.88% <ø> (ø)
trace-normalization 98.25% <ø> (ø)
trace-obfuscation 95.73% <ø> (ø)
trace-protobuf 77.67% <ø> (ø)
trace-utils 93.42% <ø> (+0.41%) ⬆️

@pr-commenter
Copy link

pr-commenter bot commented Aug 13, 2024 •
edited
Loading

Benchmarks

Comparison

Benchmark execution time: 2024-09-05 08:13:47

Comparing candidate commit 498e0ed in PR branch julio/build-system with baseline commit c035537 in branch main.

Found 7 performance improvements and 13 performance regressions! Performance is the same for 31 metrics, 2 unstable metrics.

scenario:credit_card/is_card_number/

  • 🟥 execution_time [+1.201µs; +1.202µs] or [+99.059%; +99.080%]
  • 🟥 throughput [-410376538.857op/s; -410298795.113op/s] or [-49.771%; -49.761%]

scenario:credit_card/is_card_number/37828224631

  • 🟥 execution_time [+1.201µs; +1.203µs] or [+99.091%; +99.216%]
  • 🟥 throughput [-410889557.731op/s; -410394110.745op/s] or [-49.818%; -49.757%]

scenario:credit_card/is_card_number/37828224631000521389798

  • 🟩 execution_time [-7.880µs; -7.760µs] or [-7.155%; -7.046%]
  • 🟩 throughput [+688869.088op/s; +699275.109op/s] or [+7.586%; +7.701%]

scenario:credit_card/is_card_number/x371413321323331

  • 🟥 execution_time [+1.047µs; +1.237µs] or [+4.772%; +5.637%]
  • 🟥 throughput [-2441658.126op/s; -2065642.491op/s] or [-5.355%; -4.530%]

scenario:credit_card/is_card_number_no_luhn/

  • 🟥 execution_time [+1.201µs; +1.202µs] or [+99.073%; +99.098%]
  • 🟥 throughput [-410515206.697op/s; -410347221.103op/s] or [-49.781%; -49.760%]

scenario:credit_card/is_card_number_no_luhn/37828224631

  • 🟥 execution_time [+1.202µs; +1.202µs] or [+99.139%; +99.160%]
  • 🟥 throughput [-410720143.638op/s; -410615886.857op/s] or [-49.793%; -49.780%]

scenario:credit_card/is_card_number_no_luhn/37828224631000521389798

  • 🟩 execution_time [-7.911µs; -7.789µs] or [-7.183%; -7.073%]
  • 🟩 throughput [+691670.731op/s; +702119.069op/s] or [+7.617%; +7.732%]

scenario:credit_card/is_card_number_no_luhn/x371413321323331

  • 🟥 execution_time [+1.069µs; +1.258µs] or [+4.864%; +5.724%]
  • 🟥 throughput [-2480713.948op/s; -2105780.086op/s] or [-5.447%; -4.624%]

scenario:normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...

  • 🟩 execution_time [-11.954µs; -11.352µs] or [-3.836%; -3.643%]
  • 🟩 throughput [+121473.558op/s; +128119.660op/s] or [+3.785%; +3.993%]

scenario:normalization/normalize_name/normalize_name/bad-name

  • 🟥 execution_time [+564.679ns; +633.074ns] or [+2.001%; +2.244%]

scenario:tags/replace_trace_tags

  • 🟩 execution_time [-102.776ns; -91.062ns] or [-3.714%; -3.291%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 498e0ed 1725523258 julio/build-system
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 2.413µs 2.414µs ± 0.001µs 2.414µs ± 0.001µs 2.415µs 2.416µs 2.417µs 2.419µs 0.21% 1.599 6.332 0.03% 0.000µs 1 200
credit_card/is_card_number/ throughput 413328316.034op/s 414193294.476op/s ± 143403.490op/s 414211422.877op/s ± 91586.122op/s 414303785.675op/s 414369837.639op/s 414426816.162op/s 414439674.757op/s 0.06% -1.593 6.291 0.03% 10140.158op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 111.164µs 113.371µs ± 0.900µs 113.420µs ± 0.522µs 113.918µs 114.723µs 115.851µs 116.729µs 2.92% 0.150 0.726 0.79% 0.064µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 8566823.388op/s 8821150.502op/s ± 69938.644op/s 8816821.666op/s ± 40394.162op/s 8862872.100op/s 8946043.687op/s 8977881.307op/s 8995741.679op/s 2.03% -0.087 0.634 0.79% 4945.409op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 107.070µs 107.864µs ± 0.460µs 107.805µs ± 0.294µs 108.122µs 108.699µs 109.117µs 110.112µs 2.14% 1.162 2.580 0.43% 0.033µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 9081624.173op/s 9271091.372op/s ± 39322.895op/s 9276041.839op/s ± 25309.146op/s 9298661.923op/s 9320461.397op/s 9331894.178op/s 9339653.366op/s 0.69% -1.121 2.389 0.42% 2780.549op/s 1 200
credit_card/is_card_number/37828224631 execution_time 2.413µs 2.415µs ± 0.005µs 2.414µs ± 0.001µs 2.415µs 2.415µs 2.417µs 2.483µs 2.84% 13.524 185.429 0.20% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 402801365.829op/s 414147228.004op/s ± 816927.421op/s 414225785.555op/s ± 86161.533op/s 414299566.544op/s 414361829.012op/s 414379795.664op/s 414424742.538op/s 0.05% -13.496 184.910 0.20% 57765.492op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 104.397µs 105.068µs ± 0.438µs 104.984µs ± 0.234µs 105.248µs 105.892µs 106.489µs 106.890µs 1.82% 1.312 2.287 0.42% 0.031µs 1 200
credit_card/is_card_number/378282246310005 throughput 9355420.311op/s 9517843.325op/s ± 39494.744op/s 9525270.916op/s ± 21236.302op/s 9543280.944op/s 9568181.682op/s 9576082.023op/s 9578853.723op/s 0.56% -1.280 2.161 0.41% 2792.700op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 101.802µs 102.303µs ± 0.246µs 102.260µs ± 0.151µs 102.443µs 102.742µs 102.989µs 103.271µs 0.99% 0.904 0.975 0.24% 0.017µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 9683299.776op/s 9774976.797op/s ± 23465.597op/s 9778951.787op/s ± 14439.633op/s 9792440.726op/s 9804544.750op/s 9819814.088op/s 9822944.434op/s 0.45% -0.888 0.931 0.24% 1659.268op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 21.968µs 23.082µs ± 0.511µs 23.104µs ± 0.387µs 23.431µs 23.924µs 24.166µs 24.241µs 4.92% 0.076 -0.613 2.21% 0.036µs 1 200
credit_card/is_card_number/x371413321323331 throughput 41252423.580op/s 43344957.645op/s ± 959457.891op/s 43281802.651op/s ± 733624.847op/s 44054202.306op/s 44885401.544op/s 45355479.520op/s 45520772.024op/s 5.17% 0.016 -0.621 2.21% 67843.918op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 2.413µs 2.414µs ± 0.001µs 2.414µs ± 0.000µs 2.415µs 2.415µs 2.415µs 2.416µs 0.06% 0.294 -0.826 0.03% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 413978380.196op/s 414219121.331op/s ± 103999.558op/s 414225584.376op/s ± 80841.691op/s 414306048.879op/s 414358045.353op/s 414405475.725op/s 414432203.873op/s 0.05% -0.293 -0.827 0.03% 7353.879op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 89.026µs 90.023µs ± 0.569µs 89.956µs ± 0.330µs 90.300µs 91.030µs 91.865µs 92.401µs 2.72% 1.235 2.435 0.63% 0.040µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 10822351.490op/s 11108706.875op/s ± 69627.167op/s 11116606.566op/s ± 40948.006op/s 11157244.194op/s 11198532.214op/s 11220845.840op/s 11232671.328op/s 1.04% -1.181 2.240 0.63% 4923.384op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 85.386µs 86.187µs ± 0.382µs 86.145µs ± 0.203µs 86.377µs 86.731µs 87.039µs 89.033µs 3.35% 2.219 14.272 0.44% 0.027µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 11231786.147op/s 11602934.189op/s ± 50934.871op/s 11608391.222op/s ± 27320.709op/s 11631077.174op/s 11670678.862op/s 11698097.621op/s 11711555.792op/s 0.89% -2.073 12.978 0.44% 3601.639op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 2.413µs 2.414µs ± 0.001µs 2.414µs ± 0.001µs 2.415µs 2.415µs 2.416µs 2.417µs 0.11% 0.465 0.031 0.03% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 413747487.141op/s 414190202.221op/s ± 124716.853op/s 414191743.471op/s ± 96383.246op/s 414289558.195op/s 414369016.653op/s 414411291.672op/s 414434158.262op/s 0.06% -0.464 0.027 0.03% 8818.813op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 82.259µs 83.064µs ± 0.285µs 83.031µs ± 0.154µs 83.213µs 83.585µs 83.934µs 84.160µs 1.36% 0.594 1.305 0.34% 0.020µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 11882173.617op/s 12039048.860op/s ± 41258.711op/s 12043681.345op/s ± 22288.093op/s 12063004.576op/s 12102499.054op/s 12123678.431op/s 12156706.172op/s 0.94% -0.563 1.253 0.34% 2917.431op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 101.760µs 102.277µs ± 0.233µs 102.252µs ± 0.126µs 102.376µs 102.668µs 103.090µs 103.334µs 1.06% 1.262 2.934 0.23% 0.016µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 9677353.098op/s 9777448.694op/s ± 22182.405op/s 9779732.970op/s ± 11994.344op/s 9792079.432op/s 9806279.795op/s 9811977.031op/s 9827051.908op/s 0.48% -1.240 2.841 0.23% 1568.533op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 21.881µs 23.133µs ± 0.474µs 23.097µs ± 0.312µs 23.440µs 23.953µs 24.324µs 24.603µs 6.52% 0.249 0.031 2.04% 0.034µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 40645630.737op/s 43245898.732op/s ± 882516.904op/s 43295683.761op/s ± 582977.168op/s 43861125.057op/s 44532920.386op/s 45196565.434op/s 45701565.369op/s 5.56% -0.130 -0.035 2.04% 62403.369op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [2.414µs; 2.414µs] or [-0.005%; +0.005%] None None None
credit_card/is_card_number/ throughput [414173420.131op/s; 414213168.820op/s] or [-0.005%; +0.005%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [113.246µs; 113.496µs] or [-0.110%; +0.110%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [8811457.679op/s; 8830843.326op/s] or [-0.110%; +0.110%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [107.800µs; 107.928µs] or [-0.059%; +0.059%] None None None
credit_card/is_card_number/ 378282246310005 throughput [9265641.597op/s; 9276541.147op/s] or [-0.059%; +0.059%] None None None
credit_card/is_card_number/37828224631 execution_time [2.414µs; 2.415µs] or [-0.028%; +0.028%] None None None
credit_card/is_card_number/37828224631 throughput [414034009.720op/s; 414260446.288op/s] or [-0.027%; +0.027%] None None None
credit_card/is_card_number/378282246310005 execution_time [105.007µs; 105.128µs] or [-0.058%; +0.058%] None None None
credit_card/is_card_number/378282246310005 throughput [9512369.733op/s; 9523316.916op/s] or [-0.058%; +0.058%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [102.269µs; 102.337µs] or [-0.033%; +0.033%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [9771724.691op/s; 9778228.903op/s] or [-0.033%; +0.033%] None None None
credit_card/is_card_number/x371413321323331 execution_time [23.011µs; 23.153µs] or [-0.307%; +0.307%] None None None
credit_card/is_card_number/x371413321323331 throughput [43211986.009op/s; 43477929.281op/s] or [-0.307%; +0.307%] None None None
credit_card/is_card_number_no_luhn/ execution_time [2.414µs; 2.414µs] or [-0.003%; +0.003%] None None None
credit_card/is_card_number_no_luhn/ throughput [414204707.992op/s; 414233534.669op/s] or [-0.003%; +0.003%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [89.944µs; 90.102µs] or [-0.088%; +0.088%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [11099057.219op/s; 11118356.531op/s] or [-0.087%; +0.087%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [86.134µs; 86.240µs] or [-0.061%; +0.061%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [11595875.106op/s; 11609993.272op/s] or [-0.061%; +0.061%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [2.414µs; 2.414µs] or [-0.004%; +0.004%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [414172917.664op/s; 414207486.777op/s] or [-0.004%; +0.004%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [83.024µs; 83.104µs] or [-0.048%; +0.048%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [12033330.799op/s; 12044766.920op/s] or [-0.047%; +0.047%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [102.244µs; 102.309µs] or [-0.032%; +0.032%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [9774374.426op/s; 9780522.962op/s] or [-0.031%; +0.031%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [23.068µs; 23.199µs] or [-0.284%; +0.284%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [43123590.377op/s; 43368207.087op/s] or [-0.283%; +0.283%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 498e0ed 1725523258 julio/build-system
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 9.151ms 9.186ms ± 0.015ms 9.186ms ± 0.009ms 9.194ms 9.211ms 9.227ms 9.249ms 0.69% 0.536 1.452 0.16% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [9.184ms; 9.188ms] or [-0.022%; +0.022%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 498e0ed 1725523258 julio/build-system
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.463µs 2.910µs ± 1.393µs 2.750µs ± 0.023µs 2.769µs 3.004µs 12.597µs 15.849µs 476.21% 7.966 63.625 47.76% 0.099µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [2.717µs; 3.103µs] or [-6.635%; +6.635%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 498e0ed 1725523258 julio/build-system
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 295.755ns 308.720ns ± 14.635ns 301.267ns ± 4.720ns 313.330ns 342.694ns 351.473ns 353.982ns 17.50% 1.534 1.513 4.73% 1.035ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [306.691ns; 310.748ns] or [-0.657%; +0.657%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 498e0ed 1725523258 julio/build-system
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 618.493µs 619.084µs ± 0.244µs 619.088µs ± 0.159µs 619.248µs 619.474µs 619.668µs 619.774µs 0.11% 0.057 0.016 0.04% 0.017µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1613491.196op/s 1615288.729op/s ± 637.017op/s 1615279.684op/s ± 415.112op/s 1615694.251op/s 1616399.855op/s 1616753.606op/s 1616834.045op/s 0.10% -0.055 0.016 0.04% 45.044op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 472.292µs 473.208µs ± 1.214µs 473.142µs ± 0.179µs 473.309µs 473.633µs 473.863µs 489.767µs 3.51% 12.768 171.784 0.26% 0.086µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2041787.656op/s 2113250.096op/s ± 5250.864op/s 2113530.281op/s ± 798.956op/s 2114345.472op/s 2115741.958op/s 2116610.013op/s 2117336.079op/s 0.18% -12.685 170.281 0.25% 371.292op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 190.542µs 191.080µs ± 0.619µs 191.032µs ± 0.157µs 191.195µs 191.397µs 191.564µs 199.224µs 4.29% 11.456 148.458 0.32% 0.044µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5019481.379op/s 5233471.111op/s ± 16351.109op/s 5234714.884op/s ± 4304.234op/s 5238842.351op/s 5244451.552op/s 5246497.569op/s 5248191.695op/s 0.26% -11.269 145.201 0.31% 1156.198op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 46.537µs 47.427µs ± 0.215µs 47.418µs ± 0.142µs 47.569µs 47.792µs 47.878µs 47.982µs 1.19% -0.293 0.983 0.45% 0.015µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 20841189.387op/s 21085397.119op/s ± 95534.222op/s 21088964.327op/s ± 63323.737op/s 21145498.842op/s 21229079.735op/s 21309035.207op/s 21488494.071op/s 1.89% 0.333 1.073 0.45% 6755.290op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 51.618µs 51.850µs ± 0.102µs 51.843µs ± 0.069µs 51.913µs 52.012µs 52.078µs 52.266µs 0.82% 0.509 0.511 0.20% 0.007µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 19132792.513op/s 19286485.243op/s ± 37941.669op/s 19289064.778op/s ± 25836.461op/s 19314637.340op/s 19342739.025op/s 19354410.456op/s 19372912.655op/s 0.43% -0.496 0.477 0.20% 2682.881op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [619.051µs; 619.118µs] or [-0.005%; +0.005%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [1615200.444op/s; 1615377.013op/s] or [-0.005%; +0.005%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [473.039µs; 473.376µs] or [-0.036%; +0.036%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2112522.376op/s; 2113977.815op/s] or [-0.034%; +0.034%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [190.994µs; 191.165µs] or [-0.045%; +0.045%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5231205.004op/s; 5235737.217op/s] or [-0.043%; +0.043%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [47.397µs; 47.457µs] or [-0.063%; +0.063%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [21072156.995op/s; 21098637.244op/s] or [-0.063%; +0.063%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [51.836µs; 51.864µs] or [-0.027%; +0.027%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [19281226.892op/s; 19291743.593op/s] or [-0.027%; +0.027%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 498e0ed 1725523258 julio/build-system
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.627µs 2.670µs ± 0.018µs 2.669µs ± 0.007µs 2.677µs 2.707µs 2.713µs 2.715µs 1.75% 0.291 0.480 0.66% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.668µs; 2.673µs] or [-0.091%; +0.091%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 498e0ed 1725523258 julio/build-system
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 38.438µs 38.947µs ± 0.733µs 38.621µs ± 0.056µs 38.696µs 40.487µs 40.556µs 41.883µs 8.45% 1.782 1.576 1.88% 0.052µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [38.845µs; 39.048µs] or [-0.261%; +0.261%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 498e0ed 1725523258 julio/build-system
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 1.197µs 1.373µs ± 0.054µs 1.386µs ± 0.032µs 1.416µs 1.447µs 1.465µs 1.465µs 5.70% -0.743 0.530 3.93% 0.004µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [1.365µs; 1.380µs] or [-0.546%; +0.546%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 498e0ed 1725523258 julio/build-system
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 16.639µs 22.143µs ± 13.806µs 16.782µs ± 0.088µs 16.929µs 49.591µs 52.350µs 119.468µs 611.88% 3.692 18.347 62.20% 0.976µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [20.230µs; 24.056µs] or [-8.641%; +8.641%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 498e0ed 1725523258 julio/build-system
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 135.861µs 136.259µs ± 0.168µs 136.245µs ± 0.086µs 136.332µs 136.497µs 136.995µs 137.258µs 0.74% 1.938 8.741 0.12% 0.012µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [136.235µs; 136.282µs] or [-0.017%; +0.017%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 498e0ed 1725523258 julio/build-system
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 75.008µs 75.247µs ± 0.135µs 75.234µs ± 0.029µs 75.263µs 75.326µs 75.622µs 76.865µs 2.17% 9.067 102.747 0.18% 0.010µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [75.229µs; 75.266µs] or [-0.025%; +0.025%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 498e0ed 1725523258 julio/build-system
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 295.985µs 299.972µs ± 2.059µs 299.957µs ± 1.540µs 301.587µs 303.489µs 303.879µs 304.612µs 1.55% 0.060 -0.887 0.68% 0.146µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 3282869.150op/s 3333797.708op/s ± 22875.280op/s 3333813.272op/s ± 17127.606op/s 3350331.843op/s 3369824.419op/s 3375227.048op/s 3378554.662op/s 1.34% -0.037 -0.893 0.68% 1617.527op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 28.528µs 28.817µs ± 0.107µs 28.833µs ± 0.068µs 28.899µs 28.959µs 28.994µs 29.021µs 0.65% -0.663 -0.090 0.37% 0.008µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 34457645.031op/s 34702658.768op/s ± 128704.659op/s 34682370.934op/s ± 81958.974op/s 34775463.185op/s 34978030.912op/s 35041123.148op/s 35053793.961op/s 1.07% 0.679 -0.063 0.37% 9100.794op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 16.525µs 16.665µs ± 0.110µs 16.735µs ± 0.105µs 16.770µs 16.798µs 16.854µs 16.880µs 0.86% -0.004 -1.698 0.66% 0.008µs 1 200
normalization/normalize_name/normalize_name/good throughput 59241857.437op/s 60007867.987op/s ± 397466.254op/s 59753696.324op/s ± 374029.559op/s 60394073.946op/s 60501933.433op/s 60507626.963op/s 60512983.165op/s 1.27% 0.010 -1.704 0.66% 28105.108op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [299.687µs; 300.258µs] or [-0.095%; +0.095%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [3330627.414op/s; 3336968.002op/s] or [-0.095%; +0.095%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [28.802µs; 28.831µs] or [-0.051%; +0.051%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [34684821.541op/s; 34720495.996op/s] or [-0.051%; +0.051%] None None None
normalization/normalize_name/normalize_name/good execution_time [16.650µs; 16.681µs] or [-0.092%; +0.092%] None None None
normalization/normalize_name/normalize_name/good throughput [59952782.987op/s; 60062952.987op/s] or [-0.092%; +0.092%] None None None

Baseline

Omitted due to size.

@hoolioh hoolioh force-pushed the julio/build-system branch 3 times, most recently from bb811d9 to 05bd0a3 Compare August 13, 2024 12:47
@hoolioh hoolioh changed the title Add build system infrastructure. [WIP] Add build system infrastructure. Aug 13, 2024
@hoolioh hoolioh force-pushed the julio/build-system branch 20 times, most recently from 8057d2c to c8e242b Compare August 21, 2024 08:21
bantonsson
bantonsson reviewed Aug 27, 2024
View reviewed changes
Cargo.toml Show resolved Hide resolved
@hoolioh
Add document describing the new build process.
78e00fc
@hoolioh
Add numbering so RFCs can be addressed better.
9d22bc5
@hoolioh
Add build system infrastructure.
49d0bfe 
* Set output folder via environment variable.
* Add tar generation.
* Make builder work both in release and debug profiles.
* Add version handling.
* Avoid deep cloning Strings by using Rc<str>.
@hoolioh
Modify CI files to use the new build system.
11a149d 
* Modify Alpine build to use new build system.
* Fix musl arch specialization.
* Set LIBDD_OUTPUT_FOLDER in alpine build image.
@hoolioh
Solve PR comments.
9710602 
* Fix report paths.
* Add documentation.
* Rerun build if any environment variable changes.
* Use determine_paths to figure out source directory.
@hoolioh hoolioh force-pushed the julio/build-system branch 2 times, most recently from f2586e2 to 2c438dd Compare August 28, 2024 10:58
@hoolioh
Fix bugs
a7f88b9 
* Add call to wait on spawning commands to ensure all pending modifications to the filesystem have finished.
* Fix header directory assembling the tar file.
* Avoid telemetry header inclusion when telemetry feature is not
  selected.
* Fix clippy suggestions.
* Run clippy through all workspace so the builder crate is included.
* Fix merge conflicts.
@r1viollet
Copy link
Contributor

Happy to merge this when you think we have reached parity with today's build. This is already a leap forward! Some of these can be addressed as follow ups.

@hoolioh
Copy link
Contributor Author

hoolioh commented Sep 4, 2024

Happy to merge this when you think we have reached parity with today's build. This is already a leap forward! Some of these can be addressed as follow ups.

@r1viollet during my testing the binaries generated are on par with what we're building now with the scripts. Also the examples are running fine as well as the tests in all the platforms we're currently using. Originally I wanted to modify the gitlab pipeline to use the builder crate but there were some problems with the Windows build. I talked with Gregory about that and we decided that we will make the changes in another separate PR, that's why the build scripts are still in place. Therefore if you're okay with the latest changes I think we can move forward, merge this one and start working on the next iteration.

bantonsson
bantonsson approved these changes Sep 4, 2024
View reviewed changes
Copy link
Contributor

@bantonsson bantonsson left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even the benchmarks approve 😉. And after the merges they didn't 😢. Something is broken with the benchmarks and their apparent flakiness.

@r1viollet
Copy link
Contributor

Ok, let's go! Please give a heads up to everyone when merging and we will look out for any potential issues.

r1viollet
r1viollet approved these changes Sep 4, 2024
View reviewed changes
Copy link
Contributor

@r1viollet r1viollet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hoolioh hoolioh merged commit e899c91 into main Sep 5, 2024
30 checks passed
@hoolioh hoolioh deleted the julio/build-system branch September 5, 2024 09:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@VianneyRuhlmann VianneyRuhlmann VianneyRuhlmann left review comments

@morrisonlevi morrisonlevi morrisonlevi left review comments

@r1viollet r1viollet r1viollet approved these changes

@ekump ekump ekump approved these changes

@pawelchcki pawelchcki pawelchcki approved these changes

@bantonsson bantonsson bantonsson approved these changes

Assignees
No one assigned
Labels
ci-build
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@hoolioh @codecov-commenter @r1viollet @morrisonlevi @bantonsson @pawelchcki @ekump @VianneyRuhlmann