Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Base weights can be zero due to a negative intercept value from the linear regression calculated during benchmarks #11804

Closed
koute opened this issue Jul 8, 2022 · 0 comments · Fixed by #11806
Labels
I3-bug The node fails to follow expected behavior.

Comments

@koute
Copy link
Contributor

koute commented Jul 8, 2022

By default min squares linear regression is used to calculate weights for extrinsics with components. The linear regression returns two values: a single intercept value (used as the base weight which represents the constant cost for a given extrinsic) and one or more slope values (which model the cost of the extrinsic as the values' of its components increase).

Now, depending on the exact numbers passed into the regression solver the intercept can actually be negative, for example consider this minimal reproduction: (the numbers here are the exact values I got running a benchmark for pallet_contracts/seal_transfer)

use linregress::FormulaRegressionBuilder;
use linregress::RegressionDataBuilder;

fn main() {
    let names = vec!["r"];
    let data = [
        (
            "Y",
            vec![
                374814.0, 376724.0, 377135.0, 377434.0, 377564.0, 377814.0, 378444.0, 380154.0,
                387224.0, 390674.0, 3765469.0, 3768161.0, 3773769.0, 3774359.0, 3784089.0,
                3785710.0, 3795070.0, 3796031.0, 3802661.0, 3812460.0, 7016704.0, 7030594.0,
                7033834.0, 7040853.0, 7045654.0, 7064893.0, 7075575.0, 7076156.0, 7085564.0,
                7099575.0, 10373120.0, 10373288.0, 10378050.0, 10378240.0, 10388459.0, 10403299.0,
                10406290.0, 10427740.0, 10436948.0, 10438680.0, 13673644.0, 13702855.0, 13703694.0,
                13711304.0, 13729414.0, 13733415.0, 13733584.0, 13737004.0, 13740995.0, 13768015.0,
                16941628.0, 16947068.0, 16955848.0, 16958639.0, 16971029.0, 16991829.0, 16995978.0,
                17001319.0, 17031360.0, 17031539.0, 20292943.0, 20293624.0, 20313014.0, 20318154.0,
                20322973.0, 20327884.0, 20376775.0, 20401705.0, 20434846.0, 20439215.0, 23669779.0,
                23677149.0, 23698099.0, 23704801.0, 23745839.0, 23789821.0, 23799171.0, 23804111.0,
                23809201.0, 23854362.0, 27043105.0, 27057924.0, 27063685.0, 27070555.0, 27091016.0,
                27092835.0, 27142716.0, 27164446.0, 27170856.0, 27184927.0, 30481771.0, 30506901.0,
                30543441.0, 30546172.0, 30551502.0, 30553301.0, 30562142.0, 30582572.0, 30596762.0,
                30630073.0, 33474942.0, 33475112.0, 33504753.0, 33504902.0, 33512503.0, 33521173.0,
                33536143.0, 33550523.0, 33567253.0, 33571043.0, 36952650.0, 36993129.0, 37003810.0,
                37041519.0, 37117141.0, 37147712.0, 37184142.0, 37328574.0, 37490604.0, 37499784.0,
                40398006.0, 40445396.0, 40456686.0, 40466707.0, 40502606.0, 40505037.0, 40515847.0,
                40590226.0, 40596698.0, 40608988.0, 43717840.0, 43721811.0, 43757261.0, 43763661.0,
                43763710.0, 43834342.0, 43848742.0, 43853291.0, 44014883.0, 44078014.0, 47111966.0,
                47136336.0, 47224827.0, 47290347.0, 47302558.0, 47333548.0, 47339968.0, 47362320.0,
                47393738.0, 47395809.0, 50442631.0, 50460321.0, 50469121.0, 50474943.0, 50540201.0,
                50552542.0, 50582743.0, 50583173.0, 50587693.0, 50646164.0, 52633353.0, 52654454.0,
                52697985.0, 52728045.0, 52778517.0, 52797727.0, 52805817.0, 52832776.0, 52881616.0,
                52974488.0, 56090811.0, 56117661.0, 56169311.0, 56234562.0, 56234893.0, 56264973.0,
                56280123.0, 56332303.0, 56409034.0, 56456925.0, 63243295.0, 63262456.0, 63331886.0,
                63358987.0, 63461328.0, 63597350.0, 63640791.0, 63713401.0, 63759323.0, 63923174.0,
                67128197.0, 67141107.0, 67213988.0, 67214838.0, 67221778.0, 67303219.0, 67331099.0,
                67416181.0, 67522102.0, 67534581.0, 70410161.0, 70425321.0, 70476323.0, 70685905.0,
                70795996.0, 70903397.0, 70964417.0, 71063629.0, 71193340.0, 71200919.0,
            ],
        ),
        (
            "r",
            vec![
                0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0,
                1.0, 1.0, 1.0, 1.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 3.0, 3.0,
                3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 3.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0,
                4.0, 4.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 6.0, 6.0, 6.0, 6.0,
                6.0, 6.0, 6.0, 6.0, 6.0, 6.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0, 7.0,
                8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 8.0, 9.0, 9.0, 9.0, 9.0, 9.0, 9.0,
                9.0, 9.0, 9.0, 9.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0, 10.0,
                11.0, 11.0, 11.0, 11.0, 11.0, 11.0, 11.0, 11.0, 11.0, 11.0, 12.0, 12.0, 12.0, 12.0,
                12.0, 12.0, 12.0, 12.0, 12.0, 12.0, 13.0, 13.0, 13.0, 13.0, 13.0, 13.0, 13.0, 13.0,
                13.0, 13.0, 14.0, 14.0, 14.0, 14.0, 14.0, 14.0, 14.0, 14.0, 14.0, 14.0, 15.0, 15.0,
                15.0, 15.0, 15.0, 15.0, 15.0, 15.0, 15.0, 15.0, 16.0, 16.0, 16.0, 16.0, 16.0, 16.0,
                16.0, 16.0, 16.0, 16.0, 17.0, 17.0, 17.0, 17.0, 17.0, 17.0, 17.0, 17.0, 17.0, 17.0,
                18.0, 18.0, 18.0, 18.0, 18.0, 18.0, 18.0, 18.0, 18.0, 18.0, 19.0, 19.0, 19.0, 19.0,
                19.0, 19.0, 19.0, 19.0, 19.0, 19.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0, 20.0,
                20.0, 20.0,
            ],
        ),
    ];
    let data = RegressionDataBuilder::new().build_from(data).ok().unwrap();
    let model = FormulaRegressionBuilder::new()
        .data(&data)
        .formula(format!("Y ~ {}", names.join(" + ")))
        .fit()
        .ok()
        .unwrap();

    println!("{}", model.parameters.intercept_value);
}

This returns an intercept value of -271299.7199134391. And for this particular extrinsic this was essentially random; sometimes you get a negative intercept, sometimes you don't. In both cases the input data is reasonable.

So what we do in the code is that we cast the intercept value into u128 which, due to Rust's saturating float casts, results in a base weight of zero, which, while not as bad as saying it has a negative cost, still doesn't really make sense because based on the data the extrinsic clearly has a non-zero constant cost.

For extrinsics where the range of components is bound so that their values cannot be zero this should be mostly harmless (since the component weight will pick up the slack), but imagine an extrinsic where the allowed minimum values of the components are zero, in which case we'd have (ignoring database weights) an extrinsic which wouldn't actually consume any weight but still consume resources.

cc @athei @ggwpez @shawntabrizi

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
I3-bug The node fails to follow expected behavior.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant