Constrained input for property testing / Closures as property function function

[wucke13]

I have a usecas where I would like to do property checking on a function. The input of a function is a rather big struct with many fields. For most of the cases, it only matters to check what happens when two of these fields (which hold numeric values encoded in uom) are checked for the property. Is there any way of doing this in quickcheck?

My first attempt was using shim closure, which takes only the two relevant values, modifies the two regarding fields in a clone of a template of the big struct and checks the property. However, a capturing closure does not implement quickcheck::Testable and hence won't work. Is there any way of making this work?

To clarify my use case, I'll attach some demo code:

use quickcheck::QuickCheck;
let mut qc = QuickCheck::new();
                                                                        
let test = |value_a:f64, value_b:f64| {
    let mut new_state = template_state.clone();
    new_state.a = Length::new::<foot>(value_a);
    new_state.b = Velocity::new::<foot_per_minute>(value_b);
    let result = world.process(&new_state);
    assert!(result.is_fine());
};
                                                                        
qc.quickcheck(test );

[BurntSushi]

However, a capturing closure does not implement quickcheck::Testable and hence won't work. Is there any way of making this work?

No. See: #56

From what I can tell, it looks like you should just implement Arbitrary for your big struct. Maybe assigned fixed values to all fields except for the two you care about?

[wucke13]

Thank you for the quick response! The other, fixed values are different depending on which scenario is tested. So I having different implementations of arbitrary depending on scope should not be an issue, correct?
I just realized another problem: the process function is a member of struct, can it at all be casted to fn(_)->_?

[BurntSushi]

To be honest, I don't really understand the problem you're trying to solve. So I'm not sure how to help you. Sorry.

[wucke13]

I've come up with a much simpler way to describe it: Is there any way to limit the input space with values which are not known at compile time?

[BurntSushi]

Yes, implement the Arbitrary trait and generate random values that are restricted to whatever input space you want.

[wucke13]

Buy how do I get the non compile time known values in the Arbitrary implementation?

[BurntSushi]

You generate them. If you can't generate them, then add them in the property function itself?

I think this back and forth isn't really accomplishing much. Maybe providing more code that others can compile that fits your use case would help.

[wucke13]

Sorry for the delay.

What I would like to do is investigate whether QuickCheck can be used to test invariants of a system described in Gherkin language/using the Cucumber framework. Consider the following mock up code:

struct AircraftState {
    pub height: f64,

    // Imagine many more attributes here, this is a complex state!
}

impl AircraftState {
    pub fn get_callouts(&self)-> Vec<String>;
}

struct MyWorld{
    pub height_max: f64,
}

// some cucumber stuff

pub fn steps() -> Steps<crate::MyWorld> {
    let mut builder: Steps<crate::MyWorld> = Steps::new();

    builder.when_regex(
        "the plane is less than (d+) foot above ground high",
        |mut world: crate::MyWorld, matches,_step| {
            world.height_max = matches[1].parse().unwarp();
            world
        })
    .then("a 500 foot callout is emitted", |world, _step| {
        // now some quickcheck code, whiches generates random AircraftState. However,
        // every height value is in between the constraint given by height_max in world.
        // The quickcheck test succeeds if the callout is emitted
    });
}

I want to use Gherkin to describe a scenario. This scenario contains some constraints, e.g. a certain value (height in the example) is in a certain range. Now I want to quickcheck to test whether inputs which implement the constraints of the scenario actually produce the expected output. Please ignore the Cucumber part here, that is solely to illustrate my use case but it is not relevant to my issue with using quickcheck.

The issue is: I learn about the constraints for one scenario on runtime, so this values are not compile time known. In order for quickcheck to check this, I need a way to inject these values into quickcheck. The following options have I found so far:

• I can implement Arbitrary on my own. However, this is not working for this usecase: There is no way to pass data to the arbitrary implementation that is not known at compile time.
• I could capture the values in the property function for quickcheck. This is not working, because quickcheck is using pure function pointers. Passing a FnMut simply is not working here.
• I could use some hackish global variable construct to share the constraints gathered from the scenario with the implementation of arbitrary (or the quickcheck property function). However, this is very hackish and likely cause problems if tests are ran concurrently.

I hope this makes it a bit more clear what I'm after. If you find one of my assumptions to be untrue, please let me know.

Maybe providing more code that others can compile that fits your use case would help.

I can't. As far as I can tell, with the current API it's not possible to make code which demonstrates this that does compile.

[BurntSushi]

Since closures implementing Testable would work with your use case, it is therefore possible to achieve what you want by simply implementing Testable yourself with your own custom type. Unfortunately, that also means re-implementing the shrinking logic. It's not awful to do, but it's not great either. I don't see any way to re-use existing Testable implementations to get out of this and I don't see any nice way of exposing the shrinking logic either.

I think your only other choice at present would be to utilize some kind of global state. There are likely competing designs for the crate itself that would enable your use case, but these seem pervasive and I do not think they are worth pursuing.

Regrettably, the nicest solution here would indeed probably be closures, but Rust just doesn't make it possible.