Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[enhancement] Support multiple test cases for a single test proc #1581

Open
rw1nkler opened this issue Sep 3, 2024 · 0 comments
Open

[enhancement] Support multiple test cases for a single test proc #1581

rw1nkler opened this issue Sep 3, 2024 · 0 comments
Labels
dslx DSLX (domain specific language) implementation / front-end enhancement New feature or request testing Test-infrastructure related

Comments

@rw1nkler
Copy link
Contributor

rw1nkler commented Sep 3, 2024

What's hard to do? (limit 100 words)

When using test procs, it is not possible to run different test cases, that will be eventually listed as separate entries in the report without duplicating test procs. It would be helpful to describe multiple tests cases that can reuse the same config() function.

Current best alternative workaround (limit 100 words)

Currently, the user has to either put all the tests in the single next() function, making individual test cases indistinguishable or create multiple test procs, which leads to significantly longer code.

Your view of the "best case XLS enhancement" (limit 100 words)

It would be great to define many next functions, preferably with some additional information that will help identify the test cases in the test summary. Some changes to the procs syntax were proposed in this comment, but the example given doesn’t show if specifying multiple test cases for the same design would be possible.

Here are some options for adding multiple test cases to DSLX:

  1. Custom attribute for the next() function
#[test_proc]
proc PassthroughTest {
    terminator: chan<bool> out;
    data_s: chan<u32> out;
    data_r: chan<u32> in;

    config(terminator: chan<bool> out) {
        let (data_s, data_r) = chan<u32>("data");
        spawn Passthrough(data_r, data_s);
        (terminator, data_s, data_r)
    }

    #[test_case(name="First test case")]
    next() {
        ...
        send(tok, terminator, true);
    }

    #[test_case(name="Second test case")]
    next(count: u32) {
        ...
        send(tok, terminator, true);
    }
}
  1. Custom attribute with test case name derived from a function name
#[test_proc]
proc PassthroughTest {
    terminator: chan<bool> out;
    data_s: chan<u32> out;
    data_r: chan<u32> in;

    config(terminator: chan<bool> out) {
        let (data_s, data_r) = chan<u32>("data");
        spawn Passthrough(data_r, data_s);
        (terminator, data_s, data_r)
    }

    #[test_case]
    case_one() {
        ...
        send(tok, terminator, true);
    }

    #[test_case]
    case_two(count: u32) {
        ...
        send(tok, terminator, true);
    }
}
  1. Test cases naming convention taken, for example, from Google Test.
#[test_proc]
proc PassthroughTest {
    terminator: chan<bool> out;
    data_s: chan<u32> out;
    data_r: chan<u32> in;

    config(terminator: chan<bool> out) {
        let (data_s, data_r) = chan<u32>("data");
        spawn Passthrough(data_r, data_s);
        (terminator, data_s, data_r)
    }

    test_case_one() {
        ...
        send(tok, terminator, true);
    }

    test_case_two(count: u32) {
        ...
        send(tok, terminator, true);
    }
}

One of the challenges for enabling multiple test cases is how to write the init() function for each of them. However, I believe that in general state for test procs is not required at all, since the next() function can be replaced with a for loop.

@rw1nkler rw1nkler added the enhancement New feature or request label Sep 3, 2024
@proppy proppy added dslx DSLX (domain specific language) implementation / front-end testing Test-infrastructure related labels Sep 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dslx DSLX (domain specific language) implementation / front-end enhancement New feature or request testing Test-infrastructure related
Projects
Status: No status
Development

No branches or pull requests

2 participants