Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: only skip ssz_static tests associated to missing type #6798

Merged
merged 3 commits into from
May 23, 2024

Conversation

nflaig
Copy link
Member

@nflaig nflaig commented May 16, 2024

Motivation

Motivation from #6776 to only skip ssz_static tests associated to missing type instead of all ssz_static tests.

Description

  • add separate it-statement to assert that type is defined
  • return instead of throwing an error if type is not defined

@nflaig nflaig requested a review from jeluard May 22, 2024 09:00
@@ -57,25 +57,28 @@ const sszStatic =
(ssz.altair as Types)[typeName] ||
(ssz.phase0 as Types)[typeName];

it(`SSZ type ${typeName} for fork ${fork} is defined`, function () {
expect(sszType).toBeDefined();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you specify an error message in expect()? Currently it throws AssertionError: expected undefined not to be undefined which doesn't make too much sense unless we look at the name of the test.

Something like

expect(sszType, `SSZ type ${typeName} for fork ${fork} is not defined`).toBeDefined()

and

it(`${fork} - ${typeName} existence check`)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you specify an error message in expect()?

This works but gives an eslint error

Expect takes most 1 argument eslint (vitest/valid-expect)

We have to use the custom function toEqualWithMessage in that case

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AssertionError: expected undefined not to be undefined which doesn't make too much sense unless we look at the name of the test.

Yes I totally agree with this but @nazarhussain has been pushing against having detailed assertion error messages so I kept it as is.

Right now if you wanna have a better assertion message you either have to

  • use toBeWithMessage / toEqualWithMessage which is crossed out as it is marked deprecated
  • or pass as 2nd param to expect and get an eslint error

We might wanna rethink this, imo it would be best if we override the vitest default toBe and toEqual to support the message as a 2nd param, and remove the custom toBeWithMessage and toEqualWithMessage

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tend to agree with @nazarhussain on this. The spec test name should be descriptive enough to explain what is being asserted, and there should ideally be only one expect in each test case so that its clear which assertion is failing. I tend to feel that adding more messaging to the test case means that the naming or assertion is incorrect or confusing and should be refactored

Copy link
Contributor

@ensi321 ensi321 May 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image Maybe I am nitpicking or not used how vitest outputs logs, but when I debug I first look at the message in red, which in this case is not descriptive at all. I will then need to look at the unit test name, which is white and is prepended by file name and spec test name. image On the other hand, this is good because when I see the red message, I know `deposit_requests_root` is missing right from the first glance

Copy link
Member

@wemeetagain wemeetagain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@wemeetagain wemeetagain merged commit bc071e4 into electra-fork May 23, 2024
13 of 17 checks passed
@wemeetagain wemeetagain deleted the nflaig/ssz_static-tests branch May 23, 2024 19:56
g11tech pushed a commit that referenced this pull request May 24, 2024
* test: only skip ssz_static tests associated to missing type

* More detailed error message if type is not defined
g11tech pushed a commit to g11tech/lodestar that referenced this pull request Jun 19, 2024
…e#6798)

* test: only skip ssz_static tests associated to missing type

* More detailed error message if type is not defined
g11tech pushed a commit that referenced this pull request Jun 25, 2024
* test: only skip ssz_static tests associated to missing type

* More detailed error message if type is not defined
g11tech pushed a commit that referenced this pull request Jun 25, 2024
* test: only skip ssz_static tests associated to missing type

* More detailed error message if type is not defined
g11tech pushed a commit that referenced this pull request Jul 1, 2024
* test: only skip ssz_static tests associated to missing type

* More detailed error message if type is not defined
g11tech pushed a commit that referenced this pull request Jul 30, 2024
* test: only skip ssz_static tests associated to missing type

* More detailed error message if type is not defined
g11tech pushed a commit that referenced this pull request Jul 31, 2024
* test: only skip ssz_static tests associated to missing type

* More detailed error message if type is not defined
g11tech pushed a commit that referenced this pull request Aug 2, 2024
* test: only skip ssz_static tests associated to missing type

* More detailed error message if type is not defined
g11tech pushed a commit that referenced this pull request Aug 9, 2024
* test: only skip ssz_static tests associated to missing type

* More detailed error message if type is not defined
g11tech pushed a commit that referenced this pull request Aug 9, 2024
* test: only skip ssz_static tests associated to missing type

* More detailed error message if type is not defined
g11tech pushed a commit that referenced this pull request Aug 23, 2024
* test: only skip ssz_static tests associated to missing type

* More detailed error message if type is not defined
g11tech pushed a commit that referenced this pull request Aug 27, 2024
* test: only skip ssz_static tests associated to missing type

* More detailed error message if type is not defined
philknows pushed a commit that referenced this pull request Sep 3, 2024
* test: only skip ssz_static tests associated to missing type

* More detailed error message if type is not defined
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.22.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants