EPIC: Improve test cases

[prisis]

List of test that need to be checked and adjusted:

test	developer	PR #	Done
address.spec.ts	@Shinigami92	#285	✔️
all_functional.spec.ts	can be skipped		✔️
animal.spec.ts	@Shinigami92	#293	✔️
commerce.spec.ts	@Shinigami92	#294	✔️
company.spec.ts	@Shinigami92	#493	✔️
database.spec.ts	@Shinigami92	#393	✔️
datatype.spec.ts	@Shinigami92	#344	✔️
date.spec.ts	@Shinigami92	#312	✔️
fake.spec.ts	is okay		✔️
finance_iban.spec.ts	@Shinigami92	#515	✔️
finance.spec.ts	@Shinigami92	#519	✔️
git.spec.ts	@Shinigami92	#425	✔️
helpers.spec.ts	@Shinigami92	#499	✔️
image.spec.ts	@Shinigami92	#414	✔️
internet.spec.ts	@ST-DDT, @Shinigami92	(partially) #258, #475	✔️
locales.spec.ts	can be skipped		✔️
lorem.spec.ts	@Shinigami92	#473	✔️
music.spec.ts	@piotrekn	#383	✔️
name.spec.ts	@Shinigami92	#372	✔️
phone_number.spec.ts	@piotrekn	#396	✔️
random.spec.ts	@Shinigami92	#490	✔️
system.spec.ts	@prisis	#303	✔️
time.spec.ts	@Shinigami92	#421	✔️
unique.spec.ts	@Shinigami92	#370	✔️
vehicle.spec.ts	@Shinigami92	#469	✔️
word.spec.ts	@Shinigami92	#470	✔️
hacker.spec.ts	@Shinigami92	#386	✔️

• First step would be to create some seeds and a seed map with return values to always create the same data.
• Extend the test to run on every local that faker supports.
• Remove mock calls to test the function itself and not the mocked output.

[github-actions]

Hello @prisis. We like your proposal/feedback and would appreciate a contribution via a Pull Request by you or another community member. We thank you in advance for your contribution and are looking forward to reviewing it!

[Shinigami92]

For the long run we could try to create a type + abstraction like this:

export type SeededRun<Module, K extends keyof Module = keyof Module> = {
  seed: number;
  expectations: {
    [P in K]: any;
  };
};

But it would be cool to also have somehow have the expectations values typed 🤔

The benefit would be to get typescript-notified if there is a missing test

[piotrekn]

@prisis, I can do music.spec.ts, phone_number.spec.ts and random.spec.ts. Can you assign them to me , please?

[Shinigami92]

I will do 👍 Please make them in 3 dedicated PRs, then I will update the PR numbers also

[Shinigami92]

@prisis you missed hacker cause hacker doesn't has a test at all 😱
I think I will target hacker as next module
I will also add it now to the list above

[prisis]

It was hacked 🤣 yeah, was printing all test file out

[piotrekn]

Guys, this pattern is flawed:

  // Create and log-back the seed for debug purposes
  faker.seed(Math.ceil(Math.random() * 1_000_000_000));

  describe(`random seeded tests for seed ${faker.seedValue}`, () => {

It should be more like this:

  // Create and log-back the seed for debug purposes
  const seed = Math.ceil(Math.random() * 1_000_000_000);

  describe(`random seeded tests for seed ${seed}`, () => {
    beforeAll(() => {
      faker.seed(seed);
    });

...or even beforeEach instead of beforeAll.
The problem is that faker.seedValue is a global variable, and every faker.seed() changes it. Once seededRuns have executed, the last seed set remains, not a random one we expect. The expression faker.seed(Math.ceil(Math.random() * 1_000_000_000)); is executed once before all tests, on declaration.

Please review your PRs for that case.

[Shinigami92]

Guys, this pattern is flawed:

  // Create and log-back the seed for debug purposes
  faker.seed(Math.ceil(Math.random() * 1_000_000_000));

  describe(`random seeded tests for seed ${faker.seedValue}`, () => {

It should be more like this:

  // Create and log-back the seed for debug purposes
  const seed = Math.ceil(Math.random() * 1_000_000_000);

  describe(`random seeded tests for seed ${seed}`, () => {
    beforeAll(() => {
      faker.seed(seed);
    });

...or even beforeEach instead of beforeAll. The problem is that faker.seedValue is a global variable, and every faker.seed() changes it. Once seededRuns have executed, the last seed set remains, not a random one we expect. The expression faker.seed(Math.ceil(Math.random() * 1_000_000_000)); is executed once before all tests, on declaration.

Please review your PRs for that case.

You are missing that we added faker.seed(seed) in every single it block to ensure this behavior.
Even beforeEach was not safe enough, cause we sometimes have nested describe+it blocks in this loop.
If you found a specific test were we overseen this, please report it to us.

[ST-DDT]

@Shinigami92 I think you and @piotrekn are referring to different things.

As you can see there is no faker.seed() inside this section anymore:

faker/test/animal.spec.ts

Lines 101 to 115 in f1884be

	// Create and log-back the seed for debug purposes
	faker.seed(Math.ceil(Math.random() * 1_000_000_000));
	describe(`random seeded tests for seed ${faker.seedValue}`, () => {
	for (let i = 1; i <= NON_SEEDED_BASED_RUN; i++) {
	for (const functionName of functionNames) {
	describe(`${functionName}()`, () => {
	it(`should return random value from ${functionName} array`, () => {
	const actual = faker.animal[functionName]();
	expect(faker.definitions.animal[functionName]).toContain(actual);
	});
	});
	}
	}
	});

However, this isn't needed because the main purpose of those methods is to randomly execute the methods and check for unexpected behavior. These tests should be removed or disabled by default in the medium term to remove the flaky test coverage behavior.
The faker seed is set once and logged to ensure that in the event of a failure, the test setup can be recovered by manually setting the seed. It is not required to have the exact seed for all methods.

[Shinigami92]

@ST-DDT ah yes, you are totally right

The weather is grey here and I feel a bit weak today, so sorry if I confuse some stuff

[Shinigami92]

Vehicle is driving me crazy, so I will tackle that now 😜

[Shinigami92]

For some tests I have no words...

[Shinigami92]

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.

[Shinigami92]

I will now rewrite the internet

[Shinigami92]

I will try targeting finance_iban.spec.ts now

[Shinigami92]

We are near to finish 🏁

Only reviews are needed!

[Shinigami92]

All done 🎉