Discussion about async operations, network requests and mocking.

[reggi]

I have some thoughts and ideas about "integration" tests. Would love to start an open discussion and get some insight on mocking in general. Feel free to close if this isn't the place to have this.

I posted a question on StackOverflow a while ago, read something like this.

I'm interested in creating full mocked unit tests, as well as integration tests that check if some async operation has returned correctly. I'd like one command for the unit tests and one for the integration tests that way I can run them separately in my CI tools. What's the best way to do this? Tools like mocha, and jest only seem to focus on one way of doing things.

It's my opinion that "mocking" a network request isn't the answer. It's time consuming and can not be accurate. In a perfect world what I want is the ability to make network requests (or any async request) and watch my tests pass, then store the payloads locally in the project. This would allow any API changes or changes to the actual payload to be automatic. I can write my code to produce the correct response back from the request, then I could run the test from then on mocked with this automatic response that was cached.

I've seen people pointed around to the jest.disableAutomock function as a solution. I don't think that's the solution. I think that a jest --intergation functionality is necessary to truly test async operations without mocking.

I truly don't understand how you can write a database query, mock it get your test working and think that that's the end of it without ever actually testing the request hitting the database.

I think actually making the async requests and automatically caching, generating the mocks for you all the responses is the way to go. Looking for criticism and feedback, am I missing the full picture? Is jest just primarily for front end? Does this already exist?

[thymikee]

Since Jest 15 automocking is disabled by default, so nothing gets mocked unless you tell Jest to do so. You can have another folder for integration tests, e.g. look at integration_tests dir inside Jest repo.

Also nothing prevents you to create test and test-integration npm scripts, running different configs with --config flag.

[cpojer]

@kentaromiura talked about this before, it's similar to snapshots: find a way to record remote data through Jest. @reggi can you design an API that makes sense either within Jest or as a third-party module?

[reggi]

Thanks for the comments @cpojer @thymikee!

I put this together really quick to try and get at what I want. Ideally both flags would fill 100% coverage.

Here's the test:

import fetch from 'node-fetch'
import memoeyesAsync from './memoeyes'

async function getGoogle() {
  let res = await fetch('https://google.com')
  let body = await res.text()
  return body.split('google').length
}

function add(a, b) {
  return a + b;
}

getGoogle = memoeyesAsync({getGoogle}, process.env.TEST_TYPE, {filename: './memoeyes-cache.js'})

describe ('x', () => {
  it('should add two numbers', () => {
    expect(add(1, 2)).toBe(3)
  })
  it('should work', async () => {
    let google = await getGoogle()
    expect(google).toBe(79)
  })
})

Here's the function that caches the result.

import Datastore from 'nedb'
import Promise from 'bluebird'

export default function memoeyesAsync (fnObj, testType, nedb) {
  let fnName = Object.keys(fnObj)[0]
  let fn = Object.values(fnObj)[0]
  nedb.autoload = true
  let db = new Datastore(nedb)
  db.update = Promise.promisify(db.update)
  db.find = Promise.promisify(db.find)
  const INTERGRATION = (testType === 'intergration')
  const UNIT = (testType === 'unit')
  if (INTERGRATION) {
    return async function (...args) {
      let value = await fn(...args)
      let results = await db.find({fnName, args})
      if (results.length == 0) await db.insert({fnName, args, value})
      return value
    }
  } else if (UNIT) {
    return async function (...args) {
      let results = await db.find({fnName, args})
      if (results && results[0] && results[0].value) return results[0].value
      throw new Error('need intergration')
    }
  }
  return false
}

So if you run TEST_TYPE=intergration jest it makes the network request and if you run TEST_TYPE=unit jest it uses the cached saved version of the function result so the test passes, however it does skip over the entire function. 😬 🤔 🌈

This is the kind of functionality I'm looking for.

I'd love something that was closer to node and just auto-cached all the return values, that way I wouldn't have to deliberately wrap functions and also skip over the functions and they could all be mocked.

[reggi]

I vaguely remember a talk @indutny gave on llnode that may give the ability to store the results of every function call so I could hypothetically get a copy of all returned function values that way. Not sure. 🤔 🐟 💌

[indutny]

Hello everyone!

I'm sorry, but it would be hard to do it with llnode. llnode is a plugin for debugger, not a profiling or general monitoring tool.

[Haroenv]

Would be nice that in UNIT mode, it would do a jest.fn() or another mock. Maybe some way to have the first call do an integration test might make sense?

[reggi]

One thing is preventing me from coming up with another viable API for this.

I have a conceptual issue with is the following:

Let's say I have the following function.

doSomethingAsync() // like network call

But the argument to this function is a class instance.

const instanceOfData = new Data()
doSomethingAsync(instanceOfdata)

I have no way of storing the input arguments doSomethingAsync because I am incapable of stringifying instanceOfdata and playing it back.

Does anyone have an idea on how we can stringify any set of argument, store them to the file-system then call the same function with the stored arguments to produce the same output?

[SimenB]

For network requests, something like nock and its recording might work (see also #6081), /cc @palmerj3

Not too sure about "other async work" such as FS or DB. I suppose DB should just work as that's often TCP or something, but intercepting FS access seems overkill.

I think this issue is a bit vague. It's more a discussion which seems to have died out.

If people come across this, follow #6081 for HTTP mocking, and open up a new issue if you want some sort of playback for IO that's not network.

(happy to reopen if people disagree!)