Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Receiving "Unhandled error event" with error handler #1386

Open
didomenicom opened this issue Jul 16, 2021 · 3 comments
Open

Receiving "Unhandled error event" with error handler #1386

didomenicom opened this issue Jul 16, 2021 · 3 comments

Comments

@didomenicom
Copy link

In a similar situation as described in #896 and #969, I am receiving the following error.

ERROR	[ioredis] Unhandled error event: Error: connect ETIMEDOUT
    at TLSSocket.<anonymous> (/var/task/node_modules/ioredis/built/redis/index.js:317:37)
    at Object.onceWrapper (events.js:482:28)
    at TLSSocket.emit (events.js:376:20)
    at TLSSocket.emit (domain.js:470:12)
    at TLSSocket.Socket._onTimeout (net.js:483:8)
    at listOnTimeout (internal/timers.js:555:17)
    at processTimers (internal/timers.js:498:7)

My code looks like this

const redisClient = new Redis(`rediss://<username>:<pasword>@<host>:<port>`, {
	reconnectOnError(error: Error) {
		console.log(`[Redis]: Processing reconnectOnError - "${error.message}`)
		const targetErrors = [/READONLY/, /ETIMEDOUT/];

		targetErrors.forEach((targetError) => {
			if (targetError.test(error.message)) {
				console.log("[Redis]: reconnectOnError true")
				return true;
			}
		});
	},
})

redisClient.on('connect', () => {
  logger.debug(`Connecting to redis "${process.env.REDIS_ENDPOINT}:${process.env.REDIS_PORT}"...`)
})

redisClient.on('ready', () => {
  logger.debug('Connected to redis!')
})

redisClient.on('error', (err: Error) => {
  logger.error({
    message: 'Error with redis client',
    errorMessage: err.message,
    errorStack: err.stack,
    errorName: err.name
  })
})

redisClient.on('close', () => {
  logger.debug('[Redis]:closed')
})

redisClient.on('reconnecting', (time: Number) => {
  logger.debug(`[Redis]:reconnecting - ${time}ms`)
})

redisClient.on('end', () => {
  logger.debug('[Redis]:end')
})

redisClient.on('wait', () => {
  logger.debug('[Redis]:wait')
})

From what I can tell, this function has no event listeners defined (count == 0) and is falling into the console.error().

How do I setup the code so that my error handling code will catch this error and it will not hit the console.error() section of code in silentEmit()?

@esatterwhite
Copy link

esatterwhite commented Jul 16, 2021

Yes, I'm seeing this too, w/ sentinels and I wound up at the same place in the code.

In my case we are running sentinels in kubernetes and connecting to them directly. If a pod goes down, it'll get recreated, but the IP may have changed, so the connect to the sentinel will start emitting ETIMEDOUT errors, but the error handler on the main instance is never triggered.

const redis = new Redis({
  sentinels: ['host-1:26379', 'host-2:26379', 'host-3:26379']
, name: 'sentinel'
, enableReadyCheck: true
, lazyConnect: true
, enableOfflineQueue: true
, failoverDetector: true
, retryStrategy: () => {
    return RECONNECT_TIMEOUT
  }
, sentinelRetryStrategy: () => {
    return RECONNECT_TIMEOUT
  }
, reconnectOnError: (err) => {
    /* istanbul ignore if */
    if (err.message.includes('READONLY')) {
      log.warn({err}, 'readonly event triggered')
      return 2
    }
  }
})

redis.on('error', (err) => {
  process.nextTick(() => {
    throw err
  })
})

For a stand alone connection, I would expect the error handler here to be triggered on these timeout errors.
For sentinels, I would expect the connection w/ error to be marked as dead or have some number of reconnection attempts.
And the error handler should be tripped when all of the sentinels are unreachable. Timeout errors on connections should probably be handled similarly to unreachable errors.

@kodeine
Copy link

kodeine commented May 9, 2022

hello @esatterwhite can you please tell me what value are you using for RECONNECT_TIMEOUT ?

@truemoroz
Copy link

How to catch connection timeout error?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants