BullMq Flows?

[timganter]

Hi there! Is support for BullMQ flows on the roadmap for this package? 🙂

[RomainLanz]

Hey @timganter! 👋🏻

I have never heard of it!
Skimming through the doc, it indeed seems a nice feature to have. So expect to have it land on the next major release (the one to support AdonisJS V6).

[timganter]

That's awesome to hear! Thanks @RomainLanz ! 🎉

[ThisIsMissEm]

So just some small caveats: I've just been trying to get Bullmq Flows to work, and there some issues I've encountered:

1. Flows basically destroy all types, like, everything you do with them will be unsafe. With Queues and Workers you can type them like:

const DownloadWorker = new Worker<MediaDownloadData, MediaDownloadResult>(

export const MediaDownloadQueue = new Queue<MediaDownloadData, MediaDownloadResult>(

2. FlowProducer and FlowJobs do not at all like working with MemoryDB or Redis Cluster mode, I'm not sure why, but they just hang on producing jobs that the Worker can pick up. From my experience with Bullmq, hangs tend to happen when a connection cannot be established, but even passing in the correct connection args it just didn't work.

3. Good luck with typings on connections: Bullmq ships it's own types for RedisConnection, and doesn't seem natively aware of RedisClusterConnection.

So basically: you loose typesafety with your jobs, and you'll hit into hard to debug connection issues when using MemoryDB or Redis Cluster mode.

[RomainLanz]

Thanks for the feedback! Maybe we should wait a bit before implementing them inside this package.

Also, it seems it may be "experimental".

Flows are a brand new feature in BullMQ, and although is implemented on a stable foundation there could be some unknown issues.

[ThisIsMissEm]

@RomainLanz yeah, it cost me a good few hours ripping them out and figuring out an alternative approach.