Track processes in the database

[bensheldon]

There should be a table of GoodJob processes in the database. They can be updated via a heartbeat by the Notifier, which should always hold a database connection. And a timeout value such that if the updated_at value hasn't been touched, the process would be treated as expired.

(Refactoring-wise: I think the notifier could become a more generalized 1-per-process supervisor that also contains the Poller's functionality too)

[jrochkind]

Nice! This is a feature that resque has (although I don't think sidekiq actually does?)

It is a somewhat error-prone and complex part of resque though, that ends up having a lot of lines of code behind it. Resque seems to have found it somewhat tricky to get right.

[bensheldon]

@jrochkind thank you! I'll take a look at how Resque does it. Redis does have the capability to enumerate over all processes: https://www.rubydoc.info/github/mperham/sidekiq/Sidekiq/ProcessSet

[bensheldon]

Considering this story done for the moment. Improvements are tracked in #560.

To query processes:

GoodJob::Process.all

[jrochkind]

Awesome! Are these yet revealed in the admin dashboard?

[bensheldon]

@jrochkind yep! https://goodjob-demo.herokuapp.com/good_job/processes