Paralelism x database connections

[pedrofurtado]

Hi @bensheldon! 👋

Thanks for your hard work on this gem!

If I run a good_job start process in production, with --max-threads=5, the process will have 1 database connection and 5 possible parallel job processing with pool connection?

Another scenario: If I run 3 good_job start processes in production with --max-threads=5, there will be 3 database connections and each connection with 5 possible parallels job processing?

What is the behavior of good_job standalone processes, threads in each processes considering database connections?

Thanks a lot! 🍻

[bensheldon]

Every thread requires its own database connection; this is a requirement of ActiveRecord / Postgres Adapter.

good_job start --max-threads=5 will consume 6 database connections (8 if you use cron) if you set that as data pool: size in your database.yml.

Running 3 of them in that configuration will use 6 x 3 = 18 database connections.

I'd love if you have suggestions for improvements to the documentation: https://github.com/bensheldon/good_job#database-connections

[pedrofurtado]

Hi @bensheldon ! Thank you for you feedback! 🤝

Now it's clear 💡 !

A last doubt, there is some different behavior if we define pool: false, for example, in rails config/database.yml? It can generate some weird behavior, or only is not used pooled connections? 👍

Here, we have some apps configured by this way 😢 is there some chance to produce errors or strange behavior in the scenario of standalones good_job processes with some max threads?

Thanks again for your help! 🍻

[bensheldon]

is there some chance to produce errors or strange behavior in the scenario of standalones good_job processes with some max threads?

GoodJob uses ActiveRecord's database connection pool to check out database connections. You want to make sure that ActiveRecord's database connection pool is equal-to or greater-than the number of threads/connections used by GoodJob. If the ActiveRecord database connection pool is insufficient, you'll see performance problems (while it waits for an available connection) or ActiveRecord::ConnectionTimeoutError errors (if it is unable to check out a connection).

This is analogous to matching the ActiveRecord database connection pool size to puma threads.