Refactoring threading

[koenedaele]

We are running into issues because we're using providers in non thread-safe ways. Problem manifests itself most with skosprovider_sqlalchemy ( #343 ), but can happen with other providers that do anything more than a simple in memory provider.

After some discussion, the solution would be something like this:

• Currently we're registering providers with a registry. We should also be able to register a provider_factory. This factory would be called by the registry whenever someone actually gets a provider. We could also make it so that we always need to register a factory or have some kind of default provider factory that always return the passed provider.
• When starting a new thread, the factory should get access to the pyramid reqest in some way so it can get a db session.
• These db session is currently created within atramhasis.db.data_managers. It should also just be available as request.db.
• The factory creates a provider with everything it needs. It either creates a new one or copies an existing one, but it does not return one that already exists. At the end of a request, the provider is lost and all open sessions are closed.

[koenedaele]

See OnroerendErfgoed/skosprovider_sqlalchemy#26

[cecemel]

This issue seems to be dragged along for three sprints now. Is there anyone available this sprint who could work on it (within a reasonable time frame)? Else, I would suggest to leave it out of the sprint.

[koenedaele]

Fixed by #491

Fixed in production before that by running the application single threaded.