Don't hit the database for retrieving packages

[maartenba]

Following on #2522 and inconsistency between DB / search, we should update the gallery and make sure no pages / services hit the database directly. Some front-ends to look at:

• Package details page
• OData: find packages by id
• OData: find updates

Once finished, revert #2522 in a meaningful way (e.g. owners may want to look at pending packages still)

[yishaigalatzer]

For the first item - The goal here is that once search service updates go in, the search service will no longer show packages that haven't been indexed. The plan is to make the website get the list of unindexed packages (from the catalog) and show them as pending.

[johnataylor]

It's very difficult for the web site to read the catalog directly. The data in the catalog has to be processed first by some job.

The index is the mechanism by which the web site gets the package data from the catalog.

The indexing processes also provide some parts of the index in a slightly optimized way such that clients can sometimes by-pass a computed lookup in Lucene. The web site could read that, that would be harmless, though also pointless. The point is reading the catalog directly is tricky and that's by design.

[maartenba]

Fixed OData feed with #2550
Package details page pending.

[maartenba]

Ran stress test, will need to be revisited because perf was lower than hitting the database. This does mean we still have the possible inconsitency for now, but this is not ready for prime time.

[maartenba]

@yishaigalatzer During triage, please move this one to a future sprint. We'll need to work on perf in search before we revisit this one.

[maartenba]

The feed optimization is now part of #2583. It uses raw search and is now lightning fast.
Package details page in gallery still hits the database.

[maartenba]

Fixed with #2821