Error 500 on package versions

[yzislin]

Hello,
First of all, I want to thank you for this great service. It contains enormous amount of useful data and information.

I am experiencing a lot of 500 errors when trying to pull package versions (sometimes on packages as well).
This is my api endpoint:
https://packages.ecosyste.ms/api/v1/registries/npmjs.org/versions

These are parameters that I pass:
params = {"page": 1, "per_page": 1000, "sort": "updated_at", "order": "asc", "updated_after": "2004-10-18T00:00:00.000Z"}

I use python requests to perform these operations.
headers = {"Content-Type": "application/x-www-form-urlencoded"}
response = requests.get(url, params=params, headers=headers, timeout=20)

Thank you for taking a look.

[andrew]

I saw this error come in today, there's some invalid data showing up:

File: | app/models/version.rb:25:in `download_url'
-- | --
web.1
/api/v1/registries/npmjs.org/versions

MESSAGE

undefined method `registry' for nil

SUMMARY

File:	
app/models/version.rb:25:in `download_url'
Host:	
web.1
Path:	
/api/v1/registries/[npmjs.org/versions](http://npmjs.org/versions)

[yzislin]

I've noticed if I change the timestamp by a minute or so, it will work. To me it feels like there is corruption in data or something. Maybe pagination issue.

[andrew]

I suspect it's that a version is being created and then displayed before it's been fully synced, I'm going to investigate further later today

[andrew]

Deploying an extra filter to avoid rendering invalid versions now.

[yzislin]

Great. Thank you. I will give it a try.
I believe I've seen this error on different API calls like advisories as well.

[yzislin]

I am still experiencing 500 errors on the same package versions API call. Thx

[andrew]

I recommend dropping the per_page value down, there’s a max timeout of 30 seconds

[yzislin]

It is not helping. Today for some reason it is really slow and eventually I get 500.
curl -v "https://packages.ecosyste.ms/api/v1//registries/npmjs.org/versions?page=1&per_page=4&sort=updated_at&order=asc&updated_after=2022-04-23T22:11:40Z"