IsAbsoluteLatestVersion is false for all versions in some packages

[BenLubar]

For example, here's the WindowsCompatibility package (which currently has only one version):

https://www.powershellgallery.com/api/v2/FindPackagesById()?$filter=IsAbsoluteLatestVersion&id=%27WindowsCompatibility%27&semVerLevel=2.0.0

As you can see, the API reports the latest version as not being the latest version:

[jrasch]

I would like to add that this is affecting our ProGet product, which sends the semVerLevel query string argument as per the NuGet server spec.

ProGet relies on the IsLatestVersion property being set because it is used by the UI to determine the necessary redirect when an explicit version is not supplied while browsing a feed, but more importantly it is used to serve script-related PowerShellGet module commands (e.g. Install-Script) that don't require an explicit version (which is pretty much all of them).

If the &semVerLevel=2.0.0 argument is removed from the example reproduction URL, then the IsLatestVersion element's value is true as expected.

[Liquidmantis]

This has been causing pain for our group. We use ProGet as a proxy server for the PS Gallery so that systems that don't have Internet access can install and update modules. This has been broken for a while now (I believe it corresponded with the powershellgallery.com refresh).

[jrasch]

This appears to be fixed now.

[edyoung]

That is strange, because we have not made a change to address this.

[Liquidmantis]

It hasn't changed for us. I also upgraded Proget from 5.1.10 to 5.1.15 and see the same behavior.

[jrasch]

My mistake, there was another version of that package released so the IsAbsoluteLatestVersion property became true, but IsLatestVersion still does not work when that is used as the filter:

https://www.powershellgallery.com/api/v2/FindPackagesById()?$filter=IsLatestVersion&id=%27WindowsCompatibility%27&semVerLevel=2.0.0

[apxltd]

Hey @SydneyhSmith @edyoung --

I think you weren't aware of this, but this particular issue is unfortunately causing some of your big customers a whole lot of pain and lot of lost labor because it's made PowerShellGallery effectively unusable in this "proxy" situation. As a result of this bug, they have to dedicate time to manually recreating and copying packages... and It's impossible for our software (ProGet) to work-around the bug because ultimately the data on your end is bad/unpredictable.

Can you provide us with a channel to escalate this so this gets on your radar, and gives your customers some visibility as to when or if it will be fixed?

We have spent quite a bit of time working with our customers to try to navigate Microsoft Enterprise Support, Microsoft Platform Support, etc, and ultimately we just get (very nicely) bounced with "not our problem, please call this 1800 number instead."

Who should we try officially contacting on the inside to get you guys the resources to fix this?

Thanks!

Alex

[edyoung]

I'm the engineering manager for the team which runs the powershell gallery (among other things) so there's no particular need to officially contact anyone else.

To be honest it was not obvious from the issue that this was particularly pressing, so it hasn't received much attention so far.

We will take a look shortly. If you want to provide customers an avenue to add their +1, please point them to this issue or the Powershell gallery support alias [email protected]

[apxltd]

Thanks so much Ed! I had no idea it was this kind of problem either... I had just gotten pulled into it myself.

[ephos]

To be honest it was not obvious from the issue that this was particularly pressing, so it hasn't received much attention so far.
We will take a look shortly. If you want to provide customers an avenue to add their +1, please point them to this issue or the Powershell gallery support alias [email protected]

Hi @edyoung the recent activity on this thread may stem from an issue we found using ProGet and trying to proxy the PowerShell Gallery. We originally looped in Inedo and they referenced this issue. We had also been in contact with Microsoft. I can provide any more information needed as well if needed. I also +1'd the items above.

[edyoung]

Hi folks! It would help to get a really concrete repro case which captures exactly what the problem is. In my testing https://www.powershellgallery.com/api/v2/FindPackagesById()?$filter=IsLatestVersion&id=%27WindowsCompatibility%27&semVerLevel=2.0.0 returns version 1.0.0 with both IsAbsoluteLatestVersion=true and IsLatestVersion=true, which I believe is correct.

If you can add a few queries which produce the wrong answer, and specifically what is wrong with them, that'll help.

[jrasch]

Here is an example of a package that shows false for IsLatestVersion and IsAbsoluteLatestVersion even though there's only a single version:

https://www.powershellgallery.com/api/v2/FindPackagesById()?$filter=IsLatestVersion&id=%27xDisk%27&semVerLevel=2.0.0

<d:IsLatestVersion m:type="Edm.Boolean">false</d:IsLatestVersion><d:IsAbsoluteLatestVersion m:type="Edm.Boolean">false</d:IsAbsoluteLatestVersion><d:IsPrerelease m:type="Edm.Boolean">false</d:IsPrerelease>

The values for those elements become correct if you drop the &semVerLevel=2.0.0 argument:

https://www.powershellgallery.com/api/v2/FindPackagesById()?$filter=IsLatestVersion&id=%27xDisk%27

<d:IsLatestVersion m:type="Edm.Boolean">true</d:IsLatestVersion><d:IsAbsoluteLatestVersion m:type="Edm.Boolean">true</d:IsAbsoluteLatestVersion><d:IsPrerelease m:type="Edm.Boolean">false</d:IsPrerelease>

[thijsvanbloemendaal]

I'm experiencing the same issue. I'm unable to pull the package to proget:

The package could not be installed.
The underlying connection was closed: An unexpected error occurred on a send.

And when I try to download:

Cannot download package from https://www.powershellgallery.com/api/v2

Anyone found a solution?

[apxltd]

Seems to be some kind of gallery outage. We’ve had a few customers express same. Should go away :)

[alerickson]

This issue seems to occur whenever a module version only has a major and minor number (eg https://www.powershellgallery.com/packages/NtpTime/1.1)

[awickham10]

We're also experiencing this when using Inedo ProGet as a proxy to PowerShell Gallery. Here's a module we ran into it with. The module is using correct semver (8.0.6).

https://www.powershellgallery.com/api/v2/FindPackagesById()?$filter=IsLatestVersion&id=%27IsilonPlatform%27 produces correct results (IsAbsoluteLatestVersion and IsLatestVersion are both true).

https://www.powershellgallery.com/api/v2/FindPackagesById()?$filter=IsLatestVersion&id=%27IsilonPlatform%27&semVerLevel=2.0.0 produces incorrect results (IsAbsoluteLatestVersion and IsLatestVersion are both false).

This issue has been out here for quite a while now. Is there anything else I can do to help speed things along?

[awickham10]

Some more examples:

https://www.powershellgallery.com/api/v2/FindPackagesById()?$filter=IsLatestVersion&id=%27VMware.VumAutomation%27&semVerLevel=2.0.0

https://www.powershellgallery.com/api/v2/FindPackagesById()?$filter=IsLatestVersion&id=%27VMware.VimAutomation.vROps%27&semVerLevel=2.0.0

https://www.powershellgallery.com/api/v2/FindPackagesById()?$filter=IsLatestVersion&id=%27VMware.VimAutomation.HA%27&semVerLevel=2.0.0

https://www.powershellgallery.com/api/v2/FindPackagesById()?$filter=IsLatestVersion&id=%27VMware.VimAutomation.PCloud%27&semVerLevel=2.0.0

[awickham10]

More examples:

https://www.powershellgallery.com/api/v2/FindPackagesById()?$filter=IsLatestVersion&id=%27CredentialManager%27&semVerLevel=2.0.0

https://www.powershellgallery.com/api/v2/FindPackagesById()?$filter=IsLatestVersion&id=%27Trackyon.Utils%27&semVerLevel=2.0.0

[apxltd]

Paging @edyoung @alerickson

Anyone we can escalate this to, so you can get some resources to fix it? Or at least an ETA? This is a continuing pain point for our mutual customers

[alerickson]

@apxltd actually I'd really like to prioritize this. Clearly there's a number of folks here hitting roadblocks with this issue and it's causing me some frustrations as well. Let me look into this to see how we can get the metadata properly propagated. Conservative ETA will be May, but hoping to have a fix for it sooner.

[awickham10]

@apxltd actually I'd really like to prioritize this. Clearly there's a number of folks here hitting roadblocks with this issue and it's causing me some frustrations as well. Let me look into this to see how we can get the metadata properly propagated. Conservative ETA will be May, but hoping to have a fix for it sooner.

We're on hold rolling out our internal gallery until this issue is resolved so that is fantastic news. Thank you!

[awickham10]

@apxltd actually I'd really like to prioritize this. Clearly there's a number of folks here hitting roadblocks with this issue and it's causing me some frustrations as well. Let me look into this to see how we can get the metadata properly propagated. Conservative ETA will be May, but hoping to have a fix for it sooner.

Any idea on when this fix will be available yet? Thanks!

[SydneyhSmith]

@awickham10 unfortunately still no update, but getting higher on our todo list, we are still aiming for the late May timeframe...although there are current restrictions on our deployments so it is entirely possible this may be delayed. Thanks for checking in, and we will be sure to update the issue once the fix is deployed.

[awickham10]

Just doing a quick follow up on this. Any updates on timing?

[awickham10]

I know COVID-19 has provided unique circumstances here but are there any updates? We're getting ready to cut Internet access to a bunch of servers and without this bug fixed none of our servers will be able to install modules from the PowerShell Gallery without jumping through hoops.

[apxltd]

@awickham10 one thing that might help is to escalate via your internal channels until it gets high enough that they can request some resources for the team...

ProGet doesn't have this bug, so I guess, one thing you could do is just do a download of all their packages, then re-upload them to ProGet? This would further keep your servers from the internet.

One of our customers wrote this, https://github.com/healthwise/nuget.sync/blob/master/README.md

There's less than 50k packages on the PowerShellGallery site, so maybe it won't be too bad? But you may get rate-limited, so be careful...

Anyways if you find something that's workable, please do let us know, and we'll update our documentation at the very least with a solution.

[awickham10]

@awickham10 one thing that might help is to escalate via your internal channels until it gets high enough that they can request some resources for the team...

ProGet doesn't have this bug, so I guess, one thing you could do is just do a download of all their packages, then re-upload them to ProGet? This would further keep your servers from the internet.

One of our customers wrote this, https://github.com/healthwise/nuget.sync/blob/master/README.md

There's less than 50k packages on the PowerShellGallery site, so maybe it won't be too bad? But you may get rate-limited, so be careful...

Anyways if you find something that's workable, please do let us know, and we'll update our documentation at the very least with a solution.

Thanks @apxltd - I appreciate the information. @SydneyhSmith is there any time frame on when this will be fixed should I go forward with a workaround?

[apxltd]

@awickham10 I'm curious if you were able to work-around this bug and roll out an internal gallery? This keeps hitting our customers, and I don't think it will get fixed, so I'd love hear if you found a solution...

[awickham10]

@awickham10 I'm curious if you were able to work-around this bug and roll out an internal gallery? This keeps hitting our customers, and I don't think it will get fixed, so I'd love hear if you found a solution...

I'm currently working on getting the nuget sync program you mentioned running. I had to modify the code a bit to support Windows integrated authentication with ProGet and do some validation to make sure the IsAbsoluteLatestVersion bug doesn't carry over from PowerShell Gallery into ProGet. I'll post results soon.

[awickham10]

@apxltd After some unsuccessful attempts to getting the NuGet Sync program running with our scenario I wrote a PowerShell script to pull the new packages for the day and use nuget push to push them to a "PSGallery" repo on our ProGet server.

@SydneyhSmith or @alerickson is there any update on this issue?

[apxltd]

Thanks for the update @awickham10 , glad its working... to some extent. Definitely not a fan of you having to rely on a daily push script though.

By this point, I can't imagine we will hear back from Microsoft at this point, so we may as well consider this bug WONTFIX. I don't think the PSGallery is really actively maintained anymore to be honest?

Anyways, now that you've gotten more and more familiar w/ the issue, any ideas on what might be done in ProGet to work-around it? It's been months since we've thought about it, and we were hoping the bug would get fixed.

We used to have to do weird quirks on the nuget.org side of things, and maybe we can have some logic like "if connector=powershellgallery.com, do this quirky..." or something like that. ...

[awickham10]

@awickham10 unfortunately still no update, but getting higher on our todo list, we are still aiming for the late May timeframe...although there are current restrictions on our deployments so it is entirely possible this may be delayed. Thanks for checking in, and we will be sure to update the issue once the fix is deployed.

Any update on this?

[apxltd]

@awickham10 I think the lack of updates means they won't fix this...

Anyways, let's let's "restart" this discussion at https://forums.inedo.com/ -- we'll look at how to workaround it in ProGet. Would you mind summarizing/framing as a ProGet-workaround (i.e. what can we do on our end to workaround the problem)? This thread has gotten tricky to follow, and I even lost track myself of what the problem is...

That said, my only hesitation right now is that Microsoft is working on a whole new client (and presumably gallery), which will make this particular bug moot.... but also probably add a bunch new. See: https://devblogs.microsoft.com/powershell/powershellget-3-0-preview-11-release/

When released/stable? I would anticipate mid-2022 / early-2023? Hard to say. It looks like it has a lot of ways to go..

[ephos]

@SydneyhSmith I completely forgot I had run into this issue 4 years ago until today 😅. Curious if this problem gets resolved by PSResourceGet or if this is still an issue on the PSGallery side? If it is a Gallery issue do you have an ETA on the fix?