Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems running bundler installed standard and rubocop #340

Closed
hurricup opened this issue Oct 12, 2021 · 24 comments
Closed

Problems running bundler installed standard and rubocop #340

hurricup opened this issue Oct 12, 2021 · 24 comments

Comments

@hurricup
Copy link

Gemfile:

source 'https://rubygems.org'
gem 'rubocop'
gem 'standard'

resolved to Gemfile.lock

GEM
  remote: https://rubygems.org/
  specs:
    ast (2.4.2)
    parallel (1.21.0)
    parser (3.0.2.0)
      ast (~> 2.4.1)
    rainbow (3.0.0)
    regexp_parser (2.1.1)
    rexml (3.2.5)
    rubocop (1.22.1)
      parallel (~> 1.10)
      parser (>= 3.0.0.0)
      rainbow (>= 2.2.2, < 4.0)
      regexp_parser (>= 1.8, < 3.0)
      rexml
      rubocop-ast (>= 1.12.0, < 2.0)
      ruby-progressbar (~> 1.7)
      unicode-display_width (>= 1.4.0, < 3.0)
    rubocop-ast (1.12.0)
      parser (>= 3.0.1.1)
    ruby-progressbar (1.11.0)
    standard (0.0.36)
      rubocop (>= 0.63)
    unicode-display_width (2.1.0)

PLATFORMS
  x86_64-linux

DEPENDENCIES
  rubocop
  standard

BUNDLED WITH
   2.2.26

When running:

bundle exec standardrb -a test.rb 
/home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml: Performance/RedundantSortBy has the wrong namespace - should be Style
/home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml: Performance/Sample has the wrong namespace - should be Style
/home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml: Rails/SafeNavigation has the wrong namespace - should be Style
bundler: failed to load command: standardrb (/home/user/.rvm/gems/ruby-3.0.2@rubocop/bin/standardrb)
/home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/rubocop-1.22.1/lib/rubocop/config_obsoletion.rb:43:in `reject_obsolete!': The `Layout/AlignArray` cop has been renamed to `Layout/ArrayAlignment`. (RuboCop::ValidationError)
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Layout/AlignHash` cop has been renamed to `Layout/HashAlignment`.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Layout/AlignParameters` cop has been renamed to `Layout/ParameterAlignment`.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Layout/IndentArray` cop has been renamed to `Layout/FirstArrayElementIndentation`.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Layout/IndentAssignment` cop has been renamed to `Layout/AssignmentIndentation`.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Layout/IndentHash` cop has been renamed to `Layout/FirstHashElementIndentation`.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Layout/IndentHeredoc` cop has been renamed to `Layout/HeredocIndentation`.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Layout/LeadingBlankLines` cop has been renamed to `Layout/LeadingEmptyLines`.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Layout/Tab` cop has been renamed to `Layout/IndentationStyle`.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Layout/TrailingBlankLines` cop has been renamed to `Layout/TrailingEmptyLines`.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Lint/DuplicatedKey` cop has been renamed to `Lint/DuplicateHashKey`.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Lint/EndInMethod` cop has been renamed to `Style/EndBlock`.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Lint/MultipleCompare` cop has been renamed to `Lint/MultipleComparison`.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Lint/StringConversionInInterpolation` cop has been renamed to `Lint/RedundantStringCoercion`.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Lint/UnneededRequireStatement` cop has been renamed to `Lint/RedundantRequireStatement`.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Lint/UnneededSplatExpansion` cop has been renamed to `Lint/RedundantSplatExpansion`.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Naming/UncommunicativeBlockParamName` cop has been renamed to `Naming/BlockParameterName`.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Style/UnneededCapitalW` cop has been renamed to `Style/RedundantCapitalW`.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Style/UnneededCondition` cop has been renamed to `Style/RedundantCondition`.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Style/UnneededInterpolation` cop has been renamed to `Style/RedundantInterpolation`.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Style/UnneededPercentQ` cop has been renamed to `Style/RedundantPercentQ`.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Lint/UselessComparison` cop has been removed since it has been superseded by `Lint/BinaryOperatorWithIdenticalOperands`. Please use `Lint/BinaryOperatorWithIdenticalOperands` instead.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
The `Style/MethodMissingSuper` cop has been removed since it has been superseded by `Lint/MissingSuper`. Please use `Lint/MissingSuper` instead.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
`Performance` cops have been extracted to the `rubocop-performance` gem.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
`Rails` cops have been extracted to the `rubocop-rails` gem.
(obsolete configuration found in /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/config/base.yml, please update it)
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/rubocop-1.22.1/lib/rubocop/config_validator.rb:78:in `check_obsoletions'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/rubocop-1.22.1/lib/rubocop/config_validator.rb:44:in `validate'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/rubocop-1.22.1/lib/rubocop/config.rb:49:in `check'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/rubocop-1.22.1/lib/rubocop/config.rb:38:in `create'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/rubocop-1.22.1/lib/rubocop/config_loader.rb:55:in `load_file'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/rubocop-1.22.1/lib/rubocop/config_store.rb:29:in `options_config='
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/lib/standard/creates_config_store/assigns_rubocop_yaml.rb:6:in `call'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/lib/standard/creates_config_store.rb:17:in `block in call'
        from <internal:kernel>:90:in `tap'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/lib/standard/creates_config_store.rb:16:in `call'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/lib/standard/builds_config.rb:22:in `call'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/lib/standard/cli.rb:16:in `run'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/standard-0.0.36/exe/standardrb:7:in `<top (required)>'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/bin/standardrb:23:in `load'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/bin/standardrb:23:in `<top (required)>'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/bundler-2.2.26/lib/bundler/cli/exec.rb:58:in `load'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/bundler-2.2.26/lib/bundler/cli/exec.rb:58:in `kernel_load'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/bundler-2.2.26/lib/bundler/cli/exec.rb:23:in `run'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/bundler-2.2.26/lib/bundler/cli.rb:477:in `exec'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/bundler-2.2.26/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/bundler-2.2.26/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/bundler-2.2.26/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/bundler-2.2.26/lib/bundler/cli.rb:31:in `dispatch'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/bundler-2.2.26/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/bundler-2.2.26/lib/bundler/cli.rb:25:in `start'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/bundler-2.2.26/exe/bundle:49:in `block in <top (required)>'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/bundler-2.2.26/lib/bundler/friendly_errors.rb:128:in `with_friendly_errors'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/gems/bundler-2.2.26/exe/bundle:37:in `<top (required)>'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/bin/bundle:23:in `load'
        from /home/user/.rvm/gems/ruby-3.0.2@rubocop/bin/bundle:23:in `<main>'
@searls
Copy link
Contributor

searls commented Oct 12, 2021

That's a very old version of Standard (one that had a more relaxed constraint on rubocop, which is why Bundler's solver apparently picked it. Please try running bundle update standard and if that doesn't work, set a version specifier of at least 1.0

@searls searls closed this as completed Oct 12, 2021
@michaelglass
Copy link

michaelglass commented Jan 2, 2023

something happened recently that installed exactly that ancient version of standard... Just commenting for posterity. I'm not exactly sure what I did 😅

If you add standard to your Gemfile after installing rubocop, you may end up with this ancient version which does not have an upper version bound for rubocop.

Subsequent versions of standard have max bounds on the rubocop version so if the Gemfile in your lockfile is too high, bundler will calculate that the ancient standard is a compatible version 😅

@markijbema
Copy link

Yeah I ran into this too. Took some googling to get here, but it gave me the first impression of an abandoned project. Maybe standard could release one special version in between serious releases (like rspec did with 2.99) which is compatible with all rubocop versions but just prints an instruction of what to do? Seems to me that switching from rubocop to standard is quite natural and it did not occur to me to remove the rubocop gems first

@searls
Copy link
Contributor

searls commented Feb 16, 2023

That's a good idea @markijbema. It seems like our options are pretty limited.

We should probably consider a new release:

  • With version 0.0.36.1 so it narrowly captures this edge case
  • Specifies rubocop, > 0.63 (as opposed to >=, so it doesn't break things that legitimately are that old and intentionally lock [email protected] or earlier)
  • That prints a long postinstall message to a link to a document explaining WTF is going on
  • Prints a similar message every time it's run
  • (Maybe?) prints a similar message every time it's loaded
  • (Maybe?) raises an error every time it's run
  • (Probably not?) raises an error every time it's loaded

Thoughts?

@searls searls reopened this Feb 16, 2023
@michaelglass
Copy link

Are there cases when standard is loaded outside of the context of

  • from its own executable
  • within rubocop?

If so, then I guess it would make sense to avoid raising on load.

Otherwise, raising on load would reduce the noise of the subsequent slew of rubocop errors.

@searls
Copy link
Contributor

searls commented Feb 17, 2023

Are there cases when standard is loaded outside of the context of

  • from its own executable
  • within rubocop?

No, but anyone who doesn't have require: false next to it in the Gemfile would see their process explode if they have Bundler require everything. The thinking of doing this would be that since people would only hit this if they first install or update the gem, if we can slam on the brakes for them right away before they've committed/pushed/moved on we can spare them confusion and frustration.

@searls
Copy link
Contributor

searls commented Apr 4, 2023

Working on a version 0.0.36.1 now.

I've drafted this message and think it should run on postinstall as well as with every run of the CLI or rake tasks

WARNING: You do not want to run this very, very old version of Standard!

What's going on here?
---------------------
Versions of Standard prior to 0.0.37 depended on `>= 0.63' of RuboCop, which
means that Bundler continues to resolve to those (now ancient) versions when
whenever `gem "standard"' is added to a Gemfile that has already locked
to a newer version of RuboCop than standard currently depends on.

"How do I fix this?", you might be asking.

How to fix this
---------------
If you add a version specifier that pins `standard' to a version newer
than 0.0.36.1, Bundler will resolve the newest versions of `standard',
`rubocop', and any other rubocop-dependent gems you may have installed.

1. Update your Gemfile to pin standard to be at least one such version:

  gem "standard", ">= 1.0"

2. Run `bundle`, which will install and lock more appropriate versions

  Example output:
    Using rubocop 1.48.1 (was 1.49.0)
    Using rubocop-performance 1.16.0
    Using standard 1.26.0 (was 0.0.36)

We're very sorry for this inconvenience!

UPDATE: Simplifed the message

UPDATE 2: Simplified again

Feedback on this message?

searls added a commit that referenced this issue Apr 4, 2023
This is to mitigate the risk identified in #340
@czeise
Copy link
Contributor

czeise commented Apr 4, 2023

Message is great. The problem is clearly explained as is the fix.

Small typo: There's an extra "when" in the "What's going on here?" section that should be removed:

...which means that Bundler continues to resolve to those (now ancient) versions when whenever `gem "standard"' is added to a Gemfile...

@searls
Copy link
Contributor

searls commented Apr 5, 2023

Okay, I've released v0.0.36.1

If you add standard and Bundle resolves it for whatever reason, you'll see this:

$ bundle
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies...
Using ast 2.4.2
Using bundler 2.4.10
Using json 2.6.3
Using parallel 1.22.1
Using regexp_parser 2.7.0
Using rexml 3.2.5
Using unicode-display_width 2.4.2
Using ruby-progressbar 1.13.0
Using rainbow 3.1.1
Using parser 3.2.2.0
Using rubocop-ast 1.28.0
Using rubocop 1.49.0
Fetching standard 0.0.36.1
Installing standard 0.0.36.1
Bundle complete! 2 Gemfile dependencies, 13 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
Post-install message from standard:
============================================================================
= WARNING: You do not want to run this very, very old version of Standard! =
============================================================================

What's going on here?
---------------------
Versions of Standard prior to 0.0.37 depended on `>= 0.63' of RuboCop, which
means that Bundler continues to resolve to those (now ancient) versions
whenever `gem "standard"' is added to a Gemfile that has already locked
to a newer version of RuboCop than standard currently depends on.

"How do I fix this?", you might be asking.

How to fix this
---------------
If you add a version specifier pinning `standard' to a version newer
than 0.0.36.1, Bundler will resolve appropriate versions of `standard',
`rubocop', and any other rubocop-dependent gems you may have installed.

1. Update your Gemfile to pin standard to be at least one such version:

  gem "standard", ">= 1.0"

2. Run `bundle`, which will install and lock more appropriate versions

  Example output:
    Using rubocop 1.48.1 (was 1.49.0)
    Using rubocop-performance 1.16.0
    Using standard 1.26.0 (was 0.0.36)

This version (0.0.36.1) is an inoperative placeholder gem that exists
solely to print this message.

We're very sorry for this inconvenience!

============================================================================
=                         END OF BIG SCARY WARNING                         =
============================================================================

You'll see the same message printed whenever the standardrb executable or standard rake tasks are invoked.

@sferik
Copy link

sferik commented Aug 15, 2023

Might I be so bold to suggest yanking all gem versions of Standard prior to 0.0.37? On one hand, it’s possible that someone has a production deployment locked to one of these versions, which are now more than 4 years old. On the other hand, Standard shouldn’t be part of the production bundle and if you try running that version of Standard you get:

ArgumentError: Unknown Ruby version: 3.0

All versions of Ruby prior to 3.0 are EOL as of March 31, 2023, so anyone using this code is not only running a badly outdated version of Standard, they are also running a known insecure version of Ruby. If yanking Standard breaks someone’s deployment, that’s probably a good wakeup call to update Ruby before something much worse happens.

For your convenience, here is a command to generate a list of all the gem versions that should be yanked:

gem search '^standard$' --all | sed 's/[^0-9., ]*//g' | sed 's/,/\n/g' | xargs ruby -e 'ARGV.each { |version| puts version if Gem::Version.new(version) < Gem::Version.new("0.0.37") }'

According to Rubygems, people have run into this issue nearly 5,000 times over the past 4 months.

@searls
Copy link
Contributor

searls commented Aug 16, 2023

@sferik thanks for this comment, Erik.

This is really tough for me, because I am a pretty firm believer that package repositories should be write-only and relied upon as immutable as a matter of principle. But the practical reality of how Bundler's solver currently works is that the existence of these old releases is causing a significant amount of harm to people for (if I'm being honest) an at-best theoretical benefit of keeping every release up in perpetuity on RubyGems.

After thinking about it overnight, I'm 👍 on doing this.

@MikeMcQuaid
Copy link

All versions of Ruby prior to 3.0 are EOL as of March 31, 2023, so anyone using this code is not only running a badly outdated version of Standard, they are also running a known insecure version of Ruby.

Don't have enough context here but from a random Homebrew maintainer's perspective: anyone using /usr/bin/ruby on any version of macOS after March 31, 2023 is using an EOLd Ruby. That doesn't matter to most of us who are doing "Modern Ruby Development" but to a non-trivial number people on macOS (and other platforms in similar situations): they aren't knowingly doing anything wrong.

@jamiemccarthy
Copy link
Contributor

Brownouts first? A new version 0.0.36.2, released now, could start intentionally breaking a few CI builds starting in (say) October. Just 10% of builds on weekends at first. With a verbose message saying to upgrade (or remove) itself. But it could steadily expand the days of the week / percentage, until (say) April 2024, the fifth anniversary of the dependency fix, when it breaks all builds.

@searls
Copy link
Contributor

searls commented Aug 16, 2023

@jamiemccarthy it is already a blackout. 0.0.36.1 intentionally disables the CLI and exits nonzero

@michaelglass
Copy link

My 2 cents:

Status quo

  • helps legacy projects that have an older standardrb locked.
  • hurts new adopters who install rubocop before standardrb.

Yanking flips who's helped / hurt.

@searls
Copy link
Contributor

searls commented Aug 16, 2023

Yes, thats the tradeoff, but I am dubious that there are any actual users depending on 0.0.1 to 0.0.36. As a result I believe more people are harmed by the current state than are helped

Is there a way to query RG.o for version downloads over a time range?

@jamiemccarthy
Copy link
Contributor

it is already a blackout

Okay, then I'd feel less bad about yanking.

It might not hurt to put the phrase Could not find gem standard in a section of the README that explains all this, and let Google pick that up first. That's probably what I'd search on if my bundle install suddenly started failing. And that README section wouldn't have to be permanent.

@searls
Copy link
Contributor

searls commented Aug 16, 2023

@jamiemccarthy agree. My plan would be to lock a Gemfile to 0.0.36 before yanking, yank, then copy the resulting error message into the README

@sferik
Copy link

sferik commented Aug 16, 2023

Don't have enough context here but from a random Homebrew maintainer's perspective: anyone using /usr/bin/ruby on any version of macOS after March 31, 2023 is using an EOLd Ruby. That doesn't matter to most of us who are doing "Modern Ruby Development" but to a non-trivial number people on macOS (and other platforms in similar situations): they aren't knowingly doing anything wrong.

@MikeMcQuaid Yanking existing gems should only affect new production deployments. I suppose it’s possible that some number of people deploy a 4-year-old version of Standard into production on macOS, but I suspect that number is somewhere between non-existent and vanishingly small. If someone already has the gem installed locally, they can continue using it, even if it’s yanked on Rubygems. If at some point, they decide to reinstall their app/library or update their Ruby version, bundle install will find a newer version of Standard, which will presumably be a welcome improvement over early versions.

PS: Thanks for taking this feedback. I’m happy to see swift action on this. 😊

@sferik
Copy link

sferik commented Aug 16, 2023

@jamiemccarthy agree. My plan would be to lock a Gemfile to 0.0.36 before yanking, yank, then copy the resulting error message into the README

@searls When attempting to install a yanked gem using gem install standard --version 0.0.36, the error will look like this:

ERROR:  Could not find a valid gem 'standard' (= 0.0.36) in any repository

When attempting to install a yanked gem in development via bundle install, the error will look like this:

Could not find gem 'standard (= 0.0.36)' in rubygems repository https://rubygems.org/ or installed locally.

When attempting to install a yanked gem in production via bundle install --deployment, the error will look like this:

Your bundle is locked to standard (0.0.36) from rubygems repository https://rubygems.org/ or installed locally, but that version can no longer be found in that source. That means the author of standard (0.0.36) has removed it. You'll need to update your bundle to a version other than standard (0.0.36) that hasn't been removed in order to install.

@sferik
Copy link

sferik commented Aug 16, 2023

Is there a way to query RG.o for version downloads over a time range?

@searls I’m pretty familiar with the Rubygems.org API and I don’t believe this is possible. (It is however possible to query for downloads all time for a particular gem version.)

I assume you want to check how many recent downloads there have been of Standard gems earlier than 0.0.36.1? The easiest thing is probably just to check the download stats, write the results somewhere, and check again some period of time later to get a sense of the rate. Here is a snapshot for today (taken at approximately 2:25 PM Pacific Time):

gem search '^standard$' --all | sed 's/[^0-9., ]*//g' | sed 's/,/\n/g' | xargs ruby -e 'ARGV.each { |version| puts version if Gem::Version.new(version) < Gem::Version.new("0.0.37") }' | xargs ruby -r json -r net/http -e 'puts "| Version | Downloads at #{Time.now.utc} |"; puts "| --- | --- |"; ARGV.each { |version| puts "| #{version} | #{JSON.parse(Net::HTTP.get_response(URI("https://rubygems.org/api/v1/downloads/standard-#{version}.json")).body)["version_downloads"]} |" }'
Version Downloads at 2023-08-16 21:25:20 UTC
0.0.36.1 4872
0.0.36 596315
0.0.35 1384
0.0.34 1428
0.0.33 2734
0.0.32 1816
0.0.31 2385
0.0.30 2444
0.0.29 1485
0.0.28 1392
0.0.27 1399
0.0.26 3523
0.0.25 1759
0.0.24 15268
0.0.23 2075
0.0.22 1391
0.0.21 1392
0.0.20 1414
0.0.19 4921
0.0.18 1420
0.0.17 1516
0.0.16 1531
0.0.15 1838
0.0.14 1527
0.0.13 1798
0.0.12 4613
0.0.11 1481
0.0.10 1802
0.0.9 1448
0.0.8 1529
0.0.7 1473
0.0.6 1451
0.0.5 1439
0.0.4 1491
0.0.3 1496
0.0.2 1429
0.0.1 1447

For your convenience, here is an alternative version of the command with CSV output:

gem search '^standard$' --all | sed 's/[^0-9., ]*//g' | sed 's/,/\n/g' | xargs ruby -e 'ARGV.each { |version| puts version if Gem::Version.new(version) < Gem::Version.new("0.0.37") }' | xargs ruby -r csv -r json -r net/http -e 'puts ["Version", "Downloads at #{Time.now.utc}"].to_csv; ARGV.each { |version| puts [version, JSON.parse(Net::HTTP.get_response(URI("https://rubygems.org/api/v1/downloads/standard-#{version}.json")).body)["version_downloads"]].to_csv }'
Version,Downloads at 2023-08-16 21:25:29 UTC
0.0.36.1,4872
0.0.36,596315
0.0.35,1384
0.0.34,1428
0.0.33,2734
0.0.32,1816
0.0.31,2385
0.0.30,2444
0.0.29,1485
0.0.28,1392
0.0.27,1399
0.0.26,3523
0.0.25,1759
0.0.24,15268
0.0.23,2075
0.0.22,1391
0.0.21,1392
0.0.20,1414
0.0.19,4921
0.0.18,1420
0.0.17,1516
0.0.16,1531
0.0.15,1838
0.0.14,1527
0.0.13,1798
0.0.12,4613
0.0.11,1481
0.0.10,1802
0.0.9,1448
0.0.8,1529
0.0.7,1473
0.0.6,1451
0.0.5,1439
0.0.4,1491
0.0.3,1496
0.0.2,1429
0.0.1,1447

@sferik
Copy link

sferik commented Aug 17, 2023

Okay, I ran the scripts again approximately 25 hours later and calculated the diffs. Here are the results. In summary:

  1. There appears to be a bot that downloads every version once per day.
  2. Version 0.0.36.1 had 51 non-bot downloads.
  3. Version 0.0.36 had 223 non-bot downloads.
  4. Version 0.0.24 had 6 non-bot downloads.

By comparison, the latest version of Standard (1.30.1) has been downloaded 227,650 times since its release on July 7, 2023, for an average of 5,650 non-bot downloads per day.

Does this make you reconsider your thinking about yanking 0.0.36?

Version Downloads at 2023-08-16 21:25:29 UTC Downloads at 2023-08-17 22:16:42 UTC Diff
0.0.36.1 4872 4924 52
0.0.36 596315 596539 224
0.0.35 1384 1385 1
0.0.34 1428 1429 1
0.0.33 2734 2735 1
0.0.32 1816 1817 1
0.0.31 2385 2386 1
0.0.30 2444 2445 1
0.0.29 1485 1486 1
0.0.28 1392 1393 1
0.0.27 1399 1400 1
0.0.26 3523 3524 1
0.0.25 1759 1760 1
0.0.24 15268 15275 7
0.0.23 2075 2076 1
0.0.22 1391 1392 1
0.0.21 1392 1393 1
0.0.20 1414 1415 1
0.0.19 4921 4922 1
0.0.18 1420 1421 1
0.0.17 1516 1517 1
0.0.16 1531 1532 1
0.0.15 1838 1839 1
0.0.14 1527 1528 1
0.0.13 1798 1799 1
0.0.12 4613 4614 1
0.0.11 1481 1482 1
0.0.10 1802 1803 1
0.0.9 1448 1449 1
0.0.8 1529 1530 1
0.0.7 1473 1474 1
0.0.6 1451 1452 1
0.0.5 1439 1440 1
0.0.4 1491 1492 1
0.0.3 1496 1497 1
0.0.2 1429 1430 1
0.0.1 1447 1448 1

Here’s the data in CSV format if you want to play with it yourself:

Version,Downloads at 2023-08-16 21:25:29 UTC,Downloads at 2023-08-17 22:16:42 UTC,Diff
0.0.36.1,4872,4924,52
0.0.36,596315,596539,224
0.0.35,1384,1385,1
0.0.34,1428,1429,1
0.0.33,2734,2735,1
0.0.32,1816,1817,1
0.0.31,2385,2386,1
0.0.30,2444,2445,1
0.0.29,1485,1486,1
0.0.28,1392,1393,1
0.0.27,1399,1400,1
0.0.26,3523,3524,1
0.0.25,1759,1760,1
0.0.24,15268,15275,7
0.0.23,2075,2076,1
0.0.22,1391,1392,1
0.0.21,1392,1393,1
0.0.20,1414,1415,1
0.0.19,4921,4922,1
0.0.18,1420,1421,1
0.0.17,1516,1517,1
0.0.16,1531,1532,1
0.0.15,1838,1839,1
0.0.14,1527,1528,1
0.0.13,1798,1799,1
0.0.12,4613,4614,1
0.0.11,1481,1482,1
0.0.10,1802,1803,1
0.0.9,1448,1449,1
0.0.8,1529,1530,1
0.0.7,1473,1474,1
0.0.6,1451,1452,1
0.0.5,1439,1440,1
0.0.4,1491,1492,1
0.0.3,1496,1497,1
0.0.2,1429,1430,1
0.0.1,1447,1448,1

@searls
Copy link
Contributor

searls commented Aug 17, 2023

It's interesting that 0.0.36 had 4 times as many downloads as 0.0.36.1. What that tells me is it's likely a few hundred repos/builds out there locked to it due to the OP's issue before we cut 0.0.36.1, and it's still installing because it's in the lockfile. My sincere hope is that standard isn't actually being executed by these projects (and if it is, not in a way where a working version would change anything). In short, I think they're probably just zombie installs. I mean, there are 50 installs a day of a version that we intentionally broke.

So, no, this doesn't change my mind but the data is interesting. I think I'd rather rip off the bandaid now that you've had me thinking about this and be prepared to help people deal with the error messages

searls added a commit that referenced this issue Aug 19, 2023
@searls
Copy link
Contributor

searls commented Aug 19, 2023

Ok, this is now done. I've gone through and yanked all versions prior to 0.0.37, which contained an unbounded (>=) dependency on rubocop.

Thank you very much to @sferik for the kick in the pants and the detailed analysis, data, and instructions on how to do this gracefully. I also updated the README to feed google for anyone who runs into errors trying to install old versions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants