Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unresolved or ambiguous specs during Gem::Specification.reset #7657

Closed
n-rodriguez opened this issue May 13, 2024 · 19 comments
Closed

Unresolved or ambiguous specs during Gem::Specification.reset #7657

n-rodriguez opened this issue May 13, 2024 · 19 comments
Labels

Comments

@n-rodriguez
Copy link

Hi there!

I got this warning in CI when running my app with TruffleRuby 24.0.1 :

$ bin/bundle install --jobs ${PARALLEL_TEST_PROCESSORS}
WARN: Unresolved or ambiguous specs during Gem::Specification.reset:
      stringio (>= 0)
      Available/installed versions of this gem:
      - 3.1.0
      - 3.0.4
WARN: Clearing out unresolved specs. Try 'gem cleanup <gem>'
Please report a bug if this causes problems.
Bundle complete! 204 Gemfile dependencies, 390 gems now installed.
Bundled gems are installed into `/cache/nicolas/concerto/truffleruby-wip-truffleruby`

I don't have it with Ruby 3.3.1.

stringio is declared in my Gemfile :

nicolas@MacBook-Pro-de-Nicolas:~/PROJECTS/CONCERTO/concerto$ grep stringio Gemfile Gemfile.lock
Gemfile:  gem 'stringio'
Gemfile.lock:      stringio
Gemfile.lock:    stringio (3.1.0)
Gemfile.lock:    stringio (3.1.0-java)
Gemfile.lock:  stringio

Thank you!

@deivid-rodriguez
Copy link
Member

Can you share the contents of bin/bundle? Is this project public?

@n-rodriguez
Copy link
Author

n-rodriguez commented May 13, 2024

It's the default bundler binstub :

#!/usr/bin/env ruby
# frozen_string_literal: true

#
# This file was generated by Bundler.
#
# The application 'bundle' is installed as part of a gem, and
# this file is here to facilitate running it.
#

require "rubygems"

m = Module.new do
  module_function

  def invoked_as_script?
    File.expand_path($0) == File.expand_path(__FILE__)
  end

  def env_var_version
    ENV["BUNDLER_VERSION"]
  end

  def cli_arg_version
    return unless invoked_as_script? # don't want to hijack other binstubs
    return unless "update".start_with?(ARGV.first || " ") # must be running `bundle update`
    bundler_version = nil
    update_index = nil
    ARGV.each_with_index do |a, i|
      if update_index && update_index.succ == i && a.match?(Gem::Version::ANCHORED_VERSION_PATTERN)
        bundler_version = a
      end
      next unless a =~ /\A--bundler(?:[= ](#{Gem::Version::VERSION_PATTERN}))?\z/
      bundler_version = $1
      update_index = i
    end
    bundler_version
  end

  def gemfile
    gemfile = ENV["BUNDLE_GEMFILE"]
    return gemfile if gemfile && !gemfile.empty?

    File.expand_path("../Gemfile", __dir__)
  end

  def lockfile
    lockfile =
      case File.basename(gemfile)
      when "gems.rb" then gemfile.sub(/\.rb$/, ".locked")
      else "#{gemfile}.lock"
      end
    File.expand_path(lockfile)
  end

  def lockfile_version
    return unless File.file?(lockfile)
    lockfile_contents = File.read(lockfile)
    return unless lockfile_contents =~ /\n\nBUNDLED WITH\n\s{2,}(#{Gem::Version::VERSION_PATTERN})\n/
    Regexp.last_match(1)
  end

  def bundler_requirement
    @bundler_requirement ||=
      env_var_version ||
      cli_arg_version ||
      bundler_requirement_for(lockfile_version)
  end

  def bundler_requirement_for(version)
    return "#{Gem::Requirement.default}.a" unless version

    bundler_gem_version = Gem::Version.new(version)

    bundler_gem_version.approximate_recommendation
  end

  def load_bundler!
    ENV["BUNDLE_GEMFILE"] ||= gemfile

    activate_bundler
  end

  def activate_bundler
    gem_error = activation_error_handling do
      gem "bundler", bundler_requirement
    end
    return if gem_error.nil?
    require_error = activation_error_handling do
      require "bundler/version"
    end
    return if require_error.nil? && Gem::Requirement.new(bundler_requirement).satisfied_by?(Gem::Version.new(Bundler::VERSION))
    warn "Activating bundler (#{bundler_requirement}) failed:\n#{gem_error.message}\n\nTo install the version of bundler this project requires, run `gem install bundler -v '#{bundler_requirement}'`"
    exit 42
  end

  def activation_error_handling
    yield
    nil
  rescue StandardError, LoadError => e
    e
  end
end

m.load_bundler!

if m.invoked_as_script?
  load Gem.bin_path("bundler", "bundle")
end

Is this project public?

Unfortunately no but it should be easily reproducible

@n-rodriguez
Copy link
Author

n-rodriguez commented May 14, 2024

repro script to run with truffleruby 24.0.1 :

#!/usr/bin/env ruby

# frozen_string_literal: true

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"

  gem 'psych'
end
nicolas@MacBook-Pro-de-Nicolas:~/PROJECTS/CONCERTO/test$ ./test.rb
setrlimit to increase file descriptor limit failed, errno 22
Fetching gem metadata from https://rubygems.org/........
Resolving dependencies...
WARN: Unresolved or ambiguous specs during Gem::Specification.reset:
      stringio (>= 0)
      Available/installed versions of this gem:
      - 3.1.0
      - 3.0.4
WARN: Clearing out unresolved specs. Try 'gem cleanup <gem>'
Please report a bug if this causes problems.

@deivid-rodriguez
Copy link
Member

I tried upgrading/downgrading RubyGems and it seems this is no longer an issue since RubyGems 3.4.21, particularly since RubyGems started using a pure-ruby YAML parser instead of psych: #6615.

Please do reopen if still happening after upgrading RubyGems!

@deivid-rodriguez deivid-rodriguez closed this as not planned Won't fix, can't repro, duplicate, stale Jun 18, 2024
@navidemad
Copy link

I am facing this issue with RubyGems 3.5.23.

Run bundle config path vendor/bundle
Fetching gem metadata from https://rubygems.org/........
Fetching minitest-retry 0.2.3
Fetching capybara-email 3.0.2
Installing minitest-retry 0.2.3
Installing capybara-email 3.0.2
WARN: Unresolved or ambiguous specs during Gem::Specification.reset:
      stringio (>= 0)
      Available/installed versions of this gem:
      - 3.1.2
      - 3.1.1
WARN: Clearing out unresolved specs. Try 'gem cleanup <gem>'
Please report a bug if this causes problems.

@dkav
Copy link
Contributor

dkav commented Nov 22, 2024

Like @navidemad, I am seeing this regularly when updating using bundler.

Installing rexml 3.3.9
WARN: Unresolved or ambiguous specs during Gem::Specification.reset:
      stringio (>= 0)
      Available/installed versions of this gem:
      - 3.1.2
      - 3.1.1

It also occurs on the latest HEAD too.

Installing rake 13.2.1
WARN: Unresolved or ambiguous specs during Gem::Specification.reset:
      stringio (>= 0)
      Available/installed versions of this gem:
      - 3.1.2
      - 3.1.2.dev

@deivid-rodriguez
Copy link
Member

Yeah, I've seen this too, will try to have a look.

@tisba
Copy link

tisba commented Nov 27, 2024

Is there a new issue we can subscribe too? Or can you re-open this one, @deivid-rodriguez?

@deivid-rodriguez
Copy link
Member

Nope. Can you open a new ticket with repro steps?

@tisba
Copy link

tisba commented Nov 27, 2024

Nope. Can you open a new ticket with repro steps?

I just saw the issue too and was wondering if this issue is the one to follow. But I'm happy trying to build a simplified repro and file a new issue. Thanks! 🚀

@deivid-rodriguez
Copy link
Member

Great, thank you! I did notice the issue myself at some point but I forgot to open a new ticket and I now lost track of how to repro.

@kevinoid
Copy link

For what it's worth, I've been able to reproduce the "Unresolved or ambiguous specs during Gem::Specification.reset" error on Ruby 3.2.6 with Rubygems 3.4.19 after running gem install --user-install psych stringio. With Docker/Podman:

podman run --rm docker.io/ruby:3.2 \
    sh -c 'gem install --user-install psych stringio && gem --version'

Which produces the output

WARNING:  You don't have /root/.local/share/gem/ruby/3.2.0/bin in your PATH,
	  gem executables will not run.
Building native extensions. This could take a while...
Successfully installed psych-5.2.0
Building native extensions. This could take a while...
Successfully installed stringio-3.1.2
2 gems installed
WARN: Unresolved or ambiguous specs during Gem::Specification.reset:
      stringio (>= 0)
      Available/installed versions of this gem:
      - 3.1.2
      - 3.0.4
WARN: Clearing out unresolved specs. Try 'gem cleanup <gem>'
Please report a bug if this causes problems.
3.4.19

However, the error does not occur after gem update --system (Rubygems 3.5.23) nor with docker.io/ruby:3.3 (Ruby 3.3.6 with Rubygems 3.5.22), so I'm uncertain whether there's value to opening a new issue.

@deivid-rodriguez
Copy link
Member

Yeah, not for that if upgrading RubyGems fixes it, but people are also seeing this with the latest RubyGems.

@tisba
Copy link

tisba commented Dec 2, 2024

I can confirm that this is still happening to me with Bundler 2.5.23 and rubygems 3.5.23. But so far I was unable to reproduce it reliably :( I'll try to remember to take a FS snapshot before running bundle install.

@scivola
Copy link

scivola commented Dec 3, 2024

Here is a minimal reproduction procedure:
rails/thor#889

@tisba
Copy link

tisba commented Dec 3, 2024

Thanks, @scivola. Unfortunately I think this is not enough to reproduce. I already tried basically the same thing:

  • using ruby:3.3.6 docker image (docker just to get a guaranteed clean environment)

  • running gem update --system to get the latest rubygems 3.5.23

  • Gemfile with:

    source "https://rubygems.org"
    gem "stringio"
  • running bundle install

Works fine.

@scivola
Copy link

scivola commented Dec 3, 2024

In my example rails/thor#889, it is necessary to use Thor.
Could you make a file with extension .thor such as:

require "bundler"
Bundler.require

and execute below?

% gem install thor
% thor -T

@deivid-rodriguez
Copy link
Member

I can somewhat reproduce this, although not as explained in rails/thor#889.

If stringio is not in the Gemfile, then I get no warnings. If stringio is included in the Gemfile, but the version resolved by Bundler is different from the latest version installed in the system, then I get an error reading:

✗ thor -T     
/Users/deivid/.asdf/installs/ruby/3.3.6/lib/ruby/site_ruby/3.3.0/bundler/runtime.rb:299:in `check_for_activated_spec!': You have already activated stringio 3.1.2, but your Gemfile requires stringio 3.1.1. Prepending `bundle exec` to your command may solve this. (Gem::LoadError)
	from /Users/deivid/.asdf/installs/ruby/3.3.6/lib/ruby/site_ruby/3.3.0/bundler/runtime.rb:25:in `block in setup'
	from /Users/deivid/.asdf/installs/ruby/3.3.6/lib/ruby/site_ruby/3.3.0/bundler/spec_set.rb:191:in `each'
	from /Users/deivid/.asdf/installs/ruby/3.3.6/lib/ruby/site_ruby/3.3.0/bundler/spec_set.rb:191:in `each'
	from /Users/deivid/.asdf/installs/ruby/3.3.6/lib/ruby/site_ruby/3.3.0/bundler/runtime.rb:24:in `map'
	from /Users/deivid/.asdf/installs/ruby/3.3.6/lib/ruby/site_ruby/3.3.0/bundler/runtime.rb:24:in `setup'
	from /Users/deivid/.asdf/installs/ruby/3.3.6/lib/ruby/site_ruby/3.3.0/bundler.rb:164:in `setup'
	from /Users/deivid/.asdf/installs/ruby/3.3.6/lib/ruby/site_ruby/3.3.0/bundler.rb:212:in `require'
	from /Users/deivid/code/playground/thor/tasks.thor:2:in `load_thorfile'
	from /Users/deivid/.asdf/installs/ruby/3.3.6/lib/ruby/gems/3.3.0/gems/thor-1.3.2/lib/thor/util.rb:157:in `class_eval'
	from /Users/deivid/.asdf/installs/ruby/3.3.6/lib/ruby/gems/3.3.0/gems/thor-1.3.2/lib/thor/util.rb:157:in `load_thorfile'
	from /Users/deivid/.asdf/installs/ruby/3.3.6/lib/ruby/gems/3.3.0/gems/thor-1.3.2/lib/thor/runner.rb:227:in `block in initialize_thorfiles'
	from /Users/deivid/.asdf/installs/ruby/3.3.6/lib/ruby/gems/3.3.0/gems/thor-1.3.2/lib/thor/runner.rb:226:in `each'
	from /Users/deivid/.asdf/installs/ruby/3.3.6/lib/ruby/gems/3.3.0/gems/thor-1.3.2/lib/thor/runner.rb:226:in `initialize_thorfiles'
	from /Users/deivid/.asdf/installs/ruby/3.3.6/lib/ruby/gems/3.3.0/gems/thor-1.3.2/lib/thor/runner.rb:176:in `list'
	from /Users/deivid/.asdf/installs/ruby/3.3.6/lib/ruby/gems/3.3.0/gems/thor-1.3.2/lib/thor/command.rb:28:in `run'
	from /Users/deivid/.asdf/installs/ruby/3.3.6/lib/ruby/gems/3.3.0/gems/thor-1.3.2/lib/thor/invocation.rb:127:in `invoke_command'
	from /Users/deivid/.asdf/installs/ruby/3.3.6/lib/ruby/gems/3.3.0/gems/thor-1.3.2/lib/thor.rb:538:in `dispatch'
	from /Users/deivid/.asdf/installs/ruby/3.3.6/lib/ruby/gems/3.3.0/gems/thor-1.3.2/lib/thor/base.rb:584:in `start'
	from /Users/deivid/.asdf/installs/ruby/3.3.6/lib/ruby/gems/3.3.0/gems/thor-1.3.2/bin/thor:6:in `<top (required)>'
	from /Users/deivid/.asdf/installs/ruby/3.3.6/bin/thor:25:in `load'
	from /Users/deivid/.asdf/installs/ruby/3.3.6/bin/thor:25:in `<main>'

This happens because thor activates stringio internally, but activates a different version than the one Bundler is using. I think rails/thor#892 should fix that.

@deivid-rodriguez
Copy link
Member

Hei!

So I was finally able to reproduce the problem reliably.

You have to:

  • Make sure you have two versions of stringio globally installed, like 3.1.1 and 3.1.2.
  • Use a script like the following, and run it making sure rake (or any other gem) is not pre-installed. For example, with: gem uninstall rake --force --executables && ruby script.rb:
require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"

  gem "rake"
end

Then you should get output like:

$ gem uninstall rake --force --executables && ruby script.rb
Removing rake
Successfully uninstalled rake-13.2.1
Fetching gem metadata from https://rubygems.org/.
Resolving dependencies...
Fetching rake 13.2.1
Installing rake 13.2.1
WARN: Unresolved or ambiguous specs during Gem::Specification.reset:
      stringio (>= 0)
      Available/installed versions of this gem:
      - 3.1.2
      - 3.1.1
WARN: Clearing out unresolved specs. Try 'gem cleanup <gem>'
Please report a bug if this causes problems.

The problem is that during the same process:

  • First Bundler activates psych in order to read YAML data in the .gem package it's about to install.
  • When psych is activated, its dependencies (namely, stringio) are left unresolved until they are actually used. In this case, stringio is not actually used, so the proper version is never actually resolved.
  • Then Bundler sets up the $LOAD_PATH for the inline Gemfile, and for that it calls Gem::Specification.reset to reset the set of specifications known to Bundler to those in the $LOAD_PATH.
  • Gem::Specification.reset prints this warning when it finds specifications that are waiting to be actually required before being completely resolved.

This kind of warning sometimes could make sense sometimes but I don't think it shows any actual problem in this particular situation, so I proposed #8309 to skip it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

7 participants