Heroku CI: Webdrivers::VersionError: Failed to find Chrome binary or its version.

[ryenski]

Summary

We're struggling to get our system tests running on Heroku CI, having migrated from chromedriver-helper.

Although the Chrome binary seems to be installed correctly, running the system tests on Heroku CI produces the error: Webdrivers::VersionError: Failed to find Chrome binary or its version.

Executing the test suite locally, of course everything works great. But on Heroku CI, it's unable to find the Chrome binary. Here's an excerpt from the test log:

-----> Running test command `rake`...
The HashDiff constant used by this gem conflicts with another gem of a similar name.  As of version 1.0 the HashDiff constant will be completely removed and replaced by Hashdiff.  For more information see https://github.com/liufengyun/hashdiff/issues/45.
Run options: --seed 40983
# Running:
...........sh: 1: [/usr/local/sbin,: not found
sh: 1: [/usr/local/sbin,: not found
sh: 1: [/usr/local/sbin,: not found
sh: 1: [/usr/local/sbin,: not found
sh: 1: [/usr/local/sbin,: not found
sh: 1: [/usr/local/sbin,: not found
E
Error:
My::ProfileTest#test_0002_edit profile:
Webdrivers::VersionError: Failed to find Chrome binary or its version.
    test/system/my/profile_test.rb:19:in `block in <class:ProfileTest>'
Error:
My::ProfileTest#test_0002_edit profile:
Webdrivers::VersionError: Failed to find Chrome binary or its version.
    test/support/application_system_test_case.rb:9:in `block in <class:ApplicationSystemTestCase>'
Error:
My::ProfileTest#test_0002_edit profile:
Webdrivers::VersionError: Failed to find Chrome binary or its version.
    
bin/rails test test/system/my/profile_test.rb:18
sh: 1: [/usr/local/sbin,: not found
sh: 1: [/usr/local/sbin,: not found
sh: 1: [/usr/local/sbin,: not found
sh: 1: [/usr/local/sbin,: not found
sh: 1: [/usr/local/sbin,: not found
sh: 1: [/usr/local/sbin,: not found
E

Test setup

We are using the chrome buildpacks. In our app.json file we have:

{ "url": "https://github.com/heroku/heroku-buildpack-chromedriver" },
{ "url": "https://github.com/heroku/heroku-buildpack-google-chrome" }

Here's how we're initializing Selenium:

# frozen_string_literal: true

require 'webdrivers'
require 'capybara/rails'
require 'capybara/minitest'

chrome_bin = ENV.fetch('GOOGLE_CHROME_SHIM', nil)

chrome_opts = chrome_bin ? { 'chromeOptions' => { 'binary' => chrome_bin } } : {}

Capybara.register_driver :chrome do |app|
  Capybara::Selenium::Driver.new(
    app,
    browser: :chrome,
    desired_capabilities: Selenium::WebDriver::Remote::Capabilities.chrome(chrome_opts)
  )
end

Capybara.javascript_driver = :chrome

class ActionDispatch::SystemTestCase
  # Make the Capybara DSL available in all integration tests
  include Capybara::DSL
  # Make `assert_*` methods behave like Minitest assertions
  include Capybara::Minitest::Assertions

  driven_by :selenium, using: :headless_chrome, screen_size: [1400, 1400] unless ENV['CI']

  teardown do
    Capybara.reset_sessions!
  end
end

Chrome Binary

The Chrome binary seems to be installed correctly by the buildpack. Here's the result of executing the binary (in the Heroku CI debug console):

~ $ echo $GOOGLE_CHROME_SHIM
/app/.apt/usr/bin/google-chrome-stable
~ $ echo $GOOGLE_CHROME_BIN
/app/.apt/opt/google/chrome/chrome
~ $ `$GOOGLE_CHROME_SHIM`

Executing that binary produces:

DevTools listening on ws://127.0.0.1:9222/devtools/browser/5c76de39-1a69-4193-966c-5670573eab59
^C

I was also curious what the chromedriver:version rake task would return

Here's the result (using Heroku debug console):

~ $ bundle exec rake webdrivers:chromedriver:version
The HashDiff constant used by this gem conflicts with another gem of a similar name.  As of version 1.0 the HashDiff constant will be completely removed and replaced by Hashdiff.  For more information see https://github.com/liufengyun/hashdiff/issues/45.
2019-06-12 23:59:12 WARN Webdrivers No existing chromedriver found.

... and here's what I see locally:

bundle exec rake webdrivers:chromedriver:version
2019-06-12 19:10:24 INFO Webdrivers chromedriver 74.0.3729.6

[ryenski]

Interestingly, executing Webdrivers::ChromeFinder.version in the Heroku CI debug console also raises the same exception:

~ $ rails c
irb(main):024:0> Webdrivers::ChromeFinder.version
sh: 1: [/usr/local/sbin,: not found
Traceback (most recent call last):
        1: from (irb):24
Webdrivers::VersionError (Failed to find Chrome binary or its version.)

the Chrome.path is empty:

irb(main):001:0> Selenium::WebDriver::Chrome.path
=> nil

webdrivers/lib/webdrivers/chrome_finder.rb

Lines 6 to 14 in a287cad

	def version
	location = Selenium::WebDriver::Chrome.path || send("#{System.platform}_location")
	version = send("#{System.platform}_version", location)
	raise VersionError, 'Failed to find Chrome binary or its version.' if version.nil? || version.empty?
	Webdrivers.logger.debug "Browser version: #{version}"
	version[/\d+\.\d+\.\d+\.\d+/] # Google Chrome 73.0.3683.75 -> 73.0.3683.75
	end

[twalpole]

The problem here is the shim installed by the chrome buildpack. Unfortunately when it's called with --version as an option to determine what version the installed Chrome is, it sticks a bunch of other options on the command line which means rather than returning the version of Chrome it actually starts Chrome. The way around this is to tell Webdrivers where the actual Chrome binary is

Selenium::WebDriver::Chrome.path = ENV['GOOGLE_CHROME_BIN']

This is mentioned in the heroku/circleci section of the README - https://github.com/titusfortner/webdrivers#herokucircleci-users

[twalpole]

Note -- It would be really great if the heroku/circleci shims were updated to allow them to actually return the version when queried for it.

[ryenski]

Thanks - that's a step in the right direction. These are some wild hoops to jump through for Heroku.

The test is now finding the right binary, but now it's hanging with no output. I suspect this is a problem with the buildpack.

[kapoorlakshya]

@twalpole @crispinheneise I submitted heroku/heroku-buildpack-google-chrome#73 last night to make $GOOGLE_CHROME_SHIM return the version. Let's see if it gets merged.

[kapoorlakshya]

The latest version of the google-chrome buildpack no longer needs the workaround described here. We've updated the SHIM to support webdrivers. See heroku/heroku-buildpack-google-chrome#73 for more information.