Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: driver loading when file not accessible #370

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -339,6 +339,7 @@ required you can pass them separated by a comma.
NOTE: If not provided, Plugin will look for the driver class in the Logstash Java classpath. Additionally, if the library
does not appear to be being loaded correctly via this setting, placing the relevant jar(s) in the Logstash Java
classpath rather than via this setting may help.
Please also make sure the path is readable by the Logstash process (e.g. `logstash` user when running as a service).

[id="plugins-{type}s-{plugin}-jdbc_fetch_size"]
===== `jdbc_fetch_size`
Expand Down
1 change: 1 addition & 0 deletions lib/logstash/inputs/jdbc.rb
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,7 @@ def set_value_tracker(instance)
end

def run(queue)
load_driver
if @schedule
@scheduler = Rufus::Scheduler.new(:max_work_threads => 1)
@scheduler.cron @schedule do
Expand Down
67 changes: 39 additions & 28 deletions lib/logstash/plugin_mixins/jdbc/jdbc.rb
Original file line number Diff line number Diff line change
Expand Up @@ -140,47 +140,58 @@ def jdbc_connect

private

def load_driver
if @drivers_loaded.false?
require "java"
require "sequel"
require "sequel/adapters/jdbc"

load_driver_jars
begin
Sequel::JDBC.load_driver(@jdbc_driver_class)
rescue Sequel::AdapterNotFound => e # Sequel::AdapterNotFound, "#{@jdbc_driver_class} not loaded"
# fix this !!!
message = if jdbc_driver_library_set?
"Are you sure you've included the correct jdbc driver in :jdbc_driver_library?"
else
":jdbc_driver_library is not set, are you sure you included " +
"the proper driver client libraries in your classpath?"
end
raise LogStash::PluginLoadingError, "#{e}. #{message}"
end
@drivers_loaded.make_true
end
end

def load_driver_jars
unless @jdbc_driver_library.nil? || @jdbc_driver_library.empty?
if jdbc_driver_library_set?
@jdbc_driver_library.split(",").each do |driver_jar|
@logger.debug("loading #{driver_jar}")
# load 'driver.jar' is different than load 'some.rb' as it only causes the file to be added to
# JRuby's class-loader lookup (class) path - won't raise a LoadError when file is not readable
unless FileTest.readable?(driver_jar)
raise LogStash::PluginLoadingError, "unable to load #{driver_jar} from :jdbc_driver_library, " +
"file not readable (please check user and group permissions for the path)"
end
begin
@logger.debug("loading #{driver_jar}")
# Use https://github.com/jruby/jruby/wiki/CallingJavaFromJRuby#from-jar-files to make classes from jar
# available
require driver_jar
rescue LoadError => e
raise LogStash::PluginLoadingError, "unable to load #{driver_jar} from :jdbc_driver_library, #{e.message}"
rescue StandardError => e
raise LogStash::PluginLoadingError, "unable to load #{driver_jar} from :jdbc_driver_library, #{e}"
end
end
end
end

private
def open_jdbc_connection
require "java"
require "sequel"
require "sequel/adapters/jdbc"
def jdbc_driver_library_set?
!@jdbc_driver_library.nil? && !@jdbc_driver_library.empty?
end

def open_jdbc_connection
# at this point driver is already loaded
Sequel.application_timezone = @plugin_timezone.to_sym
if @drivers_loaded.false?
begin
load_driver_jars
Sequel::JDBC.load_driver(@jdbc_driver_class)
rescue LogStash::Error => e
# raised in load_drivers, e.cause should be the caught Java exceptions
raise LogStash::PluginLoadingError, "#{e.message} and #{e.cause.message}"
rescue Sequel::AdapterNotFound => e
# fix this !!!
message = if @jdbc_driver_library.nil?
":jdbc_driver_library is not set, are you sure you included
the proper driver client libraries in your classpath?"
else
"Are you sure you've included the correct jdbc driver in :jdbc_driver_library?"
end
raise LogStash::PluginLoadingError, "#{e}. #{message}"
end
@drivers_loaded.make_true
end

@database = jdbc_connect()
@database.extension(:pagination)
if @jdbc_default_timezone
Expand Down
32 changes: 32 additions & 0 deletions spec/inputs/jdbc_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -1291,6 +1291,38 @@
end
end

context "when an unreadable jdbc_driver_path entry is present" do
let(:driver_jar_path) do
jar_file = $CLASSPATH.find { |name| name.index(Jdbc::Derby.driver_jar) }
raise "derby jar not found on class-path" unless jar_file
jar_file.sub('file:', '')
end

let(:invalid_driver_jar_path) do
path = File.join(Dir.mktmpdir, File.basename(driver_jar_path))
FileUtils.cp driver_jar_path, path
FileUtils.chmod "u=x,go=", path
path
end

let(:settings) do
{ "statement" => "SELECT * from types_table", "jdbc_driver_library" => invalid_driver_jar_path }
end

before do
plugin.register
end

after do
plugin.stop
end

it "raise a loading error" do
expect { plugin.run(queue) }.
to raise_error(LogStash::PluginLoadingError, /unable to load .*? from :jdbc_driver_library, file not readable/)
end
end

context "when using prepared statements" do
let(:last_run_value) { 250 }
let(:expected_queue_size) { 100 }
Expand Down