Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implemented close method to release scheduler resources on error #28

Conversation

andsel
Copy link
Contributor

@andsel andsel commented Apr 24, 2020

Close method is used to clean in case of error in run phase of the plugin, implemented to avoid leak of threads generated by Rufus scheduler.

Fixes #21

andsel added a commit to andsel/logstash-integration-jdbc that referenced this pull request Apr 27, 2020
close method is used to clean in case of error in run phase of the plugin, implemented to avoid leak of threads generated by Rufus scheduler.

Fixes logstash-plugins#21
Close logstash-plugins#28
@andsel andsel force-pushed the fix/scheduler_shutdown_on_do_close branch from 8b96c96 to 4f65c81 Compare April 27, 2020 09:43
@andsel andsel changed the title Implemented close method for release of resources Implemented close method to release scheduler resources on error Apr 27, 2020
@elasticsearch-bot elasticsearch-bot self-assigned this Apr 27, 2020
@andsel andsel assigned andsel and unassigned elasticsearch-bot Apr 27, 2020
Copy link
Contributor

@kares kares left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 this makes sense - we actually have been seeing stuck threads from jdbc plugin.

Copy link
Contributor

@yaauie yaauie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code-change looks sensible.

I'd like to see a small change to the spec to eliminate the monkey-patch introduced here.

We also need a version bump (patch-level is okay) and an entry in the changelog, maybe something like:

Fixed potential resource leak by ensuring scheduler is shut down when a pipeline containing this plugin is shut down or restarted

Comment on lines 194 to 205
it "should cleanup resources on close" do
class LogStash::Inputs::Jdbc
def scheduler_down?
@scheduler.down?
end
end

runner = Thread.new do
plugin.run(queue)
end
sleep 1

plugin.do_close

expect(plugin.scheduler_down?).to be_truthy
end

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather avoid monkey-patching the plugin in our specs, since this modification to the class will persist beyond the scope of this individual spec.

An alternate solution would be:

Suggested change
it "should cleanup resources on close" do
class LogStash::Inputs::Jdbc
def scheduler_down?
@scheduler.down?
end
end
runner = Thread.new do
plugin.run(queue)
end
sleep 1
plugin.do_close
expect(plugin.scheduler_down?).to be_truthy
end
it 'cleans up scheduler resources on close' do
runner = Thread.new do
plugin.run(queue)
end
sleep 1
plugin.do_close
scheduler = plugin.instance_variable_get(:@scheduler)
expect(scheduler).to_not be_nil
expect(scheduler.down?).to be_truthy
end

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank's @yaauie good point, I missed it!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yaauie I've added the suggested changes, if you want you can go forward with the review

close method is used to clean in case of error in run phase of the plugin, implemented to avoid leak of threads generated by Rufus scheduler.

Fixes logstash-plugins#21
Close logstash-plugins#28
@andsel andsel force-pushed the fix/scheduler_shutdown_on_do_close branch from 4f65c81 to df09843 Compare April 28, 2020 08:15
@andsel andsel requested a review from yaauie April 28, 2020 08:15
@andsel
Copy link
Contributor Author

andsel commented Apr 28, 2020

Jenkins test this please

Copy link
Contributor

@yaauie yaauie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@elasticsearch-bot
Copy link

Andrea Selva merged this to master!

@elasticsearch-bot elasticsearch-bot merged commit df09843 into logstash-plugins:master May 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Leak of scheduler thread under not catched errors in plugin run
4 participants