Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More than one collection as input #88

Open
ghost opened this issue Jan 4, 2018 · 2 comments
Open

More than one collection as input #88

ghost opened this issue Jan 4, 2018 · 2 comments

Comments

@ghost
Copy link

ghost commented Jan 4, 2018

Hi there, I'm trying to get data from two collections "logs" and "servers_output" from a mongodb database and insert it into an ES.

I works for one collection but when I try to use two collection it doesn't work.

The documentation says that is an string, "A regex that will be used to find desired collecitons."

So this is my cfg :

mongodb {
    uri => 'mongodb://xxxx:[email protected]/pritunl?ssl=false'
    placeholder_db_dir => '/opt/logstash-mongodb/'
    placeholder_db_name => 'servers_output.db'
    collection => '(logs,servers_output)'
    batch_size => 5000
 }

But i get this error:

{:timestamp=>"2018-01-04T15:51:33.138000+0000", :message=>"Pipeline aborted due to error", :exception=>"NoMethodError", :error=>"undefined method `[]' for nil:NilClass", :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-mongodb-0.3.3/lib/logstash/inputs/mongodb.rb:89:in `init_placeholder'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-mongodb-0.3.3/lib/logstash/inputs/mongodb.rb:99:in `get_placeholder'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-mongodb-0.3.3/lib/logstash/inputs/mongodb.rb:146:in `update_watched_collections'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-mongodb-0.3.3/lib/logstash/inputs/mongodb.rb:144:in `update_watched_collections'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-mongodb-0.3.3/lib/logstash/inputs/mongodb.rb:168:in `register'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:330:in `start_inputs'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:329:in `start_inputs'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:180:in `start_workers'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/pipeline.rb:136:in `run'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.4.1-java/lib/logstash/agent.rb:491:in `start_pipeline'"], :level=>:error}
{:timestamp=>"2018-01-04T15:51:36.145000+0000", :message=>"stopping pipeline", :id=>"main"}

==> /var/log/logstash/logstash.stdout <==
{:timestamp=>"2018-01-04T15:51:36.145000+0000", :message=>"stopping pipeline", :id=>"main"}

Thanks in advance.

@191801737
Copy link

mongodb {
uri => 'mongodb://xxxx:[email protected]/pritunl?ssl=false'
placeholder_db_dir => '/opt/logstash-mongodb/'
placeholder_db_name => 'servers_output.db'
collection => 'logs'
batch_size => 5000
}
mongodb {
uri => 'mongodb://xxxx:[email protected]/pritunl?ssl=false'
placeholder_db_dir => '/opt/logstash-mongodb/'
placeholder_db_name => 'servers_output.db'
collection => 'servers_output'
batch_size => 5000
}

try this

@zpol
Copy link

zpol commented Apr 26, 2018

It worked! thanks, I'm also using:

(collection1|collection2)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants