Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add setting to use custom Facter implementation #16

Merged
merged 1 commit into from
Nov 10, 2021

Conversation

GabrielNagy
Copy link

Starting with versions 7.12.0/6.25.0, Puppet was changed not to directly
depend on Facter anymore, but to use a Puppet::Runtime implementation
instead (e.g. calls to Facter were changed to
Puppet.runtime[:facter] to allow for pluggable Facter backends).

rspec-puppet stubs facts from facterdb by setting custom facts with
higher weights, meaning that Facter 4 will still resolve the underlying
core facts (which is by design), leading to noticeable performance hits
when compiling catalogs with Facter 4 as opposed to Facter 2 (which just
returned the custom fact).

This means we can achieve a pretty big performance improvement with
rspec-puppet by registering a custom Facter implementation that bypasses
Facter altogether and just saves facts to hash.

This behavior cand be activated by setting facter_implementation to
rspec in RSpec.configure. By default, the setting has the value of
facter which maintains the old behavior of going through Facter.

@GabrielNagy GabrielNagy force-pushed the facter-impl branch 2 times, most recently from 5963b7d to e31be81 Compare November 4, 2021 13:30
README.md Outdated Show resolved Hide resolved
lib/rspec-puppet/adapters.rb Outdated Show resolved Hide resolved
lib/rspec-puppet.rb Outdated Show resolved Hide resolved
@GabrielNagy GabrielNagy force-pushed the facter-impl branch 2 times, most recently from f490be6 to 0c3a14a Compare November 5, 2021 11:36
Copy link

@ekohl ekohl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall 👍

What is your next step? Implement setting a different default should be done somewhere, but what's the appropriate place?

I'm also looking at how to best set it. Right now the user should either implement the same compatibility code or force a new enough version. Does supporting an implementation that prefers rspec but falls back to facter on older versions make sense?

lib/rspec-puppet/adapters.rb Outdated Show resolved Hide resolved
@GabrielNagy GabrielNagy force-pushed the facter-impl branch 2 times, most recently from 8075ac3 to 0ba5f75 Compare November 5, 2021 13:12
@GabrielNagy
Copy link
Author

What is your next step? Implement setting a different default should be done somewhere, but what's the appropriate place?

I see that the puppet-nginx module for example uses the spec helper from https://github.com/voxpupuli/voxpupuli-test/blob/master/lib/voxpupuli/test/spec_helper.rb, so maybe that could be a good place?

I'm also looking at how to best set it. Right now the user should either implement the same compatibility code or force a new enough version. Does supporting an implementation that prefers rspec but falls back to facter on older versions make sense?

I think falling back to facter if rspec is not supported makes sense, this way of doing things should involve the least amount of changes in downstream modules.

lib/rspec-puppet/adapters.rb Outdated Show resolved Hide resolved
lib/rspec-puppet/adapters.rb Outdated Show resolved Hide resolved
Starting with versions 7.12.0/6.25.0, Puppet was changed not to directly
depend on Facter anymore, but to use a `Puppet::Runtime` implementation
instead (e.g. calls to `Facter` were changed to
`Puppet.runtime[:facter]` to allow for pluggable Facter backends).

rspec-puppet stubs facts from facterdb by setting custom facts with
higher weights, meaning that Facter 4 will still resolve the underlying
core facts (which is by design), leading to noticeable performance hits
when compiling catalogs with Facter 4 as opposed to Facter 2 (which just
returned the custom fact).

This means we can achieve a pretty big performance improvement with
rspec-puppet by registering a custom Facter implementation that bypasses
Facter altogether and just saves facts to hash.

This behavior cand be activated by setting `facter_implementation` to
`rspec` in `RSpec.configure`. By default, the setting has the value of
`facter` which maintains the old behavior of going through Facter. If
`rspec` is set but the Puppet version does not support Facter
implementations, rspec will warn and fall back to using Facter.
@GabrielNagy
Copy link
Author

@ekohl I think this is pretty much done, let me know if I missed any of your suggestions. Thanks for the reviews!

Copy link

@ekohl ekohl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have merge permission here, but this looks good to me.

@ciprianbadescu ciprianbadescu merged commit 796c4b2 into puppetlabs:master Nov 10, 2021
@ekohl
Copy link

ekohl commented Nov 11, 2021

I see that the puppet-nginx module for example uses the spec helper from https://github.com/voxpupuli/voxpupuli-test/blob/master/lib/voxpupuli/test/spec_helper.rb, so maybe that could be a good place?

I opened voxpupuli/voxpupuli-test#62 to experiment with this.

@bastelfreak
Copy link
Collaborator

Hi people. I merged and released the change from @ekohl . This passed on our nginx module: voxpupuli/puppet-nginx#1485

I tried the same on our bird module and a single test fails:

Failures:

  1) bird on archlinux-5-x86_64 with only IPv4 is expected not to contain Yumrepo[bird]
     Failure/Error: it { is_expected.not_to contain_yumrepo('bird') }

     Puppet::PreformattedError:
       Evaluation Error: Error while evaluating a Resource Statement, Could not autoload puppet/type/service: Could not autoload puppet/provider/service/bsd: Could not autoload puppet/provider/service/init: undefined method `downcase' for nil:NilClass (file: /home/bastelfreak/code/modulesync_config/modules/voxpupuli/puppet-bird/spec/fixtures/modules/bird/manifests/init.pp, line: 156, column: 5) on node bastelfreak-nb.fritz.box
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/provider/service/init.rb:24:in `block in <top (required)>'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/classgen.rb:132:in `class_eval'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/classgen.rb:132:in `genthing'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/classgen.rb:33:in `genclass'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/type.rb:1847:in `provide'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/provider/service/init.rb:3:in `<top (required)>'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/autoload.rb:78:in `load'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/autoload.rb:78:in `load_file'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/monkey_patches.rb:232:in `block in load_file'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/consts.rb:53:in `without_stubs'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/monkey_patches.rb:231:in `load_file'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/autoload.rb:182:in `load'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/type.rb:1780:in `provider'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/type.rb:1832:in `provide'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/provider/service/bsd.rb:1:in `<top (required)>'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/autoload.rb:78:in `load'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/autoload.rb:78:in `load_file'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/monkey_patches.rb:232:in `block in load_file'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/consts.rb:53:in `without_stubs'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/monkey_patches.rb:231:in `load_file'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/autoload.rb:93:in `block in loadall'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/autoload.rb:91:in `each'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/autoload.rb:91:in `loadall'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/autoload.rb:196:in `loadall'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/metatype/manager.rb:126:in `block in newtype'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/concurrent/lock.rb:10:in `synchronize'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/metatype/manager.rb:73:in `newtype'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/type/service.rb:10:in `<module:Puppet>'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/type/service.rb:8:in `<top (required)>'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/autoload.rb:78:in `load'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/autoload.rb:78:in `load_file'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/monkey_patches.rb:232:in `block in load_file'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/consts.rb:53:in `without_stubs'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/monkey_patches.rb:231:in `load_file'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/autoload.rb:182:in `load'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/metatype/manager.rb:171:in `block in type'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/concurrent/lock.rb:10:in `synchronize'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/metatype/manager.rb:154:in `type'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/runtime3_resource_support.rb:107:in `find_builtin_resource_type'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/runtime3_resource_support.rb:71:in `find_resource_type'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/runtime3_resource_support.rb:23:in `create_resources'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/runtime3_support.rb:340:in `create_resources'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/evaluator_impl.rb:884:in `block in eval_ResourceExpression'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/evaluator_impl.rb:881:in `map'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/evaluator_impl.rb:881:in `eval_ResourceExpression'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/visitor.rb:94:in `visit_this_1'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/evaluator_impl.rb:81:in `evaluate'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/evaluator_impl.rb:1058:in `block in eval_IfExpression'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/scope.rb:985:in `with_guarded_scope'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/evaluator_impl.rb:1056:in `eval_IfExpression'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/visitor.rb:94:in `visit_this_1'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/evaluator_impl.rb:81:in `evaluate'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/evaluator_impl.rb:679:in `block in eval_BlockExpression'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/evaluator_impl.rb:679:in `each'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/evaluator_impl.rb:679:in `reduce'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/evaluator_impl.rb:679:in `eval_BlockExpression'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/visitor.rb:94:in `visit_this_1'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/evaluator_impl.rb:81:in `evaluate'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/parser/evaluating_parser.rb:60:in `evaluate'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/ast/pops_bridge.rb:27:in `evaluate'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/ast/pops_bridge.rb:63:in `block (2 levels) in evaluate'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/ast/pops_bridge.rb:62:in `catch'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/ast/pops_bridge.rb:62:in `block in evaluate'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/ast/pops_bridge.rb:61:in `catch'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/ast/pops_bridge.rb:61:in `evaluate'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/ast.rb:30:in `safeevaluate'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/resource/type.rb:80:in `evaluate_code'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/resource.rb:79:in `block in evaluate'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/profiler/around_profiler.rb:58:in `profile'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/profiler.rb:51:in `profile'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/resource.rb:71:in `evaluate'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/compiler.rb:259:in `each'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/compiler.rb:259:in `evaluate_classes'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/runtime3_resource_support.rb:62:in `block in create_resources'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/runtime3_resource_support.rb:37:in `map'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/runtime3_resource_support.rb:37:in `create_resources'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/runtime3_support.rb:340:in `create_resources'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/evaluator_impl.rb:884:in `block in eval_ResourceExpression'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/evaluator_impl.rb:881:in `map'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/evaluator_impl.rb:881:in `eval_ResourceExpression'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/visitor.rb:49:in `block in visit_this'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/visitor.rb:43:in `each'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/visitor.rb:43:in `visit_this'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/visitor.rb:96:in `visit_this_1'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/evaluator_impl.rb:81:in `evaluate'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/puppet_stack.rb:42:in `stack'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/evaluator_impl.rb:756:in `eval_Program'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/visitor.rb:94:in `visit_this_1'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/evaluator/evaluator_impl.rb:81:in `evaluate'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/pops/parser/evaluating_parser.rb:60:in `evaluate'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/ast/pops_bridge.rb:109:in `evaluate'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/ast.rb:30:in `safeevaluate'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/resource/type.rb:80:in `evaluate_code'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/resource.rb:79:in `block in evaluate'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/profiler/around_profiler.rb:58:in `profile'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/profiler.rb:51:in `profile'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/resource.rb:71:in `evaluate'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/compiler.rb:399:in `evaluate_main'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/compiler.rb:132:in `block (2 levels) in compile'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/profiler/around_profiler.rb:58:in `profile'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/profiler.rb:51:in `profile'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/compiler.rb:132:in `block in compile'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/context.rb:62:in `override'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet.rb:302:in `override'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/compiler.rb:123:in `compile'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/parser/compiler.rb:34:in `compile'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/indirector/catalog/compiler.rb:328:in `block (2 levels) in compile'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/profiler/around_profiler.rb:58:in `profile'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util/profiler.rb:51:in `profile'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/indirector/catalog/compiler.rb:326:in `block in compile'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util.rb:233:in `block in benchmark'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/util.rb:232:in `benchmark'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/indirector/catalog/compiler.rb:324:in `compile'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/indirector/catalog/compiler.rb:68:in `block in find'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/node/environment.rb:440:in `with_text_domain'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/indirector/catalog/compiler.rb:64:in `find'
     # ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/indirector/indirection.rb:223:in `find'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/adapters.rb:86:in `catalog'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/adapters.rb:176:in `catalog'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/adapters.rb:264:in `catalog'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/support.rb:476:in `build_catalog_without_cache_v2'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/support.rb:488:in `block in build_catalog'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/cache.rb:23:in `get'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/support.rb:486:in `build_catalog'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/support.rb:100:in `block in load_catalogue'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/support.rb:410:in `with_vardir'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/support.rb:85:in `load_catalogue'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/example/class_example_group.rb:7:in `catalogue'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/support.rb:14:in `block in subject'
     # ./.vendor/ruby/3.0.0/gems/rspec-puppet-2.11.0/lib/rspec-puppet/matchers/create_generic.rb:84:in `matches?'
     # ./spec/classes/bird_spec.rb:42:in `block (5 levels) in <top (required)>'
     # ------------------
     # --- Caused by: ---
     # NoMethodError:
     #   undefined method `downcase' for nil:NilClass
     #   ./.vendor/ruby/3.0.0/gems/puppet-7.12.1/lib/puppet/provider/service/init.rb:24:in `block in <top (required)>'

Finished in 9.1 seconds (files took 4.87 seconds to load)
201 examples, 1 failure

Failed examples:

rspec ./spec/classes/bird_spec.rb[1:1:2:1] # bird on archlinux-5-x86_64 with only IPv4 is expected not to contain Yumrepo[bird]


249 examples, 1 failure

I saw similar issues in the past when we had a broken fact, but I cannot remember the details. This is reproducible with the rspec facter implementation.

The mentioned puppet code:
https://github.com/puppetlabs/puppet/blob/0fc096f07ecc1b4a36ae6eabe5716b3f03bd9c4e/lib/puppet/provider/service/init.rb#L23-L27

I guessed that the facts from facterdb don't contain the legacy facts osfamily/operatingsystem, but they do:
https://github.com/voxpupuli/facterdb/blob/master/facts/3.14/archlinux-x86_64.facts#L389-L416

the mentioned puppet code is a serice resource:

  if $manage_service {
    service { $daemon_name_v4:
      ensure     => $service_v4_ensure,
      enable     => $service_v4_enable,
      hasrestart => false,
      restart    => '/usr/sbin/birdc configure',
      pattern    => $daemon_name_v4,
      require    => Package[$package_name_v4],
    }
  }

So I'm not sure why it fails. The nginx module also has Archlinux in the metadata.json, but the tests seem to pass.

@ekohl
Copy link

ekohl commented Nov 27, 2021

It is an interesting case. When I change it to call pry:

it do
  require 'pry' ; binding.pry
  is_expected.not_to contain_yumrepo('bird')
end

Run it as SPEC_FACTS_OS=arch be rspec spec/classes/bird_spec.rb then I can see:

[1] pry(#<RSpec::ExampleGroups::Bird::OnArchlinux5X8664::WithOnlyIPv4>)> Puppet.runtime[:facter]
=> #<RSpec::Puppet::FacterTestImpl:0x00005642f1d48e08 @facts={}>

So somehow the facts are not being set. I'm not sure, but one warning is interesting:

No facts were found in the FacterDB for Facter v4.2.4 on {:operatingsystem=>"Archlinux", :hardwaremodel=>"x86_64"}, using v3.14.14 instead

However, it really appears to only be the first testcase within the nesting. If I add another one before it, that fails and then the next testcase (the one that now fails) does pass. Then I call pry again and see:

[1] pry(#<RSpec::ExampleGroups::Bird::OnArchlinux5X8664::WithOnlyIPv4>)> Puppet.runtime[:facter]
=> #<RSpec::Puppet::FacterTestImpl:0x000055cc3cffa1c8 @facts={}>

So not having facts there may be a false positive.

The next thing I checked if it's actually related to the OS at all. So I ran SPEC_FACTS_OS=centos be rspec spec/classes/bird_spec.rb which interestingly fails on exactly the same thing.

When you set manage_service to false (so the service type isn't used), the tests don't fail on that. That makes me think that perhaps the Puppet.runtime[:facter] not having any facts may actually be an issue. How it then does manage to pass is something I can't answer.

@GabrielNagy
Copy link
Author

@bastelfreak @ekohl I just opened #19 which appears to fix this issue; can you take a look and confirm?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants