## Name ##

ylookup - parse yaml within puppet from a yaml format data file

## Install ##

The ylookup.rb code should be copied into /usr/lib/ruby/site_ruby/1.8/puppet/parser/functions/ylookup.rb

## Description ##

A simple ruby function which read yaml files inside puppet modules.
This is originally based on rip work on extlookup, by ohad levy.
Supplemented by xyntrix.

## Details ##

It is preferred to place .yaml files relative to your module, in a data/ directory.  You must define a relative path search order within site.pp, and those paths will be searched. 

Given module name "examplemod" and the data file is "foo.yaml":

   examplemod/data/foo.yaml

A fact driven path ordering example:

 $lookup_order_path = [
    "%{sitemodulename}/data/hostname/%{hostname}",
    "%{sitemodulename}/data/%{hosttype}/%{hostmode}",
    "%{sitemodulename}/data/%{hosttype}/default",
    "%{sitemodulename}/data/%{hostmode}",
    "%{sitemodulename}/data/default",
    "%{modulename}/data/%{company}",
    "%{modulename}/data/default"
 ]

Built in variable ordering example [1]:

 $lookup_order_path = [
   "$caller_module_name/data/$hostname",
   "$caller_module_name/data/$hostname/default",
   "$caller_module_name/data",
   "$caller_module_name/data/default",
   "$name/data",
   "$name/data/default"
 ]

By default ylookup scans for data.yaml and values.yaml.  You can optionally over-ride the fix prefix by setting $lookup_order_files in your site.pp. The prefix will still have .yaml appended to it.

 $lookup_order_files = [ "customdata", "foodefaultdata" ]

Once your $lookup_order_path is set, you just need to specify the yaml variable to locate.  

 $myvar = ylookup("myvar")

If you would like to use a different $lookup_order_path, without setting the variable in the global nameespace, you may provide it as a second argument to ylookup().

 $myvar = ylookup("myvar", ["%{environment}/data","common/data/default"])

## Example ##

---------- begin YAML example, data/data.yaml ----------

singlevalue : legs

examplestring :
  Here is a line of 
  text which spans
  multiple lines

examplelist: 
 - cat,dog,hampster
 - pig,goat,sheep
 - duck,goose,swan
 - ant,beetle,millepede

hash_of_hashes:
 subhash_key : subhash1_value
 subhash_key : subhash2_value 
 
array_of_hashes:
 - hash_key_A : value_A
   hash_key_B : value_B
 - hash_key_A : value_C
   hash_key_B : value_D

# recursive variable example
my_app_config_dir : /etc/my_app
my_app_conf : %{my_app_config_dir}/my_app.conf

---------- cut YAML example, data/data.yaml ----------

#### Examples from manifest.pp ####

 ylookup("singlevalue")
   - will return: "legs" 
 ylookup("examplestring")
   - will return: "Here is a line of text which spans multiple lines" 
 ylookup("examplelist")
   - will return: ["cat,dog,hampster","pig,goat,sheep","duck,goose,swan","ant,beetle,millepede"] 

#### Example in a Template (.erb) ####

 <%= scope.function_ylookup(["singlevalue","default"]) %>

 <%= scope.function_lookup(["array_of_hashes"]).each do |hash| -%>
  'hash_key_A' is set to value <%= hash['hash_key_A'] %>
 <% end -%>

#### Sample puppet class ####

Array of string data example (which just happens to be comma seperated). This technique can be useful in many ways (for example, maintaining a list of shares, mountpoints and mount_options which puppet can then ensure are mounted).

----- begin puppet example, example_yaml_lookup/manifests/init.pp -----

 class example_yaml_lookup {
    $examplelist = ylookup("examplelist")
 define do_stuff () {
   $param0=inline_template('<%= name.split(",")[0] %>')
   $param1=inline_template('<%= name.split(",")[1] %>')
   $param2=inline_template('<%= name.split(",")[2] %>')
   notify{"parma0: ${param0} - param1:  ${param1} - param2: ${param2}":}
 }

 do_stuff { $examplelist: }

----- end puppet example, example_yaml_lookup/manifests/init.pp -----


## Expected output ##

 [root@rhel6-puppettest puppet-lookup]# puppetd -t
 info: Caching catalog for <computer_name>
 info: Applying configuration version '1300797371'
 notice: parma0: cat - param1:  dog - param2: hampster
 notice: /Stage[main]/Example_yaml_lookup/example_yaml_lookup::Do_stuff[cat,dog,hampster]/Notify[parma0: cat - param1:  dog - param2: hampster]/message: defined 'message' as 'parma0: cat - param1:  dog - param2: hampster'
 notice: parma0: pig - param1:  goat - param2: sheep
 notice: /Stage[main]/Example_yaml_lookup/Example_yaml_lookup::Do_stuff[pig,goat,sheep]/Notify[parma0: pig - param1:  goat - param2: sheep]/message: defined 'message' as 'parma0: pig - param1:  goat - param2: sheep'
 notice: parma0: duck - param1:  goose - param2: swan
 notice: /Stage[main]/Example_yaml_lookup/Example_yaml_lookup::Do_stuff[duck,goose,swan]/Notify[parma0: duck - param1:  goose - param2: swan]/message: defined 'message' as 'parma0: duck - param1:  goose - param2: swan'
 notice: parma0: ant - param1:  beetle - param2: millepede
 notice: /Stage[main]/Example_yaml_lookup/Example_yaml_lookup::Do_stuff[ant,beetle,millepede]/Notify[parma0: ant - param1:  beetle - param2: millepede]/message: defined 'message' as 'parma0: ant - param1:  beetle - param2: millepede'
 notice: Finished catalog run in 2.05 seconds
 

## Debugging ##

You can debug the behavior of ylookup() using several variables in your manifest (generally site.pp):

 $lookup_debug = true		# General debugging (all debugging is enabled)
 $lookup_file_debug = true	# File lookup debugging only - print what files are being read
 $lookup_hash_debug = true	# Debug hashes - print their keys and their values
 $lookup_type_debug = true	# Debug return types - print the type (scalar, array, hash, number) of the value being returned.
 
You must run your puppetmaster in debug mode to view the debug output.

  puppet master --debug --no-daemonize

#### Example Debug Output ####

 $ puppet apply --noop -v --debug manifests/init.pp --modulepath=/usr/share/puppet/modules
 debug: Scope(Class[example]): ylookup(): --------------------------------------------------------------------
 debug: Scope(Class[example]): ylookup(): Looking for subject 'things'
 debug: Scope(Class[example]): ylookup(): Relative Lookup Path: '[ "/data", "/data/default", "/example/data", "/example/data/default" ]'
 debug: Scope(Class[example]): ylookup(): Suffix Lookup Filespec: '["data"]'
 debug: Scope(Class[example]): ylookup(): --------------------------------------------------------------------
 debug: Scope(Class[example]): ylookup(): Environment: 'production'
 debug: Scope(Class[example]): ylookup(): Environment path: '["/usr/share/puppet/modules"]'
 debug: Scope(Class[example]): ylookup(): --------------------------------------------------------------------
 debug: Scope(Class[example]): ylookup(): Looking for '/usr/share/puppet/modules/data/data.yaml'
 debug: Scope(Class[example]): ylookup(): Looking for '/usr/share/puppet/modules/data/default/data.yaml'
 debug: Scope(Class[example]): ylookup(): Looking for '/usr/share/puppet/modules/example/data/data.yaml'
 debug: Scope(Class[example]): ylookup(): Looking for '/usr/share/puppet/modules/example/data/default/data.yaml'
 debug: Scope(Class[example]): ylookup(): Found the following relevant data files: /usr/share/puppet/modules/example/data/data.yaml
 debug: Scope(Class[example]): ylookup(): scanning /usr/share/puppet/modules/example//data/data.yaml
 debug: Scope(Class[example]): ylookup(): YAML.load_file(/usr/share/puppet/modules/example//data/data.yaml)[lookup_name] result: 'apple,banana,carrot,dog,elephant,fox'
 debug: Scope(Class[example]): ylookup(): result is a Array
 debug: Scope(Class[example]): ylookup():  - 'apple,banana,carrot'
 debug: Scope(Class[example]): ylookup():  - 'dog,elephant,fox'
 debug: Scope(Class[example]): ylookup(): 'things' => 'apple,banana,carrot,dog,elephant,fox' : from '/usr/share/puppet/modules/example//data/data.yaml'
 debug: Scope(Class[example]): ylookup(): RESULT: 'things' => 'apple,banana,carrot,dog,elephant,fox'
 debug: Scope(Class[example]): ylookup():  Recursive calls: 1
 debug: Creating default schedules
 debug: Finishing transaction 22222222894160
 debug: Loaded state in 0.00 seconds
 debug: Loaded state in 0.00 seconds
 info: Applying configuration version '1234973671'
 debug: Finishing transaction 22222222896161
 debug: Storing state
 debug: Stored state in 0.01 seconds

## Known Issues ##

* Numbers are returned as integers or floating point, not strings. Work around: surround value in YAML with quotes (' or ").
* Use of common Ruby idioms are likely lacking
* Debugging variables could/should be coalesced into something more expressive.
* var_to_fact() debug variable is local, not global.  


## References ##

[1] Built in variables, http://docs.puppetlabs.com/guides/faq.html#are-there-variables-available-other-than-those-provided-by-facter