zookeeper::config manages datastore directory recursively - puppet takes 1700+ seconds to complete run

[danielbenzvi]

Hello
We experience an issue where Puppet runs take over 1700 seconds on our zookeeper instances. We found out Puppet's state.yaml is over 500MB in size due to a file resource managing $datastore recursively in config.pp:

  file { $datastore:
    ensure  => directory,
    owner   => $user,
    group   => $group,
    mode    => '0644',
    recurse => true,
  }

Our state.yaml contains mostly ZooKeeper snapshots:

  File[/var/lib/zookeeper/version-2/snapshot.18730c54a5]:
    !ruby/sym checked: 2016-01-07 18:40:26.182027 -05:00
  File[/var/lib/zookeeper/version-2/log.17d2e5191f]:
    !ruby/sym checked: 2015-12-29 05:17:05.593017 -05:00
  File[/var/lib/zookeeper/version-2/snapshot.139d2c865a]:
    !ruby/sym checked: 2015-10-26 17:17:32.388460 -05:00
  File[/var/lib/zookeeper/version-2/snapshot.170f6fdcc1]:

Seems like Puppet keeps state.yaml entries forever, even if the file was already deleted: https://tickets.puppetlabs.com/browse/PUP-3647

Is there any specific reason for managing $datastore recursively?

[deric]

Right, I see, that's quite annoying. It should be ZooKeepers responsibility to create files with proper owner/group. So, I think we should remove this.