Slave mode broken for WMTS layers with dimensions

[pvalsecc]

My layer is configured like that:

defaults:
    layer: &layer
        grid: swissgrid
        type: wms
        url: http://mapserver/mapserv
        extension: png
        mime_type: image/png
        wmts_style: default
        dimensions:
        -    name: DATE
             default: '20160518'
             generate: ['20160518']
             values: ['20140603', '20150309', '20160429', '20160518']
        meta: on
        meta_size: 2
        meta_buffer: 50
        cost:
            tileonly_generation_time: 40
            tile_generation_time: 30
            metatile_generation_time: 30
            tile_size: 20

layers:
    Mtbland:
        <<: *layer
        layers:  MtblandAll

When starting the slave, it fails wih this stacktrace (I've included the 2 debug prints I've added to understand the problem:

ERROR:tilecloud_chain:get_store:layer['dimensions']=[{'default': '20160518', 'values': ['20140603', '20150309', '20160429', '20160518'], 'name': 'DATE', 'generate': ['20160518']}]
ERROR:tilecloud_chain:get_store:dimensions={}
Traceback (most recent call last):
  File "/usr/local/bin/generate_tiles", line 11, in <module>
    load_entry_point('tilecloud-chain', 'console_scripts', 'generate_tiles')()
  File "/src/tilecloud_chain/generate.py", line 466, in main
    generate.gene(options, gene)
  File "/src/tilecloud_chain/generate.py", line 49, in gene
    self._gene(options, gene)
  File "/src/tilecloud_chain/generate.py", line 86, in _gene
    self.cache_tilestore = gene.get_tilesstore(options.cache, dimensions)
  File "/src/tilecloud_chain/__init__.py", line 570, in get_tilesstore
    for lname, layer in self.layers.items()
  File "/src/tilecloud_chain/__init__.py", line 570, in <dictcomp>
    for lname, layer in self.layers.items()
  File "/src/tilecloud_chain/__init__.py", line 457, in get_store
    for dimension in layer['dimensions']
KeyError: 'DATE'

I can trace back the empty dimensions parameter to this line:

tilecloud-chain/tilecloud_chain/generate.py

Line 49 in 29ca2f1

self._gene(options, gene)

But I don't know what to do, here.

[sbrunner]

Isn't related to #299?

[sbrunner]

For me this:
https://github.com/alexbrault/tilecloud-chain/blob/48677e7483ec2ee5f94616cffb631c4fe325e168/tilecloud_chain/__init__.py#L562-L565
should be replaced by:

    def get_tilesstore(self, cache_name):
        cache = self.caches[cache_name]
        cache_tilestore = MultiTileStore({
            lname: self.get_store(cache, layer, dimensions=layer.dimensions)

or better:

    def get_tilesstore(self, cache_name):
        cache = self.caches[cache_name]
        cache_tilestore = MultiTileStore({
            lname: self.get_store(cache, layer)

and the dimension will be get in the get_store.

[pvalsecc]

No, it's not related to #299. All my layers have the same dimension.

[pvalsecc]

It's not that easy. In that context, dimensions is supposed to be a dictionary of {dimension_name} -> {value}.
In slave mode, I don't see how to support multi-dimension values.
To be perfect, in master slave mode, the master must add SQS messages for each dimension values combination and the dimension values must be included in the message.

[sbrunner]

Effectively, but I don't understand what's different with local one...

[pvalsecc]

In local mode, we generate for every combination:

tilecloud-chain/tilecloud_chain/generate.py

Lines 41 to 47 in c3b2a2b

	all_dimensions = gene.get_all_dimensions()
	if len(all_dimensions) == 0: # pragma: no cover
	self._gene(options, gene)
	else:
	for dimensions in all_dimensions:
	self._gene(options, gene, dimensions)

By the way, I see that the master is doing that as well. But I guess the values oare not passed in SQS and the slave, with my PR, will do only the same set of values every times.

[sbrunner]

And it's don on master, then we will have an SQS messages for each dimension, not?
The missing thing is that the dimensions should be added to the tiles metadatas...

[pvalsecc]

I've just tried to do it, but it's a huge change that involves changes in tilecloud as well: a lot of places take only the TileCoord as parameter and expect the dimensions as member of the class. We need all those places to take the Tile as parameter and remove the dimensions member. We don't have time for that. Can't we just take my hackish PR for the moment and leave this issue open to remember to do a proper fix?

[sbrunner]

Fixed with #314