Upon start up, tessera spends several minutes at 100% CPU before returning anything

[amandasaurus]

I have ported the openstreetmap-carto style to vector tiles, and use tessera to serve it. I have pre-generated the vector tiles with tilelive-copy. I'm serving these over http. I'm using tessera to apply the osm-carto style to those vector tiles.

And when I start tessera, and hit a URL for the map JS page, or an individual tile, I get mapnik log messages about depreciated min/maxzoom. And then tessera sits there for a long time (20 minutes?), at 100% of one CPU core, before it returns anything. Once it's gotten over this initial CPU thing, raster tile images are generated in a sensibly quick, snappy, manner. It's just the first tile that takes about 30 minutes.

From looking at the http request logs of the vector tiles, tessera is loading the tilejson file, and nothing else. So it's not loading the actual vector tiles files (ie the MVT pbf's).

What's going on? Why does it take so long to initialize? How can I debug this? stracing the process just shows futex stuff, so I presume it's doing all the things in Javascript.

[amandasaurus]

Further investigation appears to show that the CPU intensive part takes place before it reads the tm2source. I had a typo in a tessera config, pointing to an non-existant tm2source, and the error about that appear after several minutes of sitting there churning away.

Also I think Kosmtik has the same/similar problem.

[amandasaurus]

Print statement debugging is telling me that the biggest slow down is from the call to carto.Renderer() from this part of tilelive-tmstyle.

[mojodna]

Ah, that makes total sense, particularly since openstreetmap-carto is quite complex.

The potential way around that would be to pre-compile the CartoCSS to XML, but that won't work with the existing tilelive-mapnik module (which is what is typically used for Mapnik XML) since it doesn't understand how to configure tilelive-vector.

tilelive-tmstyle could potentially be adapted to read XML-formatted versions that were prepared ahead of time, otherwise...

[amandasaurus]

Ah, that makes total sense, particularly since openstreetmap-carto is quite complex.

I know that the openstreetmap-carto style is very complicated (some say it's one of the most complex cartocss styles out there), however the times I'm seeing are very strange. It's taking 4 minutes for it to convert the style! Surely something is going wrong?

When I run the carto command line on the (non-vector tiled version) of openstreetmap-carto it takes about 16 seconds. This is with version 0.9 from apt, and 0.16 from the source github repository. How comes this is taking literally more than ten times slower?

I don't know enough about nodejs or npm in order to be do a git bisect or something, in order to trace things down. You can get the vectortile osm-carto here: https://github.com/geofabrik/openstreetmap-carto-vector-tiles

[amandasaurus]

Ah, that makes total sense, particularly since openstreetmap-carto is quite complex.

Another weird data point: You can check out the openstreetmap-carto style, and render it with tessera with this command: ./node_modules/.bin/tessera carto+file://./project.mml, and it can serve images after ~20 seconds. This style doesn't use vectortiles. So the carto is able to render/parse/process the osm-carto style in ~20 seconds. But it takes ten times as long to process when using tilelive-tmstyle and vector tiles.....

[mojodna]

Very odd, especially since some of the initialization time for database-backed styles is checking whether tables exist, etc., which isn't necessary with vector tiles.

tilelive-tmstyle is currently locked to ~0.14.0, so upgrading that may help things some. I'm not familiar with carto internals, but I do know that @JesseCrocker has been able to find some performance improvements in tilelive-tmstyle already, so there may be more that can be done there.

Nice talk at SotM, btw!

[pnorman]

osm-carto should take 10 seconds or so to compile. After my blog post we found some big gains in how we were referring to layers by classes, but you'd already have those gains. This is what I found in Kosmtik as well as with the command line, and across a reasonable range of carto versions.

[amandasaurus]

I have made a local git clone of tilelive-tmsource and tilelive-tmstyle, and changed the "carto" dependency on both to "latest". It has installed 0.17.2. And I still get the same problem. It takes ~3½ minutes to start up. npm list tells me there is only one version of carto installed (the 0.17.2 one).

[amandasaurus]

I have opened an issue on the carto project, hence the problem isn't here.