Skip to content

Commit

Permalink
Merge branch 'master' into buildings_rewrite_3
Browse files Browse the repository at this point in the history 
Conflicts:
	project.mml
  • Loading branch information
@matthijsmelissen
matthijsmelissen committed Oct 4, 2014
2 parents 00edc02 + 555c868 commit 6b2a4de
Show file tree
Hide file tree
Showing 183 changed files with 4,755 additions and 2,448 deletions.
22 changes: 22 additions & 0 deletions .travis.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
language: node_js
cache: apt
node_js:
- "0.10"
before_install:
- sudo apt-get update -qq
- sudo apt-get install -y -q python-demjson
install:
- npm install [email protected]
- mkdir -p data/world_boundaries
- mkdir -p data/simplified-land-polygons-complete-3857
- mkdir -p data/ne_110m_admin_0_boundary_lines_land
- mkdir -p data/ne_10m_populated_places
- mkdir -p data/land-polygons-split-3857
- touch data/world_boundaries/builtup_area.shp
- touch data/simplified-land-polygons-complete-3857/simplified_land_polygons.shp
- touch data/ne_110m_admin_0_boundary_lines_land/ne_110m_admin_0_boundary_lines_land.shp
- touch data/ne_10m_populated_places/ne_10m_populated_places_fixed.shp
- touch data/land-polygons-split-3857/land_polygons.shp
script:
- jsonlint project.mml
- ./node_modules/carto/bin/carto project.mml | xmllint - | wc -l
25 changes: 24 additions & 1 deletion CONTRIBUTING.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,16 @@ You should fork the project into your own repo, create a topic branch
there and then make one or more pull requests back to the gravitystorm repository.
Your pull requests will then be reviewed and discussed.

## Editing Layers

OpenStreetMap Carto uses a YAML file for defining layers. Some of the rationale is outlined in [a GitHub issue](https://github.com/gravitystorm/openstreetmap-carto/issues/711). Editing multi-line SQL statements in a YAML file is much friendlier than editing escaped SQL in a JSON file.

This requires a preprocessing step to convert the YAML to JSON. A script is provided at `scripts/yaml2mml.py`, which depends on PyYAML, available through `pip install pyyaml` or packaged on Ubuntu as `python-yaml`.

After editing the YAML file, run `./scripts/yaml2mml.py < project.yaml > project.mml && touch project.mml` to update the file and force TileMill to reload it.

When committing changes, add both the `project.yaml` and `project.mml` files to the commit. One of the big advantages of this system is that to resolve any layer merge conflicts, they only need to be resolved in the YAML file where they are easier to handle, then the JSON file can be regenerated, while at the same time the styles work with Tilemill out-of-the-box without needing to run the `yaml2mml` script.

## CartoCSS Style Guidelines

* Always specify zoom levels as either >= or < . Don't use = or =< or >
Expand All @@ -18,7 +28,6 @@ Your pull requests will then be reviewed and discussed.
* Two space indents. No tabs.
* space after : but not before
* Dashes, not underscores, in layer names
* Name SQL subqueries after the layer name (but use underscores)
* Avoid restating defaults, e.g. don't add `point-allow-overlap = false`
* Avoid repeating the layer name for layers with mutiple attachments, i.e., prefer

Expand Down Expand Up @@ -48,6 +57,20 @@ instead of
```
* Order the selectors in a style-sheet in rough order of importance (i.e., highway=primary, then highway=secondary) and beyond that, add layers that are rendered later (i.e., higher) lower in the file.

## SQL Style Guidelines
Because SQL within JSON or YAML will not generally be syntax highlighted, indentation and caps are particularly important.

* SQL keywords in caps, as in PostgreSQL documentation
* Two space indents. No tabs.
* Start with `(SELECT` and start the columns on the next line.
* Two indents for columns, to bring them to the same indent level as later clause contents
* Add indentation after `SELECT`s until the end of the sub-select.
* Add indentation for contents of `FROM`, `WHERE`, `ORDER BY` and other clauses
* Put content with WHERE, etc if it's short
* Add indentation if necessary for complex function calls, WHERE parenthesis, and CASE statements
* One space before and after = etc
* Name SQL subqueries after the layer name (but use underscores)

## Previews

Some changes benefit from a review from a wider audience. In most cases some static images are sufficient, but sometimes a demo layer is necessary. pnorman has a private server which can host layers and has some data from parts of the world loaded. Before requesting this in a pull request, make sure that you don't anticipate any more changes to it.
Expand Down
95 changes: 95 additions & 0 deletions INSTALL.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# Installation

You need OpenStreetMap data loaded into a PostGIS database (see below for [dependencies](https://github.com/gravitystorm/openstreetmap-carto#dependencies)). These stylesheets currently work only with the osm2pgsql defaults (i.e. database name is `gis`, table names are `planet_osm_point`, etc).

It's probably easiest to grab an PBF of OSM data from [Mapzen](https://mapzen.com/metro-extracts/) or [geofabrik](http://download.geofabrik.de/). Once you've set up your PostGIS database, import with osm2pgsql:

```
osm2pgsql -d gis ~/path/to/data.osm.pbf --style openstreetmap-carto.style
```

You can find a more detailed guide to setting up a database and loading data with osm2pgsql at [switch2osm.org](http://switch2osm.org/loading-osm-data/).

Additionally you need some shapefiles.

## Scripted download

To download the shapefiles you can run the following script from this directory. No further steps should be needed as the data has been processed and placed in the requisite directories.

```
./get-shapefiles.sh
```

## Manual download

You can also download them manually at the following paths:

* [`simplified-land-polygons.shp`](http://data.openstreetmapdata.com/simplified-land-polygons-complete-3857.zip) (updated daily)
* [`land-polygon.shp`](http://data.openstreetmapdata.com/land-polygons-split-3857.zip) (updated daily)
* [`builtup_area.shp`](http://planet.openstreetmap.org/historical-shapefiles/world_boundaries-spherical.tgz)
* [`ne_110m_admin_0_boundary_lines_land.shp`](http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/110m/cultural/ne_110m_admin_0_boundary_lines_land.zip)
* [`ne_10m_populated_places_fixed.shp`](http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/cultural/ne_10m_populated_places.zip) (and see [below](#populated-places-shapefile))

The repeated www.naturalearthdata.com in the Natural Earth shapefiles is correct.

Put these shapefiles at `path/to/openstreetmap-carto/data`.

### Populated places shapefile

The Natural Earth 2.0 populated places shapefile contains data that triggers a bug in mapnik. As
a workaround we run the shapefile through ogr2ogr to clean up the data.

```
ogr2ogr ne_10m_populated_places_fixed.shp ne_10m_populated_places.shp
```

See https://github.com/mapnik/mapnik/issues/1605 for more details.

## Fonts
The stylesheet depends on a number of openly licensed fonts for support of all the languages found on the map. The package which supplies these fonts on Ubuntu is indicated.

If a font is missing, it will skip to the next available font which contains those characters. If you are not concerned with a particular language, you do not need its fonts. DejaVu Sans and Unifont are the two required fonts, and included on most systems.

### Global
* DejaVu Sans, for most languages (`ttf-dejavu`)
* Droid Sans Fallback, as a reasonable fallback (`fonts-droid`)
* Unifont, as a last resort fallback (`ttf-unifont`)

### Southeast Asia
* Arundina Sans, for Thai (`fonts-sipa-arundina`)
* Padauk, for Burmese (`fonts-sil-padauk`)
* Khmer OS Metal Chrieng Regular, for Khmer (`fonts-khmeros`)

### South Asia

* Mukti Narrow, for Bangali (`ttf-indic-fonts-core`)
* Gargi Medium, for Devanagari (`ttf-indic-fonts-core`)
* TSCu_Paranar, for Tamil (`ttf-tamil-fonts`)
* Mallige, for Kannada (`ttf-indic-fonts-core` for normal and bold and `ttf-kannada-fonts` for oblique) *The filename uses "Malige" but the font name uses "Mallige"*

On Ubuntu you can install all the fonts with

```
sudo apt-get install ttf-dejavu fonts-droid ttf-unifont fonts-sipa-arundina fonts-sil-padauk fonts-khmeros \
ttf-indic-fonts-core ttf-tamil-fonts ttf-kannada-fonts
```

## Dependencies

* [TileMill](http://mapbox.com/tilemill) - This is a TileMill project you can copy (or symlink) directly into your Mapbox/project directory

If you aren't using TileMill, you can compile the CartoCSS stylesheets into Mapnik XML using the command-line `carto` command.

* [carto](https://github.com/mapbox/carto) >= 0.9.3 (we're using instances with cascading rules)
* [mapnik](https://github.com/mapnik/mapnik/wiki/Mapnik-Installation) >= 2.1.0

---

* [osm2pgsql](http://wiki.openstreetmap.org/wiki/Osm2pgsql) to import your data into a PostGIS database
* [PostgreSQL](http://www.postgresql.org/)
* [PostGIS](http://postgis.org/)
* [ogr2ogr](http://www.gdal.org/) command line GDAL utility for processing vector data. here we use it to work around a encoding bug in the Nautral Earth data.
* curl, unzip for downloading and decompressing files
* shapeindex (a companion utility to Mapnik found in the mapnik-utils package) for indexing downloaded shapefiles

* [PyYAML](http://pyyaml.org/wiki/PyYAML) if editing the MML (layer definition) file (packaged as `python-yaml` on Ubuntu, or installed with `pip install pyyaml`)
147 changes: 50 additions & 97 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,104 +1,40 @@
# OpenStreetMap Carto

The standard stylesheet on [OpenStreetMap.org](http://www.openstreetmap.org) An implemention of the standard OpenStreetMap mapnik style, in CartoCSS.
![screenshot](https://raw.github.com/gravitystorm/openstreetmap-carto/master/preview.png)

These are the CartoCSS map stylesheets for the Standard map layer on [OpenStreetMap.org](http://www.openstreetmap.org).

These stylesheets can be used in your own cartography projects, and are designed to be easily
customised. They work with [TileMill](http://www.mapbox.com/tilemill/) and also with the command-line [CartoCSS](https://github.com/mapbox/carto) processor.

Since August 2013 these stylesheets are used on the OSMF tileservers (tile.openstreetmap.org), and
are updated from each point release. They supersede the previous [XML-based stylesheets](https://trac.openstreetmap.org/browser/subversion/applications/rendering/mapnik)

# Setup

You need OpenStreetMap data loaded into a PostGIS database (see below for [dependencies](https://github.com/gravitystorm/openstreetmap-carto#dependencies)). These stylesheets currently work only with the osm2pgsql defaults (i.e. database name is ``gis``, table names are ``planet_osm_point``, etc).

It's probably easiest to grab an PBF of OSM data from [metro.teczno.com](http://metro.teczno.com/) or [geofabrik](http://download.geofabrik.de/). Once you've set up your PostGIS database, import with osm2pgsql:

```
osm2pgsql -d gis ~/path/to/data.osm.pbf --style openstreetmap-carto.style
```

You can find a more detailed guide to setting up a database and loading data with osm2pgsql at [switch2osm.org](http://switch2osm.org/loading-osm-data/)

Additionally you need some shapefiles.

## Scripted download

To download the shapefiles you can run the following script from this directory. No further steps should be needed as the data has been processed and placed in the requisite directories.

```
./get-shapefiles.sh
```

## Manual download

You can also download them manually at the following paths:

* `simplified-land-polygons.shp` [download](http://data.openstreetmapdata.com/simplified-land-polygons-complete-3857.zip)
* `land-polygon.shp` [download](http://data.openstreetmapdata.com/land-polygons-split-3857.zip)
* `builtup_area.shp` [download](http://planet.openstreetmap.org/historical-shapefiles/world_boundaries-spherical.tgz)
* `ne_110m_admin_0_boundary_lines_land.shp` [download](http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/110m/cultural/ne_110m_admin_0_boundary_lines_land.zip)
* `ne_10m_populated_places_fixed.shp` [download](http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/cultural/ne_10m_populated_places.zip) (and see below)

Put these shapefiles at path/to/opentreetmap-carto/data

### Populated places shapefile
Since August 2013 these stylesheets have been used on the OSMF tileservers (tile.openstreetmap.org), and
are updated from each point release. They supersede the previous [XML-based stylesheets](https://trac.openstreetmap.org/browser/subversion/applications/rendering/mapnik).

The Natural Earth 2.0 populated places shapefile contains data that triggers a bug in mapnik. As
a workaround we run the shapefile through ogr2ogr to clean up the data.
# Installation

```
ogr2ogr ne_10m_populated_places_fixed.shp ne_10m_populated_places.shp
```
You need a PostGIS database populated with OpenStreetMap data in the standard
osm2pgsql database layout, along with auxillary shapefiles. See [INSTALL.md](INSTALL.md).

See https://github.com/mapnik/mapnik/issues/1605 for more details.
# Contributing

## Fonts
The stylesheet depends on a number of openly licensed fonts for support of all the languages found on the map. The package which supplies these fonts on Ubuntu is indicated.
Contributions to this project are welcome, see [CONTRIBUTING.md](CONTRIBUTING.md)
for full details.

If a font is missing, it will skip to the next available font which contains those characters. If you are not concerned with a particular language, you do not need its fonts. DejaVu Sans and Unifont are the two required fonts, and included on most systems.
# Versioning

### Global
* DejaVu Sans, for most languages (``ttf-dejavu``)
* Droid Sans Fallback, as a reasonable fallback (``fonts-droid``)
* Unifont, as a last resort fallback (``ttf-unifont``)
This project follows a MAJOR.MINOR.PATCH versioning system. In the context of a
cartographic project you can expect the following:

### Southeast Asia
* Arundina Sans, for Thai (``fonts-sipa-arundina``)
* Padauk, for Burmese (``fonts-sil-padauk``)
* Khmer OS Metal Chrieng Regular, for Khmer (``fonts-khmeros``)

### South Asia

* Mukti Narrow, for Bangali (``ttf-indic-fonts-core``)
* Gargi Medium, for Devanagari (``ttf-indic-fonts-core``)
* TSCu_Paranar, for Tamil (``ttf-tamil-fonts``)
* Mallige, for Kannada (``ttf-indic-fonts-core`` for normal and bold and ``ttf-kannada-fonts`` for oblique) *The filename uses "Malige" but the font name uses "Mallige"*

On Ubuntu you can install all the fonts with

```
sudo apt-get install ttf-dejavu fonts-droid ttf-unifont fonts-sipa-arundina fonts-sil-padauk fonts-khmeros \
ttf-indic-fonts-core ttf-tamil-fonts ttf-kannada-fonts
```

## Dependencies

* [TileMill](http://mapbox.com/tilemill) - This is a TileMill project you can copy (or symlink) directly into your Mapbox/project directory

If you aren't using TileMill, you can compile the CartoCSS stylesheets into Mapnik XML using the command-line `carto` command.

* [carto](https://github.com/mapbox/carto) >= 0.9.3 (we're using instances with cascading rules)
* [mapnik](https://github.com/mapnik/mapnik/wiki/Mapnik-Installation) >= 2.1.0

---

* [osm2pgsql](http://wiki.openstreetmap.org/wiki/Osm2pgsql) to import your data into a PostGIS database
* [PostgreSQL](http://www.postgresql.org/)
* [PostGIS](http://postgis.org/)
* [ogr2ogr](http://www.gdal.org/) command line GDAL utility for processing vector data. here we use it to work around a encoding bug in the Nautral Earth data.
* curl, unzip for downloading and decompressing files
* shapeindex (a companion utility to Mapnik found in the mapnik-utils package) for indexing downloaded shapefiles
* PATCH: When a patch version is released, there would be no reason not to
upgrade. PATCH versions contain only bugfixes e.g. stylesheets won't compile,
features are missing by mistake, etc.
* MINOR: These are routine releases and happen every 1-3 weeks. They will
contain changes to what's shown on the map, how they appear, new features
added and old features removed. They may rarely contain changes to assets i.e.
shapefiles and fonts but will not contain changes that require software or
database upgrades.
* MAJOR: Any change the requires reloading a database, or upgrading software
dependecies will trigger a major version change.

# Roadmap

Expand All @@ -107,25 +43,42 @@ If you aren't using TileMill, you can compile the CartoCSS stylesheets into Mapn
This was a full re-implementation of the original OSM style, with only a few bugs discovered later. There's been
no interest in creating further point releases in the v1.x series.

## Easier to wrangle (v2.x)
## Current work (v2.x)

There are a number of refactorings that can be made to the style, either to fix glitches
with the current style, or to leverage new features in carto / mapnik to simplify the stylesheets
with only small changes to the output. It's also appropriate to pull out some of the 'old-skool'
The v2.x series focuses on refactoring the style, both to to fix glitches and to
leverage new features in CartoCSS / mapnik to simplify the stylesheets with only
small changes to the output. It's also appropriate to pull out the 'old-skool'
tagging methods that are now rarely used.

Care is being taken to not get too clever with variables and expressions. While these often make
it easier to customise, experience has shown that over-cleverness (e.g. [interpolated entities][cleverness])
can discourage contributions.

The end goal will be a style that hews close to the current look of the standard style, but is
much more suitable for further development, and/or forking for third-parties to customise.
The end goal will be a style that remains familiar but is much more suitable for
further development, and/or forking for third-parties to customise.

## Tackle the backlog (v3.x)
## Future (v3.x)

There are over [400 open requests][trac] on trac, some that have been open for years. These need
There are over [300 open requests][issues], some that have been open for years. These need
reviewing and dividing into obvious fixes, or additional new features that need some cartographic
judgement. The work done already in v1.0 and v2.0 will make it much easier to process these.

[trac]: https://trac.openstreetmap.org/query?component=mapnik&status=!closed&order=changetime&desc=1&max=500
[issues]: https://github.com/gravitystorm/openstreetmap-carto/issues
[cleverness]: https://github.com/openstreetmap/mapnik-stylesheets/blob/master/inc/settings.xml.inc.template#L16

# Alternatives

There are many open-source stylesheets written for creating OpenStreetMap-based
maps using mapnik, many based on this project. Some alternatives are:

* [OSM-Bright](https://github.com/mapbox/osm-bright)
* [XML-based stylesheets](https://trac.openstreetmap.org/browser/subversion/applications/rendering/mapnik)
* [osmfr-cartocss](https://github.com/cquest/osmfr-cartocss)
* [openstreetmap-carto-german](https://github.com/woodpeck/openstreetmap-carto-german)

# Maintainers

* Andy Allan [@gravitystorm](https://github.com/gravitystorm/)
* Matthijs Melissen [@math1985](https://github.com/math1985/)
* Paul Norman [@pnorman](https://github.com/pnorman/)
* Mateusz Konieczny [@mkoniecz](https://github.com/mkoniecz/)
Loading

0 comments on commit 6b2a4de

Please sign in to comment.