Skip to content

Commit

Permalink
docs: Update contributing guide to reflect the latest data process
Browse files Browse the repository at this point in the history
  • Loading branch information
@gilmoreorless
gilmoreorless committed May 21, 2023
1 parent 8b39cdc commit 4b1419b
Showing 1 changed file with 98 additions and 77 deletions.
175 changes: 98 additions & 77 deletions contributing.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ demonstrating the issue before opening a ticket on GitHub.
## Contributing

Fork the library, install grunt, and install dependencies.
Note that all pull requests will require a CLA.
Note that all pull requests will require a signed [CLA (Contributor License Agreement)](https://openjsf.org/about/the-openjs-foundation-cla/).

```
npm install -g grunt-cli
Expand All @@ -19,9 +19,10 @@ Additions to `moment-timezone.js` or `moment-timezone-utils.js` should have matc
## Running tests

```sh
grunt nodeunit # run all tests
grunt nodeunit:core # run all core tests
grunt nodeunit:zones # run all zone tests
grunt nodeunit # run all tests (includes the 3 commands below)
grunt nodeunit:core # run all core tests
grunt nodeunit:zones # run all zone tests
grunt nodeunit:countries # run all country tests
```

If all the tests are passing, submit a pull request.
Expand All @@ -40,13 +41,20 @@ dependencies.
which curl tar zic zdump
```

To compile the latest iana release, run `grunt data`.
To compile the latest IANA release, run `grunt data`.

To compile a specific tagged release, run `grunt data:2014d`, replacing `2014d`
with the desired release tag.

The compilation process involves 7 steps. For each of the steps, a tag name can
be passed to grunt. If no tag name was passed, it defaults to the latest release.
The compilation process involves 8 steps, each of which can be run separately.
For each of the steps, a tag name can be passed to grunt. If no tag name was passed,
it defaults to the latest release.

Some tasks create files within the `temp` directory. These are only used by other
parts of the process and are ignored by git.

Other tasks create files in the `data` or `tests` directories. These files should
be committed to git.

### 1. Download data from iana.org/time-zones.

Expand All @@ -57,7 +65,64 @@ grunt data-download:2014d
This downloads the data to `temp/curl/2014d/data.tar.gz` and unzips into
`temp/download/2014d`.

### 2. Compile the data to a binary format using `zic(8)`
### 2. Extract zone & country meta data

```
grunt data-meta:2014d
```

This task creates a JSON file containing a list of timezones and countries.
The extracted meta data can be used to list all timezones, list all countries, and filter zones by a country.

The output is saved in `data/meta/2014d.json`

Sample output:

```
{
"countries": {
"AD": {
"name": "Andorra",
"abbr": "AD",
"zones": [
"Europe/Andorra"
]
},
"AE": {
"name": "United Arab Emirates",
"abbr": "AE",
"zones": [
"Asia/Dubai"
]
},
...
},
"zones": {
"Europe/Andorra": {
"name": "Europe/Andorra",
"lat": 42.5,
"long": 1.5167,
"countries": [
"AD"
],
"comments": ""
},
"Asia/Dubai": {
"name": "Asia/Dubai",
"lat": 25.3,
"long": 55.3,
"countries": [
"AE",
"OM"
],
"comments": ""
},
...
}
}
```

### 3. Compile the data to a binary format using `zic(8)`

```
grunt data-zic:2014d
Expand All @@ -66,7 +131,13 @@ grunt data-zic:2014d
This compiles each of the source files in the `temp/download/2014d` folder to
`temp/zic/2014d`.

### 3. Dump the changes from the binary format into a text format using `zdump(8)`.
An alternative path to the `zic` binary can be provided using the `--zic-path` option.

```
grunt data-zic:2014d --zic-path=/path/to/custom/zic
```

### 4. Dump the changes from the binary format into a text format using `zdump(8)`.

```
grunt data-zdump:2014d
Expand All @@ -82,7 +153,14 @@ This dumps a list of timezone changes for each zone in `temp/zic/2014d` to
>
> via `man zdump`
### 4. Collect the changes into a single JSON file.
An alternative path to the `zdump` binary can be provided using the `--zdump-path` option.

```
grunt data-zdump:2014d --zdump-path=/path/to/custom/zdump
```


### 5. Collect the changes into a single JSON file.

```
grunt data-collect:2014d
Expand Down Expand Up @@ -113,7 +191,7 @@ to `temp/collect/2014b.json`.
...
```

### 5. Remove duplicate entries.
### 6. Remove duplicate entries.

```
grunt data-dedupe:2014d
Expand All @@ -123,11 +201,9 @@ grunt data-dedupe:2014d
each time discontinuity. In order to save space, we remove the
_one second before_ entries.

The output is saved in `data/unpacked/2014d.json`. This is the first file in the
process that is committed to git. Steps 1-4 are stored in an ignored `temp/`
directory.
The output is saved in `data/unpacked/2014d.json`.

### 6. Pack the data into the compressed format.
### 7. Pack the data into the compressed format.

```
grunt data-pack:2014d
Expand All @@ -148,85 +224,30 @@ Pack each zone in `data/unpacked/2014d.json` and save in `data/packed/2014d.json
"America/Phoenix|MST MDT MWT|70 60 60|01010202010|-261r0 1nX0 11B0 1nX0 SgN0 4Al1 Ap0 1db0 SWqX 1cL0"
```

### 7. Extract zone & country meta data

```
grunt data-meta:2014d
```

This task creates a JSON file containing list of timezones and countries. The extracted meta data can be used to list all timezones, list all countries, and filter zones by a country.

The output is saved in `data/meta/2014d.json`

Sample output:

```
{
"countries": {
"AD": {
"name": "Andorra",
"abbr": "AD",
"zones": [
"Europe/Andorra"
]
},
"AE": {
"name": "United Arab Emirates",
"abbr": "AE",
"zones": [
"Asia/Dubai"
]
},
...
},
"zones": {
"Europe/Andorra": {
"name": "Europe/Andorra",
"lat": 42.5,
"long": 1.5167,
"countries": [
"AD"
],
"comments": ""
},
"Asia/Dubai": {
"name": "Asia/Dubai",
"lat": 25.3,
"long": 55.3,
"countries": [
"AE",
"OM"
],
"comments": ""
},
...
}
}
```

### 8. Build tests for each zone.

```
grunt data-tests
```

This task does not support tagged releases, and is only run on the latest
timezone data.

Reads data from `temp/collect/latest.json` and constructs unit tests for each
zone. This uses `temp/collect/latest.json` instead of `data/unpacked/latest.json`
in order to test both the times _one second before_ and _exactly at_
each time discontinuity.

Without a version, this will write files to the `tests` directory.
If this is run with a specific version (`grunt data-tests:2014d`), the files will
be written to `temp/tests/<version>` instead.

### All tasks

For sanity, all these tasks are bundled into one task. This is the preferred way
For ease of use, all these tasks are bundled into one task. This is the preferred way
to compile the data.

```
grunt data # run tasks 1-8 on the latest release
grunt data:2014c # run tasks 1-7 on the 2014c release
grunt data:2014d # run tasks 1-7 on the 2014d release
grunt data:2014c # run tasks 1-8 on the 2014c release
grunt data:2014d # run tasks 1-8 on the 2014d release
```


Expand Down

0 comments on commit 4b1419b

Please sign in to comment.