Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vlib: add toml module + tests #11964

Merged
merged 65 commits into from
Sep 24, 2021
Merged
Show file tree
Hide file tree
Changes from 60 commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
da1f39f
vlib: add toml support
larpon Jun 8, 2021
d46ae81
toml: add types and position
larpon Jun 8, 2021
7946481
toml: switch to heap based node tree
larpon Jun 8, 2021
c357c7e
toml: parse assignment
larpon Jun 8, 2021
ad899ed
toml: add identifier and assign parser support
larpon Jun 11, 2021
9fd04c6
toml: rewrite parsing logic
larpon Jun 15, 2021
7967ca3
toml: add missing util module
larpon Jun 15, 2021
4031d5c
toml: run v fmt over all files
larpon Jun 16, 2021
0c4fbed
toml: don't use Table and Array type aliases
larpon Jul 2, 2021
8001d67
toml: add parsing of boolean values
larpon Jul 2, 2021
507642c
toml: simplify boolean parse logic
larpon Jul 2, 2021
e96f129
toml: prepare parsing of time in date time
larpon Jul 2, 2021
58e0703
toml: parse arrays
larpon Jul 13, 2021
53f4bb3
toml: add value extraction, add more tests
larpon Aug 2, 2021
1945bec
toml: rename test file
larpon Aug 2, 2021
5795546
toml: add test that should pass
larpon Aug 2, 2021
b44dd0d
toml: initial map value retrieval
larpon Aug 2, 2021
956cc66
toml: improve value retrieval, fix double key parsing code paths
larpon Aug 3, 2021
18872b2
ast: remove unused scope
larpon Aug 19, 2021
8f46bfc
toml: remove unused walker module
larpon Sep 2, 2021
a8c1990
toml: add Null type as sumtype value for dead ends
larpon Sep 15, 2021
28811bd
toml: fix key scanning, add excerpt function
larpon Sep 15, 2021
614ab6c
toml: fix recent compile warnings, edit panic messages
larpon Sep 15, 2021
d181f53
toml: add test for single letter key values
larpon Sep 15, 2021
6e894f7
examples: add simple toml example
larpon Sep 15, 2021
6ff39a2
toml: add more functions for converting toml.Any type(s)
larpon Sep 15, 2021
b801e2d
time: add rfc3339 parsing
larpon Sep 21, 2021
b483b89
toml: add some documentation
larpon Sep 22, 2021
6bd11df
toml: add documentation, re-structure a bit
larpon Sep 22, 2021
b0c710d
toml: add more tests
larpon Sep 22, 2021
250425e
toml: fix parsing, fix json output, add test
larpon Sep 22, 2021
2618784
toml: reduce most panics to errors
larpon Sep 22, 2021
d6878b2
toml: add invalid tests
larpon Sep 22, 2021
2e44d79
toml: add README.md
larpon Sep 22, 2021
cae378f
toml: fix example
larpon Sep 22, 2021
4067cac
toml: better error propergation from parse()
larpon Sep 22, 2021
619717d
toml: run vfmt
larpon Sep 22, 2021
b137bd2
toml: rename peek_n -> peek
larpon Sep 22, 2021
ed9e1fa
toml: rename ast.Value -> ast.Node
larpon Sep 22, 2021
64732c4
toml: add ast.Node walker
larpon Sep 22, 2021
1e3c4a1
toml: add checker module
larpon Sep 22, 2021
7a632c8
toml: run checks on all ast.Node
larpon Sep 22, 2021
8cf8279
toml: squashed some more invalid checks in BurntSushi tests
larpon Sep 22, 2021
06c0a76
toml: small fixes
larpon Sep 22, 2021
dcddc44
toml: squash more invalid tests
larpon Sep 22, 2021
c04de91
toml: add more invalid checks
larpon Sep 23, 2021
c524419
toml: run vfmt over files
larpon Sep 23, 2021
52e7837
ci: add TOML tests for external testsuite
larpon Sep 23, 2021
a1688bc
ci: add -stats to TOML tests
larpon Sep 23, 2021
54eca8b
toml: run vfmt over tests
larpon Sep 23, 2021
511f5d8
toml: move from vlib/x to vlib
larpon Sep 23, 2021
e088d45
toml: fix README.md example
larpon Sep 23, 2021
8a241c1
toml: fix checker comment
larpon Sep 24, 2021
ef446c8
toml: house keeping
larpon Sep 24, 2021
5a014dd
toml: fix house keeping glitch, more coffee needed
larpon Sep 24, 2021
b1e10c0
toml: run vfmt over types, can I have one more coffee please...
larpon Sep 24, 2021
5bc895c
toml: small key.str() optimization as suggested by @spytheman, fix co…
larpon Sep 24, 2021
3378abb
toml: add defer to clean up after test
larpon Sep 24, 2021
8987aaf
toml: use [if trace_toml?] instead of [if debug] for util.printdbg()
larpon Sep 24, 2021
13ea338
toml: use match for rest of value unwrapping in toml.ast_to_any
larpon Sep 24, 2021
a124e6c
ci: restructure external toml testing a bit
larpon Sep 24, 2021
71d6926
ci: revert restructure, add TOML text from some tests to testdata
larpon Sep 24, 2021
55d1048
toml: add missing test files, good thing it's friday...
larpon Sep 24, 2021
85cfb1a
toml: fix some tests, use external toml files
larpon Sep 24, 2021
1a93e1c
toml: fix byte -1 return as @JalonSolov suggests
larpon Sep 24, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .github/workflows/toml_ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: toml CI

on:
push:
paths-ignore:
- "**.md"
pull_request:
paths-ignore:
- "**.md"

jobs:
toml-module-pass-external-test-suites:
runs-on: ubuntu-20.04
timeout-minutes: 121
steps:

- uses: actions/checkout@v2
- name: Build V
run: make -j2 && ./v -cc gcc -o v cmd/v

- name: Clone BurntSushi/toml-test
run: |
cd vlib/toml/tests/testdata
git clone --depth 1 https://github.com/BurntSushi/toml-test.git burntsushi/toml-test

- name: Run tests
run: ./v -stats test vlib/toml
48 changes: 48 additions & 0 deletions examples/toml.v
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
import toml

// Complete text from the example in the README.md:
// https://github.com/toml-lang/toml/blob/3b11f6921da7b6f5db37af039aa021fee450c091/README.md#Example
const toml_text = '# This is a TOML document.

title = "TOML Example"

[owner]
name = "Tom Preston-Werner"
dob = 1979-05-27T07:32:00-08:00 # First class dates

[database]
server = "192.168.1.1"
ports = [ 8000, 8001, 8002 ]
connection_max = 5000
enabled = true

[servers]

# Indentation (tabs and/or spaces) is allowed but not required
[servers.alpha]
ip = "10.0.0.1"
dc = "eqdc10"

[servers.beta]
ip = "10.0.0.2"
dc = "eqdc10"

[clients]
data = [ ["gamma", "delta"], [1, 2] ]

# Line breaks are OK when inside arrays
hosts = [
"alpha",
"omega"
]'

fn main() {
doc := toml.parse(toml_text) or { panic(err) }
title := doc.value('title').string()
println('title: "$title"')
ip := doc.value('servers.alpha.ip').string()
println('Server IP: "$ip"')

toml_json := doc.to_json()
println(toml_json)
}
55 changes: 55 additions & 0 deletions vlib/time/parse.c.v
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,61 @@ pub fn parse_rfc2822(s string) ?Time {
}
}

// ----- rfc3339 -----
const (
err_invalid_3339 = 'Invalid 3339 format'
)

// parse_rfc3339 returns time from a date string in RFC 3339 datetime format.
pub fn parse_rfc3339(s string) ?Time {
if s == '' {
return error(time.err_invalid_3339 + ' cannot parse empty string')
}
mut t := parse_iso8601(s) or { Time{} }
// If parse_iso8601 DID NOT result in default values (i.e. date was parsed correctly)
if t != Time{} {
return t
}

t_i := s.index('T') or { -1 }
parts := if t_i != -1 { [s[..t_i], s[t_i + 1..]] } else { s.split(' ') }

// Check if s is date only
if !parts[0].contains_any(' Z') && parts[0].contains('-') {
year, month, day := parse_iso8601_date(s) ?
t = new_time(Time{
year: year
month: month
day: day
})
return t
}
// Check if s is time only
if !parts[0].contains('-') && parts[0].contains(':') {
mut hour_, mut minute_, mut second_, mut microsecond_, mut unix_offset, mut is_local_time := 0, 0, 0, 0, i64(0), true
hour_, minute_, second_, microsecond_, unix_offset, is_local_time = parse_iso8601_time(parts[0]) ?
t = new_time(Time{
hour: hour_
minute: minute_
second: second_
microsecond: microsecond_
})
if is_local_time {
return t // Time is already local time
}
mut unix_time := t.unix
if unix_offset < 0 {
unix_time -= (-unix_offset)
} else if unix_offset > 0 {
unix_time += unix_offset
}
t = unix2(i64(unix_time), t.microsecond)
return t
}

return error(time.err_invalid_3339 + '. Could not parse "$s"')
}

// ----- iso8601 -----
const (
err_invalid_8601 = 'Invalid 8601 Format'
Expand Down
55 changes: 55 additions & 0 deletions vlib/toml/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# TOML module
`toml` is a fully fledged TOML v1.0.0 compatible parser written in pure V.

## Usage

```v
import toml

// Complete text from the example in the README.md:
// https://github.com/toml-lang/toml/blob/3b11f6921da7b6f5db37af039aa021fee450c091/README.md#Example
const toml_text = '# This is a TOML document.

title = "TOML Example"

[owner]
name = "Tom Preston-Werner"
dob = 1979-05-27T07:32:00-08:00 # First class dates

[database]
server = "192.168.1.1"
ports = [ 8000, 8001, 8002 ]
connection_max = 5000
enabled = true

[servers]

# Indentation (tabs and/or spaces) is allowed but not required
[servers.alpha]
ip = "10.0.0.1"
dc = "eqdc10"

[servers.beta]
ip = "10.0.0.2"
dc = "eqdc10"

[clients]
data = [ ["gamma", "delta"], [1, 2] ]

# Line breaks are OK when inside arrays
hosts = [
"alpha",
"omega"
]'

fn main() {
doc := toml.parse(toml_text) or { panic(err) }
title := doc.value('title').string()
println('title: "$title"')
ip := doc.value('servers.alpha.ip').string()
println('Server IP: "$ip"')

toml_json := doc.to_json()
println(toml_json)
}
```
Loading