diff --git a/README.md b/README.md index 6354fb1..3675e70 100644 --- a/README.md +++ b/README.md @@ -1,172 +1,235 @@ # Hial -Hial is an uniform data API, particularly suitable for textual data. It is a CLI tool backed by a library. +Hial is a general purpose data API and CLI tool. It is a programmatic interface to different types of data represented in an uniform manner (a general tree/graph structure). This makes the data easy to read, explore and modify using a small number of functions. Hial proposes a relatively simple mental model, suitable for most use cases, which improves the user comfort and speed. -An uniform data API is a programmatic interface to different types of data represented in an uniform manner (a general tree/graph structure). This makes the data easy to read, explore and modify using a small number of functions. The types of data that can be supported by this API are the file system structure, usual configuration files (json, yaml, toml), markup files (xml, html), programs written in various programming languages, operating system configurations and runtime parameters, database tables and records, etc. +The types of data that can be supported by this API are the file system, configuration files (json, yaml, toml), markup files (xml, html), programs written in various programming languages, operating system configurations and runtime parameters, database tables and records, etc. -### Why is it needed? +The API can be seen as an generalization of the concepts behind xpath, json path, file system path, and other similar path languages. It is a concise way to express data operations and transformations which are common in programming and system administration. -The uniform data API aims to make common data read/search/write operations easy to declare and execute. It maximizes user comfort and speed because it requires a single mental model, which is suitable for most use cases. A simple and uniform data model makes it easy to create, transform, filter and delete any kind of data, manually or programmatically. +:warning: Hial is **currently under construction.** Some things don't work yet and some things will change. -The following tasks should be easy with such an API: -- pinpoint changes in configuration files (e.g. change a specific value in a json/yaml/toml/text file) -- interactive or automated data exploration (structured grep) -- semantic diffs -- custom program refactoring - +### What can it do? -## The data model +##### 1. Select or search for precise pieces of data in a structured way. -🚧 **Hial is currently under construction. Some things do not work yet and some other things will change**. 🚧 +Print a value embedded in a json file or from a url that returns json or xml data: -The data model is that of a tree of simple data nodes. The tree has a root node and a hierarchy of children nodes. +```bash +hial './config.json^json/services/webapp' +hial 'http://www.phonetik.uni-muenchen.de/cgi-bin/BASRepository/oaipmh/oai.pl^http^xml' +hial 'http://api.github.com^http^json/rate_limit_url^http^json/resources/core' +``` -Each data node is called a **cell**. It may have a **value** (a simple data type like string, number, bool or blob). + -All cells except the root cell have a **super** cell and are part of the **super** group (all the cells with the same parent, or the sub group of the super cell). A cell may have an **index** (a number) or a **label** (usually a string) to identify it in the super group. +Print all services with inaccessible images in a Docker compose file: -A cell is always an **interpretation** of some underlying data. For example a series of bytes `7b 22 61 22 3a 31 7d` can be interpreted as a byte array (a single cell with a blob value of `7b 22 61 22 3a 31 7d`) or as an utf-8 encoded string (another cell with a string value of `{"a":1}`) or as a json tree of cells (the root cell being the json object `{}` with a sub cell with label `a` and value `1`). A cell with some value can be always explicitly re-interpreted as another type of cell. +```bash +# shell +hial './config.yaml^yaml/services/*[ /image^split[":"]/[0]^http[HEAD]@status/code>=400 ]' +# 🚧 wip: split interpretation (regex[( ([^:]*): )*] +# 🚧 wip: HEAD param for http +``` -A cell also has a string **type** describing its kind, depending on the interpretation. Such types can be: "file" or "folder" (in the *fs* interpretation), "array" (in the *json* interpretation), "function_item" (in the *rust* interpretation), "response" (in the *http* interpretation), etc. +```rust +// rust (native) +for service in Cell::from("./config.yaml").all("^yaml/services") { + let image = service.to("/image"); + if image.to("^http[HEAD]@status/code") >= 400 { + println!("service {} has an invalid image: {}", + service.read().value()?, + image.read().value()? + ); + } +} +``` + +Print the structure of a rust file (struct, enum, type, functions) as a tree: -```ascii - ┌----------┐ - | Cell | - |----------| - | [index] | - | [label] | - | [value] | - | [type] | - └----------┘ - / \ - / \ - ┌-----------┐ ┌------------┐ - | Sub Group | | Attr Group | - └-----------┘ └------------┘ - / \ - / \ - Cell Cell - Cell ... - ... +```bash +hial './src/tests/rust.rs^rust/**[#type^split["_"]/[-1]=="item"]/*[name|parameters|return_type]' +# 🚧 wip: search results as tree ``` -### Examples: +##### 2. Modify data selected as above. -- A *folder* of the file system is a cell. It has a *sub* group and may have *sub* cells (files or folders which it contains); it may also have a *super* cell (parent folder). Its *attr* items are creation/modification date, access rights, size, etc. The folder name is the *label* and has no *value*. +Change the default mysql port systemwide: +```bash +# shell +hial '/etc/mysql/my.cnf^fs[rw]^ini/mysqld/port = 3307' +# 🚧 wip: rw parameter for fs interpretation +``` -- A *file* of the file system is a cell. It has no *sub* items, may have one *super*, has the same *attr* as a folder and the *label* as its name. A file cell can be *interpreted* in many other ways (string cell, json/yaml/xml cell tree, programming cell trees). +```bash +// rust +Cell::from("/etc/mysql/my.cnf") + .to("^fs[rw]^ini/mysqld/port") + .write() + .set_value(3307)?; +``` -- An entry into a json object is a cell. The parent object is its *super* group. The json key in the key/value pair is the cell *label*. If the value of this json object entry is null or bool or number, then the cell will have a corresponding value and no *sub*; if it's an array or object then the cell will have a *sub* group with the content of the array or object. +Change the user's docker configuration: +```bash +# shell +hial '~/.docker/config.json^json/auths/docker.io/username = "newuser"' +# 🚧 wip: support ~ +``` +```rust +// rust +Cell::from("~/.docker/config.json") + .to("^fs[rw]^json/auths/docker.io/username") + .write() + .set_value("newuser")?; +``` -- A method in a java project is a cell. It has a parent class (*super*), access attributes (*attr*), and arguments, return type and method body as children (*sub*). +##### 3. Copy pieces of data from one place to another. -- An http call response is a cell. It has status code and headers as *attr* and the returned body data as its value (a blob). It is usually further interpreted as either a string or json or xml etc. +Copy a string from some json object entry which is embedded in a zip file, into a rust string: -### Path language +```bash +# shell +hial 'copy( ./assets.zip^zip/data.json^json/meshes/sphere ./src/assets/sphere.rs^rust/**[:let_declaration][/pattern=sphere]/value )' +# 🚧 wip: support copy +# 🚧 wip: support zip +# 🚧 wip: support :type filter +# 🚧 wip: /**[filter] should match leaves only +``` -This unified data model naturally supports a path language similar to a file system path, xpath or json path. A cell is always used as a starting point (e.g. the file system current folder). The `/` symbol designates moving to the *sub* group; the `@` symbol to the *attr* group. Jumping to a different interpretation is done using the `^` (elevate) symbol. +Split a markdown file into sections and put each in a separate file: -As a special case, the starting point of a path is allowed to be a valid url (starting with `http://` or `https://`) or a file system path (must be either absolute, starting with `/`, or relative, starting with `.`). +```bash +# shell +`hial 'copy ./book.md^md/*[:heading1][as x] ./{label(x)}.md' +# 🚧 wip: support markdown +``` -Other special operators are: the `*` operator which selects any cell in the current group and the `**` operator which selects any cell in current group and any cell descendants in the current interpretation. Filtering these cells is done by boolean expressions in brackets. +##### 4. Transform data from one format or shape into another. -Examples: +Transform a json file into an xml file with the same format and vice versa: -- `.^file` is the current folder ("." in the `file` interpretation). It is equivalent to just `.`. -- `./src/main.rs` is the `main.rs` file in the ./src/ folder. -- `./src/main.rs@size` is the size of this file (the `size` attribute of the file). +```bash +hial 'copy file.json^json^tree^xml ./file.xml' +hial 'copy file.xml^xml^tree^json ./file.json' +# 🚧 wip: support tree implementation and conversion +``` -- `./src/main.rs^rust` represents the rust AST tree. -- `./src/main.rs^rust/*[#type=='function_item']` are all the top-level cells representing functions in the `main.rs` rust file. +##### 5. Structured diffs -- `http://api.github.com` is a url cell. -- `http://api.github.com^http` is the http response of a GET request to this url. -- `http://api.github.com^http^json` is the json tree interpretation of a http response of a GET request to this url. -- `http://api.github.com^http^json/rate_limit_url^http^json/resources/core/remaining` makes one http call and uses a field in the respose to make another http call, then select a subfield in the returning json. +Compare two files in different formats and print the resulting diff tree: -- `./src/**^rust` returns a list of all rust files (all files that have a rust interpretation) descending from the `src` folder. -- `./src/**^rust/**/*[#type=="function_item"]` lists all rust functions in all rust files in the `src` folder. -- `./src/**^rust/**/*[#type=="function_item"]/**/*[#type=="let_declaration"]` lists all occurences of *let* declarations in all functions in all rust files in the src folder. -- `./src/**^rust/**/*[#type=="function_item"]/**/*[#type=="let_declaration"][/pattern/*]` lists only destructuring patterns in all occurences of *let* declarations in all functions in all rust files in the src folder. The destructuring patterns are the only ones that have a descendant of `pattern`, for which the filter `[/pattern/*]` is true. +```bash +hial 'diff ./file.json^json^tree ./file.xml^xml^tree' +# 🚧 wip: support diff +``` -To test the examples yourself, run the `hial` command line tool, e.g.: `hial ls 'http://api.github.com^http^json'` +## Installation and usage -## What's the current feature implementation status? +To test the examples or use the library from a shell, build the project: `cargo build --release`. Then run the `hial` command, e.g.: `hial 'http://api.github.com^http^json'` -See [issues.md](./issues.md). +## The data model -## What languages are supported? +The data model is that of a tree of simple data nodes. The tree has a root node and a hierarchy of children nodes. -- Rust API is natively available; Rust is also the implementation language. -- C interop: work in progress. -- Python, Java, Go, Javascript wrappers are planned. +Each data node is called a **cell**. It may have a **value** (a simple data type like string, number, bool or blob). A cell is part of a **group** of cells. The cell may have an **index** (a number) or a **label** (usually a string) to identify it in this group. -## API examples, use cases +A cell may have subordinate cells (children in the tree structure) which are organized into a **group**. We call this the **sub** group. A cell may also have attributes or properties which also cells and are put into the **attr** group. The children cells have the first cell as their **parent**. -#### - Explore files on the file system +A cell is always an **interpretation** of some underlying data. For example a series of bytes `7b 22 61 22 3a 31 7d` can have multiple interpretations: -```bash -# shell, works -hial ls "." +1. a simple byte array which is represented by a single cell with the data as a blob value: ``` - -```rust -// rust works natively -for cell in Cell::from_path(".^file/**").all() { - // list all file names at the same level - // indenting children requires more work - println!("{}: ", cell.read().label()); -} +Cell: value = Blob([7b 22 61 22 3a 31 7d]), ``` +2. a string of utf-8 encoded characters which is represented by a single cell with the data as a string value: - +``` +Cell: value = String("{\"a\":1}"), +``` -#### - Read a list of services from a Docker compose file and print those that have inaccessible images +3. a json object which is represented by a tree of cells, the root cell being the json object `{}` with a sub cell with label `a` and value `1`: -```bash -# shell, works -echo "Bad images:" -hial ls "./examples/productiondump.json^json/stacks/*/services/*[/image^http@status/code!=200]/name" +```json +Cell: + type: "object", + sub: + Cell: + label: "a", + value: 1, + type: "number", ``` -```rust -// rust, works natively -for service in Cell::from_path("./config.yaml^yaml/services").all() { - let image = service.to("/image"); - if image.to('^http[@method=HEAD]@status/code') >= 400 { - println("service {} has an invalid image: {}", service.read().value()?, image.read().value()?); +Usually a piece of data has a humanly obvious best interpretation (e.g. json in the previous example), but the data can be always explicitly reinterpreted differently. + +A cell also has a string **type** describing its kind, depending on the interpretation. Such types can be: "file" or "folder" (in the *fs* interpretation), "array" (in the *json* interpretation), "function_item" (in the *rust* interpretation), "response" (in the *http* interpretation), etc. + +````mermaid +--- +title: Data model diagram +--- +erDiagram + Cell 1--o| "Sub Group" : "sub()" + Cell 1--o| "Attr Group" : "attr()" + Cell { + int index + value label + value value + string type } -} -``` + "Sub Group" 1--0+ "Cell" : "at(), get()" + "Attr Group" 1--0+ "Cell" : "at(), get()" +```` - +#### Examples: -#### - Change the default mysql port +- A *folder* of the file system is a cell. It has a *sub* group and may have *sub* cells (files or folders which it contains); it may also have a *parent* cell (parent folder). Its *attr* items are creation/modification date, access rights, size, etc. The folder name is the *label* and has no *value*. -```bash -# shell, wip -hial "/etc/mysql/my.cnf^ini/mysqld/port = 3307" -``` +- A *file* of the file system is a cell. It has no *sub* items, may have one *parent*, has the same *attr* as a folder and the *label* as its name. A file cell can be *interpreted* in many other ways (string cell, json/yaml/xml cell tree, programming cell trees). -```rust -// rust, wip -Cell::from_path('/etc/mysql/my.cnf^toml/mysqld/port').write().set_value(3307)?; -``` +- An entry into a json object is a cell. The json key in the key/value pair is the cell *label*. If the value of this json object entry is null or bool or number, then the cell will have a corresponding value and no *sub*; if it's an array or object then the cell will have a *sub* group with the content of the array or object. -```python -# python, wip: python interop not done -hial.to('/etc/mysql/my.cnf^toml/mysqld/port').write().set_value(3307) -``` +- A method in a java project is a cell. It has a parent class, access attributes (*attr*), and arguments, return type and method body as children (*sub*). + +- An http call response is a cell. It has status code and headers as *attr* and the returned body data as its value (a blob). It is usually further interpreted as either a string or json or xml etc. + +### Path language + +This unified data model naturally supports a path language similar to a file system path, xpath or json path. A cell is always used as a starting point (e.g. the file system current folder). The `/` symbol designates moving to the *sub* group; the `@` symbol to the *attr* group. Jumping to a different interpretation is done using the `^` (elevate) symbol. + +As a special case, the starting point of a path is allowed to be a valid url (starting with `http://` or `https://`) or a file system path (which must be either absolute, starting with `/`, or relative, starting with `.`). + +Other special operators are the `*` operator which selects any cell in the current group and the `**` operator which selects any cell in current group and any cell descendants in the current interpretation. Filtering these cells is done by boolean expressions in brackets. + +Examples: + +- `.^fs` is the current folder ("." in the file system interpretation). It is equivalent to just `.`. +- `./src/main.rs` is the `main.rs` file in the ./src/ folder. +- `./src/main.rs@size` is the size of this file (the `size` attribute of the file). + +- `./src/main.rs^rust` represents the rust AST tree. +- `./src/main.rs^rust/*[#type=='function_item']` are all the top-level cells representing functions in the `main.rs` rust file. + +- `http://api.github.com` is a url cell. +- `http://api.github.com^http` is the http response of a GET request to this url. +- `http://api.github.com^http^json` is the json tree interpretation of a http response of a GET request to this url. +- `http://api.github.com^http^json/rate_limit_url^http^json/resources/core/remaining` makes one http call and uses a field in the respose to make another http call, then selects a subfield in the returning json. + +- `./src/**^rust` returns a list of all rust files (all files that have a rust interpretation) descending from the `src` folder. +- `./src/**^rust/**[#type=="function_item"]` lists all rust functions in all rust files in the `src` folder. +- `./src/**^rust/**[#type=="function_item"]/**[#type=="let_declaration"]` lists all occurences of *let* declarations in all functions in all rust files in the src folder. +- `./src/**^rust/**[#type=="function_item"]/**[#type=="let_declaration"][/pattern/*]` lists only destructuring patterns in all occurences of *let* declarations in all functions in all rust files in the src folder. The destructuring patterns are the only ones that have a descendant of `pattern`, for which the filter `[/pattern/*]` is true. + +## What's the current project status? + +See [status.md](doc/status.md) and [issues.md](doc/issues.md). + +The implementation language is Rust, and a Rust API is natively available. + +As a command line tool hial can be used from any language that can call shell commands. + +C, Python, Java, Go, Javascript wrappers are planned. diff --git a/implementation_guidelines.md b/doc/implementation_guidelines.md similarity index 100% rename from implementation_guidelines.md rename to doc/implementation_guidelines.md diff --git a/issues.md b/doc/issues.md similarity index 54% rename from issues.md rename to doc/issues.md index 9fca4cd..bb70f7c 100644 --- a/issues.md +++ b/doc/issues.md @@ -1,18 +1,29 @@ # List of Todos and other Issues -## TODOs +- replace set_value(x) with value(x) +- add split(":") and regex interpretations +- /*[type().ends_with()] ?? -> or not, just replace it with #type^split("_")/[-1]=="xx" +- /*[name|parameters|return_type] ?? +- ^fs[rw] ?? +- should blobs/bytes be part of value? they are only useful by reinterpretation +- set value on the command line: '/username = "newuser"' +- https://raw.githubusercontent.com/rust-lang/rust/master/src/tools/rustfmt/src/lib.rs^http^rust does not work +- support zip, markdown +- support 'copy source destination' +- support ^json^tree^xml +- support diff ./file.json^json^tree ./file.xml^xml^tree +- '**[filter]' must be work as '**/*[filter]' (filter to be applied only on leaves) +- support type selector: `hial './src/tests/rust.rs^rust/*[:function_item]'` +- support rust/ts write: `hial './src/tests/rust.rs^rust/*[:function_item].label = "modified_fn_name"'` +- new structure: /api, /api/impl, /interpretations/api, /interpretations/*, /search +- add http interpretation params: method=HEAD, accept="" +- functions -! focus on releasing a first minimal version, then improve +- release first minimal version: - interpretations: path+fs, json+yaml+toml+xml, rust+js, url?+http - explicit and implicit write support (policy, include readonly) - fix tests, todo!() and TODO: in code -- support type selector: `hial './src/tests/rust.rs^rust/*[:function_item]'` -- support rust/ts write: `hial './src/tests/rust.rs^rust/*[:function_item].label = "modified_fn_name"'` -- set value on the command line -- new structure: /api, /api/impl, /interpretations/api, /interpretations/*, /search - -- - operations: - assign to variables; @@ -30,50 +41,10 @@ - ?explore python implementation and usage - ?search should return all matches embedded in a delegation cell, which has all results as subs and delegates write operations to all the subs -- ?rename XCell, Cell, CellTrait to Nex/NexIn/NexInTrait +- ?rename XCell to Xell - later: python, git, database, ical, zip, markdown -### Feature implementation status - -| *Feature* | *Readable* | *Writeble* | -|------------|------------|------------| -| url | yes | yes | -| path | yes | yes | -| fs | yes | yes | -| http | yes | yes | -| json | yes | yes | -| yaml | yes | yes | -| toml | yes | yes | -| xml | yes | yes | -| rust | yes | | -| | | | -| git | | | -| database | | | -| ical | | | -| zip | | | -| | | | -| plain text | | | -| markdown | | | -| | | | -| python | | | -| javascript | | | -| go | | | -|------------|------------|------------| - - - - -| *Feature* | *Support* | -|-----------------|-----------| -| path lang | partial | -| | | -| C interop | | -| Python interop | | - - -### Todos, Issues, Problems - - todo: c interop and a small c test - cell must implement partialeq, eq (same pointed location) - todo CLI: @@ -88,9 +59,7 @@ - todo: interpretations parameters - todo: custom tree datastructure? - todo: cell symlinks -- todo: cell path - todo: path bindings -- todo: diffs - unclear: we should have some internal language: - Usecase: json: `/question[/answer_entities/*.is_empty()].count()` @@ -100,13 +69,3 @@ './**[.name=='config.yaml'][as composefile]^yaml/services/*/image[^string^http@status/code!=200] tree 'result' -> [composefile] -> image ``` - -### Examples - -1. Extract the general structure of a rust file. Get the struct/enum/type definitions (just the name and the type) and the function definitions (just the name and the signature). Get all implementations of traits and the functions inside them, as a tree. - -``` -hial 'item = ./src/tests/rust.rs^rust/**[:struct_item|:enum_item|:type_item|:function_item]; item/' - -' -``` diff --git a/doc/status.md b/doc/status.md new file mode 100644 index 0000000..801ee27 --- /dev/null +++ b/doc/status.md @@ -0,0 +1,36 @@ +# Feature implementation status + +### Interpretations + +| *Feature* | *Readable* | *Writeble* | +|------------|------------|------------| +| url | yes | yes | +| path | yes | yes | +| fs | yes | yes | +| http | yes | yes | +| json | yes | yes | +| yaml | yes | yes | +| toml | yes | yes | +| xml | yes | yes | +| rust | yes | | +| | | | +| git | | | +| database | | | +| ical | | | +| zip | | | +| | | | +| plain text | | | +| markdown | | | +| | | | +| python | | | +| javascript | | | +| go | | | +|------------|------------|------------| + + +### Language bindings + +| *Feature* | *Support* | +|-----------------|-----------| +| C interop | | +| Python interop | | diff --git a/doc/use-cases.md b/doc/use-cases.md new file mode 100644 index 0000000..b9865e5 --- /dev/null +++ b/doc/use-cases.md @@ -0,0 +1,58 @@ +# List of potential use cases + +This list of potential use cases should drive the development of the library. + +See also the "### What can it do?" section in the README.md file. + +### Functions + +Special: path(cell path), label, value, serial +General: sort, unique, filter, map, reduce, grouping +Text: concat, regex, len, casing, substrings, split, join, replace, trim, padding, ends_with, starts_with +Aggregation: count, sum, avg, min, max +Math: round, abs +Date: parse, format, add, subtract, diff, duration + +``` +/question[count(/answer_entities/*)==0] +``` + +### Python requirements + +Update a python module version in a requirements.txt file: + +```bash +# change the version of the requests module to 1.2.3 +hial './requirements.txt^python.reqs/*[/[0]=="requests"] = "1.2.3"' + +# increment the minor version of the requests module +hial 'x = ./requirements.txt^python.requirements/*[/[0]=="requests"]; $x/[2]^version/:minor += 1' +``` + + +### Search with results structured into a tree + +Unclear: what is the accepted language? +``` +x = './**/*[.name=='config.yaml'] (as composefile)]^yaml/services/*/image[^string^http@status/code!=200] +tree 'result' / [composefile] / image +``` + +### Transform one format to another + +Transform a json file to an xml file and vice versa. + +### Structured diff between two files in different formats + +``` +hial 'diff x y' +``` + + +### - Extract the general structure of a rust file + +Get the struct/enum/type definitions (just the name and the type) and the function definitions (just the name and the signature). Get all implementations of traits and the functions inside them, as a tree. + +``` +hial 'item = ./src/tests/rust.rs^rust/**[:struct_item|:enum_item|:type_item|:function_item]; item/' +``` diff --git a/src/base/extra.rs b/src/base/extra.rs index add665a..e7b25a5 100644 --- a/src/base/extra.rs +++ b/src/base/extra.rs @@ -947,6 +947,46 @@ impl Iterator for CellIterator { } } +impl DoubleEndedIterator for CellIterator { + fn next_back(&mut self) -> Option { + match &mut self.cell_iterator { + CellIteratorKind::DynCellIterator { dyn_cell, domain } => { + dispatch_dyn_cell_iterator!(dyn_cell, |x| { + x.next_back().map(|cell_res| Cell { + dyn_cell: match cell_res { + Ok(cell) => DynCell::from(cell), + Err(err) => DynCell::from(err), + }, + domain: Rc::clone(domain), + }) + }) + } + CellIteratorKind::Elevation(cell_res) => match cell_res { + Ok(cell) => { + let cell = cell.clone(); + *cell_res = nores(); + Some(cell) + } + Err(err) => { + if err.kind == HErrKind::None { + None + } else { + Some(Cell { + dyn_cell: DynCell::from(err.clone()), + domain: Rc::new(Domain { + write_policy: cell::Cell::new(WritePolicy::ReadOnly), + origin: None, + dyn_root: OnceCell::new(), + dirty: cell::Cell::new(false), + }), + }) + } + } + }, + } + } +} + impl CellIterator { pub fn err(self) -> Res { match self.cell_iterator { diff --git a/src/base/intra.rs b/src/base/intra.rs index 0fb40f1..8c353e0 100644 --- a/src/base/intra.rs +++ b/src/base/intra.rs @@ -61,7 +61,7 @@ pub trait CellWriterTrait: Debug { pub trait GroupTrait: Clone + Debug { type Cell: CellTrait; - type CellIterator: Iterator>; + type CellIterator: DoubleEndedIterator>; fn label_type(&self) -> LabelType; fn len(&self) -> Res; diff --git a/src/interpretations/xml.rs b/src/interpretations/xml.rs index c5581e7..d4e4205 100644 --- a/src/interpretations/xml.rs +++ b/src/interpretations/xml.rs @@ -31,6 +31,7 @@ pub(crate) struct Cell { pub(crate) struct CellIterator { group: Group, next_pos: usize, + next_back_pos: usize, key: OwnValue, } @@ -603,6 +604,33 @@ impl Iterator for CellIterator { } } } +impl DoubleEndedIterator for CellIterator { + fn next_back(&mut self) -> Option { + fn inner(this: &mut CellIterator) -> Res { + loop { + if this.next_back_pos == 0 { + return nores(); + } + this.next_back_pos -= 1; + let cell = this.group.at(this.next_back_pos)?; + let reader = cell.read()?; + if Some(this.key.as_value()) == reader.label().ok() { + return Ok(cell); + } + } + } + match inner(self) { + Ok(cell) => Some(Ok(cell)), + Err(e) => { + if e.kind == HErrKind::None { + None + } else { + Some(Err(e)) + } + } + } + } +} impl GroupTrait for Group { type Cell = Cell; @@ -642,6 +670,7 @@ impl GroupTrait for Group { Ok(CellIterator { group: self.clone(), next_pos: 0, + next_back_pos: self.len()?, key: key.to_owned_value(), }) } diff --git a/src/pathlang/parse.rs b/src/pathlang/parse.rs index 5c72e52..e3cfc9c 100644 --- a/src/pathlang/parse.rs +++ b/src/pathlang/parse.rs @@ -165,10 +165,10 @@ fn path_item_selector(input: &str) -> NomRes<&str, Selector> { .map(|(next_input, res)| (next_input, Selector::from(res))) } -fn path_item_index(input: &str) -> NomRes<&str, usize> { +fn path_item_index(input: &str) -> NomRes<&str, isize> { context( "path_item_index", - delimited(tag("["), number_usize, tag("]")), + delimited(tag("["), number_isize, tag("]")), )(input) } @@ -212,8 +212,21 @@ fn path_item_start(input: &str) -> NomRes<&str, char> { } fn number_usize(input: &str) -> NomRes<&str, usize> { - context("number", recognize(many1(one_of("0123456789"))))(input).map(|(next_input, res)| { - let n = usize::from_str(res).unwrap_or_else(|_| panic!("parse error, logic error")); + context("positive number", recognize(many1(one_of("0123456789"))))(input).map( + |(next_input, res)| { + let n = usize::from_str(res).unwrap_or_else(|_| panic!("parse error, logic error")); + (next_input, n) + }, + ) +} + +fn number_isize(input: &str) -> NomRes<&str, isize> { + context( + "number", + recognize(tuple((opt(one_of("+-")), many1(one_of("0123456789"))))), + )(input) + .map(|(next_input, res)| { + let n = isize::from_str(res).unwrap_or_else(|_| panic!("parse error, logic error")); (next_input, n) }) } diff --git a/src/pathlang/path.rs b/src/pathlang/path.rs index 6915b14..6f33f2e 100644 --- a/src/pathlang/path.rs +++ b/src/pathlang/path.rs @@ -19,8 +19,8 @@ pub enum PathStart<'a> { pub struct PathItem<'a> { pub(crate) relation: Relation, pub(crate) selector: Option>, // field name (string) or '*' or '**' - pub(crate) index: Option, // or index - pub(crate) filters: Vec>, // [@size>0] or [.name.endswith('.rs')] + pub(crate) index: Option, + pub(crate) filters: Vec>, // [@size>0] or [.name.endswith('.rs')] } #[derive(Clone, Debug, PartialEq)] diff --git a/src/pathlang/search.rs b/src/pathlang/search.rs index 3199d6c..980b7b4 100644 --- a/src/pathlang/search.rs +++ b/src/pathlang/search.rs @@ -206,7 +206,16 @@ impl<'s> Searcher<'s> { } (None | Some(Selector::Star) | Some(Selector::DoubleStar), Some(index)) => { ifdebug!(println!("get child by index")); - let cell = guard_ok!(group.at(index).err(), err => { + let at_index = if index < 0 { + let len = group.len().unwrap_or_else(|e| { + warning!("Error while searching: cannot get group length: {:?}", e); + 0 + }); + (len as isize + index) as usize + } else { + index as usize + }; + let cell = guard_ok!(group.at(at_index).err(), err => { if err.kind != HErrKind::None { warning!("Error while searching: cannot get cell: {:?}", err); } @@ -230,7 +239,12 @@ impl<'s> Searcher<'s> { return ; }); if let Some(index) = opt_index { - if let Some(cell) = iter.nth(index) { + let opt_cell = if index < 0 { + iter.rev().nth((-index - 1) as usize) + } else { + iter.nth(index as usize) + }; + if let Some(cell) = opt_cell { Self::process_cell(stack, path, cell, path_index, true, next_max_path_index) } } else { diff --git a/src/tests/rust.rs b/src/tests/rust.rs index 1454250..c17b1f6 100644 --- a/src/tests/rust.rs +++ b/src/tests/rust.rs @@ -42,21 +42,22 @@ fn rust_write_and_save() -> Res<()> { assert_eq!(root.to("/[7]/[1]").read().value()?, "editable_rust_fn"); - root.to("/[7]/[1]") - .write() - .set_value("modified_rust_fn".into())?; - assert_eq!(root.to("/[7]/[1]").read().value()?, "modified_rust_fn"); + // TODO: writable rust + // root.to("/[7]/[1]") + // .write() + // .set_value("modified_rust_fn".into())?; + // assert_eq!(root.to("/[7]/[1]").read().value()?, "modified_rust_fn"); - root.save(&root.origin())?; - assert_eq!(file.to("^rust/[7]/[1]").read().value()?, "modified_rust_fn",); + // root.save(&root.origin())?; + // assert_eq!(file.to("^rust/[7]/[1]").read().value()?, "modified_rust_fn",); - root.to("/[7]/[1]") - .write() - .set_value("editable_rust_fn".into())?; - assert_eq!(root.to("/[7]/[1]").read().value()?, "editable_rust_fn"); + // root.to("/[7]/[1]") + // .write() + // .set_value("editable_rust_fn".into())?; + // assert_eq!(root.to("/[7]/[1]").read().value()?, "editable_rust_fn"); - root.save(&file.clone())?; - assert_eq!(file.to("^rust/[7]/[1]").read().value()?, "editable_rust_fn",); + // root.save(&file.clone())?; + // assert_eq!(file.to("^rust/[7]/[1]").read().value()?, "editable_rust_fn",); Ok(()) } diff --git a/src/tests/search.rs b/src/tests/search.rs index 88219ce..1a24efe 100644 --- a/src/tests/search.rs +++ b/src/tests/search.rs @@ -17,6 +17,17 @@ fn path_simple_item() -> Res<()> { filters: vec![], },] ); + + let path = Path::parse("/a[-2]")?; + assert_eq!( + path.0.as_slice(), + &[PathItem { + relation: Relation::Sub, + selector: Some(Selector::Str("a")), + index: Some(-2), + filters: vec![], + },] + ); Ok(()) } @@ -142,6 +153,12 @@ fn search_simple_search_with_index() -> Res<()> { let eval = str_eval(root.clone(), "/test/a[2]/*")?; assert_eq!(eval, ["z:3"]); + let eval = str_eval(root.clone(), "/test/a[-1]/*")?; + assert_eq!(eval, ["z:3"]); + + let eval = str_eval(root.clone(), "/test/a[-2]/*")?; + assert_eq!(eval, ["y:2"]); + Ok(()) }