feat: Schema list API #1625

nasdf · 2023-07-11T20:01:34Z

Relevant issue(s)

Resolves #1624

Description

This PR adds an API for listing schemas.

HTTP

curl http://localhost:9181/api/v0/schema/list

Output

{
  "data": {
    "collections": [
      {
        "name": "User",
        "id": "bafkreibpnvkvjqvg4skzlijka5xe63zeu74ivcjwd76q7yi65jdhwqhske",
        "fields": [
          {
            "id": "1",
            "name": "age",
            "kind": "Int"
          },
          {
            "id": "2",
            "name": "name",
            "kind": "String"
          },
          {
            "id": "3",
            "name": "points",
            "kind": "Float"
          },
          {
            "id": "4",
            "name": "verified",
            "kind": "Boolean"
          }
        ]
      }
    ],
    "result": "success"
  }
}

CLI

defradb client schema list

Output

type User {
    name: String 
    age: Int 
    verified: Boolean 
    points: Float
}

Tasks

I made sure the code is well commented, particularly hard-to-understand areas.
I made sure the repository-held documentation is changed accordingly.
I made sure the pull request title adheres to the conventional commit style (the subset used in the project can be found in tools/configs/chglog/config.yml).
I made sure to discuss its limitations such as threats to validity, vulnerability to mistake and misuse, robustness to invalidation of assumptions, resource requirements, ...

How has this been tested?

Manual testing

Specify the platform(s) on which this was tested:

MacOS

…emas

codecov · 2023-07-11T20:11:24Z

Codecov Report

Patch coverage: 53.85% and project coverage change: -0.21 ⚠️

Comparison is base (6978664) 76.15% compared to head (57c29ef) 75.94%.

@@             Coverage Diff             @@
##           develop    #1625      +/-   ##
===========================================
- Coverage    76.15%   75.94%   -0.21%     
===========================================
  Files          192      193       +1     
  Lines        19852    19964     +112     
===========================================
+ Hits         15118    15161      +43     
- Misses        3707     3768      +61     
- Partials      1027     1035       +8

Flag	Coverage Δ
all-tests	`75.94% <53.85%> (-0.21%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
cli/schema_list.go	`14.71% <14.71%> (ø)`
client/descriptions.go	`72.97% <43.75%> (-22.27%)`	⬇️
cli/schema_patch.go	`31.90% <50.00%> (-0.25%)`	⬇️
api/http/handlerfuncs.go	`84.07% <90.00%> (+0.93%)`	⬆️
api/http/router.go	`100.00% <100.00%> (ø)`
cli/cli.go	`72.82% <100.00%> (+0.27%)`	⬆️
cli/schema_add.go	`30.51% <100.00%> (ø)`

... and 7 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6978664...57c29ef. Read the comment docs.

fredcarle

This is an nice functionality. I like the improvements from what you had for it on the playground PR :)

I do have a few things I'd like to address as you'll see bellow. Nothing major though and once they're resolved we should be good to merge.

fredcarle · 2023-07-11T21:04:28Z

api/http/handlerfuncs.go

+	Fields []fieldResponse `json:"fields,omitempty"`
+}
+
+func listSchemaHandler(rw http.ResponseWriter, req *http.Request) {


todo: I would be really nice to have a unit test for this handler to see what the expected output is and that we catch future changes to the schema structure. Ideally the error paths would also be tested. You can base yourself off of the other unit test for this.

Good call. I added tests for the error path as well as the happy path.

fredcarle · 2023-07-11T21:06:23Z

api/http/handlerfuncs.go

+	sendJSON(
+		req.Context(),
+		rw,
+		simpleDataResponse("result", "success", "collections", colResp),


suggestion: You can probably remove the result and success for the response as those are superfluous in this case.

fredcarle · 2023-07-11T21:10:08Z

api/http/router.go

@@ -30,6 +30,7 @@ const (
 	DumpPath        string = versionedAPIPath + "/debug/dump"
 	BlocksPath      string = versionedAPIPath + "/blocks"
 	GraphQLPath     string = versionedAPIPath + "/graphql"
+	SchemaListPath  string = versionedAPIPath + "/schema/list"


suggestion: We started changing out paths to be a bit more RESTful with the introduction of the index handlers. If you don't mind, please convert the 3 schema paths to a single SchemaPath string = versionedAPIPath + "/schema" and use the relevant http verbs in the setRoutes method bellow (Get, Post, Patch).

I simplified the paths like you mentioned. Do we need to make an announcement when changing the API?

At this stage I don't think we do as most users probably use the cli for this but the python sdk might need to be updated.

fredcarle · 2023-07-11T21:11:50Z

cli/schema_list.go

+func MakeSchemaListCommand(cfg *config.Config) *cobra.Command {
+	var cmd = &cobra.Command{
+		Use:   "list",
+		Short: "List schema types from DefraDB",


suggestion: List schema types with their respective fields. No a strong opinion but I though it would be a bit more descriptive of what it does.

fredcarle

Good job Keenan :) Thanks for the changes.

islamaliev · 2023-07-19T12:38:43Z

bug bash result:
ran locally defradb nodes with different rootdir with different data. Ran list CLI command for them and observed the list of collections. Also added a new collection to make sure the result is not cached.

## Relevant issue(s) Resolves sourcenetwork#1624 ## Description This PR adds an API for listing schemas. The schema API routes have also been simplified to be more RESTful. ### HTTP `curl http://localhost:9181/api/v0/schema` #### Output ``` { "data": { "collections": [ { "name": "User", "id": "bafkreibpnvkvjqvg4skzlijka5xe63zeu74ivcjwd76q7yi65jdhwqhske", "fields": [ { "id": "1", "name": "age", "kind": "Int" }, { "id": "2", "name": "name", "kind": "String" }, { "id": "3", "name": "points", "kind": "Float" }, { "id": "4", "name": "verified", "kind": "Boolean" } ] } ], "result": "success" } } ``` ### CLI `defradb client schema list` #### Output ``` type User { name: String age: Int verified: Boolean points: Float } ``` ## Tasks - [x] I made sure the code is well commented, particularly hard-to-understand areas. - [x] I made sure the repository-held documentation is changed accordingly. - [x] I made sure the pull request title adheres to the conventional commit style (the subset used in the project can be found in [tools/configs/chglog/config.yml](tools/configs/chglog/config.yml)). - [x] I made sure to discuss its limitations such as threats to validity, vulnerability to mistake and misuse, robustness to invalidation of assumptions, resource requirements, ... ## How has this been tested? Automated test + manual testing Specify the platform(s) on which this was tested: - MacOS

nasdf added 3 commits July 10, 2023 15:39

add schema list api route. add String method to FieldKind type.

f95747b

add schema list cli command. ignore generated fields when listing sch…

69b9271

…emas

Merge branch 'develop' into nasdf/feat/schema-list

cc23fdc

fredcarle assigned nasdf Jul 11, 2023

fredcarle requested a review from a team July 11, 2023 20:50

fredcarle added feature New feature or request area/api Related to the external API component labels Jul 11, 2023

fredcarle requested changes Jul 11, 2023

View reviewed changes

nasdf added 2 commits July 11, 2023 15:39

simplify schema api routes

fc9e9e1

add list schema http api test.

f48888b

nasdf requested a review from fredcarle July 12, 2023 16:01

fredcarle approved these changes Jul 12, 2023

View reviewed changes

Merge branch 'develop' into nasdf/feat/schema-list

57c29ef

nasdf merged commit ac2fc49 into sourcenetwork:develop Jul 12, 2023

nasdf deleted the nasdf/feat/schema-list branch July 12, 2023 17:19

fredcarle added this to the DefraDB v0.6 milestone Jul 17, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Schema list API #1625

feat: Schema list API #1625

nasdf commented Jul 11, 2023

codecov bot commented Jul 11, 2023 •

edited

Loading

fredcarle left a comment

fredcarle Jul 11, 2023

nasdf Jul 11, 2023

fredcarle Jul 11, 2023

fredcarle Jul 11, 2023

nasdf Jul 11, 2023

fredcarle Jul 12, 2023

fredcarle Jul 11, 2023

fredcarle left a comment

islamaliev commented Jul 19, 2023

feat: Schema list API #1625

feat: Schema list API #1625

Conversation

nasdf commented Jul 11, 2023

Relevant issue(s)

Description

HTTP

Output

CLI

Output

Tasks

How has this been tested?

codecov bot commented Jul 11, 2023 • edited Loading

Codecov Report

fredcarle left a comment

Choose a reason for hiding this comment

fredcarle Jul 11, 2023

Choose a reason for hiding this comment

nasdf Jul 11, 2023

Choose a reason for hiding this comment

fredcarle Jul 11, 2023

Choose a reason for hiding this comment

fredcarle Jul 11, 2023

Choose a reason for hiding this comment

nasdf Jul 11, 2023

Choose a reason for hiding this comment

fredcarle Jul 12, 2023

Choose a reason for hiding this comment

fredcarle Jul 11, 2023

Choose a reason for hiding this comment

fredcarle left a comment

Choose a reason for hiding this comment

islamaliev commented Jul 19, 2023

codecov bot commented Jul 11, 2023 •

edited

Loading