Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix go namespace slashes #886

Merged
merged 8 commits into from
Nov 12, 2021
Merged

Fix go namespace slashes #886

merged 8 commits into from
Nov 12, 2021

Conversation

nellshamrell
Copy link
Contributor

@nellshamrell nellshamrell commented Nov 11, 2021

Fix go namespace slashes

Background

This is a fix for #884

When go support was deployed to staging and production, I discovered that when someone requests a definition with url encoded slashes, such as:

definitions/go/golang/github.com%2fgorilla/mux/v1.7.3

The load balancer decodes the slashes before the request is passed to the service, which means the coordinates requested are:

definitions/go/golang/github.com/gorilla/mux/v1.7.3

Which results in a 404 not found, as there are more parameters in the url than the route expect.

Implementation

This fix is a bit brute force, but it does work. It allows for up to three additional parameters in the URL and, for go components only, strings the relevant arguments together, then parses the revision, name, and namespace out of that string.

Signed-off-by: Nell Shamrell <[email protected]>
@nellshamrell nellshamrell marked this pull request as ready for review November 11, 2021 22:50
@nellshamrell nellshamrell changed the title [WIP] Fix go namespace slashes Fix go namespace slashes Nov 11, 2021
// Painful way of handling go namespaces with multiple slashes
// Unfortunately, it seems the best option without doing a massive
// rearchitecture of the entire coordinate system
if (request.params.type == "go" && request.params.provider == "golang") {
if (request.params.type == 'go' && request.params.provider == 'golang') {
Copy link
Contributor

@MichaelTsengLZ MichaelTsengLZ Nov 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly some lint rule is not set up very well.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Michael, do you mean that these should be triple equals to be the best JavaScript it can be? ===
(Agree if so)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. the == should be ===. What I mean is the double quote can be caught by lint rules. I think == could also be caught in lint rules.

@MichaelTsengLZ
Copy link
Contributor

MichaelTsengLZ commented Nov 12, 2021

Feels a little brute force and we only support the go component with up to three slashes. Could we replace slash in the namespace with another char, like _ or whitespace, and then call encodeURIComponent()?

@jeffwilcox for more inputs.

@nellshamrell
Copy link
Contributor Author

nellshamrell commented Nov 12, 2021

@MichaelTsengLZ it is, indeed, brute force. I tried a few experiments with replacing the slashes with other characters. The problem is that there any character that can be used in a URL can be used in a go namespace (including underscores, hyphens, periods, etc.).

We could consider something like whitespace, but I feel that would make the API more difficult to use (it's not intuitive to include whitespace in an API call). I do think we would run into trouble when the load balancer decoded the encoding before passing the url to the service.

I feel this approach introduces the fewest compromises and we can add more "extra" arguments to support more slashes in go namespaces.

@nellshamrell
Copy link
Contributor Author

Much of the discussion around what to do about slashes in go namespaces can be found in these three pull requests:

#862
#864
#865

Copy link
Contributor

@jeffwilcox jeffwilcox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! One minor note.

// Painful way of handling go namespaces with multiple slashes
// Unfortunately, it seems the best option without doing a massive
// rearchitecture of the entire coordinate system
if (request.params.type == "go" && request.params.provider == "golang") {
if (request.params.type == 'go' && request.params.provider == 'golang') {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Michael, do you mean that these should be triple equals to be the best JavaScript it can be? ===
(Agree if so)

@jeffwilcox
Copy link
Contributor

As to the brute force and other aspect... sorry, I don't have enough background on ClearlyDefined to help here, but I trust Nell and the design she has in mind for the implementation.

@MichaelTsengLZ
Copy link
Contributor

MichaelTsengLZ commented Nov 12, 2021

@MichaelTsengLZ it is, indeed, brute force. I tried a few experiments with replacing the slashes with other characters. The problem is that there any character that can be used in a URL can be used in a go namespace (including underscores, hyphens, periods, etc.).

We could consider something like whitespace, but I feel that would make the API more difficult to use (it's not intuitive to include whitespace in an API call). I do think we would run into trouble when the load balancer decoded the encoding before passing the url to the service.

I feel this approach introduces the fewest compromises and we can add more "extra" arguments to support more slashes in go namespaces.

Whitespace will be encoded as %20 in the URL while slash will be encoded as %2F. Therefore, I don't think it will complicate the URL that much. But I agree this is also very hacky and not sure whether it will cause any issue when nginx parse it to really whitespace in URL and then pass it to node app.

Signed-off-by: Nell Shamrell <[email protected]>
@nellshamrell
Copy link
Contributor Author

@MichaelTsengLZ I tested how the API behaves when it receives an API call with whitespace in it, it results in it not being able to find the route (similar to what was previously happening when the slashes were decoded). We would need to add in special handling for the whitespaces similar to what we are doing with slashes here:

(The 404 not found error occurs when the express app cannot find a route that corresponds to the request)

curl http://localhost:4000/definitions/go/golang/github.com containerd/containerd/v1.5.4
{"error":{"code":"404","message":"An error has occurred","innererror":{"status":404,"name":"Error","message":"Not Found","stack":"Error: Not Found\n    at requestHandler (/opt/service/app.js:159:17)\n    at Layer.handle [as handle_request] (/opt/service/node_modules/express/lib/router/layer.js:95:5)\n    at trim_prefix (/opt/service/node_modules/express/lib/router/index.js:317:13)\n    at /opt/service/node_modules/express/lib/router/index.js:284:7\n    at Function.process_params (/opt/service/node_modules/express/lib/router/index.js:335:12)\n    at next (/opt/service/node_modules/express/lib/router/index.js:275:10)\n    at /opt/service/node_modules/express/lib/router/index.js:635:15\n    at next (/opt/service/node_modules/express/lib/router/index.js:260:14)\n    at Function.handle (/opt/service/node_modules/express/lib/router/index.js:174:3)\n    at router (/opt/service/node_modules/express/lib/router/index.js:47:12)"}}}curl: (6) Could not resolve host: containerd

@nellshamrell
Copy link
Contributor Author

I have corrected the "==" to "===". That does, indeed, make it the best Javascript it can be :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants