Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add local keymanager API #151

Closed
wants to merge 29 commits into from
Closed

Add local keymanager API #151

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
730a0d5
Add validator manager API
dapplion Jun 4, 2021
d433e14
Fix issues
dapplion Jun 5, 2021
0ff09a3
Simplify routes
dapplion Oct 4, 2021
51861f8
Generalize language
dapplion Oct 4, 2021
2fd107b
Delete accounts routes
dapplion Oct 4, 2021
37490ab
Remove validator-oapi.yaml
dapplion Oct 4, 2021
db508c2
Merge branch 'master' into validator-manager-api
dapplion Oct 4, 2021
817a24e
Rename accounts for keystores
dapplion Oct 5, 2021
b920615
Remove slashing_protection_last_entry property
dapplion Oct 5, 2021
44c772a
Merge branch 'validator-manager-api' of github.com:dapplion/eth2.0-AP…
dapplion Oct 5, 2021
c0feb5f
Address PR comments
dapplion Oct 13, 2021
6f56ae9
Update key-manager-oapi.yaml
dapplion Oct 15, 2021
bd3ffe7
Update key-manager-oapi.yaml
dapplion Oct 27, 2021
5e8dded
Update key-manager-oapi.yaml
dapplion Oct 27, 2021
82ba2b9
Update key-manager-oapi.yaml
dapplion Oct 27, 2021
5d27a29
Merge branch 'master' into validator-manager-api
dapplion Oct 27, 2021
e1f5232
Clarify formats
dapplion Oct 27, 2021
3168f46
Remove keystore notation for keys
dapplion Oct 27, 2021
ec9f7d4
Allow to import with an array of passwords
dapplion Oct 27, 2021
43581ac
Fix indentation
dapplion Nov 2, 2021
aa3ff0a
Add not_deletable property
dapplion Nov 2, 2021
84465af
Clarify DeleteKeysResponse
dapplion Nov 4, 2021
c6e0daf
Update not_deletable prop name
dapplion Nov 4, 2021
ef02cd2
Wrap response in data prop
dapplion Nov 4, 2021
08f627f
Add type array to password requests
dapplion Nov 4, 2021
be49b64
Update ImportKeystores description
dapplion Nov 4, 2021
3b14523
Remove request. prefix in ImportKeystores description
dapplion Nov 4, 2021
32f7d68
Update DeleteKeysResponse enum values
dapplion Nov 5, 2021
c048b4c
Merge branch 'master' into validator-manager-api
dapplion Nov 10, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions apis/accounts/auth/change_password.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
post:
tags:
- AccountAuth
operationId: "ChangePassword"
summary: "Change password."
description: |
The validator client MUST re-encrypt the local representation of the keystores if applicable.
dapplion marked this conversation as resolved.
Show resolved Hide resolved
requestBody:
content:
application/json:
schema:
type: object
properties:
current_password:
type: string
password:
type: string
password_confirmation:
type: string
responses:
"200":
description: Successfully signed up
"500":
$ref: "../../../validator-oapi.yaml#/components/responses/InternalError"
55 changes: 55 additions & 0 deletions apis/accounts/auth/login.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
get:
tags:
- AccountAuth
operationId: "LoginStatus"
summary: "Check login status."
description: |
Convenience method to check if user has signed up, and their token is valid.
security:
- cookieAuth: []
responses:
"200":
content:
application/json:
schema:
title: LoginStatusResponse
type: object
properties:
has_signed_up:
type: boolean
"500":
$ref: "../../../validator-oapi.yaml#/components/responses/InternalError"

post:
tags:
- AccountAuth
operationId: "Login"
summary: "Login"
description: |
MUST validate the provided passwords against a local “Record”. After successful validation MUST return a Set-Cookie header
requestBody:
content:
application/json:
schema:
type: object
properties:
password:
type: string
password_confirmation:
type: string
responses:
"200":
description: |
Successfully authenticated. The session ID is returned in a cookie named `SESSION`. You need to include this cookie in subsequent requests.
- HttpOnly: MUST be included
Copy link

@cheatfate cheatfate Jul 1, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to avoid cookies? Because as soon as you introduce cookies for authentication server become stateful. Also because Max-Age is not specified its possible to create cookies which will never be deleted, and so can be used if cookie become stolen.
What if you use token authentication instead?

Copy link
Collaborator Author

@dapplion dapplion Jul 2, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you describe in more detail what you refer to with "token authentication"?

Given that each node will likely implement different flavours of authentication, using HttpOnly cookies would simplify the UI implementation. The browser will just re-send whatever that specific node has sent in the Set-Cookie header.

  • You can make the cookie stateless by signing it with a known key and adding data to it. Cookies fit 4096 bytes, so it should be enough for whatever session data a validator UI may need to know.
  • You can also add a timestamp to the cookie's content and use that to invalidate it in the backend, irrespective of the Max-Age directive

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can check for Json Web Token about token authentication which could be passed via headers or request string. The biggest difference is that tokens will not be managed by browsers.
This feature protects validators from loosing cookies through malicious browser extensions or malware.
As soon as communication finishes, tokens become not valid anymore.

Copy link
Contributor

@paulhauner paulhauner Jul 6, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lighthouse has an auth scheme in its existing VC API:

Here's an example of a curl request using this scheme:

curl -v localhost:5062/lighthouse/version -H "Authorization: Basic api-token-0x03eace4c98e8f77477bb99efb74f9af10d800bd3318f92c33b719a4644254d4123"

This approach is simpler for the VC, but it doesn't allow for user specified passwords. Our design ethos is to keep the VC HTTP server as simple as possible and to handle all the user-facing authentication in the GUI. Here's a design doc we drew up a while ago: https://hackmd.io/zS3h-Kk_TTmYhvyoHap6DQ

Basically, we expect the VC and BN to be hiding behind a reverse-proxy provided by the GUI web app. This avoids needing to provide HTTPS certificates for the BN, VC and web application.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@paulhauner This Signature Header is very cool idea! Is this like a roll-your-own TLS strategy to guarantee integrity of the responses? If so, could we take it a step further and make the client encrypt the requests with the pubkey so that the keystores + password are not sent in plain text? https://lighthouse-book.sigmaprime.io/api-vc-sig-header.html

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The beacon client should not provide extensive user authentication & authorization. A simple API key mechanism is enough for putting a UI-based user authenticatoin & authorization on top. In the case of implementing authentication & authorization on the beacon client, where do you draw the line with features? What about 2FA? Different storage techniques for sensible data? Etc.

Handling a JWT (Json Web Token) only makes sense when complex user management is needed.

When HTTPS is needed because access is provided on an unsecure network, a simple SSL termination service (e. g. nginx, haproxy, ...) would suffice and is more secure than handling SSL on the beacon client itself.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's a new tilt at an authentication scheme:

  1. Add an endpoint /eth/v1/keystores/auth that is not protected by any form of authentication. It returns a response with an absolute path to a file containing an auth token. The assumption here is that the user of the VC API is a privileged entity with access to the machine the VC is running on, and permission to read this file.
{
    "token_path": "/home/michael/.lighthouse/mainnet/validators/api-token.txt"
}
  1. All other endpoints require an Authorization: Bearer $token header with the value of the token loaded from token_path. The intention is that the token remains the same across multiple VC restarts, unless the user specifically regenerates a new token (this can happen out of band, e.g. by deleting the file and restarting the VC, or using a non-standard API).

Open questions:

On the JWT issue I tend to agree with @stefa2k that it's overkill for the VC. The main thing that JWT adds is the ability for the client to verify that the token was issued by the VC's public key... which does not seem useful. We could take a JWT-agnostic approach by permitting the VC to return extra metadata in the /auth response if it wishes (such as its JWT-signing public key). JWT-aware clients could use this to verify if they wanted to, while JWT-oblivious implementations could just use the JWT as an arbitrary token without parsing or verifying it.

I also think not requiring signed responses is fine.

Addendum:

The path that the VC stores the token in, and its permissions could be configurable via the CLI. This would enable things like running the VC and the UI as different users if desired, but I think we can get a long way even before those config options are added, and they needn't be standardised (the CLIs are already non-standard).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm so on the Prysm side we are switching to something like this already though there's no API to expose the path...

this is what happens ( feature in develop prysm/ master prysm-web-ui)

you can generate the jwt either by running ./validator web generate-auth-token which will write to a file called auth-token in the prysm wallet directory and return a custom URL in the CLI (i.e. localhost:7500/initialize?token=) they can also change the path using --wallet-dir though it's for the whole wallet.

they should get the same thing when running ./validator --web in the console as well as that same file location but there is a cli command to regenerate

the frontend stores jwt this in local storage and injects it throughout the front end if it's designed with an initialize page 🤔

wondering if we can do something similar in this case

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michaelsproul So providing the path through an API route would be for the front-end to tell the user to "Go to $path to grab your authenticating token" ? Maybe that's a very specific implementation of auth, not sure if it should be part of the standard. Maybe as a complimentary note or suggestion for same fs setups auth?

Copy link
Collaborator

@michaelsproul michaelsproul Oct 28, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@james-prysm Perfect! I think if Prysm can accept the JWT in an Authorization: Bearer <JWT> header on requests to the key manager API then that would be compatible. Not all programs (other clients, tools, etc) interacting with the JWT need to understand that it's a JWT, they can just treat it as an opaque API token if they wish, i.e. we are JWT-agnostic 😅

@dapplion I was thinking that the backend for the GUI could fetch the token automatically if it's running on the same machine, although you're right that it wouldn't work with the GUI and VC running on different hosts. It would be nice if we could provide a "no touch" set up that's client agnostic for the API token for the majority of use cases, and this was the only way I could think of achieving that. The alternative seems to be to leave the step where the user obtains the token client-specific, in which case the GUI has to hard-code a bunch of token-access heuristics, or up to the user, which is worse for UX (although should exist as a fallback option in case the fetch-from-fs fails).

- SameSite: MUST be set to Strict
- Max-Age: MAY be included
- Secure: SHOULD not be included. Validator client servers may not have TLS/SSL certificates
- Domain: MAY be included
headers:
Set-Cookie:
schema:
type: string
example: SESSION=abcde12345; Path=/; HttpOnly; SameSite=Strict
"500":
$ref: "../../../validator-oapi.yaml#/components/responses/InternalError"
12 changes: 12 additions & 0 deletions apis/accounts/auth/logout.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
post:
tags:
- AccountAuth
operationId: "Logout"
summary: "Logout."
description: |
MUST invalidate the token set by the Login route if found in the request headers.
responses:
"200":
description: Successfully logged out.
"500":
$ref: "../../../validator-oapi.yaml#/components/responses/InternalError"
28 changes: 28 additions & 0 deletions apis/accounts/auth/signup.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
post:
tags:
- AccountAuth
operationId: "Signup"
summary: "Signup."
description: |
Register a new password to be used to encrypt the local wallet. The password MUST:
dapplion marked this conversation as resolved.
Show resolved Hide resolved

Be stored locally hashed with salt to authenticate future logins.
Be used to encrypt and decrypt the local representation of the imported keystores
The validator binary MUST expose a CLI argument --persist-password to enable persisting
the password such that it can be restarted and perform validator duties without requiring
interaction by the user.
requestBody:
content:
application/json:
schema:
type: object
properties:
password:
type: string
password_confirmation:
type: string
responses:
"200":
description: Successfully signed up
"500":
$ref: "../../../validator-oapi.yaml#/components/responses/InternalError"
148 changes: 148 additions & 0 deletions apis/accounts/index.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
get:
tags:
- Accounts
operationId: "ListAccounts"
summary: "List accounts."
description: |
List all accounts known to and decrypted to this validator client
security:
- cookieAuth: []
parameters:
- name: page_size
in: query
required: false
description: "The maximum number of accounts to return in the response."
schema:
type: number
- name: page_token
in: query
required: false
description: "A pagination token returned from a previous call to `ListAccounts`. that indicates where this listing should continue from."
dapplion marked this conversation as resolved.
Show resolved Hide resolved
schema:
type: string
responses:
"200":
description: Success response
content:
application/json:
schema:
title: ListAccountsResponse
type: object
properties:
next_page_token:
type: string
description: "A pagination token returned from a previous call that indicates from where listing should continue."
example: "150"
total_size:
type: number
description: "Total count matching the request."
accounts:
type: array
items:
type: object
properties:
validating_pubkey:
type: string
description: "The validating public key."
derivation_path:
type: string
description: "The derivation path (if present in the imported keystore)."
slashing_protection_last_entry:
type: string
description: |
Latest slashing protection DB data item associated with this pubkey.
Signal no slashing protection DB data with a value of TBD (0, null, "")

"401":
$ref: "../../validator-oapi.yaml#/components/responses/Unauthorized"
"403":
$ref: "../../validator-oapi.yaml#/components/responses/Forbidden"
"500":
$ref: "../../validator-oapi.yaml#/components/responses/InternalError"

post:
tags:
- Accounts
operationId: "ImportAccounts"
summary: "Import accounts."
description: |
Import keystores generated by the Eth2.0 deposit CLI tooling. All keystores MUST be encrypted with the same password.
security:
dapplion marked this conversation as resolved.
Show resolved Hide resolved
- cookieAuth: []
requestBody:
content:
application/json:
schema:
type: object
properties:
keystores_imported:
type: array
description: "JSON-encoded keystore files to import during wallet creation."
items:
type: object
keystores_password:
description: "Password to unlock imported keystore files."
type: string
responses:
"200":
description: Success response
content:
application/json:
schema:
title: ImportAccountsResponse
type: object
properties:
imported_pubkeys:
type: array
description: "List of successfully imported pubkeys."
items:
type: string
"401":
$ref: "../../validator-oapi.yaml#/components/responses/Unauthorized"
"403":
$ref: "../../validator-oapi.yaml#/components/responses/Forbidden"
"500":
$ref: "../../validator-oapi.yaml#/components/responses/InternalError"

delete:
tags:
- Accounts
operationId: "DeleteAccounts"
summary: "Delete accounts."
description: |
Delete keystores for the pubkeys provided in the request. The validator client MUST stop validating
with these keystores immediately and MUST return a 200 status only after confirming no more signatures
can be created with those keys.
security:
dapplion marked this conversation as resolved.
Show resolved Hide resolved
- cookieAuth: []
requestBody:
content:
application/json:
schema:
type: object
properties:
pubkeys:
type: array
description: "List of public keys to delete."
items:
type: string
responses:
"200":
description: Success response
content:
application/json:
schema:
title: DeleteAccountsResponse
type: object
properties:
deleted_pubkeys:
type: array
description: "List of public keys successfully deleted."
items:
type: string
"401":
$ref: "../../validator-oapi.yaml#/components/responses/Unauthorized"
"403":
$ref: "../../validator-oapi.yaml#/components/responses/Forbidden"
"500":
$ref: "../../validator-oapi.yaml#/components/responses/InternalError"
70 changes: 70 additions & 0 deletions apis/accounts/slashing_protection.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
get:
tags:
- SlashingProtection
operationId: "ExportSlashingProtection"
summary: "Export slashing protection data."
description: |
MUST return a file with the format defined in EIP-3076: Slashing Protection Interchange Format.
If pubkeys is not specified MUST return data for all pubkeys in the local slashing protection DB.
dapplion marked this conversation as resolved.
Show resolved Hide resolved
security:
- cookieAuth: []
parameters:
- name: id
description: "Hex encoded public key (with 0x prefix) array"
in: query
required: false
schema:
type: array
maxItems: 30
uniqueItems: true
items:
description: "Hex encoded public key (with 0x prefix)"
type: string
responses:
"200":
description: Success response
content:
application/json:
schema:
title: ExportSlashingProtectionResponse
type: object
properties:
file:
type: string
description: "JSON representation of the slash protection"
"401":
$ref: "../../validator-oapi.yaml#/components/responses/Unauthorized"
"403":
$ref: "../../validator-oapi.yaml#/components/responses/Forbidden"
"500":
$ref: "../../validator-oapi.yaml#/components/responses/InternalError"

post:
tags:
- SlashingProtection
operationId: "ImportSlashingProtection"
summary: "Import slashing protection data."
description: |
MUST accept a file with the format defined in EIP-3076: Slashing Protection Interchange Format.
Users are encouraged to use the slashing_protection_last_entry field to confirm that they successfully
added slashing protection data for their accounts.
security:
- cookieAuth: []
requestBody:
content:
application/json:
schema:
type: object
properties:
slashing_protection_json:
type: string
description: "JSON representation of the slash protection."
responses:
"200":
description: Success response
"401":
$ref: "../../validator-oapi.yaml#/components/responses/Unauthorized"
"403":
$ref: "../../validator-oapi.yaml#/components/responses/Forbidden"
"500":
$ref: "../../validator-oapi.yaml#/components/responses/InternalError"
35 changes: 35 additions & 0 deletions types/http.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,3 +124,38 @@ IndexedErrorMessage:
type: string
example: "invalid signature"

Unauthorized:
description: "Unauthorized, no token is found"
content:
application/json:
schema:
type: object
properties:
code:
description: "Either specific error code in case of invalid request or http status code"
type: number
example: 401
message:
description: "Message describing error"
type: string
example:
code: 401
message: "Token not found"

Forbidden:
description: "Forbidden, a token is found but is invalid"
content:
application/json:
schema:
type: object
properties:
code:
description: "Either specific error code in case of invalid request or http status code"
type: number
example: 403
message:
description: "Message describing error"
type: string
example:
code: 403
message: "Token invalid"
Loading