Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add keyfile support for RLN credentials secure storage #1285

Merged
merged 28 commits into from
Oct 28, 2022
Merged

Conversation

s1fr0
Copy link
Contributor

@s1fr0 s1fr0 commented Oct 20, 2022

This PR adds support to keyfile defined according to Web3 Secret Storage Definition but extended in order to allow

  • secure storage of arbitrary-long input data rather than fixed-size ethereum private key;
  • multiple keyfiles stored in same file, which could be encrypted with different passwords;
  • a retrieval mechanism to automatically decrypt the first keyfile encrypted with the input password among many;
  • a skip mechanism to iterate among multiple keyfiles stored in same file and encrypted with same password;

The keyfile implementation is adapted from nim-eth for this purpose. In order to allow flexibility/compatibility with go-waku implementation, the keyfile fields id and version are omitted by default, but can be re-enabled by changing their respective flags.

Since the main purpose of keyfile introduction is to encrypt RLN credentials, this PR further implements their storage through keyfiles (re)definining the procs writeRlnCredentials and readRlnCredentials. Given that keyfiles process RLN credentials as arbitrary byte data, the format of RLN credentials can be changed anytime with minor modification to the code. Here a related comment.

In order to set a password to encrypt/decrypt RLN credentials, it suffices to pass the --rln-relay-cred-password=<PASSWORD> flag to the chat2 or wakunode2 instances. When this flag is not passed, the encryption/decryption password is set to "" (empty string).

This PR addresses and solve with respect to nwaku and go-waku cross-clients credentials the issue #1278

Note: keyfile.nim, by being a code adaptation coming from nim-eth doesn't seem to follow rigorously our nim guidelines, e.g. note use the result variable. Let me know if it is worth to align it to guidelines (would require some effort though).

@s1fr0 s1fr0 added enhancement New feature or request track:rln RLN Track (Secure Messaging/Applied ZK), e.g. relay and applications labels Oct 20, 2022
@s1fr0 s1fr0 self-assigned this Oct 20, 2022
@s1fr0 s1fr0 changed the title feat(utils): add keyfile (WIP) feat(utils): add keyfile support (WIP) Oct 20, 2022
@status-im-auto
Copy link
Collaborator

status-im-auto commented Oct 20, 2022

Jenkins Builds

Click to see older builds (41)
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 7abe2fd #1 2022-10-20 18:26:18 ~16 min linux 📦bin
✔️ 7abe2fd #1 2022-10-20 18:26:24 ~16 min macos 📦bin
a6bf2be #2 2022-10-21 15:37:21 ~18 min linux 📄log
✔️ a6bf2be #2 2022-10-21 15:37:43 ~18 min macos 📦bin
✔️ 0ded30f #3 2022-10-22 00:32:27 ~15 min linux 📦bin
0ded30f #3 2022-10-22 00:32:32 ~15 min macos 📄log
2a6464b #4 2022-10-22 01:17:37 ~17 min linux 📄log
2a6464b #4 2022-10-22 01:18:52 ~18 min macos 📄log
fc5f48f #5 2022-10-22 01:19:19 ~18 min macos 📄log
fc5f48f #5 2022-10-22 01:20:34 ~19 min linux 📄log
✔️ 7c2c1f9 #6 2022-10-22 01:22:50 ~15 min linux 📦bin
✔️ 7c2c1f9 #6 2022-10-22 01:27:46 ~20 min macos 📦bin
✔️ 589ee0d #7 2022-10-24 10:00:36 ~15 min macos 📦bin
✔️ 589ee0d #7 2022-10-24 10:01:10 ~16 min linux 📦bin
✔️ 517c3b1 #8 2022-10-24 23:19:34 ~15 min linux 📦bin
✔️ 517c3b1 #8 2022-10-24 23:19:36 ~15 min macos 📦bin
5a47ec3 #9 2022-10-25 17:18:04 ~16 min macos 📄log
5a47ec3 #9 2022-10-25 17:20:50 ~19 min linux 📄log
e74cc7e #10 2022-10-25 17:20:26 ~17 min linux 📄log
✔️ e74cc7e #10 2022-10-25 17:23:42 ~20 min macos 📦bin
✔️ e74cc7e #11 2022-10-25 17:49:41 ~14 min linux 📦bin
✔️ 08be67c #12 2022-10-26 21:34:11 ~15 min linux 📦bin
✔️ 08be67c #11 2022-10-26 21:34:43 ~15 min macos 📦bin
✔️ d814f17 #13 2022-10-26 22:04:56 ~16 min linux 📦bin
✔️ d814f17 #12 2022-10-26 22:06:29 ~17 min macos 📦bin
✔️ 4ff8a7f #14 2022-10-26 22:11:04 ~16 min linux 📦bin
✔️ 4ff8a7f #13 2022-10-26 22:12:00 ~17 min macos 📦bin
✔️ 01ebc54 #14 2022-10-26 22:25:23 ~14 min macos 📦bin
✔️ 01ebc54 #15 2022-10-26 22:25:26 ~14 min linux 📦bin
✔️ 9b6850e #16 2022-10-27 13:15:34 ~16 min linux 📦bin
✔️ 9b6850e #15 2022-10-27 13:15:54 ~16 min macos 📦bin
✔️ 6aa7805 #16 2022-10-27 14:27:53 ~14 min macos 📦bin
✔️ 6aa7805 #17 2022-10-27 14:31:34 ~18 min linux 📦bin
75c6ff6 #19 2022-10-27 15:28:06 ~9 min linux 📄log
75c6ff6 #18 2022-10-27 15:29:42 ~11 min macos 📄log
✔️ 112de23 #19 2022-10-27 17:26:11 ~14 min macos 📦bin
✔️ 112de23 #20 2022-10-27 17:26:33 ~14 min linux 📦bin
✔️ 239113c #20 2022-10-27 18:46:14 ~14 min macos 📦bin
✔️ 239113c #21 2022-10-27 18:46:45 ~15 min linux 📦bin
✔️ ffad08b #22 2022-10-27 20:24:47 ~15 min linux 📦bin
✔️ ffad08b #21 2022-10-27 20:25:23 ~16 min macos 📦bin
Commit #️⃣ Finished (UTC) Duration Platform Result
78f2151 #23 2022-10-27 22:35:22 ~14 min linux 📄log
✔️ 78f2151 #22 2022-10-27 22:35:31 ~14 min macos 📦bin
✔️ 78f2151 #25 2022-10-27 23:38:52 ~14 min linux 📦bin
✔️ def10d0 #23 2022-10-28 07:51:35 ~15 min macos 📦bin
✔️ def10d0 #26 2022-10-28 07:53:55 ~17 min linux 📦bin

@s1fr0 s1fr0 linked an issue Oct 22, 2022 that may be closed by this pull request
1 task
@s1fr0 s1fr0 marked this pull request as ready for review October 22, 2022 01:26
@s1fr0 s1fr0 removed the request for review from richard-ramos October 22, 2022 01:26
@s1fr0 s1fr0 changed the title feat(utils): add keyfile support (WIP) feat: add keyfile support for RLN credentials secure storage Oct 22, 2022
Copy link
Contributor

@LNSD LNSD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, check my comments (and sorry for the nitpicking 🙏🏼)

tests/v2/test_keyfile.nim Outdated Show resolved Hide resolved
waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim Outdated Show resolved Hide resolved
waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim Outdated Show resolved Hide resolved
waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim Outdated Show resolved Hide resolved
tests/v2/test_keyfile.nim Outdated Show resolved Hide resolved
waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim Outdated Show resolved Hide resolved
# retrieve rln-relay credential
credentials = some(readPersistentRlnCredentials(rlnRelayCredPath))
credentials = readRlnCredentials(rlnRelayCredPath, conf.rlnRelayCredentialsPassword)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here you are accessing a conf field. Using the conf object everywhere makes refactoring/decoupling work harder.

Can you specify a parameter in this function named rlnRelayCredentialsPassword and use it here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I got the suggestion correctly: conf is passed as parameter to the mount function which in turn calls readRlnCredentials. If I add rlnRelayCredentialsPassword to mount I'll get no advantage. I got it wrong?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The advantage is that you decouple the mount() procedure from the WakuConf type (char2 or wakunode2 conf type), which will ease the task of decoupling the RLN module from the chat2 and wakunode2 configuration types.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand, but currently the WakuConf is passed to mount(), and other conf options are used for other purposes. Unless you're thinking (in the long term) to move all these conf parameters used as parameters to mount?
If so does it make sense to do it in this PR or better a separate "decoupling" one?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tracking issue #1294

@@ -1256,8 +1281,8 @@ proc mount(node: WakuNode,
# persist rln credential
credentials = some(RlnMembershipCredentials(rlnIndex: node.wakuRlnRelay.membershipIndex,
membershipKeyPair: node.wakuRlnRelay.membershipKeyPair))
writeFile(rlnRelayCredPath, pretty(%credentials.get()))

if writeRlnCredentials(rlnRelayCredPath, credentials.get(), conf.rlnRelayCredentialsPassword).isErr():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here:

Can you specify a parameter in this function named rlnRelayCredentialsPassword and use it here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tracking issue #1294

Comment on lines +294 to +297
rlnRelayCredentialsPassword* {.
desc: "Password for encrypting RLN credentials",
defaultValue: ""
name: "rln-relay-cred-password" }: string
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this configuration option be within a when defined(rln): compilation guard?

The rest of the RLN conf options should also be under the when compilation guard; if this is allowed by Nim.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possible, but the full Chat2Conf/WakuNodeConf object has to be redefined under/outside the flag, i.e. we duplicate all the remaining field definition. I agree that it should be the case, i.e. not have fields unless supported, but nim seems a bit unhandy for these tasks. No problem for me, but maybe not the right PR. Wdyt?

Copy link
Contributor

@LNSD LNSD Oct 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok moving this to another PR. Not a blocker on this PR, but certainly, in the near future, we should put a when guard to RLN-specific configuration options (the same way we have it in the implementation).

cc @rymnc @staheri14

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tracking issue #1294

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more relevant issue about the visibility of config options is #999 where it was suggested to use sub-commands to group all the rln-relay-related configs and make them available only when the rln-relay config option is set to true. Would that address your comment @LNSD ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but nim seems a bit unhandy for these tasks.

Now I see why you said that. It is an issue in nim-confutils macros 😫 Then, let's leave it as it is until we have a better way of doing this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh! I thought was just not possible in nim (i.e. have object fields defined at compilation-time), but indeed with some macros it might be possible to achieve the same goal.

I agree that this has to be done in followup PRs! Thanks for pointing it out!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@LNSD I see your point, that makes sense.

On a side note, I don't understand how subcommands will help here. We are not using subcommands in the nwaku node app (aka wakunode2). And I don't see any use case for them in wakunode2, at least in the near future.

It was suggested in one of the PRs and my understanding was that it is part of the plan to use subcommands: #992 (comment) @jm-clius

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please let us know if the plan re subcommands has changed so that we deprioritize that issue i.e., #999 cc: @jm-clius @LNSD

Copy link
Contributor

@staheri14 staheri14 Oct 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand how subcommands will help here.

@LNSD
I'd say all the rln-relay-related config options will get encapsulated (and only accessible) under the rln-relay command/subcommand, hence the CLI interface becomes less confusing for the users (I assume this is the desirable feature we are looking for)

Comment on lines +178 to +181
rlnRelayCredentialsPassword* {.
desc: "Password for encrypting RLN credentials",
defaultValue: ""
name: "rln-relay-cred-password" }: string
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this configuration option be within a when defined(rln): compilation guard?

The rest of the RLN conf options should also be under the when compilation guard; if this is allowed by Nim.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comment: #1285 (comment)

tests/v2/test_keyfile.nim Outdated Show resolved Hide resolved
tests/v2/test_waku_rln_relay.nim Outdated Show resolved Hide resolved
Copy link
Contributor

@alrevuelta alrevuelta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking great, just some final questions/comments


# We generate 6 different secrets and we encrypt them using 3 different passwords, and we store the obtained keystore

let secret1 = randomSeqByte(rng[], 300)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since there is a clear pattern here, perhaps just use a for loop?

for pass in @[pass1, passn]:
  let secret = randomSeqByte(rng[], 300)
  let keyfile = createKeyFileJson(secret, pass)
  check:
      keyfile.isOk()
      saveKeyFile(filepath, keyfile.get()).isOk()

you can even randomize the password for each iteration, or even include %,Aa-? and weird stuff just in case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but then later checks on number of encryption under same password, the decrypted values and so one will get slightly more complicated. I would have been more open to make it more general in case we needed it in nwaku or needed to test hundreds of cases, but this is just a test and I don't see it as a valuable effort for just a stylistic advantage (which is unclear given the second part of the test).

tests/v2/test_utils_keyfile.nim Show resolved Hide resolved
tests/v2/test_utils_keyfile.nim Show resolved Hide resolved
tests/v2/test_utils_keyfile.nim Outdated Show resolved Hide resolved
tests/v2/test_utils_keyfile.nim Outdated Show resolved Hide resolved
waku/v2/protocol/waku_rln_relay/waku_rln_relay_utils.nim Outdated Show resolved Hide resolved
Copy link
Contributor

@jm-clius jm-clius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for addressing the error handling. :)

@alrevuelta alrevuelta self-requested a review October 27, 2022 14:48
Copy link
Contributor

@alrevuelta alrevuelta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! thank you for addressing the comments. I left this one open as a nice to have but not blocking.

@s1fr0 s1fr0 requested a review from staheri14 October 27, 2022 15:03
Copy link
Contributor

@staheri14 staheri14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, I just left some minor comments.

"r": 8,
"salt": "247797c7a357b707a3bdbfaa55f4c553756bca09fec20ddc938e7636d21e4a20",
},
"mac": "5a3ba5bebfda2c384586eda5fcda9c8397d37c9b0cc347fea86525cf2ea3a468",
Copy link
Contributor

@staheri14 staheri14 Oct 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have a test for decrypting a key file with a right password but with a tampered/wrong mac?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. Test added in 239113c

expectedSecret = randomSeqByte(rng[], 300)
password = "miawmiawcat"

# By default keyfiles are created using PBKDF2. Here we test keyfiles created using scrypt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By keyfiles are created, you mean the encryption key is derived from a supplied password using PBKDF2?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Clarified in 239113c

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request track:rln RLN Track (Secure Messaging/Applied ZK), e.g. relay and applications
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat(rln): How to securely store RLN credentials cross-client
7 participants