Touch Screen Stenography Keyboard

[DanielAtCosmicDNA]

Hi, I developed an open source touch screen stenography keyboard in React Three Fiber and I would like to discuss recommended ways to connect it to Plover for evaluation. The source code can be found at: https://github.com/CosmicDNA/touchscreen-stenography-keyboard.

[user202729]

One way is to modify Plover source code, of course.

Another way is to use my plugin user202729/plover_websocket_server: A server for exposing Plover engine events and functionality, connect to the websocket, and send strokes.

Unfortunately the documentation is rather poor right now.

Related: iStenopad Overlay: A Bust | The Plover Blog

[DanielAtCosmicDNA]

As you already have a web-socket plugin, I prefer to stick with it for the time being, unless we find out later we need something else.

Many people are developing REL due to years of fast typing with staggered keyboards and the actuation force might be a factor. With touch screens there is close to no actuation force especially if you train yourself not to exert too much pressure on the screen. In this regard I think every initiative has merit. It may be possible to use touch screens in one way or another to steno type in the long term given compromises are thoughtfully made.

For now I am happy typing and practicing on my laptop screen.

[DanielAtCosmicDNA]

I also like the idea to modify Plover's source code, maybe to have the touch screen keyboard there as an option would be nice for the wider community. I am not sure on this, happy to support though.

[user202729]

Actually thinking about it it's probably better to use Plover machine plugin. https://plover.readthedocs.io/en/latest/plugin-dev/machines.html

[DanielAtCosmicDNA]

@user202729 , can you try it out? The last commit I made to the branch feature-plover-websocket triggered a branch deploy to https://feature-plover-websocket--touch-stenography-keyboard.netlify.app/.

[user202729]

Unfortunately I don't have a large-enough touchscreen device.

[DanielAtCosmicDNA]

@user202729 , I just created a pull request at your plugin to allow SSL in the web-socket. With this setting users will be able to connect to Plover from another machine within the network, what opens many possibilities like the ability to control Plover from an external tablet.

[DanielAtCosmicDNA]

@user202729 , can you bump the version of the plover_websocket_server so that I can test the new functionality out?

[user202729]

Alright (but you can also manually install the plugins not on PyPI as explained in https://plover.wiki/index.php/Plugins#Manually_Installing_Plugins )

[DanielAtCosmicDNA]

@user202729 , I am working on the fork from your plugin. I think it would be a good way to secure the communication with the user interface to use assymetric encryption. As soon I have figured out the problem here, I think a pull request is interesting to merge it back to the repo.

[user202729]

What's your threat model? (i.e. in which threat model will this cause a trouble compared to just use wss?)

[DanielAtCosmicDNA]

The reasons for using both TLS (Transport Layer Security) and Ed25519 asymmetric encryption in this context:

TLS (Transport Layer Security):

• Purpose: TLS is a widely adopted security protocol designed to facilitate privacy and data security for communications over the Internet.
• Usage: It encrypts communication between web applications and servers, such as web browsers loading a website 1.
• Benefits:
  • Authentication: TLS ensures that the other party in a connection is who they claim to be.
  • Integrity: It verifies whether data retains its initial integrity.
  • Confidentiality: TLS provides encryption to protect data from prying eyes.
• How It Works:
  • During the TLS handshake, authentication occurs, and cryptographic keys are established.
  • The connection is secured using encryption algorithms like AES (Advanced Encryption Standard) 2.
• Comparison to End-to-End Encryption:
  • TLS terminates at the server, allowing the server operator to view messages since they are not encrypted end-to-end 3.

Ed25519 Asymmetric Encryption:

• Purpose: Ed25519 is an elliptic curve cryptography algorithm used for digital signatures and key exchange.
• Usage: It can be employed for secure communication between parties.
• Benefits:
  • Strong Security: Ed25519 provides high security with relatively short keys.
  • Efficiency: It is computationally efficient for key generation and signing.
• How It Works:
  • Alice and Bob generate their key pairs (private and public keys).
  • They derive a shared secret using their private keys and each other’s public keys.
  • The shared secret is used for subsequent encryption 4.

Combined Usage:

• Defense in Depth: Using both TLS and Ed25519 provides layered security.
• TLS for Transport: TLS secures the communication channel between the web keyboard application and Plover via the WebSocket plugin.
• Ed25519 for Message-Level Security:
  • Each message is additionally encrypted using Ed25519.
  • This ensures that even if the TLS connection is compromised, the individual messages remain confidential and tamper-proof.

I believe combining TLS and Ed25519 provides a robust security posture, protecting both the channel and the content of messages in a Touch screen web keyboard application, especially because the user may not be willing to deal with the fuss of setting up a SSL certificate. In that specific case ed25519 by default would secure the application and network.

Another attack vector would be for a deployed keyboard app as a website to a major infrastructure provider. There is the possibility that the TLS key would be compromised by a breach in the provider's security. Even if that happens, the Ed25519 would protect Plover from receiving data from malicious players. Of course the private key of the server needs to be kept safe at all times.

This solution compared to the plugin's default solution would represent a major improvement in the overall security in my understanding as the security key would not have to be sent unprotected through the network which might or not be subject to malicious sniffing. Instead only the server's and the client's public keys would have to be shared.

[user202729]

Don't tell me the text is AI-generated? 😅

[DanielAtCosmicDNA]

Indeed, it was augmented.

[DanielAtCosmicDNA]

It is also important to consider that prior to the web-socket connection being stablished, there would be a http(s) connection in place secured by Ed25519 asymmetric encryption. Once the connection is upgraded to web-socket after the handshake, I believe it is important that each keystroke be also encrypted. But this may well be an optional feature as the most important security concern for a web-socket connection is to make sure that only authorised connections are accepted to the web-socket in the first place.

[DanielAtCosmicDNA]

@user202729 , I finished implementing the changes here to have the asymmetric encryption in place and following the best web socket practices. I have validated it with the strategy outlined here and tested it out with the touchscreen keyboard. Let me know your thoughts...