💡 Feature request: Verbose HTTPS debug session: dump all traffic to and from Reddit

[Tokarak]

Redlib uses undocumented private API endpoints for OAuth spoofing. This API is subject to change without warning, which can break Redlib (see #229). Currently, there is no descriptive error messages or logging, apart from the basics, such as the HTTP status code. For debugging, it is essential to have more information, such as request and response headers.

There are two ways to do this:

1.) Programmatic logging. The internal http library can log its request/response data structure as soon as it is constructed. I have not looked further into this.

2.) Https sniffer. This involves adding a self-signed CA certificate to the host, Redlib trusting that certificate, and proxying Redlib through a https sniffer proxy which uses that certificate to communicate with Redlib. This can be done manually, but is very inconvenient. I suggest adding a feature flag (or command-line flag) to Redlib which disables https verification (maybe enable this feature it by default for debug builds — but consider security), which will make Redlib ready to plug in into any https sniffer software through the built-in socks proxy feature.

No 1., I think, is a symptom of a wider issue which is a consequence of a silly "no logs" privacy policy and unidiomatic internal data and error structures, and hence way out of scope.

No 2. is simple to implement, without affecting regular hosts. I would also like to placate any privacy concerns with No. 2 in advance: it's a simple patch to just disable certificate verification — it does not introduce a logger into the project; any sufficiently determined (the level of sufficiency is just a tad above convenience) instance could use a sniffer with the current code base.

I would like input from the maintainers and the community on:

• Is 1. feasible, and is the logging features better than 2.? HTTPS sniffers are a competitive niche, mature, and full of features which a stderr log will not match.
• If 1. is feasible, and though I disagree with those policies, 1. would violate the historical privacy stance of Redlib and Libreddit. Now what?
• For 2.: feature flag, or command line flag? Default in debug builds? Feature flag which enables the command-line flag, and is default in debug builds?
• Could 2. be made even easier with a container image?

[sigaloid]

This would be a tricky one, because many of the requirements for features state that no identifying information should be logged. Maybe a large banner saying this instance is in debug mode, requests will be logged? Then save it into a network file. I wholeheartedly agree that logging requests would make my job so much easier in these cases. Definitely something to add in the future. Or at least a mode to bypass TLS verification so we can use something like mitmproxy/HTTP toolkit to visualize it easier.

[pimlie]

Although a valid point, anybody can already build their own redlib server and add any code patches they want. So some level of trust between the user and the server admin already needs to exist. Locally I just added a bunch of trace!( calls to see all uri's & headers while debugging, there doesn't seem to be anything preventing me from publishing a public instance with an https forwarder in front of it and just log all the traces.

If that's a strong requirement then we'd probably need to distribute a signed server binary with some type of attestation so remote users can validate the server's signature.

[sigaloid]

Very true. Definitely not a strong requirement to the point of attestation, but ideally the default container would enforce informing the user if that mode is enabled.

[Tokarak]

If that's a strong requirement then we'd probably need to distribute a signed server binary with some type of attestation so remote users can validate the server's signature.

You have the same problem: you can spoof the signature.

---

It seems that the best solution is to add an option to disable https cert verification. Good First Issue.

[sigaloid]

I'd also add #184 to this project - we'd likely need a Hyper client adapter compatible with 0.14.x that would inherit the proper HTTP/S client environment variables. Something like https://lib.rs/crates/mz-http-proxy perhaps? (Of course not in scope of your PR but just under the same initiative of improving debugging)

https://github.com/orgs/redlib-org/projects/2

[argium]

Redlib does not guarantee privacy between the user and reddit - the host being used will always be in a position to intercept traffic. You're hidden from reddit, not the owner.

That aside, options:

• Decouple the web UI from the request logic and allow it to be invoked from the command line
• Flag to disable TLS validation would be fine
• Flag to log the contents of the response from reddit also fine