Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Print esp-println generated defmt messages #466

Merged
merged 10 commits into from
Oct 11, 2023
Merged

Conversation

bugadani
Copy link
Contributor

@bugadani bugadani commented Aug 26, 2023

Closes #316

Counterpart of esp-rs/esp-println#48 as it relies on the injected framing bytes. Not sure how to best lift this curse yet.

This PR adds a new compile-time defmt flag that enables defmt support in the binaries and the library. For this, the PR raises MSRV to 1.70. By default, the defmt feature is enabled. defmt detection is done by parsing the elf file.

The PR refactors serial port input handling so library users can define their own processing if they so choose. The default processor is able to recognize and parse frames generated by esp-println. Library users can disable this processing by using the Serial input processor or by disabling the defmt feature (which internally does the same besides disabling the dependencies).

@bugadani bugadani force-pushed the defmt branch 5 times, most recently from 0fc516b to 090c2d5 Compare August 26, 2023 20:07
@bugadani bugadani marked this pull request as ready for review August 26, 2023 20:26
@bugadani bugadani marked this pull request as draft August 27, 2023 09:23
@bugadani bugadani force-pushed the defmt branch 2 times, most recently from b816ac1 to 9162ef6 Compare August 28, 2023 14:00
@bugadani

This comment was marked as outdated.

@bugadani bugadani marked this pull request as ready for review August 28, 2023 15:33
@bugadani bugadani marked this pull request as draft August 28, 2023 15:33
@bugadani
Copy link
Contributor Author

bugadani commented Sep 12, 2023

Oh come on... @jessebraham do you think we can bump MSRV for defmt, either selectively or otherwise? (defmt has at least one dependency that is 1.67+. I'm not sure if we can use an earlier parser to work around it)

@MabezDev
Copy link
Member

Dependency hell is real :D. Are these not breaking changes from the library(s)? Surely bumping rust-version indicates a breaking change? Idk but that's really annoying, not sure what to do here other than bump out MSRV too.

@bugadani
Copy link
Contributor Author

bugadani commented Sep 12, 2023

In this case, defmt-parser is a new dependency so it's probably not a case of a patch-with-breaking-MSRV-bump. CI just wasn't passing for the last few weeks and I didn't notice this PR requires a bump.

@jessebraham
Copy link
Member

jessebraham commented Sep 12, 2023

Strictly speaking, I don't think this situation is technically covered by SemVer itself.

There is, however, some documentation in The Cargo Book which states that this is allowed if we bump the minor version:
https://doc.rust-lang.org/cargo/reference/semver.html#possibly-breaking-changing-the-minimum-version-of-rust-required

We would need to do a minor release to include these changes anyway, so no harm done.

I have no issue bumping the MSRV as long as it's not the current stable version. I'd prefer to have at least a few versions back, if possible.

Edit: don't know why I didn't check the CI logs 😂 1.70.0 if fine with me!

@MabezDev
Copy link
Member

Oh yes, of course. In that case we either bump the MSRV and be done with it, or we say we don't make any MSRV guarentees behind a defmt feature (that we'd need to introduce).

Imo bumping MSRV is okay, it realistically only affects library users which (publically) is just me: https://crates.io/crates/espflash/reverse_dependencies :D.

@bugadani
Copy link
Contributor Author

bugadani commented Sep 12, 2023

There is, however, some documentation in The Cargo Book which states that this is allowed if we bump the minor version:

Oh hell please do not start following this madness. While not covered by Semver, an MSRV change should absolutely be treated as major breaking. For an application it doesn't matter, but a library bumping MSRV with a minor version is absolutely inconsiderate and espflash is a library 🥺

@jessebraham
Copy link
Member

Okay, well we can't merge this then and can't update a number of dependencies ever. So I'm not sure that's preferable.

@bugadani
Copy link
Contributor Author

Okay, well we can't merge this then and can't update a number of dependencies ever

Not sure I follow. Sorry for being way more emotional here than probably warranted.

An MSRV bump being a major change doesn't mean we can't update dependencies, the release just needs to be a major version after that, in my opinion. SemVer considers minor bumps with 0.x major versions as major releases.

@bugadani bugadani marked this pull request as ready for review September 12, 2023 16:08
@bugadani bugadani changed the title Print defmt messages Print esp-println generated defmt messages, raise MSRV Sep 12, 2023
@bugadani
Copy link
Contributor Author

Have you guys a recommendation how I could featurize defmt? I think it would be nice to ship the binaries with it enabled, but I also don't want to force defmt on libraries. I believe I can extract the code itself to make parsing in monitor opt-in, hide that behind a feature flag and call the library part done. But for the binaries, I think it would be better to have the feature default on, as opposed to passing a flag in CI.

I also don't know if the feature should be controllable in the binaries, via a --defmt switch or something similar. I tend to think it'd just complicate the implementation without much value, but I'd love to hear your take.

@bugadani
Copy link
Contributor Author

bugadani commented Sep 15, 2023

I'd consider this done now, although there are a few open questions.

I've been using this PR for the last few days/weeks without any issue, though I'm a bit unsure if defmt-by-default is the polite way to handle the issue.

Because of the MSRV bump I'd suggest a major release (3.0.0) at least for the library, but considering the low number of public dependent maybe not doing this isn't the end of the world. For binaries, semver is a lot less relevant.

The defmt feature is enabled by default mainly because I wanted to avoid editing the CI files for this.
I considered adding a runtime switch but the implementation complexity doesn't seem to be worth it.

Let me know if you'd like to instead keep defmt off by default. Doing that would allow keeping MSRV low, which would avoid the semver-related headaches. From-source installations could still enable the feature, however one downside is that it's not very feasible to install from source on a Raspberry Pi.

@bugadani bugadani marked this pull request as ready for review September 15, 2023 11:36
CHANGELOG.md Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
Co-authored-by: Sergio Gasquez Arcos <[email protected]>
@bugadani bugadani marked this pull request as draft October 9, 2023 08:19
@bugadani bugadani marked this pull request as ready for review October 9, 2023 09:14
@SergioGasquez
Copy link
Member

Just tried testing your branch and couldnt monitor a esp-tempalte project (tried c3 and s3)

cargo install --force espflash --git https://github.com/bugadani/espflash --branch defmt
cargo generate esp-rs/esp-template --name test -d mcu=esp32c3 -d advanced=false
cd test
cargo r -r

Also tried updating the esp-println: esp-println = { git = "https://github.com/esp-rs/esp-println", features = ["esp32c3", "defmt"] }

But always getting the following:

Hello world!
Error:   × Main thread panicked.
                                  ├─▶ at /rustc/bf9a1c8a193fc373897196321215794c8bebbeec/library/std/src/io/mod.rs:1630:36
                                                                                                                            ╰─▶ range start index 78 out of range for slice of length 77
                                                                                                                                                                                          help: set the `RUST_BACKTRACE=1` environment variable to display a backtrace.

Not sure if its failing, or I am doing something wrong?

@bugadani
Copy link
Contributor Author

Interesting find. I have experienced a few panics myself, but so far the impl has been stable enough for me. The last commit has resolved a similar issue for me but maybe newlines weren't the root cause. I'll investigate.

@bugadani
Copy link
Contributor Author

bugadani commented Oct 11, 2023

Okay so the panic happened like this:

  • println printed a non-defmt frame
  • The frame delimiter has correctly handled this frame as Raw
  • The contents were written correctly to ResolvingPrinter. The frame ended with a \n character.
  • In ResolvingPrinter::write, the \n was expanded to \r\n using Normalized, increasing the length of the string by one
  • ResolvingPrinter::write returned the length of the normalized string instead of the input's, causing write_all to incorrectly index during the next iteration, causing the panic.

@SergioGasquez
Copy link
Member

I did some further testing and seems to be working fine! Just one question/discussion, shall we have the defmt feature enable by default? @jessebraham @MabezDev @bjoernQ

@bjoernQ
Copy link
Contributor

bjoernQ commented Oct 11, 2023

I did some further testing and seems to be working fine! Just one question/discussion, shall we have the defmt feature enable by default? @jessebraham @MabezDev @bjoernQ

I guess it shouldn't be a default. While it's not very likely that someone tries to use the framing marker by accident it's not impossible. Additionally, I think that a user who wants to use defmt is probably a more advanced user and it should be okay if they need some extra steps in enabling it. But maybe I'm thinking too cautiously

@MabezDev
Copy link
Member

I agree with @bjoernQ's points. I think that if we get a lot of requests to enable it by default then we can re-assess later.

Copy link
Member

@SergioGasquez SergioGasquez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just did some testing and everything was working! Thanks for the contribution!

@SergioGasquez SergioGasquez merged commit 1dfe0a1 into esp-rs:main Oct 11, 2023
19 checks passed
SergioGasquez added a commit to SergioGasquez/espflash that referenced this pull request Oct 11, 2023
* Print defmt frames output by esp-println

* Add changelog entry

* Display log level

* Bump MSRV to 1.70

* Refactor for opt-in defmt

* Make defmt a default feature

* Update CHANGELOG.md

Co-authored-by: Sergio Gasquez Arcos <[email protected]>

* Fix panic on frame containing multiple lines

* Fix panic due to newline normalization

* Don't enable defmt by default

---------

Co-authored-by: Sergio Gasquez Arcos <[email protected]>
SergioGasquez added a commit to SergioGasquez/espflash that referenced this pull request Nov 2, 2023
* Print defmt frames output by esp-println

* Add changelog entry

* Display log level

* Bump MSRV to 1.70

* Refactor for opt-in defmt

* Make defmt a default feature

* Update CHANGELOG.md

Co-authored-by: Sergio Gasquez Arcos <[email protected]>

* Fix panic on frame containing multiple lines

* Fix panic due to newline normalization

* Don't enable defmt by default

---------

Co-authored-by: Sergio Gasquez Arcos <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

add defmt support
5 participants