Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Switches to no color when running for CI so that console logs can be processed by JUnit test reports #123

Open
marcellodesales opened this issue Jan 4, 2022 · 1 comment

Comments

@marcellodesales
Copy link

marcellodesales commented Jan 4, 2022

Problem

  • Tests reports for Continuous Integration usually use JUnit test reports
    • Jenkins, Gitlab, Github
  • When converting an output of test cases to JUNit, while preserving the formatted output of test reports, we get errors
  • Since the output is in XML, the PCDATA embedded as texts must be escaped as it will fail any XML lint

NOTE: This was identified in a CI build for Mobile apps using a junit dart formatter https://gitlab.com/gitlab-org/gitlab/-/issues/268035#note_801222659

Possible Solution

  • Use an alternative output without color formatting for the test reports
  • Switch the use a logger formatting that does NOT show the colors, special characters, etc

Question: How can we use flutter test and format the output of the tests to be without colors?

JUnit converted test report with output with colors

flutter test --coverage --machine | tojunit --output build/app/reports/junit-tests-report.xml
cat output build/app/reports/junit-tests-report.xml
  • The output of the junit test reports include the raw output from logger
....
....
┌───────────────────────────────────────────────────────────────────────────────
│ #0   UserRepository.saveUserAddress (package:parking_web_app_maicero_shop/infrastructure/user/user_repository.dart:573:17)
│ #1   <asynchronous suspension>
│ #2   StackZoneSpecification._registerUnaryCallback.<anonymous closure> (package:stack_trace/src/stack_zone_specification.dart)
├┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
│ 12:51:49.751 (+0:00:03.910572)
├┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄
│ ⛔ Critical server error: Http status error [500] in the function: saveUserAddress() on user_repository
└──────────────────────────────

XML Linter errors with special characters from colored output

  • Since the output is in XML, the PCDATA embedded as texts must be escaped...
$ docker run -i --rm -v $PWD:/data mribeiro/xmllint build/app/reports/junit-tests-report.xml
Unable to find image 'mribeiro/xmllint:latest' locally
latest: Pulling from mribeiro/xmllint
Image docker.io/mribeiro/xmllint:latest uses outdated schema1 manifest format. Please upgrade to a schema2 image for better future compatibility. More information at https://docs.docker.com/registry/spec/deprecated-schema-v1/
a3ed95caeb02: Pull complete
5f6cc4b8eedc: Pull complete
7ebc7bd59630: Pull complete
Digest: sha256:e8293a7906daf6bba7cc2812e9cc54c0e0932afe6a162987cf999bcfd08c93a2
Status: Downloaded newer image for mribeiro/xmllint:latest
build/app/reports/junit-tests-report.xml:63: parser error : PCDATA invalid Char value 27
┌──────────────────────
^
build/app/reports/junit-tests-report.xml:63: parser error : PCDATA invalid Char value 27
��──────────────────────────
┌──────────────────────
^
build/app/reports/junit-tests-report.xml:63: parser error : PCDATA invalid Char value 27
��──────────────────────────
                                                                               ^
build/app/reports/junit-tests-report.xml:64: parser error : PCDATA invalid Char value 27
│ #0   UserRepository.signUp (package:parking_web_app_maicero_shop/
^
build/app/reports/junit-tests-report.xml:64: parser error : PCDATA invalid Char value 27
ge:parking_web_app_maicero_shop/infrastructure/user/user_repository.dart:178:17)
                                                                               ^
build/app/reports/junit-tests-report.xml:65: parser error : PCDATA invalid Char value 27
│ #1   <asynchronous suspension>
^
build/app/reports/junit-tests-report.xml:65: parser error : PCDATA invalid Char value 27
│ #1   <asynchronous suspension>
                                                ^
build/app/reports/junit-tests-report.xml:66: parser error : PCDATA invalid Char value 27
│ #2   StackZoneSpecification._registerUnaryCallback.<anonymous
^

Requirement

  • We should be able to switch the formatting via command-line
  • We should be able to switch the formatting via configuration
@marcellodesales marcellodesales changed the title Switch to no color when running for CI Switch to no color when running for CI so that console logs can be processed by JUnit test reports Jan 4, 2022
@marcellodesales marcellodesales changed the title Switch to no color when running for CI so that console logs can be processed by JUnit test reports Add Switches to no color when running for CI so that console logs can be processed by JUnit test reports Jan 4, 2022
@haarts
Copy link
Collaborator

haarts commented Jun 3, 2022

I would not use the PrettyPrinter in this case. You are able to switch formatting via configuration by passing a different Printer based on a env var, cli argument, whatever.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants