Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TravisCI: More robust parsing of TravisCI log #395

Merged
merged 10 commits into from
Jun 20, 2016

Conversation

tammoippen
Copy link
Contributor

@tammoippen tammoippen commented Jun 13, 2016

Solves #392.
This PR makes the parsing of the TravisCI more robust and allows to order the different steps of the build.sh in any order (for the parsing). It is also possible to remove different steps completely and the parser will not break. A summary of the changes:

In build.sh:

  • Every block is surrounded by tags like

    ======= Extract changed files start =======
    ...
    ======= Extract changed files end =======
    
  • Output is reduced:

    • Comment set -x out, as we do not need to debug the travis log regularly. One can uncomment it, if needed.
    • Remove verbose flag of extracting clang-format.
  • Make static analysis optional with env. variable xSTATIC_ANALYSIS.

  • Fix regex bug (see line 4188) for if [[ $f =~ $EXAMPLE_DIRS ]]. I think, the TravisCI environment does not allow if [[ ... ]].

  • Add output that tells if results are not uploaded, if you are working on your fork.

In extras/parse_travis_log.py:

  • No state on reading/parsing the log. Every function opens the file itself and completely parses it for its issues. The actual parsing happens only within the starting/ending tags relevant for this function.
  • Separate vera++, cppcheck and clang-format parsing into standalone functions.
  • Add print_... functions for nicer output.
  • Cleanup "main" function.

I propose @heplesser and @gtrensch as reviewers.

@gtrensch
Copy link
Contributor

👍 Everything works fine! Just one remark from my side: There is a comment in parse_travis_log.py referring to the dependencies on build.sh. A similar comment in build.sh would be good.

@@ -1,7 +1,7 @@
#!/bin/sh

set -e
set -x
#set -x
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why commented out instead of deleted?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@heplesser I documented the 'out-commenting'. When debugging the script, the flag is very informative.

@heplesser
Copy link
Contributor

@tammoippen Very nice work. Have you tested that errors are handled properly, by introducing changes that will provoke errors/warning from vera, clang-format, cppcheck, pep8, building, testsuite? I think that we need some intentionally provoked Travis failures to be sure things work as intended.

@tammoippen
Copy link
Contributor Author

tammoippen commented Jun 20, 2016

@gtrensch I added the comment in 2e10e14. Thanks.

@tammoippen
Copy link
Contributor Author

tammoippen commented Jun 20, 2016

@heplesser in 30d525b I introduced some intentional failures that should trigger 'vera, clang-format, cppcheck, pep8, building, testsuite'. I will remove it afterwards.

And make less verbose build.sh.
Pep8ify parse_travis_log
Fix skip error return value
And add comment to `fi` of `if [ "$xSTATIC_ANALYSIS" = "1" ] ; then`
* strip() startswith queries
* standardize PEP8 start output in build.sh to other static analysis
* unique parsing of start-outputs
* make static analysis run for all jobs
* do not exit build.sh when there are files with changes
@tammoippen
Copy link
Contributor Author

See this build for forced errors.

@heplesser
Copy link
Contributor

👍 and merging.

@heplesser heplesser merged commit 830084d into nest:master Jun 20, 2016
@tammoippen tammoippen deleted the travisci branch July 1, 2016 07:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ZC: Infrastructure DO NOT USE THIS LABEL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants