Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add AStyle definition file and travis astyle check #6014

Closed
wants to merge 22 commits into from

Conversation

0xc0170
Copy link
Contributor

@0xc0170 0xc0170 commented Feb 5, 2018

@geky Your latest PR was going the good direction. I defined astyle rc based on the style we have in this codebase and added it to Travis

This is work in progress, as we need to actually define what we should be ignoring (I defined few I could think of, however there are external code in our codebase like SDK, network stacks, etc). Therefore this needs more work to be done (I could not find actually that --exclude would support wildcards?? for astyle).

I run locally on drivers only, lot of changes done in files. Thus travis should fail and point to all files that need changes.

@0xc0170 0xc0170 changed the title AStyle definition file + travis AStyle definition file + travis astyle check Feb 5, 2018
@0xc0170 0xc0170 changed the title AStyle definition file + travis astyle check Add AStyle definition file and travis astyle check Feb 5, 2018
.travis.yml Outdated
@@ -93,6 +93,24 @@ matrix:
# Report success since we have overridden default behaviour
- bash -c "$STATUS" success "Local $NAME testing has passed"

- env:
- TEST="astyle"
before_install:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, Travis doesn't inherit from multiple before_installs.

If you want the status update + pipefail + caching (and probably just future compatibility), you'll need to copy them from the global before_install or move this to the install. Maybe a YAML reference will work?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will check, and fix somehow.. I noticed that using default version it complains about options not available but their webpage lists them, so need to get update somehow , will fix

.travis.yml Outdated
install:
- sudo apt-get install astyle
script:
- export ARTISTIC_STYLE_OPTIONS=".astylerc" && astyle `find . | egrep "\.c|\.h|\.hpp|\.cpp"` > astyle.out
Copy link
Contributor

@geky geky Feb 5, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tiny nit: find can do this for you:

+ find -regex '.*\.\(h\|c\|hpp\|cpp\)'
- find . | egrep "\.c|\.h|\.hpp|\.cpp"

.travis.yml Outdated
install:
- sudo apt-get install astyle
script:
- export ARTISTIC_STYLE_OPTIONS=".astylerc" && $ find -regex '.*\.\(h\|c\|hpp\|cpp\)' -type f ! -path "./cmsis/*" ! -path "./features/mbedtls/*" ! -path "./features/FEATURE_LWIP/lwip/*" ! -path "./rtos/TARGET_CORTEX/rtx5/RTX/*" ! -path "./BUILD/*" ! -path "./tools/*" | xargs -n 100 -I {} bash -c "astyle --options=.astylerc \"{}\"" > astyle.out
Copy link
Contributor Author

@0xc0170 0xc0170 Feb 6, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@geky what magic can we use to exclude some directories ? I defined only some generic ones, but lot of targets contain their own drivers.

I tried to use --exclude, but always end up with error unmatched pattern from astyle 😢

Shall we define exclude paths in a file, or provide .astyleignore file that would contain those, we would read them and exclude from astyle.

@0xc0170
Copy link
Contributor Author

0xc0170 commented Feb 6, 2018

Note: I've noticed AStyle in ubuntu that is used in travis is around v2.05, the reference manual on their official webpage is for 3.x :-) their latest version is published for unstable versions at the moment 😀

@0xc0170
Copy link
Contributor Author

0xc0170 commented Feb 6, 2018

Looking at travis, the command seems to be working, I can see lot of diffs there for files that needs to be changed.

Based on the output, here are proposed exclude dirs (along the ones that I already defined):


.\features\FEATURE_BLE\targets\TARGET_CORDIO\stack\cordio_stack
.\features\FEATURE_BLE\targets\TARGET_Maxim\exactLE
.\features\FEATURE_BLE\targets\TARGET_NORDIC\TARGET_MCU_NRF51822\sdk\
.\features\FEATURE_LWIP\lwip-interface\lwip

.\features\unsupported\

.\targets\TARGET_Analog_Devices\TARGET_ADUCM302X\TARGET_ADUCM3029\bsp
.\targets\TARGET_Analog_Devices\TARGET_ADUCM4X50\TARGET_ADUCM4050\bsp
.\targets\TARGET_ARM_SSG\TARGET_BEETLE\cordio
.\targets\TARGET_Atmel\TARGET_SAM_CortexM0P\config
.\targets\TARGET_Atmel\TARGET_SAM_CortexM0P\drivers
.\targets\TARGET_Atmel\TARGET_SAM_CortexM0P\utils
.\targets\TARGET_Atmel\TARGET_SAM_CortexM4\config\
.\targets\TARGET_Atmel\TARGET_SAM_CortexM4\drivers\
.\targets\TARGET_Atmel\TARGET_SAM_CortexM4\services\
.\targets\TARGET_Atmel\TARGET_SAM_CortexM4\utils\cmsis\TARGET_SAMG55\include
.\targets\TARGET_Freescale\TARGET_MCUXpresso_MCUS\TARGET_K66F\drivers
.\targets\TARGET_Freescale\TARGET_MCUXpresso_MCUS\TARGET_K82F\drivers
.\targets\TARGET_Freescale\TARGET_MCUXpresso_MCUS\TARGET_KL27Z\drivers
.\targets\TARGET_Freescale\TARGET_MCUXpresso_MCUS\TARGET_KL43Z\drivers
.\targets\TARGET_Freescale\TARGET_MCUXpresso_MCUS\TARGET_KL82Z\drivers
.\targets\TARGET_Freescale\TARGET_MCUXpresso_MCUS\TARGET_KW24D\drivers
.\targets\TARGET_Freescale\TARGET_MCUXpresso_MCUS\TARGET_KW41Z\drivers
.\targets\TARGET_Freescale\TARGET_MCUXpresso_MCUS\TARGET_MCU_K22F\drivers
.\targets\TARGET_Freescale\TARGET_MCUXpresso_MCUS\TARGET_MCU_K24F\drivers
.\targets\TARGET_Freescale\TARGET_MCUXpresso_MCUS\TARGET_MCU_K64F\drivers
.\targets\TARGET_Maxim\TARGET_MAX32600\device
.\targets\TARGET_Maxim\TARGET_MAX32610\device
.\targets\TARGET_Maxim\TARGET_MAX32620\device
.\targets\TARGET_Maxim\TARGET_MAX32625\device
.\targets\TARGET_Maxim\TARGET_MAX32625\mxc
.\targets\TARGET_Maxim\TARGET_MAX32630\device
.\targets\TARGET_Maxim\TARGET_MAX32630\mxc
.\targets\TARGET_NORDIC\TARGET_MCU_NRF51822\Lib\nordic_sdk
.\targets\TARGET_NORDIC\TARGET_NRF5\TARGET_MCU_NRF51822_UNIFIED\sdk
.\targets\TARGET_NORDIC\TARGET_NRF5\TARGET_MCU_NRF52832\sdk
.\targets\TARGET_NORDIC\TARGET_NRF5\TARGET_MCU_NRF52840\sdk
.\targets\TARGET_NORDIC\TARGET_NRF5\TARGET_SDK11
.\targets\TARGET_NORDIC\TARGET_NRF5\TARGET_SDK13
.\targets\TARGET_NUVOTON\TARGET_M451\device\StdDriver
.\targets\TARGET_NUVOTON\TARGET_M480\device\StdDriver
.\targets\TARGET_NUVOTON\TARGET_NANO100\device\StdDriver
.\targets\TARGET_NUVOTON\TARGET_NUC472\device\StdDriver
.\targets\TARGET_NXP\TARGET_MCUXpresso_MCUS\TARGET_LPC54114\drivers
.\targets\TARGET_NXP\TARGET_MCUXpresso_MCUS\TARGET_LPC546XX\drivers
.\targets\TARGET_Realtek\TARGET_AMEBA\sdk
.\targets\TARGET_RENESAS\TARGET_RZ_A1XX\TARGET_GR_LYCHEE\device\inc
.\targets\TARGET_RENESAS\TARGET_RZ_A1XX\TARGET_RZ_A1H\device\inc
.\targets\TARGET_RENESAS\TARGET_RZ_A1XX\TARGET_VK_RZ_A1H\device\inc
.\targets\TARGET_Silicon_Labs\TARGET_EFM32\emlib
.\targets\TARGET_Silicon_Labs\TARGET_EFM32\TARGET_EFM32GG\device
.\targets\TARGET_Silicon_Labs\TARGET_EFM32\TARGET_EFM32HG\device
.\targets\TARGET_Silicon_Labs\TARGET_EFM32\TARGET_EFM32LG\device
.\targets\TARGET_Silicon_Labs\TARGET_EFM32\TARGET_EFM32PG\device
.\targets\TARGET_Silicon_Labs\TARGET_EFM32\TARGET_EFM32PG12\device
.\targets\TARGET_Silicon_Labs\TARGET_EFM32\TARGET_EFM32WG\device
.\targets\TARGET_Silicon_Labs\TARGET_EFM32\TARGET_EFM32ZG\device
.\targets\TARGET_Silicon_Labs\TARGET_EFM32\TARGET_EFR32MG1\device
.\targets\TARGET_Silicon_Labs\TARGET_EFM32\TARGET_EFR32MG12\device
.\targets\TARGET_Silicon_Labs\TARGET_SL_RAIL\efr32-rf-driver
.\targets\TARGET_STM\TARGET_STM32F0\device
.\targets\TARGET_STM\TARGET_STM32F1\device
.\targets\TARGET_STM\TARGET_STM32F2\device
.\targets\TARGET_STM\TARGET_STM32F3\device
.\targets\TARGET_STM\TARGET_STM32F4\device
.\targets\TARGET_STM\TARGET_STM32F7\device
.\targets\TARGET_STM\TARGET_STM32L0\device
.\targets\TARGET_STM\TARGET_STM32L1\device
.\targets\TARGET_STM\TARGET_STM32L4\device
.\targets\TARGET_TOSHIBA\TARGET_TMPM066\Periph_Driver
.\targets\TARGET_WIZNET\TARGET_W7500x\W7500x_Peripheral_Library

.travis.yml Outdated
@@ -93,6 +95,20 @@ matrix:
# Report success since we have overridden default behaviour
- bash -c "$STATUS" success "Local $NAME testing has passed"

- env:
- TEST="astyle"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean "NAME" here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

.travis.yml Outdated
install:
- sudo apt-get install astyle
script:
- export ARTISTIC_STYLE_OPTIONS=".astylerc" && find -regex '.*\.\(h\|c\|hpp\|cpp\)' -type f ! -path "./cmsis/*" ! -path "./features/mbedtls/*" ! -path "./features/FEATURE_LWIP/lwip/*" ! -path "./rtos/TARGET_CORTEX/rtx5/RTX/*" ! -path "./BUILD/*" ! -path "./tools/*" | xargs -n 100 -I {} bash -c "astyle --options=.astylerc \"{}\"" > astyle.out
Copy link
Contributor

@geky geky Feb 6, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ARTISTIC_STYLE_OPTIONS=".astylerc" could go in the env section

.astyleignore Outdated
features\FEATURE_BLE\targets\TARGET_Maxim\exactLE
features\FEATURE_BLE\targets\TARGET_NORDIC\TARGET_MCU_NRF51822\sdk\
features\FEATURE_LWIP\lwip-interface\lwip
features\unsupported\
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are your slashes backwards?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

.astylerc Outdated
attach-classes

# Folders to be excluded
# exclude=cmsis
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will remove

@0xc0170
Copy link
Contributor Author

0xc0170 commented Feb 7, 2018

I sent docs update to cover this addition and automatic check - ARMmbed/mbed-os-5-docs#400

Thanks @geky for help, the command works locally, and files changed with astyle are only those we care about (ignorefile is working) !

Once travis finishes, we review all changes and I can add a new commits here fixing the code until we make AStyle green

@0xc0170
Copy link
Contributor Author

0xc0170 commented Feb 8, 2018

The last travis run works with astyle, we can see all the styling issues now displayed t here. I'll add commits to fix those issues to make travis green

@0xc0170 0xc0170 force-pushed the dev_astyle branch 2 times, most recently from 8b1e16c to bc0091b Compare February 8, 2018 16:25
touched = true;
TEST_ASSERT_EQUAL(a0 | a1, 0x3);
}

void func1(int a0) {
void func1(int a0)
{
Copy link
Contributor

@geky geky Feb 8, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was curious about this so I tried astyle with both brace styles:

with style=kr:      1841070 line changes
with style=attach:  1819522 line changes

It would actually take fewer code changes to change the existing KR braces than to standardize on KR. We should reconsider our stance on brace placement.

I've never understood why our brace style is inconsistent between functions and every other syntactic construct.

Copy link
Contributor Author

@0xc0170 0xc0170 Feb 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would actually take fewer code changes to change the existing KR braces than to standardize on KR. We should reconsider our stance on brace placement.

I have testing online compiler - we need to match what we have here and what online compiler does. It seems it defines a rule to actually attach { in the header files, will run few more tests and get its options that it defines (only for C++ functions), AStyle has this option (attach-inlines) that seems to be active in the online compiler

The line 59 is correct, what we need to align is C++ inlines. From experience, not always it was clear when to attach { for users thus our initial coding style set it always new line for functions (no matter if C++ or C). This however as we can see it is not aligned with online compiler, will check

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated this PR, attached C++ as the online compiler does.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This contradict the guidelines and guidelines are made to be followed; the purpose of this PR is to enforce them. If the online compiler is wrong; fix the online compiler. Not the other way around.

Copy link
Contributor

@geky geky Feb 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

KR style braces are still the minority, although only slightly:

with style=kr:     1819759 line changes
with style=attach: 1819522 line changes

I'm suggesting we change the guidelines to match the style of code users are contributing. I had no idea C++ inlines are a separate style, and if we don't know what our rules are on braces how are our users supposed to?

I mean what is this:

namespace hi {

class Thing {
    int dothing() {
        if (thing) {
            blahblahblah;
        } else {
            blahblahblah;
        }
    }
};

int dothing()
{
    while (thing) {
        blahblahblah;
    }
}

}

What's the reason for not making squiggly brackets consistent?

@kjbracey
Copy link
Contributor

kjbracey commented Feb 9, 2018

Nanostack and the related "COMMON_PAL" code are subtrees from master external repos - I think it would be less disruptive to format them over there then merge, initially. Incoming subtree merges can certainly be checked for formatting.

@0xc0170
Copy link
Contributor Author

0xc0170 commented Feb 9, 2018

Nanostack and the related "COMMON_PAL" code are subtrees from master external repos - I think it would be less disruptive to format them over there then merge, initially. Incoming subtree merges can certainly be checked for formatting.

I'll remove nanostack and common pal here

@kjbracey
Copy link
Contributor

kjbracey commented Feb 9, 2018

To be a little more precise - these are from out-of-tree:

  • FEATURE_NANOSTACK/coap-service
  • FEATURE_NANOSTACK/sal-stack-nanostack
  • FEATURE_COMMON_PAL/(almost everything)

These are mbed OS specific components, mastered in-tree and can be reformatted:

  • FEATURE_COMMON_PAL/nanostack-hal-mbed-cmsis-rtos
  • FEATURE_NANOSTACK/mbed-mesh-api
  • FEATURE_NANOSTACK/nanostack-interface
  • FEATURE_NANOSTACK/targets

@0xc0170
Copy link
Contributor Author

0xc0170 commented Feb 9, 2018

Comparing to the online IDE formatter, what we do in the codebase and define even in our code style:

attach-namespaces
attach-classes

What we had earlier defined was --style=kr --indent=spaces=4 --indents-switches.

I rebased this, to clean the history a bit and reapply the coding style.

@geky
Copy link
Contributor

geky commented Mar 14, 2018

image
@0xc0170, looks like it already caught something 😛

Also damn you grammar!

@0xc0170
Copy link
Contributor Author

0xc0170 commented Mar 15, 2018

This is good (showcase how it works when it fails)! I'll rebase to fix the code style issue.

@cmonr
Copy link
Contributor

cmonr commented Mar 21, 2018

@0xc0170 Since this touches a whole bunch of files, I'm thinking this will need to either

  1. come in around 5.9.0-rc1 -and/or-
  2. be synchronized with other PR activity from the next time you rebase to when this PR comes in in relation to other PRs.

It looks like it will be ready after another rebase.

Copy link
Contributor

@cmonr cmonr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excited about the changes.
Not excited with how much PR coordination/fallout this might cause with currently opened PRs.

@0xc0170
Copy link
Contributor Author

0xc0170 commented Mar 21, 2018

come in around 5.9.0-rc1 -and/or-

I fear that this might be the case. However if the style is approved, still this can land partially (astyle check for new code only - so we start slowly adapting the new code - should be fairly simple to update and send via a new PR), and we update the older code for 5.9. Open for suggestions.

I am not rebasing again, so will wait until we finalize the plan

@cmonr
Copy link
Contributor

cmonr commented Mar 21, 2018

@0xc0170 I think the style checks as they are now are fine, and think everyone else is now onboard with them (at least, no one as raised any other concerns with them).

I think we could take the following approach:

  1. Start by adding the astyle check into Travis, but don't have it actually break builds. It should only report warnings, but always report a "Success" status.
  2. Using the list above, cherry pick this PR's commits until the need to rebase goes away. This should help get us get a bulk of the way there without needing to rebase and update the commits. The caveat with this is that PRs that are currently open will need to be rebased once this step is completed (most likely).
  3. Add an astyle commit one by one, until the rest of the codebase is up to date.

(In case it wasn't clear, each of those above steps would be an individual PR. #3 would involve multiple PRs.)

@geky
Copy link
Contributor

geky commented Mar 21, 2018

+1 for the astyle travis check as a warning until 5.9!

@screamerbg
Copy link
Contributor

screamerbg commented Mar 27, 2018

@0xc0170 +1 for the idea and the energy put in this.

To make this less disruptive I think that we need to:

  1. enable astyle check as info (not warnings nor errors)
  2. get OS code piece by piece aligned/updated with astyle (not as one mindblowing PR)
  3. finalize OS code transition to astyle to a point where there are no astyle issues
  4. once complete, get alignment with Mbed Partner teams and expand astyle to their HAL ports
  5. switch from info to warnings
  6. beyond mbed OS 5.9 switch to errors

Copy link
Contributor

@screamerbg screamerbg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my previous comment how to make this less disruptive

@0xc0170
Copy link
Contributor Author

0xc0170 commented Apr 10, 2018

This serves as an example for astyle to see all the code that is affected. I am preparing smaller one that is only adding travis and astyle config. It will check only files that has changed, once I have it running will reference here and close this one.

@0xc0170
Copy link
Contributor Author

0xc0170 commented Apr 11, 2018

enable astyle check as info (not warnings nor errors)

This should do it #6590 . Once this one is in, I'll start with the code base one by one

I am closing this for now

@0xc0170 0xc0170 closed this Apr 11, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants