[Exporters] New export-build tests #3172

sarahmarshy · 2016-10-31T21:09:18Z

Description

Adds additional export-build tests that use subprocess commands mbed import and mbed export. Additionally, I have revised how exporters store supported targets, and, as a result, how project.py handles unsupported targets. I did this to make supported targets more visible for the new tests.

Status

READY

Todos

@bridadan review
@sg- review
@theotherjimmy - review

Deploy notes

Build tests of any kind require the IDEs executable to be in the path.

Steps to test or reproduce

In a clean subdirectory of mbed-os\.build
python ..\..\tools\test\examples\examples.py import
python ..\..\tools\test\examples\examples.py export iar uvision make_armc5 make_iar make_gcc_arm

sarahmarshy · 2016-10-31T21:10:43Z

/morph export-build

bridadan · 2016-10-31T21:21:59Z

tools/export/makefile/__init__.py

@@ -102,6 +105,38 @@ def generate(self):
        else:
            raise NotSupportedException("This make tool is in development")

+    @staticmethod
+    def build(project_name, build_log="build_log.txt", project_loc=None, clean=True):


Can you remove the project_loc parameter here? Doesn't look like it's used.

Also, can build_log be renamed to log_name to match uvision's build function?

bridadan · 2016-10-31T21:32:19Z

tools/export/iar/__init__.py

@@ -120,22 +121,13 @@ def generate(self):
        self.gen_file('iar/ewd.tmpl', ctx, self.project_name + ".ewd")
        self.gen_file(self.get_ewp_template(), ctx, self.project_name + ".ewp")

-    def build(self):
+    @staticmethod
+    def build(project_name, clean=True):


The rest of our APIs use the parameter clean to mean "delete a bunch of stuff before doing the thing I asked you to do". But in this case, it means "delete all the stuff you created when you're done doing the thing I asked you to do".

For this reason, maybe we should call this parameter cleanup instead of clean?

mbed-bot · 2016-10-31T22:28:49Z

Result: FAILURE

Your command has finished executing! Here's what you wrote!

/morph export-build

bridadan · 2016-10-31T23:51:09Z

/morph export-build

mbed-bot · 2016-11-01T00:55:46Z

Result: SUCCESS

Your command has finished executing! Here's what you wrote!

/morph export-build

bridadan · 2016-11-01T04:17:06Z

@sarahmarshy There's a change in here that I only noticed after CI ran. There are targets now I believe that don't have CMSIS packs but do have IAR definitions (the NCS36510 being one of them). This target was skipped in the above CI run. Can we change the supported check for IAR so that if the device_name appears in the iar_definitions.json it is considered supported? Is there any time where this would not be the case?

0xc0170 · 2016-11-01T10:15:21Z

tools/export/cmsis/__init__.py

@@ -30,38 +30,40 @@ class DeviceCMSIS():
    """CMSIS Device class

    Encapsulates target information retrieved by arm-pack-manager"""
+    cache = Cache(True, False)


Nit: Isn't the style in our code base that classes members are in capital letters, CACHE ? I recall its in most places

0xc0170 · 2016-11-01T10:26:19Z

tools/export/makefile/__init__.py

+                shutil.rmtree('.build')
+        if ret_code != 0:
+            # Seems like something went wrong.
+            return -1


Is this documented, now that we have return values, that -1 is an error, 0 succcess ? I can't find it in this changeset. Previously build function raised an exception, no return value.

0xc0170

LGTM

sarahmarshy · 2016-11-01T15:52:26Z

@bridadan, all targets rely on CMSIS pack to work out what the debugger should be. For IAR, it uses it to find the vendor of the device, as debugger is set per vendor. @sg- is this correct? Or should vendors also be stored in targets.json?

bridadan

Thanks for the explanation @sarahmarshy, as long as we're clear on what is necessary to have exporter support then I'm happy. Any and all additional docs on the CMSIS pack dependency would be good too :)

bridadan · 2016-11-01T17:27:30Z

/morph export-build

sarahmarshy · 2016-11-01T19:53:21Z

@bridadan #3182 :)

Allows command line workflow of: mbed import mbed export Also revises exporter supported target checks

…wargs

…son for each target).

…lasses.

sarahmarshy · 2016-11-01T20:56:12Z

/morph export-build

0xc0170 · 2016-11-02T11:11:33Z

Thanks for addressing the docs API comment . I dont understand the commit - Rebase export-tests on examples.py changes - 0a4171a . If it's rebase, it should not display any changes, but it shows lot of changes incorporated without any description.

mbed-bot · 2016-11-02T14:16:45Z

Result: ABORTED

Your command has finished executing! Here's what you wrote!

/morph export-build

sarahmarshy · 2016-11-02T14:38:08Z

@0xc0170 #3048 made a lot of changes. I needed to rebase after that was merged. Consequently, I changed my tests to match the style of that PR. Not sure what a good commit message would be. Maybe I can squash all of this into one commit.

bridadan · 2016-11-02T14:49:35Z

@sarahmarshy You could make the message give details about adding the exporters and export flag to the examples.json file. Probably don't want to make this one big commit if you can help it since it's quite a bit of changes.

mbed-bot · 2016-11-02T15:20:18Z

Result: SUCCESS

Your command has finished executing! Here's what you wrote!

/morph export-build

This will determine whether the example should be exported. Additionally, relocated export logic to examples_lib.py.

Release mbed OS 5.2.2 and mbed lib v129 Known issues in this release There is currently a DNS resolution failure in Thread mode with this release. This causes a failure in the mbed-os-example-client. This will be fixed in a subsequent release. This can be worked around by reverting to mbed-os-5.2.0 Ports for Upcoming Targets 3011: Add u-blox Sara-N target. ARMmbed#3011 3099: MAX32625 ARMmbed#3099 3151: Add support for FRDM-K82F ARMmbed#3151 3177: New mcu k22512 fixing pr 3136 ARMmbed#3177 Fixes and Changes 2990: [tools] Parallel building of tests ARMmbed#2990 3008: NUCLEO_F072RB: Fix wrong timer channel number on pwm PB_5 pin ARMmbed#3008 3013: STM32xx - Change how the ADC internal pins are checked before pinmap_ ARMmbed#3013 3023: digital_loop tests update for STM32 ARMmbed#3023 3041: [nRF5] - added implementation of API of serial port flow control configuration. ARMmbed#3041 3092: [tools + tests] Adding parallelized build option for iar and uvision exporters ARMmbed#3092 3084: [nrf5] fix in Digital I/O : a gpioe pin was uninitialized badly ARMmbed#3084 3009: TRNG enabled. TRNG APIs implemented. REV A/B/C/D flags removed. Warnings removed ARMmbed#3009 3139: Handle [NOT_SUPPORTED] exception in make.py ARMmbed#3139 3074: Target stm init gcc alignement ARMmbed#3074 3140: [tests] Replacing getchar with RawSerial getc in greentea-client ARMmbed#3140 3158: Added support for 6lowpan PAN ID filter to mbed mesh api configuration ARMmbed#3158 2988: Update of can_api.c fixing ARMmbed#2987 ARMmbed#2988 3175: Updating IAR definition for the NCS36510 for IAR EW v7.8 ARMmbed#3175 3170: [tests] Preventing test from printing before Greentea __sync ARMmbed#3170 3169: [Update of ARMmbed#3014] Usb updates ARMmbed#3169 3143: CFStore fix needed for the Cloud Client ARMmbed#3143 3135: lwip - Fix memory leak in k64f cyclic-buffer overflow ARMmbed#3135 3048: Make update.py test compile examples prior to updating mbed-os version. ARMmbed#3048 3162: lwip/nsapi - Clean up warnings in network code ARMmbed#3162 3161: nsapi - Add better heuristic for the default record of DNS queries ARMmbed#3161 3173: [Exporters] Add a device_name to microbit entry in targets.json ARMmbed#3173 3072: i2c_loop tests update for STM32 ARMmbed#3072 2958: Allowing mbed_app.json files to be discovered for tests. ARMmbed#2958 2969: [nRF52] - switch irq priorities of driver handlers to the lowest level ARMmbed#2969 3078: lwip: Allow several configuration macros to be set externally (bis) ARMmbed#3078 3165: Add address type checks to NanostackInterface ARMmbed#3165 3166: nsapi_dns: Provide 2 IPv6-hosted default servers ARMmbed#3166 3171: [tools] Fixing project.py -S printing problem ARMmbed#3171 3172: [Exporters] New export-build tests ARMmbed#3172 3184: ARMmbed#3183 Compiler warning in trng_api.c with K64F ARMmbed#3184 3185: Update tests to fix build failures. Also make the code similar to oth ARMmbed#3185 3104: [NuMaker] Support CAN and fix PWM CLK error ARMmbed#3104 3182: Exporter documentation ARMmbed#3182 3186: MultiTech mDot - add back SPI3 pins ARMmbed#3186 3187: [Export-Make] Use internal class variable for resolving templates in makefiles ARMmbed#3187 3195: [Exporters - Make-based] Quote the shell call in mkdir and rmdir ARMmbed#3195 3204: [Export build-test] Directory traversal error ARMmbed#3204 3189: [Exporters - Make-based] Force make exporter to search PATH for compilers ARMmbed#3189 3200: Using Popen for uVision and unifying the structure of the build function ARMmbed#3200 3075: nsapi - Add standardized return types for size and errors ARMmbed#3075 3221: u-blox odin w2 drivers update ARMmbed#3221

sarahmarshy force-pushed the export-test-rev branch from e82fc41 to a3b8974 Compare October 31, 2016 21:15

bridadan reviewed Oct 31, 2016

View reviewed changes

sarahmarshy force-pushed the export-test-rev branch from a4cacc3 to d7e0d28 Compare October 31, 2016 21:58

sarahmarshy force-pushed the export-test-rev branch from 0f52f21 to 407e893 Compare October 31, 2016 22:43

sarahmarshy changed the title ~~New export-build tests.~~ [Exporters] New export-build tests Oct 31, 2016

This was referenced Oct 31, 2016

[Exporters] Add a device_name to microbit entry in targets.json #3173

Merged

Confusing "Target not supported" errors from absent device_name field in targets.json #3174

Closed

bridadan mentioned this pull request Oct 31, 2016

[Exporters][morph export-build] Adds build tests for make exporters and suppresses TargetNotSupported traceback #3134

Closed

2 tasks

0xc0170 reviewed Nov 1, 2016

View reviewed changes

0xc0170 approved these changes Nov 1, 2016

View reviewed changes

sg- added the needs: work label Nov 1, 2016

bridadan approved these changes Nov 1, 2016

View reviewed changes

sarahmarshy added 4 commits November 1, 2016 15:03

New export-build tests.

986af0d

Allows command line workflow of: mbed import mbed export Also revises exporter supported target checks

[Exporter tests] changing variable names and respecting requirement k…

b9bac05

…wargs

Make ArmPackManager cache a static variable (to avoid reading index.j…

d8fc362

…son for each target).

[Exporters] Build method documentation. Removal of unused Exception c…

cc154a4

…lasses.

sarahmarshy force-pushed the export-test-rev branch from cbc8d6e to 0a4171a Compare November 1, 2016 20:47

sarahmarshy force-pushed the export-test-rev branch from 0a4171a to d71ac0e Compare November 2, 2016 15:30

Add boolean 'export' to examples.json.

9624ccf

This will determine whether the example should be exported. Additionally, relocated export logic to examples_lib.py.

sarahmarshy force-pushed the export-test-rev branch from d71ac0e to 9624ccf Compare November 2, 2016 15:30

bridadan added needs: review and removed needs: work labels Nov 2, 2016

theotherjimmy approved these changes Nov 2, 2016

View reviewed changes

bridadan added ready for merge and removed needs: review labels Nov 2, 2016

sg- merged commit 2502a79 into ARMmbed:master Nov 3, 2016

sg- added release-version: 5.2.2 and removed ready for merge labels Nov 3, 2016

bridadan mentioned this pull request Nov 3, 2016

[Export-Make] Use internal class variable for resolving templates in makefiles #3187

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Exporters] New export-build tests #3172

[Exporters] New export-build tests #3172

sarahmarshy commented Oct 31, 2016 •

edited by sg-

Loading

sarahmarshy commented Oct 31, 2016

bridadan Oct 31, 2016 •

edited

Loading

bridadan Oct 31, 2016

mbed-bot commented Oct 31, 2016

bridadan commented Oct 31, 2016

mbed-bot commented Nov 1, 2016

bridadan commented Nov 1, 2016

0xc0170 Nov 1, 2016 •

edited

Loading

0xc0170 Nov 1, 2016

0xc0170 left a comment

sarahmarshy commented Nov 1, 2016 •

edited

Loading

bridadan left a comment

bridadan commented Nov 1, 2016

sarahmarshy commented Nov 1, 2016

sarahmarshy commented Nov 1, 2016

0xc0170 commented Nov 2, 2016

mbed-bot commented Nov 2, 2016

sarahmarshy commented Nov 2, 2016

bridadan commented Nov 2, 2016

mbed-bot commented Nov 2, 2016

[Exporters] New export-build tests #3172

[Exporters] New export-build tests #3172

Conversation

sarahmarshy commented Oct 31, 2016 • edited by sg- Loading

Description

Status

Todos

Deploy notes

Steps to test or reproduce

sarahmarshy commented Oct 31, 2016

bridadan Oct 31, 2016 • edited Loading

Choose a reason for hiding this comment

bridadan Oct 31, 2016

Choose a reason for hiding this comment

mbed-bot commented Oct 31, 2016

Result: FAILURE

bridadan commented Oct 31, 2016

mbed-bot commented Nov 1, 2016

Result: SUCCESS

bridadan commented Nov 1, 2016

0xc0170 Nov 1, 2016 • edited Loading

Choose a reason for hiding this comment

0xc0170 Nov 1, 2016

Choose a reason for hiding this comment

0xc0170 left a comment

Choose a reason for hiding this comment

sarahmarshy commented Nov 1, 2016 • edited Loading

bridadan left a comment

Choose a reason for hiding this comment

bridadan commented Nov 1, 2016

sarahmarshy commented Nov 1, 2016

sarahmarshy commented Nov 1, 2016

0xc0170 commented Nov 2, 2016

mbed-bot commented Nov 2, 2016

Result: ABORTED

sarahmarshy commented Nov 2, 2016

bridadan commented Nov 2, 2016

mbed-bot commented Nov 2, 2016

Result: SUCCESS

sarahmarshy commented Oct 31, 2016 •

edited by sg-

Loading

bridadan Oct 31, 2016 •

edited

Loading

0xc0170 Nov 1, 2016 •

edited

Loading

sarahmarshy commented Nov 1, 2016 •

edited

Loading