Skip to content
This repository has been archived by the owner on Apr 24, 2019. It is now read-only.

mbed-os-example-client does not work on STM Nucleo F401RE #37

Closed
markus-becker-tridonic-com opened this issue Jul 8, 2016 · 41 comments
Closed

Comments

@markus-becker-tridonic-com

I am trying to get the mbed-os-example-client to run on an STM Nucleo F401RE with the 6LoWPAN shield and the configuration.

In order to do so, I modified

diff --git a/mbed_app.json b/mbed_app.json
index 48fbd8d..f4f737b 100644
--- a/mbed_app.json
+++ b/mbed_app.json
@@ -2,7 +2,7 @@
     "config": {
         "network-interface":{
             "help": "options are ETHERNET,WIFI,MESH_LOWPAN_ND,MESH_THREAD",
-            "value": "ETHERNET"
+            "value": "MESH_LOWPAN_ND"
         },
         "wifi-ssid": {
             "help": "WiFi SSID",
@@ -15,9 +15,10 @@
     },
     "target_overrides": {
         "*": {
-            "target.features_add": ["IPV6", "CLIENT", "IPV4"],
+            "target.features_add": ["IPV6", "CLIENT"],
             "mbed-client.reconnection-count": 3,
-            "mbed-client.reconnection-interval": 5
+            "mbed-client.reconnection-interval": 5,
+            "nanostack.configuration": "lowpan_router"
         }
     }
-}
+}

and

diff --git a/tools/toolchains/gcc.py b/tools/toolchains/gcc.py
index 027a956..27dc11f 100644
--- a/tools/toolchains/gcc.py
+++ b/tools/toolchains/gcc.py
@@ -86,7 +86,7 @@ class GCC(mbedToolchain):
             self.flags["common"].append("-g")
             self.flags["common"].append("-O0")
         else:
-            self.flags["common"].append("-O2")
+            self.flags["common"].append("-Os")

         main_cc = join(tool_path, "arm-none-eabi-gcc")
         main_cppc = join(tool_path, "arm-none-eabi-g++")

so that it actually fits into ROM.

On the K64:
mbed compile -c -t GCC_ARM -m K64F -j 4

Starting mbed Client example...
Using Mesh
Connecting to Mesh..
Connected to Network successfully
IP address ....
SOCKET_MODE : UDP
Connecting to coap://[...
Registered object successfully!

On the F401RE:
mbed compile -c -t GCC_ARM -m NUCLEO_F401RE

Starting mbed Client example...
Using Mesh
Connecting to Mesh..
Connection to Network Failed -3009! Exiting application....
@markus-becker-tridonic-com
Copy link
Author

@mlnx @bcostm

@yogpan01
Copy link
Contributor

yogpan01 commented Jul 8, 2016

@SeppoTakalo @kjbracey-arm Can you please check this ?

@kjbracey
Copy link
Contributor

kjbracey commented Jul 8, 2016

Can we enable some trace?

@MarceloSalazar
Copy link
Contributor

It seems tracing is not working as expected (ticket here ARMmbed/mbed-os#449)
Could we please help to investigate the issue?

@markus-becker-tridonic-com
Copy link
Author

For which module should I enable tracing?

@kjbracey
Copy link
Contributor

kjbracey commented Jul 8, 2016

Simplest thing to do is add mbed-trace.enable to your mbed_app.json, in this section:

"target_overrides": {
    "*": {
        "mbed-trace.enable": 1
    }
}

@markus-becker-tridonic-com
Copy link
Author

Debug output on F401RE:

<\n>
<\r>Connection to Network Failed -3009! Exiting application....<\r><\n>
<\0>Starting mbed Client example...<\r><\n>
Using Mesh<\r><\n>
<\n>
<\r>Connecting to Mesh..<\r><\n>
<\r><27>[2K<27>[90m[DBG ][core]: Allocate Root Tasklet<27>[0m<\r><\n>
<\r><27>[2K<27>[90m[DBG ][6lo ]: P.Init<\n>
<27>[0m<\r><\n>
<\r><27>[2K<27>[90m[DBG ][m6La]: init()<27>[0m<\r><\n>
<\r><27>[2K<27>[90m[DBG ][m6La]: connect()<27>[0m<\r><\n>
<\n>
<\r>Connection to Network Failed -3009! Exiting application....<\r><\n>

@kjbracey
Copy link
Contributor

kjbracey commented Jul 8, 2016

Did you clean after changing the config? (There's no autodependency on it). I'm expecting trace from the network stack.

@markus-becker-tridonic-com
Copy link
Author

I assume so. My commandline was mbed compile -c -t GCC_ARM -m NUCLEO_F401RE -j 4.

@markus-becker-tridonic-com
Copy link
Author

Somehow I suspect that the nd_tasklet_main does not start.

@kjbracey
Copy link
Contributor

kjbracey commented Jul 8, 2016

Seems likely, as you cleaned. It didn't manage to start the stack at all.

@markus-becker-tridonic-com
Copy link
Author

Any hint on what could be going wrong?

@kjbracey
Copy link
Contributor

kjbracey commented Jul 8, 2016

Got to go now, but perhaps lack of memory for the Nanostack (nsdynmemlib) pool? Insufficent threads configured in RTX (I don't think error returns from thread creation are checked)?

@markus-becker-tridonic-com
Copy link
Author

Any hints on what one can tweak and where?

@SeppoTakalo
Copy link
Contributor

Way back, I recall seeing timing problems with atmel-rf-driver.
I don't think we have tested with ST board for a while.

In order to verify actually what happened, we need to check with RF sniffer if the device is able to send anything at all.

There is also assumption on the driver that 'CS' release required specific timeouts, not sure if these are true on every platform.
Please see: https://github.com/ARMmbed/atmel-rf-driver/blob/master/source/driverAtmelRFInterface.cpp#L175

@markus-becker-tridonic-com
Copy link
Author

@SeppoTakalo Previously with mbed3 this was working on an ST board. Are there changes from mbed3 to mbed5 that could break it?

@kjbracey-arm Where could I start looking for nsdynmemlib and Insufficent threads? Highly likely that this is the source since the ST board has less RAM.

@markus-becker-tridonic-com
Copy link
Author

Seems that the F401RE has "default_build": "small" in targets.json. This sets MBED_RTOS_SINGLE_THREAD which does not allow to start any threads. Shouldn't the inability to spawn new threads be caught? Does default_build small has other side effects?

When removing it, the example does not fit anymore into ROM. What has increased the ROM so much since mbed3?

@ciarmcom
Copy link
Member

ciarmcom commented Aug 1, 2016

ARM Internal Ref: IOTCLT-879

@MarceloSalazar
Copy link
Contributor

problem also reported during OOB RC#4

@bogdanm
Copy link
Contributor

bogdanm commented Aug 2, 2016

Should be fixed by #58, needs verify.

@sg-
Copy link
Contributor

sg- commented Aug 3, 2016

What has increased the ROM so much since mbed3?

@markus-becker-tridonic-com using newlib which is thread safe versus newlib-nano which isn't. Its about a 20k ROM penalty with GCC

@markus-becker-tridonic-com
Copy link
Author

Investigated more. mbedtls is a lot smaller now. Thanks.

The ETHERNET build is size-wise fine:
272724 bytes for K64F/GCC_ARM.

The MESH_LOWPAN_ND build is still big:
484478 bytes for NUCLEO_F401RE/GCC_ARM
484478 bytes for K64F/GCC_ARM

Now with the preparations for the public release, it seems you have put in the binary nanostack, with every option enabled (Thread + 6LoWPAN + BorderRouter?).

Is there a possibility to have a Thread Router, 6LoWPAN Router or Border Router nanostack binary and being able to select with e.g. NANOSTACK_BINARY_USER_CONFIG? If not, what is the supported way of putting in the nanostack source with the correct config selectable from the config NANOSTACK_USER_CONFIG_FILE like mbedtls does now with #58?

@sg-
Copy link
Contributor

sg- commented Aug 5, 2016

@javier-moreno-tridonic-com
Copy link

javier-moreno-tridonic-com commented Aug 9, 2016

@sg Thanks for the pointer!

Cloning sal-stack-nanostack-private to replace sal-stack-nanostack doesn't work without any tweaking (compilation gives a bunch of errors about type redefinitions, for instance). I'm trying to sort it out but I was wondering: Is there any specific branch/tag that we should use with the current mbed-os release? (so far I've tried master and the v5.0.5 tag)

@markus-becker-tridonic-com
Copy link
Author

@MarceloSalazar would you have a look at @javier-moreno-tridonic-com's comment?

Do we need to clone all of

  • mbed-mesh-api
  • mbed-trace
  • nanostack-hal-mbed-cmsis-rtos
  • nanostack-libservice
  • sal-stack-nanostack-eventloop
  • sal-stack-nanostack-private

Or is just sal-stack-nanostack-private enough? Which tags/versions should work?

@hasnainvirk
Copy link
Contributor

@markus-becker-tridonic-com @javier-moreno-tridonic-com

I just tried 5.0.5 and everything built smoothly. The version of Nanostack included in mbed-os is 5.0.5. It was checked out from branch 5.0-maint. Can you please have a Skype session session with me ?
You don't need to clone anyting else. Just clone nanostack and checkout either v5.0.5 or 5.0-maint.
I tried with lowpan_router config:

+-----------------------------+--------+-------+-------+
| Module | .text | .data | .bss |
+-----------------------------+--------+-------+-------+
| Fill | 692 | 26 | 83 |
| Misc | 84121 | 2377 | 2936 |
| features/FEATURE_CLIENT | 66627 | 11 | 53 |
| features/FEATURE_COMMON_PAL | 20538 | 93 | 8404 |
| features/frameworks | 3815 | 92 | 848 |
| features/mbedtls | 75930 | 51 | 8755 |
| features/net | 142494 | 302 | 34053 |
| hal/common | 5657 | 20 | 365 |
| hal/targets | 10739 | 4 | 1472 |
| rtos/rtos | 451 | 4 | 0 |
| rtos/rtx | 7037 | 20 | 2683 |
| Subtotals | 418101 | 3000 | 59652 |
+-----------------------------+--------+-------+-------+
Allocated Heap: unknown
Allocated Stack: unknown
Total Static RAM memory (data + bss): 62652 bytes
Total RAM memory (data + bss + heap + stack): 62652 bytes
Total Flash memory (text + data + misc): 421101 bytes

@hasnainvirk
Copy link
Contributor

Had a Skype session with @javier-moreno-tridonic-com . The problem was the same file names on the include path. Its a known-issue on mbed-os5.
When you have nanostack binaries you will not encounter this issue. However when you use sources, there are files like socket.h and thread.h whereas mbed-OS will bring in Socket.h and Thread.h. On unix based syatems you would still get away as they are case sensitive. Thread.h (mbed-os) and thread.h(nanostack) will be treated differently on unix. However, windows is case insensitive. Anyone building on windows with nanostack sources will encounter these issues.
Conversely speaking, anyone adding a custom library with colliding names with any other module in the mbed-OS will suffer the saimilar fate.
cc @markus-becker-tridonic-com

@javier-moreno-tridonic-com
Copy link

javier-moreno-tridonic-com commented Aug 10, 2016

I opened a new issue for better tracking.

For completion, the compilation errors can be fixed in windows by replacing the following lines:


mbed-os

File
mbed-os\features\FEATURE_CLIENT\mbed-client-classic\mbed-client-classic\m2mconnectionhandlerpimpl.h

Replace

#include "Socket.h"

With

#include "network-socket/Socket.h"

atmel-rf-driver

File
source\driverAtmelRFInterface.cpp

Replace

#include "Mutex.h"
#include "Thread.h"
using namespace rtos;

With

#include "rtos.h"

@javier-moreno-tridonic-com

New addition to this target issue. Using GCC is running great for us now. Today I tried to use the same code but compiled with armcc.

The example crashes with a HardFault in osEventObs->pre_start();

mbed-os/rtos/rtx/TARGET_CORTEX_M/rt_CMSIS.c

osStatus osKernelStart (void) {
  uint32_t stack[8];

  if (__get_IPSR() != 0U) {
    return osErrorISR;                          // Not allowed in ISR
  }

  /* Call the pre-start event (from unprivileged mode) if the handler exists
   * and the kernel is not running. */
  /* FIXME osEventObs needs to be readable but not writable from unprivileged
   * code. */
  if (!osKernelRunning() && osEventObs && osEventObs->pre_start) {
    osEventObs->pre_start(); //LINE CAUSING THE CRASH
  }

  switch (__get_CONTROL() & 0x03U) {
    case 0x00U:                                 // Privileged Thread mode & MSP
      __set_PSP((uint32_t)(stack + 8));         // Initial PSP
      if (os_flags & 1U) {                       
        __set_CONTROL(0x02U);                   // Set Privileged Thread mode & PSP
      } else {
        __set_CONTROL(0x03U);                   // Set Unprivileged Thread mode & PSP
      }
      __DSB();
      __ISB();
      break;
    case 0x01U:                                 // Unprivileged Thread mode & MSP
      return osErrorOS;
    case 0x02U:                                 // Privileged Thread mode & PSP
      if ((os_flags & 1U) == 0U) {              // Unprivileged Thread mode requested
        __set_CONTROL(0x03U);                   // Set Unprivileged Thread mode & PSP
        __DSB();
        __ISB();
      }
      break;
    case 0x03U:                                 // Unprivileged Thread mode & PSP
      if  (os_flags & 1U) { return osErrorOS; } // Privileged Thread mode requested
      break;
  }
  return __svcKernelStart();
}

Stack trace

mbed-os-example-client.elf

Thread #1 (Suspended : Signal : SIGINT:Interrupt)

HardFault_Handler() at startup_stm32f401xe.S:186 0x800051e  
<signal handler called>() at 0xfffffff9 
0x51520 
osKernelStart() at rt_CMSIS.c:572 0x8040a58 
__rt_entry() at RTX_CM_lib.h:689 0x8000338  

@javier-moreno-tridonic-com

Any updates on this?

@yogpan01
Copy link
Contributor

yogpan01 commented Sep 1, 2016

@SeppoTakalo @hasnainvirk Any progress on this issue ?

@hasnainvirk
Copy link
Contributor

@javier-moreno-tridonic-com which version of armcc are you using ?

@javier-moreno-tridonic-com

@hasnainvirk

This one

Product: MDK-ARM Standard Cortex-M only 5.20 (Flex)
Component: ARM Compiler 5.06 update 2 (build 183)
Tool: armcc [4d35cd]

@hasnainvirk
Copy link
Contributor

@yogpan01 , @SeppoTakalo

Seemingly 96K RAM is not enough for mbed-os-example-client (with 6LoWPAN-ND) to work well.

i) With GCC, RAM gets consumed while mbedTLS starts writing ecp encryption tables in RAM. This is the error code MBEDTLS_ERR_SSL_ALLOC_FAILED -32512 (-0x7F00). But the system does not crash because with GCC there is a sane mechanism of Heap and stack overlap control. Anyway, app does not connect to mDS successfully because right after handshake, while writing some ecp stuff , memory gets exhausted.

ii) With armcc, RAM gets consumed just like th eprevious case. But there is no sane mechanism to keep check on heap and stack overlap. And hence you get the Hard fault.

An issue for sane handling of heap and stack collision is made on mbed OS ARMmbed/mbed-os#2618.

However, we should now look how to reduce RAM consumption ? ROM Tables @SeppoTakalo @kjbracey-arm . Moreover, it seems we are low on ROM too as this platform have 512K of Flash.

@hasnainvirk
Copy link
Contributor

@javier-moreno-tridonic-com Our Client Lead suggested a quick fix, https://github.com/ARMmbed/mbed-os-example-client/blob/master/mbedtls_mbed_client_config.h#L108
Change the undef to define , it will save 8K RAM for you at the expense of Flash. I tested it with ARMCC and GCC.

@javier-moreno-tridonic-com

Good news, I'll try that ASAP.

On the meantime I made a test to check the RAM usage. To do so I tried modifying the startup_stm32f401xe.S to fill the RAM with a pattern.

; Reset handler
Reset_Handler    PROC
                 EXPORT  Reset_Handler             [WEAK]
        IMPORT  SystemInit
        IMPORT  __main

                 LDR     R0, =0x20000000  // lowest RAM position
                 LDR     R1, =0x20018000  // highest RAM position F401
                 SUBS    R1, R0           // RAM size
                 LDR     R2, =0xEFBEADDE  // fill pattern (DEADBEEF) in the dump file

FillRam          STR     R2, [R0]
                 ADDS    R0, R0, #4
                 SUBS    R1, #4
                 BNE     FillRam

                 LDR     R0, =SystemInit
                 BLX     R0
                 LDR     R0, =__main
                 BX      R0
                 ENDP

I dumped RAM contents on crash and used the following script to check how many 4 Bytes are different from this "DEADBEEF" pattern:

import argparse
import re

def main():
    parser = argparse.ArgumentParser(description='Counts used RAM from a RAM dump as plain text')
    parser.add_argument('-f', '--dumpfile', type=str, help='The dump file')
    args = parser.parse_args()

    dumpfile = open(args.dumpfile, 'rb')
    print dumpfile

    tokens = re.split(' |\n', dumpfile.read())
    usedram = 0

    print('Procesing ' + str(len(tokens)) + ' tokens...')
    for token in tokens:
        if token != 'DEADBEEF' and token != '':
            usedram += 4

    print('Used RAM: ' + str(usedram) + ' Bytes')

if __name__ == "__main__":
    main()

This was the result:

<open file 'F4RAMarmcc.dump', mode 'rb' at 0x028215A0>
Procesing 24576 tokens...
Used RAM: 61172 Bytes

I looks like on crash time there should be ~30KB free, assuming the startup file from armcc does the same as the one for gcc and fills with zeros the statically allocated variables. If that is not the case and the memory is not zeroed, it might be that the heap for the nanostack is taking those 30KB.

Please keep in mind that is the first time that I fiddle with this kind of assembler so there could be errors, I'd be happy if somebody could review this carefully

@hasnainvirk
Copy link
Contributor

@javier-moreno-tridonic-com It seems that the hard fault issue because of RAM exhaustion with ARMCC is fixed in mbed-o-5.1.3

@TuomoHautamaki
Copy link

@javier-moreno-tridonic-com can you close this as the original issue has been solved now. Thank you

@javier-moreno-tridonic-com

@TuomoHautamaki Well today I tried again a clean checkout of the example using mbed-os 5.1.3 and the fix proposed by @hasnainvirk to save 8KB of RAM but the example still crashes with a HardFault. on the same line.

If you mean that @markus-becker-tridonic-com initial problem is fixed I can close this issue and open a new one about the HardFault and reference this one. Is that what you want?

@hasnainvirk
Copy link
Contributor

hasnainvirk commented Sep 9, 2016

@TuomoHautamaki @markus-becker-tridonic-com I had tested it with mbed-OS 5.1.3 and the memory work arounds. It worked fine for me. I just had a Skype session with @javier-moreno-tridonic-com and I demonstrated him the solution. I suspect the OpenOCD tool that Javier is using causes troubles because it uses JTAG. By design Nucleo F401RE uses PB_4(i.e., D5 in terms of Arduino) for JTAG whereas the firefly radio shield is also using D5 for SPI operation. I would suggest a test without using external tools. Use mbed-cli to build and flash like drag and drop. Do not build with debug-info.

@javier-moreno-tridonic-com

I can finally confirm @hasnainvirk theory. Apparently my debug tools cause some sort of problem that end in a hard fault crash.

@markus-becker-tridonic-com can you close this issue???

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests