mbed import and mbed add broken on cygwin

[bmcdonnell-ionx]

In my experience so far, most mbed-cli commands work as well in Cygwin as they do from the Windows command line. The import and add commands don't work, though, as shown below. (I tried a few different ways, to see if there was a workaround.)

Looks like it's just an issue with the way the output directory path is generated.

Issue #299 also relates to paths on cygwin, so be careful not to introduce a regression there when fixing this.

Originally posted on mbed forum: mbed-cli on cygwin.

me@pc /cygdrive/c/dev
$ mbed import https://github.com/ARMmbed/mbed-os-example-blinky
fatal: Invalid path '/cygdrive/c/dev/C:\dev\mbed-os-example-blinky': No such file or directory
abort: HTTP Error 406: Not Acceptable
[mbed] Importing program "mbed-os-example-blinky" from "https://github.com/ARMmbed/mbed-os-example-blinky" at latest revision in the current branch
[mbed] ERROR: Unable to clone repository (https://github.com/ARMmbed/mbed-os-example-blinky)
---
 
me@pc /cygdrive/c/dev
$ mbed import mbed-os-example-blinky ./mbed-os-example-blinky
fatal: Invalid path '/cygdrive/c/dev/C:\dev\mbed-os-example-blinky': No such file or directory
abort: HTTP Error 406: Not Acceptable
[mbed] Importing program "mbed-os-example-blinky" from "https://github.com/ARMmbed/mbed-os-example-blinky" at latest revision in the current branch
[mbed] ERROR: Unable to clone repository (https://github.com/ARMmbed/mbed-os-example-blinky)
---
 
me@pc /cygdrive/c/dev
$ mbed import https://github.com/ARMmbed/mbed-os-example-blinky ./test1
fatal: Invalid path '/cygdrive/c/dev/C:\dev\test1': No such file or directory
abort: HTTP Error 406: Not Acceptable
[mbed] Importing program "test1" from "https://github.com/ARMmbed/mbed-os-example-blinky" at latest revision in the current branch
[mbed] ERROR: Unable to clone repository (https://github.com/ARMmbed/mbed-os-example-blinky)
---

[do import from Windows cmd prompt...]

me@pc /cygdrive/c/dev
$ cd mbed-os-example-blinky/

me@pc /cygdrive/c/dev/mbed-os-example-blinky
$ mbed add sd-driver
fatal: Invalid path '/cygdrive/c/dev/mbed-os-example-blinky/C:\dev\mbed-os-example-blinky\sd-driver': No such file or directory
abort: HTTP Error 406: Not Acceptable
[mbed] Adding library "sd-driver" from "https://github.com/ARMmbed/sd-driver" at latest revision in the current branch
[mbed] ERROR: Unable to clone repository (https://github.com/ARMmbed/sd-driver/)
---

[theotherjimmy]

@bmcdonnell-ionx It looks like you hit an odd corner case there. Did you try the workarounds mentioned in #299 after it was closed?

[bmcdonnell-ionx]

Oh, I hadn't realized that #299 is a "won't fix (for now)".

The workarounds didn't work for this issue, though.

I just tried the work-around by @mukymuk:

sidestep the cygwin path issue by avoiding absolute paths. In mbed-os/tools/settings.py I set ROOT = ".."

But it made no difference.

[EDIT: If it matters, the line in settings.py before I changed it was ROOT = abspath(join(dirname(__file__), "..")).]

@tkstreet said

As an update to @mukymuk, the settings.py doesn't exist anymore. The relevant code is now contained in ../Python27/Lib/site-packages/mbed/mbed.py

...but I do have settings.py, and I do not have /cygdrive/c/Python27/Lib/site-packages/mbed/mbed.py.

[theotherjimmy]

Thanks for looking at that. You can find the executable for Mbed CLI with which mbed, but this may not be very useful.

[bmcdonnell-ionx]

$ which mbed
/cygdrive/c/Python27/Scripts/mbed

Why do you ask?

[theotherjimmy]

I did not ask. I was just trying to help you find your mbed.py. As an alternative, you could clone this repo, modify mbed.py, and do pip install -U . in the root.

[bmcdonnell-ionx]

Oh. The path to mbed returned by which is for mbed.exe, not mbed.py. I was running mbed-cli v1.1.1, though...

I just upgraded to mbed-cli v1.2.2, and I see that I do have /cygdrive/c/Python27/Lib/site-packages/mbed/mbed.py. But what needs to be changed in there?

[theotherjimmy]

@bmcdonnell-ionx They seemed to indicate that adding ROOT= ".." would do something, but I am doubtful.

[theotherjimmy]

As you mentoined, cygwin is not supported at this time. You could use cmd.exe instead.

[bmcdonnell-ionx]

There is no ROOT= statement to edit, though.

[theotherjimmy]

@bmcdonnell-ionx That's why I'm doubtful.

[bmcdonnell-ionx]

As you mentoined, cygwin is not supported at this time.

Is there a place where a list of things broken on cygwin is kept?

[theotherjimmy]

The very bottom of README.md contains a mention that Mbed CLI does not work from within cygwin. It's not a list though.

[bmcdonnell-ionx]

I discovered that mention between when I opened this issue and now.

Is there a list somewhere else?

On #299, @screamerbg said

PRs welcome! :)

"PR" means, "problem report", right? Is that the same or different from these issue reports here on GitHub?

• If they're the same, why are cygwin-related issues being closed? I'd think they should be given a "cygwin" and/or "future" (or specific milestone) tag, and left open.
• If they're different, then where are the PRs, so I can add this to them?

[theotherjimmy]

@bmcdonnell-ionx Nope, PR means Pull Request. Issues are different, though the numbers are unique between Issues and PRs.

[bmcdonnell-ionx]

Thanks for clarifying.

Is there a good place to maintain the list of things broken on cygwin?

[theotherjimmy]

If they're the same, why are cygwin-related issues being closed? I'd think they should be given a "cygwin" and/or "future" (or specific milestone) tag, and left open.

They are closed when the original authors problem is resolved, which in the case of #299, was a documentation addition.

If they're different, then where are the PRs, so I can add this to them?

You can fork mbed-cli, checkout a feature branch git checkout -b cygwin-support, make changes that resolve you problem, commit the changes git commit, push the changes to your fork git push -u bmcdonnell-ionx, and then submit a PR (there's a button on the mbed-cli home page after you push your branch).

[theotherjimmy]

@bmcdonnell-ionx We could maintain a list in that section in the readme I mentioned.

[bmcdonnell-ionx]

I'm not sure how much sense forking makes, since I don't currently have the intent to fix this issue myself.

[theotherjimmy]

@bmcdonnell-ionx If you're contributing at all, forking makes sense.

[bmcdonnell-ionx]

Thus far, I'm only filing issue reports, not making any direct code contributions.

I'm concerned about visibility. I don't want to maintain a list of cygwin-related issues for me. It should be something that others can find relatively easily, and contribute to.

[theotherjimmy]

@bmcdonnell-ionx Then please make edits to the Readme. I think that's the best place for such a list.

[bmcdonnell-ionx]

Or did you mean that this issue can stay open? Having this here and open is good enough, IMO. I just thought you were going to close it like the other one...

[screamerbg]

@bmcdonnell-ionx I just installed the latest Cygwin 64bit v2.10 from https://cygwin.com/install.html. Importing with Mbed CLI works just fine.

From within Cygwin64 Terminal

Screamer@Y5070-M ~
$ ls -la
total 29
drwxr-xr-x+ 1 Screamer None    0 Apr 28 14:35 .
drwxrwxrwt+ 1 Screamer None    0 Apr 28 11:53 ..
-rw-------  1 Screamer None 2641 Apr 28 14:34 .bash_history
-rwxr-xr-x  1 Screamer None 1494 Apr 28 11:53 .bash_profile
-rwxr-xr-x  1 Screamer None 6054 Apr 28 13:11 .bashrc
-rwxr-xr-x  1 Screamer None 1919 Apr 28 11:53 .inputrc
-rw-r--r--  1 Screamer None   63 Apr 28 13:18 .minttyrc
-rwxr-xr-x  1 Screamer None 1265 Apr 28 13:17 .profile
drwxr-xr-x+ 1 Screamer None    0 Apr 28 12:01 .ssh

Screamer@Y5070-M ~
$ mbed --version
1.5.1

Screamer@Y5070-M ~
$ mkdir test

Screamer@Y5070-M ~
$ cd test

Screamer@Y5070-M ~/test
$ mbed import mbed-os-example-wifi
[mbed] Importing program "mbed-os-example-wifi" from "https://github.com/ARMmbed/mbed-os-example-wifi" at latest revision in the current branch
[mbed] Adding library "esp8266-driver" from "https://github.com/armmbed/esp8266-driver" at rev #264468722a97
[mbed] Adding library "mbed-os" from "https://github.com/ARMmbed/mbed-os" at rev #c05d72c3c005
[mbed] Adding library "wifi-x-nucleo-idw01m1" from "https://github.com/ARMmbed/wifi-x-nucleo-idw01m1" at rev #5871f7011d7f

Everything works fine except for the output buffering coming from Python itself. It can be disabled by running export PYTHONUNBUFFERED=1 inside the Cygwin Terminal.

Windows Command Prompt:

Microsoft Windows [Version 10.0.16299.371]
(c) 2017 Microsoft Corporation. All rights reserved.

C:\Users\Screamer>c:\cygwin\bin\bash.exe

Screamer@Y5070-M /cygdrive/c/Users/Screamer
$ cd ~

Screamer@Y5070-M ~
$ cd test

Screamer@Y5070-M ~/test
$ mbed import mbed-os-example-lorawan
[mbed] Importing program "mbed-os-example-lorawan" from "https://github.com/ARMmbed/mbed-os-example-lorawan" at latest revision in the current branch
[mbed] Adding library "mbed-lora-radio-drv" from "https://github.com/ARMmbed/mbed-semtech-lora-rf-drivers" at rev #4691c37c6b2d
[mbed] Adding library "mbed-os" from "https://github.com/ARMmbed/mbed-os" at rev #c05d72c3c005

Again, everything works as expected and there are no output buffering issues. Note that I'm using Mbed CLI 1.5.1

cc @donatieng @BlackstoneEngineering

Related issues #352 #635

[screamerbg]

And here's Mbed CLI import to a non-home dir inside the Cygwin environment (using Windows Prompt)

Microsoft Windows [Version 10.0.16299.371]
(c) 2017 Microsoft Corporation. All rights reserved.

C:\Users\Screamer>c:\cygwin\bin\bash.exe

Screamer@Y5070-M /cygdrive/c/Users/Screamer
$ cd AppData/Local/Temp

Screamer@Y5070-M /cygdrive/c/Users/Screamer/AppData/Local/Temp
$ mkdir test

Screamer@Y5070-M /cygdrive/c/Users/Screamer/AppData/Local/Temp
$ cd test

Screamer@Y5070-M /cygdrive/c/Users/Screamer/AppData/Local/Temp/test
$ mbed cache purge
[mbed] Purging cached repositories in "C:\cygwin\home\Screamer\.mbed"...
[mbed] Purge complete!

Screamer@Y5070-M /cygdrive/c/Users/Screamer/AppData/Local/Temp/test
$ mbed import mbed-os-example-filesystem
[mbed] Importing program "mbed-os-example-filesystem" from "https://github.com/ARMmbed/mbed-os-example-filesystem" at latest revision in the current branch
[mbed] Adding library "dataflash-driver" from "https://github.com/ARMmbed/dataflash-driver" at rev #efcb8a1a5c11
[mbed] Adding library "mbed-os" from "https://github.com/ARMmbed/mbed-os" at rev #c05d72c3c005
[mbed] Adding library "sd-driver" from "https://github.com/ARMmbed/sd-driver" at rev #c16fa2bf36b8
[mbed] Adding library "spif-driver" from "https://github.com/ARMmbed/spif-driver" at rev #e5e6616914a2

Everything worked just fine, including progress bars (the Python buffered issue is present only in the standard Cygwin Terminal app)

[bmcdonnell-ionx]

@screamerbg

I just installed the latest Cygwin 64bit v2.10 from https://cygwin.com/install.html. Importing with Mbed CLI works just fine.

I just did the same, and updated Mbed CLI to v1.5.1, but the problem persists for me.

me@pc /cygdrive/c/dev_temp
$ mbed import -vv mbed-os-example-wifi
Cloning into 'C:\dev_temp\mbed-os-example-wifi'...
fatal: Invalid path '/cygdrive/c/dev_temp/C:\dev_temp\mbed-os-example-wifi': No such file or directory
abort: HTTP Error 406: Not Acceptable
[mbed] Working path "C:\dev_temp" (directory)
[mbed] Importing program "mbed-os-example-wifi" from "https://github.com/ARMmbed/mbed-os-example-wifi" at latest revision in the current branch
[mbed] Trying to guess source control management tool. Supported SCMs: bld, git, hg
[mbed] Exec "git clone https://github.com/ARMmbed/mbed-os-example-wifi C:\dev_temp\mbed-os-example-wifi -v" in C:\dev_temp
[mbed] Exec "hg clone https://github.com/ARMmbed/mbed-os-example-wifi C:\dev_temp\mbed-os-example-wifi -v" in C:\dev_temp
[mbed] ERROR: Unable to clone repository (https://github.com/ARMmbed/mbed-os-example-wifi)

me@pc /cygdrive/c/dev_temp
$ uname -r
2.10.0(0.325/5/3)

me@pc /cygdrive/c/dev_temp
$ git --version
git version 2.17.0

me@pc /cygdrive/c/dev_temp
$ mbed --version
1.5.1

[bridadan]

I recently encountered an issue regarding git and cygwin on a Windows machine due to permissions. It eventually boiled down to cygwin's version of git was being used in my Windows command prompt (higher up in the system PATH) and caused all cloned files to be inaccessible. I'm curious if something along these lines are happening here.

Might be worth checking which instance of git is being used in your environments. Could you both run the following and post the output?

On the Windows Command prompt: where git.
And on Cygwin: which git.

[bmcdonnell-ionx]

@bridadan,

Looks like Cygwin is using its own git.

Windows Command prompt:

C:\Users\me>where git
C:\Users\me\AppData\Local\Programs\Git\cmd\git.exe

Cygwin:

$ where git
C:\cygwin64\bin\git.exe
C:\Users\me\AppData\Local\Programs\Git\cmd\git.exe

$ which git
/usr/bin/git

[bridadan]

Are you sure you have the latest verison of mbed-cli installed from pip? In your log I found this print:

[mbed] Trying to guess source control management tool. Supported SCMs: bld, git, hg

I can't find that log line anywhere in the source, so not sure where that's coming from.

Assuming that's all ok, I can also see its trying to run the following:

git clone https://github.com/ARMmbed/mbed-os-example-wifi C:\dev_temp\mbed-os-example-wifi -v

If you run that in your current terminal, does that command fail? If it is then it may be an issue of not using Cygwin formatted paths. I admit I don't use Cygwin so this is pretty much the limit of my debugging tips.

[bridadan]

Ah, looks like you might be using this branch? #642

(log line is here: https://github.com/ARMmbed/mbed-cli/pull/642/files#diff-dad37ed6e3ea3a963df6ff7090c2562fR1184)

Try uninstalling that branch first and reinstalling from pip to see if the issue is resolved.

[bmcdonnell-ionx]

@bridadan

Ah, looks like you might be using this branch? #642

Yes. I merged 1.5.1 into it, and installed from source (python setup.py install).

Try uninstalling that branch first

How?

[bridadan]

pip uninstall mbed-cli
pip install mbed-cli

That should do the trick

[bmcdonnell-ionx]

It hangs, and does not uninstall.

$ pip uninstall mbed-cli
Uninstalling mbed-cli-1.5.1:
  c:\python27\lib\site-packages\mbed_cli-1.5.1-py2.7.egg
  c:\python27\scripts\mbed-cli-script.py
  c:\python27\scripts\mbed-cli.exe
  c:\python27\scripts\mbed-cli.exe.manifest
  c:\python27\scripts\mbed-script.py
  c:\python27\scripts\mbed.exe
  c:\python27\scripts\mbed.exe.manifest

Should pip still be able to uninstall what was installed from source (i.e. not via pip)?

[bridadan]

Yeah it definitely should be able to uninstall. I'm afraid I don't know how to help on this one, this problem seems pretty specific to your environment. You may try uninstalling it from outside of cygwin, but that may cause more issues. But I think its probably safe to say the uninstall issue is not a problem with mbed-cli specifically.

[bmcdonnell-ionx]

You may try uninstalling it from outside of cygwin, but that may cause more issues.

I uninstalled Mbed CLI by deleting its files from c:\Python27\Lib\site-packages and c:\Python27\Scripts. I reinstalled version 1.5.1 via pip.

I also uninstalled Cygwin, and reinstalled by running setup-x86_64.exe --no-admin from a Windows 10 command prompt.

The problem persists.

I included Python 2 & 3 in my Cygwin install. python --version tells me Python 2.7.14. What does yours say?

this problem seems pretty specific to your environment.

What does which python tell you? Mine says /usr/bin/python.

Know anyone who might have any ideas?

[screamerbg]

@bmcdonnell-ionx Apologies for the late reply. I've somehow missed this.

which python returns:

Screamer@Y5070-M ~
$ which python
/cygdrive/c/PROGRA~1/Python/python

Screamer@Y5070-M ~
$ python -V
Python 2.7.11

This is on Windows 10, where Python has been installed as a native application.

I'm not sure how your python is under /usr/bin/. How did you set up your cygwin box so python appears as a native cygwin app?

[bmcdonnell-ionx]

How did you set up your cygwin box so python appears as a native cygwin app?

I installed python from the Cygwin setup-x86_64.exe, like any other package. I do also have python installed natively on Windows, though my Cygwin apparently isn't using that.

[ciarmcom]

Internal Jira reference: https://jira.arm.com/browse/MBOCUSTRIA-70

[kask01]

@here - I am observing the same problem too. Has there been any more updates on this?

[mark-edgeworth]

Hi all, I'm sorry but cygwin is not a supported platform for the mbed CLI tools; although it seems that some users have managed to achieve some level of success here. We'd welcome PRs from the community that help support this platform in future.