Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unexpected end of /proc/mounts line local_test Ubuntu 16.04 #187

Open
fsimonis opened this issue Feb 26, 2020 · 0 comments
Open

Unexpected end of /proc/mounts line local_test Ubuntu 16.04 #187

fsimonis opened this issue Feb 26, 2020 · 0 comments

Comments

@fsimonis
Copy link
Member

Open MPI fails to parse /proc/mounts in docker containers.

Unexpected end of /proc/mounts line `overlay / overlay rw,relatime,lowerdir=/work_fast/docker/overlay2/l/LSSTTVRAJFQKCH2EL7TQVVWNE3:/work_fast/docker/overlay2/l/E7SVFSB2LOQJGXPCGQCHO5K6SH:/work_fast/docker/overlay2/l/L7U5E72OZD4B4LZKEHIWZGCE3U:/work_fast/docker/overlay2/l/Y3SYJIFHIFOAQWJ3LFON6MG7ZV:/work_fast/docker/overlay2/l/OZ73REB4NO3ABTCYS5SSBKZJW5:/work_fast/docker/overlay2/l/NWNNQ7LIB7ORFKU2WSLHKOYW25:/work_fast/docker/overlay2/l/VTXPAOYTUU37HUCNW4TDB366FA:/work_fast/docker/overlay2/l/ORNABTXUGD45GN72M7TYCLKNBA:/work_fast/docker/overla'
Unexpected end of /proc/mounts line `y2/l/JBC52A3KAYFS5IGDSAQZEQDZWA:/work_fast/docker/overlay2/l/6RX7WB6SBCX45SZGJPKRAJTGXS:/work_fast/docker/overlay2/l/PQB5NLGOFB55TALSFQBK677G2R:/work_fast/docker/overlay2/l/OD4ZGRBVUPHR72PNT7PSHBXMEP:/work_fast/docker/overlay2/l/JXXTOUPWYSD7ERRBBOCMLSNIMT:/work_fast/docker/overlay2/l/K3UIJ7TRKKNAHFP7B73AOKNT6E:/work_fast/docker/overlay2/l/DNMY6ES6VFNGCCHXC3YRIXX7CF:/work_fast/docker/overlay2/l/HY7I5VKZ7NETM2ELRL2TXHH5FL:/work_fast/docker/overlay2/l/WOCT4SDQFU3FUL2KQYY422BV74:/work_fast/docker/overlay2/l/FS'
Unexpected end of /proc/mounts line `Q632SSW7MNLZBM34IYA3NFXM:/work_fast/docker/overlay2/l/RQQIBQPL4XPU46MYTYYY6HCFC4:/work_fast/docker/overlay2/l/FSZQCOMJ7OQDWB2MXJY4FV5W6P:/work_fast/docker/overlay2/l/347YFWCIEAUKI343NA523HAITN:/work_fast/docker/overlay2/l/FCFHPBRAIRJPB4TNYDFYHBY3CG:/work_fast/docker/overlay2/l/F7OOHA44URRVMIUHXVZU6SBQX6:/work_fast/docker/overlay2/l/65QQ5EWZASWNX5FSIWOQWXSSHQ:/work_fast/docker/overlay2/l/4KCZ6ODKL54DGJOPU76BKY2URH:/work_fast/docker/overlay2/l/XSNUIMBARL236HC5ARDQHBDRSG:/work_fast/docker/overlay2/l/GDGOS634G'
Unexpected end of /proc/mounts line `overlay / overlay rw,relatime,lowerdir=/work_fast/docker/overlay2/l/LSSTTVRAJFQKCH2EL7TQVVWNE3:/work_fast/docker/overlay2/l/E7SVFSB2LOQJGXPCGQCHO5K6SH:/work_fast/docker/overlay2/l/L7U5E72OZD4B4LZKEHIWZGCE3U:/work_fast/docker/overlay2/l/Y3SYJIFHIFOAQWJ3LFON6MG7ZV:/work_fast/docker/overlay2/l/OZ73REB4NO3ABTCYS5SSBKZJW5:/work_fast/docker/overlay2/l/NWNNQ7LIB7ORFKU2WSLHKOYW25:/work_fast/docker/overlay2/l/VTXPAOYTUU37HUCNW4TDB366FA:/work_fast/docker/overlay2/l/ORNABTXUGD45GN72M7TYCLKNBA:/work_fast/docker/overla'
Unexpected end of /proc/mounts line `y2/l/JBC52A3KAYFS5IGDSAQZEQDZWA:/work_fast/docker/overlay2/l/6RX7WB6SBCX45SZGJPKRAJTGXS:/work_fast/docker/overlay2/l/PQB5NLGOFB55TALSFQBK677G2R:/work_fast/docker/overlay2/l/OD4ZGRBVUPHR72PNT7PSHBXMEP:/work_fast/docker/overlay2/l/JXXTOUPWYSD7ERRBBOCMLSNIMT:/work_fast/docker/overlay2/l/K3UIJ7TRKKNAHFP7B73AOKNT6E:/work_fast/docker/overlay2/l/DNMY6ES6VFNGCCHXC3YRIXX7CF:/work_fast/docker/overlay2/l/HY7I5VKZ7NETM2ELRL2TXHH5FL:/work_fast/docker/overlay2/l/WOCT4SDQFU3FUL2KQYY422BV74:/work_fast/docker/overlay2/l/FS'
Unexpected end of /proc/mounts line `Q632SSW7MNLZBM34IYA3NFXM:/work_fast/docker/overlay2/l/RQQIBQPL4XPU46MYTYYY6HCFC4:/work_fast/docker/overlay2/l/FSZQCOMJ7OQDWB2MXJY4FV5W6P:/work_fast/docker/overlay2/l/347YFWCIEAUKI343NA523HAITN:/work_fast/docker/overlay2/l/FCFHPBRAIRJPB4TNYDFYHBY3CG:/work_fast/docker/overlay2/l/F7OOHA44URRVMIUHXVZU6SBQX6:/work_fast/docker/overlay2/l/65QQ5EWZASWNX5FSIWOQWXSSHQ:/work_fast/docker/overlay2/l/4KCZ6ODKL54DGJOPU76BKY2URH:/work_fast/docker/overlay2/l/XSNUIMBARL236HC5ARDQHBDRSG:/work_fast/docker/overlay2/l/GDGOS634G

Reason why can b found here:

This is likely due to your /proc/mount file having a line in it greater than 512 characters, causing the hwloc module of OpenMPI to fail to parse it correctly. Docker has a tendency to put very long lines into /proc/mounts.

And a possible solution here:

This issue should be fixed in hwloc since 1.11.3 (released 2 years ago). You can either upgrade to OpenMPI 3.0 which contains a hwloc 1.11.7 >= 1.11.3. Or recompile OpenMPI to use an external hwloc instead of the old embedded one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant