Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New bugs exposed while building images #236

Open
hsd-dev opened this issue Feb 27, 2019 · 11 comments
Open

New bugs exposed while building images #236

hsd-dev opened this issue Feb 27, 2019 · 11 comments

Comments

@hsd-dev
Copy link
Contributor

hsd-dev commented Feb 27, 2019

I had this issue while building care-o-bot/ac6a181 (mentioned in #214) and care-o-bot/c8091b6.

  1. In case of care-o-bot/ac6a181, the docker image building failed:
Step 42/44 : RUN echo "[ROBUST] attempting to build PUT..."  && echo "[ROBUST] is a build failure expected? ${IS_BUILD_FAILURE}."  && ./build.sh || [ "${IS_BUILD_FAILURE}" = "yes" ]
 ---> Running in acfe3340c7c7
[ROBUST] attempting to build PUT...
[ROBUST] is a build failure expected? yes.
Error(s) in package '/ros_ws/src/repo-under-test/cob_teleop_cob4/package.xml':
The license tag must neither be empty nor only contain whitespaces

Base path: /ros_ws
Source space: /ros_ws/src
Build space: /ros_ws/build
Devel space: /ros_ws/devel
Install space: /ros_ws/install
Creating symlink "/ros_ws/src/CMakeLists.txt" pointing to "/opt/ros/hydro/share/catkin/cmake/toplevel.cmake"
 ---> 6e581e697565
Removing intermediate container acfe3340c7c7
Step 43/44 : COPY test.sh .
 ---> 30569d285e05

This is because in package.xml :

The license tag must neither be empty nor only contain whitespaces

  1. In case of care-o-bot/c8091b6, after building the image, the "problematic" script fails with the error:

no module named gtk

This error was fixed by installing python-gtk within the container and only then I got the expected bug of missing dependency of pynotify.

@wasowski mentioned in today's meeting that these kind of issues were faced earlier where new bugs are exposed while building the images. Is it possible to automate these fixes or should be the responsibility of whoever is building the image?

@gavanderhoorn
Copy link
Member

gavanderhoorn commented Mar 1, 2019

If there are errors in packages that are not the PUT(s), then it could be that the maintainers/devs were using a source checkout around the time the bug was reported in which the "unrelated issues" were already fixed (we've seen something like this in care-o-bot packages before, which caused me to log rosin-project/rosinstall_generator_time_machine#11).

The released version would not have the fixes yet, which would lead to the kind of situation that you encountered (ie: fix seemingly unrelated bug before getting to the one that the report is about).

Is it possible to automate these fixes or should be the responsibility of whoever is building the image?

Do you have any suggestions for how we could automate things like this? It would seem rather difficult to me.

@gavanderhoorn
Copy link
Member

re: license tag empty issue: that was fixed in ipa320/cob_command_tools@dd1ab32, which was committed on Jul 11, 2014. The package.xml you refer to is from May 13, 2014. That's pretty close.

@andersfischernielsen
Copy link

andersfischernielsen commented Mar 4, 2019

@ipa-hsd How would I go about building dd1ab32? Or have you gotten that to build?
It seems the bug is still present in dd1ab32 (where the license section of the package.xml was added), which would enable building a buggy version of the package.

@andersfischernielsen
Copy link

After having checked out your fork and modified the .bug to contain the commit where the bug is present and the license field is filled out I get the following error when building the image:

bugzoo bug build robust:c8091b6                                                               
building bug: robust:c8091b6
Step 1/43 : ARG UBUNTU_VERSION

Step 2/43 : FROM alpine:3.7 as fork



================================================================================
An error occurred during execution:

  failed to build Docker image: robustrosin/robust:c8091b6

See log file for details: /Users/fischer/.bugzoo/logs/2019-03-06_09:32:06.bugzoo.log

The log contains no useful information:

INFO:bugzoo.mgr.source:2019-03-06 09:41:31,595: refreshed sources
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,595: initialising container manager
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,595: connecting to low-level Docker API
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,599: connected to low-level Docker API
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,599: initialised container manager
DEBUG:bugzoo.mgr.build:2019-03-06 09:41:31,607: request to build image: robustrosin/robust:c8091b6
INFO:bugzoo.mgr.build:2019-03-06 09:41:31,607: building image: robustrosin/robust:c8091b6
ERROR:bugzoo.cli.app:2019-03-06 09:41:33,637: An error occurred
Traceback (most recent call last):
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/cli/app.py", line 72, in main
    app.run()
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/cement/core/foundation.py", line 916, in run
    return_val = self.controller._dispatch()
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/cement/ext/ext_argparse.py", line 808, in _dispatch
    return func()
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/cli/controllers/bug.py", line 100, in build
    bugs.build(bug)
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/mgr/bug.py", line 94, in build
    quiet=quiet)
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/mgr/build.py", line 130, in build
    raise ImageBuildFailed(name, log)
bugzoo.exceptions.ImageBuildFailed: failed to build Docker image: robustrosin/robust:c8091b6

I'm unsure if this is a bugzoo or a configuration issue.
There is no Dockerfile in the folder for c8091b6. Could that be what's preventing the build?

@gavanderhoorn
Copy link
Member

/Users/fischer/SQUARE -> are you running this on OSX?

@andersfischernielsen
Copy link

@gavanderhoorn Yes.

@gavanderhoorn
Copy link
Member

It's all Python, so theoretically it could work, but you're "on your own", as none of us use OSX, so we haven't tested it.

@gavanderhoorn
Copy link
Member

gavanderhoorn commented Mar 5, 2019

Oh and the Dockerfile not being present in the bug dir is not an issue. The tool reuses a single - parameterised - Dockerfile for all bugs.

Perhaps @ChrisTimperley can shed some light on the errors in the log.

@andersfischernielsen
Copy link

@gavanderhoorn Okay, sweet. Thanks! 👍 I'll spin up an Ubuntu VM and see how it looks there.

Regarding the logs, I tried to add extra login in the bugzoo Python build scripts, but the response from the docker daemon is pretty useless (at least the output I got).

@ChrisTimperley
Copy link
Member

After having checked out your fork and modified the .bug to contain the commit where the bug is present and the license field is filled out I get the following error when building the image:

bugzoo bug build robust:c8091b6                                                               
building bug: robust:c8091b6
Step 1/43 : ARG UBUNTU_VERSION

Step 2/43 : FROM alpine:3.7 as fork



================================================================================
An error occurred during execution:

  failed to build Docker image: robustrosin/robust:c8091b6

See log file for details: /Users/fischer/.bugzoo/logs/2019-03-06_09:32:06.bugzoo.log

The log contains no useful information:

INFO:bugzoo.mgr.source:2019-03-06 09:41:31,595: refreshed sources
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,595: initialising container manager
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,595: connecting to low-level Docker API
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,599: connected to low-level Docker API
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,599: initialised container manager
DEBUG:bugzoo.mgr.build:2019-03-06 09:41:31,607: request to build image: robustrosin/robust:c8091b6
INFO:bugzoo.mgr.build:2019-03-06 09:41:31,607: building image: robustrosin/robust:c8091b6
ERROR:bugzoo.cli.app:2019-03-06 09:41:33,637: An error occurred
Traceback (most recent call last):
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/cli/app.py", line 72, in main
    app.run()
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/cement/core/foundation.py", line 916, in run
    return_val = self.controller._dispatch()
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/cement/ext/ext_argparse.py", line 808, in _dispatch
    return func()
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/cli/controllers/bug.py", line 100, in build
    bugs.build(bug)
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/mgr/bug.py", line 94, in build
    quiet=quiet)
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/mgr/build.py", line 130, in build
    raise ImageBuildFailed(name, log)
bugzoo.exceptions.ImageBuildFailed: failed to build Docker image: robustrosin/robust:c8091b6

I'm unsure if this is a bugzoo or a configuration issue.
There is no Dockerfile in the folder for c8091b6. Could that be what's preventing the build?

It looks like you might be running a slightly older version of Docker that doesn't support multi-stage builds (https://docs.docker.com/develop/develop-images/multistage-build/).

@andersfischernielsen
Copy link

My docker engine on macOS is version 18.09.2, build 6247962. Multi-stage builds should be supported on that.
It does seem to bee a macOS issue, though. I can build the bug just fine on my Ubuntu VM. Oh, well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants