New bugs exposed while building images #236

hsd-dev · 2019-02-27T17:40:50Z

I had this issue while building care-o-bot/ac6a181 (mentioned in #214) and care-o-bot/c8091b6.

In case of care-o-bot/ac6a181, the docker image building failed:

Step 42/44 : RUN echo "[ROBUST] attempting to build PUT..."  && echo "[ROBUST] is a build failure expected? ${IS_BUILD_FAILURE}."  && ./build.sh || [ "${IS_BUILD_FAILURE}" = "yes" ]
 ---> Running in acfe3340c7c7
[ROBUST] attempting to build PUT...
[ROBUST] is a build failure expected? yes.
Error(s) in package '/ros_ws/src/repo-under-test/cob_teleop_cob4/package.xml':
The license tag must neither be empty nor only contain whitespaces

Base path: /ros_ws
Source space: /ros_ws/src
Build space: /ros_ws/build
Devel space: /ros_ws/devel
Install space: /ros_ws/install
Creating symlink "/ros_ws/src/CMakeLists.txt" pointing to "/opt/ros/hydro/share/catkin/cmake/toplevel.cmake"
 ---> 6e581e697565
Removing intermediate container acfe3340c7c7
Step 43/44 : COPY test.sh .
 ---> 30569d285e05

This is because in package.xml :

The license tag must neither be empty nor only contain whitespaces

In case of care-o-bot/c8091b6, after building the image, the "problematic" script fails with the error:

no module named gtk

This error was fixed by installing python-gtk within the container and only then I got the expected bug of missing dependency of pynotify.

@wasowski mentioned in today's meeting that these kind of issues were faced earlier where new bugs are exposed while building the images. Is it possible to automate these fixes or should be the responsibility of whoever is building the image?

The text was updated successfully, but these errors were encountered:

gavanderhoorn · 2019-03-01T15:29:20Z

If there are errors in packages that are not the PUT(s), then it could be that the maintainers/devs were using a source checkout around the time the bug was reported in which the "unrelated issues" were already fixed (we've seen something like this in care-o-bot packages before, which caused me to log rosin-project/rosinstall_generator_time_machine#11).

The released version would not have the fixes yet, which would lead to the kind of situation that you encountered (ie: fix seemingly unrelated bug before getting to the one that the report is about).

Is it possible to automate these fixes or should be the responsibility of whoever is building the image?

Do you have any suggestions for how we could automate things like this? It would seem rather difficult to me.

gavanderhoorn · 2019-03-01T15:32:45Z

re: license tag empty issue: that was fixed in ipa320/cob_command_tools@dd1ab32, which was committed on Jul 11, 2014. The package.xml you refer to is from May 13, 2014. That's pretty close.

andersfischernielsen · 2019-03-04T06:17:39Z

@ipa-hsd How would I go about building dd1ab32? Or have you gotten that to build?
It seems the bug is still present in dd1ab32 (where the license section of the package.xml was added), which would enable building a buggy version of the package.

andersfischernielsen · 2019-03-05T20:47:40Z

After having checked out your fork and modified the .bug to contain the commit where the bug is present and the license field is filled out I get the following error when building the image:

bugzoo bug build robust:c8091b6                                                               
building bug: robust:c8091b6
Step 1/43 : ARG UBUNTU_VERSION

Step 2/43 : FROM alpine:3.7 as fork



================================================================================
An error occurred during execution:

  failed to build Docker image: robustrosin/robust:c8091b6

See log file for details: /Users/fischer/.bugzoo/logs/2019-03-06_09:32:06.bugzoo.log

The log contains no useful information:

INFO:bugzoo.mgr.source:2019-03-06 09:41:31,595: refreshed sources
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,595: initialising container manager
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,595: connecting to low-level Docker API
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,599: connected to low-level Docker API
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,599: initialised container manager
DEBUG:bugzoo.mgr.build:2019-03-06 09:41:31,607: request to build image: robustrosin/robust:c8091b6
INFO:bugzoo.mgr.build:2019-03-06 09:41:31,607: building image: robustrosin/robust:c8091b6
ERROR:bugzoo.cli.app:2019-03-06 09:41:33,637: An error occurred
Traceback (most recent call last):
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/cli/app.py", line 72, in main
    app.run()
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/cement/core/foundation.py", line 916, in run
    return_val = self.controller._dispatch()
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/cement/ext/ext_argparse.py", line 808, in _dispatch
    return func()
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/cli/controllers/bug.py", line 100, in build
    bugs.build(bug)
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/mgr/bug.py", line 94, in build
    quiet=quiet)
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/mgr/build.py", line 130, in build
    raise ImageBuildFailed(name, log)
bugzoo.exceptions.ImageBuildFailed: failed to build Docker image: robustrosin/robust:c8091b6

I'm unsure if this is a bugzoo or a configuration issue.
There is no Dockerfile in the folder for c8091b6. Could that be what's preventing the build?

gavanderhoorn · 2019-03-05T20:49:55Z

/Users/fischer/SQUARE -> are you running this on OSX?

andersfischernielsen · 2019-03-05T20:53:54Z

@gavanderhoorn Yes.

gavanderhoorn · 2019-03-05T20:56:47Z

It's all Python, so theoretically it could work, but you're "on your own", as none of us use OSX, so we haven't tested it.

gavanderhoorn · 2019-03-05T21:03:00Z

Oh and the Dockerfile not being present in the bug dir is not an issue. The tool reuses a single - parameterised - Dockerfile for all bugs.

Perhaps @ChrisTimperley can shed some light on the errors in the log.

andersfischernielsen · 2019-03-05T21:06:42Z

@gavanderhoorn Okay, sweet. Thanks! 👍 I'll spin up an Ubuntu VM and see how it looks there.

Regarding the logs, I tried to add extra login in the bugzoo Python build scripts, but the response from the docker daemon is pretty useless (at least the output I got).

ChrisTimperley · 2019-03-05T21:43:53Z

After having checked out your fork and modified the .bug to contain the commit where the bug is present and the license field is filled out I get the following error when building the image:

bugzoo bug build robust:c8091b6                                                               
building bug: robust:c8091b6
Step 1/43 : ARG UBUNTU_VERSION

Step 2/43 : FROM alpine:3.7 as fork



================================================================================
An error occurred during execution:

  failed to build Docker image: robustrosin/robust:c8091b6

See log file for details: /Users/fischer/.bugzoo/logs/2019-03-06_09:32:06.bugzoo.log

The log contains no useful information:

INFO:bugzoo.mgr.source:2019-03-06 09:41:31,595: refreshed sources
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,595: initialising container manager
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,595: connecting to low-level Docker API
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,599: connected to low-level Docker API
DEBUG:bugzoo.mgr.container:2019-03-06 09:41:31,599: initialised container manager
DEBUG:bugzoo.mgr.build:2019-03-06 09:41:31,607: request to build image: robustrosin/robust:c8091b6
INFO:bugzoo.mgr.build:2019-03-06 09:41:31,607: building image: robustrosin/robust:c8091b6
ERROR:bugzoo.cli.app:2019-03-06 09:41:33,637: An error occurred
Traceback (most recent call last):
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/cli/app.py", line 72, in main
    app.run()
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/cement/core/foundation.py", line 916, in run
    return_val = self.controller._dispatch()
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/cement/ext/ext_argparse.py", line 808, in _dispatch
    return func()
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/cli/controllers/bug.py", line 100, in build
    bugs.build(bug)
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/mgr/bug.py", line 94, in build
    quiet=quiet)
  File "/Users/fischer/SQUARE/ipa-hsd/robust/bugzoo_venv/lib/python3.7/site-packages/bugzoo/mgr/build.py", line 130, in build
    raise ImageBuildFailed(name, log)
bugzoo.exceptions.ImageBuildFailed: failed to build Docker image: robustrosin/robust:c8091b6

I'm unsure if this is a bugzoo or a configuration issue.
There is no Dockerfile in the folder for c8091b6. Could that be what's preventing the build?

It looks like you might be running a slightly older version of Docker that doesn't support multi-stage builds (https://docs.docker.com/develop/develop-images/multistage-build/).

andersfischernielsen · 2019-03-05T22:20:26Z

My docker engine on macOS is version 18.09.2, build 6247962. Multi-stage builds should be supported on that.
It does seem to bee a macOS issue, though. I can build the bug just fine on my Ubuntu VM. Oh, well.

hsd-dev mentioned this issue Feb 27, 2019

building care-o-bot c8091b6 #215

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New bugs exposed while building images #236

New bugs exposed while building images #236

hsd-dev commented Feb 27, 2019

gavanderhoorn commented Mar 1, 2019 •

edited

Loading

gavanderhoorn commented Mar 1, 2019

andersfischernielsen commented Mar 4, 2019 •

edited

Loading

andersfischernielsen commented Mar 5, 2019

gavanderhoorn commented Mar 5, 2019

andersfischernielsen commented Mar 5, 2019

gavanderhoorn commented Mar 5, 2019

gavanderhoorn commented Mar 5, 2019 •

edited

Loading

andersfischernielsen commented Mar 5, 2019

ChrisTimperley commented Mar 5, 2019

andersfischernielsen commented Mar 5, 2019

New bugs exposed while building images #236

New bugs exposed while building images #236

Comments

hsd-dev commented Feb 27, 2019

gavanderhoorn commented Mar 1, 2019 • edited Loading

gavanderhoorn commented Mar 1, 2019

andersfischernielsen commented Mar 4, 2019 • edited Loading

andersfischernielsen commented Mar 5, 2019

gavanderhoorn commented Mar 5, 2019

andersfischernielsen commented Mar 5, 2019

gavanderhoorn commented Mar 5, 2019

gavanderhoorn commented Mar 5, 2019 • edited Loading

andersfischernielsen commented Mar 5, 2019

ChrisTimperley commented Mar 5, 2019

andersfischernielsen commented Mar 5, 2019

gavanderhoorn commented Mar 1, 2019 •

edited

Loading

andersfischernielsen commented Mar 4, 2019 •

edited

Loading

gavanderhoorn commented Mar 5, 2019 •

edited

Loading