Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installs files into the same directory where build is performed from #2875

Closed
yurivict opened this issue Jul 14, 2019 · 29 comments
Closed

Installs files into the same directory where build is performed from #2875

yurivict opened this issue Jul 14, 2019 · 29 comments
Assignees
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@yurivict
Copy link

The build was performed in /wrkdirs/usr/ports/devel/google-cloud-cpp/work, so it installed the following files:

===> Checking for items in STAGEDIR missing from pkg-plist
Error: Orphaned: /wrkdirs/usr/ports/devel/google-cloud-cpp/work/.build/external/include/google/api/annotations.grpc.pb.h
Error: Orphaned: /wrkdirs/usr/ports/devel/google-cloud-cpp/work/.build/external/include/google/api/annotations.pb.h
Error: Orphaned: /wrkdirs/usr/ports/devel/google-cloud-cpp/work/.build/external/include/google/api/annotations.proto
Error: Orphaned: /wrkdirs/usr/ports/devel/google-cloud-cpp/work/.build/external/include/google/api/auth.grpc.pb.h
Error: Orphaned: /wrkdirs/usr/ports/devel/google-cloud-cpp/work/.build/external/include/google/api/auth.pb.h
Error: Orphaned: /wrkdirs/usr/ports/devel/google-cloud-cpp/work/.build/external/include/google/api/auth.proto
Error: Orphaned: /wrkdirs/usr/ports/devel/google-cloud-cpp/work/.build/external/include/google/api/http.grpc.pb.h

All installed files are supposed to be under ${PREFIX}, ex. /usr/local/include, /usr/local/lib, etc.

@ghost
Copy link

ghost commented Jul 14, 2019

.build is not where the files are installed. The files are installed in ${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_INCLUDEDIR}. Essentially the proto files are compiled into header files and when you run the install target the destination will be something like /usr/include/google/api/annotations.grpc.pb.h.

@yurivict
Copy link
Author

yurivict commented Jul 14, 2019

Unfortunately it installs files into ${CMAKE_INSTALL_PREFIX}/{build-dir}.

@ghost
Copy link

ghost commented Jul 14, 2019

I'm not sure what you mean. These are the commands I'm running:

cmake -DCMAKE_INSTALL_PREFIX=$PWD/install -H. -Bcmake-out .
cmake --build cmake-out --target install -- -j 16

And the output:

# ... snip ...
-- Installing: /home/tom/second_drive/google-cloud-cpp-1/install/include/google/api/http.pb.h
-- Installing: /home/tom/second_drive/google-cloud-cpp-1/install/include/google/api/http.grpc.pb.h
-- Installing: /home/tom/second_drive/google-cloud-cpp-1/install/include/google/api/http.proto
-- Installing: /home/tom/second_drive/google-cloud-cpp-1/install/include/google/api/annotations.pb.h
-- Installing: /home/tom/second_drive/google-cloud-cpp-1/install/include/google/api/annotations.grpc.pb.h
-- Installing: /home/tom/second_drive/google-cloud-cpp-1/install/include/google/api/annotations.proto
-- Installing: /home/tom/second_drive/google-cloud-cpp-1/install/include/google/api/auth.pb.h
-- Installing: /home/tom/second_drive/google-cloud-cpp-1/install/include/google/api/auth.grpc.pb.h
-- Installing: /home/tom/second_drive/google-cloud-cpp-1/install/include/google/api/auth.proto
# ... snip ...

As you can see my ${CMAKE_INSTALL_PREFIX} is /home/tom/second_drive/google-cloud-cpp-1/install and ${CMAKE_INSTALL_INCLUDEDIR} is include. The intermediary files will be in cmake-out/external/include/google/api/ (in your case .build instead of cmake-out) but that is not the install directory.

@yurivict
Copy link
Author

The end result is that it installs files into its current directory, prefixed with the stagedir. I don't know which instruction does this.

@ghost
Copy link

ghost commented Jul 14, 2019

Maybe I am missing something, but I don't see where files are being installed in your example. .build/external/ contains intermediary files part of the CMake build for the external projects and is not an install prefix. If files were being installed to the current directory, it wouldn't be in external/include but $PWD/include as none of our targets install anything to an external directory. You will probably need to somehow exclude .build in your plist.

@yurivict
Copy link
Author

yurivict commented Jul 14, 2019

I am only reporting the unambiguously wrong outcome, without looking at the code. For example, it installs this file:

/wrkdirs/usr/ports/devel/google-cloud-cpp/work/.build/external/include/google/api/annotations.grpc.pb.h

which is certainly wrong. I don't know where this is coming from and how does this happen.

@coryan
Copy link
Contributor

coryan commented Jul 14, 2019

Thank you for the bug report. I am not sure I understand the problem.

The build system generates a number of intermediate header files in the build directory (which presumably is /wrkdirs/usr/ports/devel/google-cloud-cpp/work/.build in your case?)

As far as I can tell, those files are installed to $PREFIX (or more accurately $CMAKE_INSTALL_PREFIX, or $DESTDIR) when you perform a make install.

Why is it a problem to create intermediate files in the build directory?

@yurivict
Copy link
Author

yurivict commented Jul 14, 2019

Why is it a problem to create intermediate files in the build directory?

It installs files into the intermediate directory.
The build directory is /wrkdirs/usr/ports/devel/google-cloud-cpp/work/.build/.
The stage directory is /wrkdirs/usr/ports/devel/google-cloud-cpp/work/stage/.
The install target creates files in /wrkdirs/usr/ports/devel/google-cloud-cpp/work/stage/wrkdirs/usr/ports/devel/google-cloud-cpp/work/.build/.
After installation, they would end up in the same location as build was performed: /wrkdirs/usr/ports/devel/google-cloud-cpp/work/stage/.
All installed files should be under /usr/local/ and not under /wrkdirs/usr/ports, etc.

@coryan
Copy link
Contributor

coryan commented Jul 14, 2019

I still do not follow, probably because I do not know the FreeBSD ports framework.

I just ran the Dockerfile shown below, the .build/external/include/google/api/annotations.grpc.pb.h file is created as part of the build step, and it is not modified as part of the install step. The file is copied to
$CMAKE_PREFIX_PATH as part of the install step.

Naturally, if you just perform a make install (or cmake --build --target install) the file is both generated in .build/external/, and then copied to $CMAKE_PREFIX_PATH. As you probably know the install target depends on all.

The install target creates files in /wrkdirs/usr/ports/devel/google-cloud-cpp/work/stage/wrkdirs/usr/ports/devel/google-cloud-cpp/work/.build/.

Hmmm... Based on the diff at the end of this comment, the only files that are touched by cmake --target install seem to be install_manifest.txt and a number of directories. But assuming if the install target copies some files around in the build directory. Why is this a problem? The install target does create a full set of headers and libraries in $CMAKE_PREFIX_PATH. How does it affect the port if some additional files are touched or modified in the build directory?

I am sorry if this is obvious to you, but as I said, I am not familiar with BSD ports (beyond knowing of its existence).

ARG DISTRO_VERSION=30
FROM fedora:${DISTRO_VERSION} AS devtools

RUN dnf makecache && \
    dnf install -y cmake gcc-c++ git make openssl-devel pkgconfig \
        zlib-devel
RUN dnf makecache && \
    dnf install -y grpc-devel grpc-plugins \
        libcurl-devel protobuf-compiler tar wget zlib-devel

WORKDIR /var/tmp/build
RUN wget -q https://github.com/google/crc32c/archive/1.0.6.tar.gz
RUN tar -xf 1.0.6.tar.gz
WORKDIR /var/tmp/build/crc32c-1.0.6
RUN cmake \
      -DCMAKE_BUILD_TYPE=Release \
      -DBUILD_SHARED_LIBS=yes \
      -DCRC32C_BUILD_TESTS=OFF \
      -DCRC32C_BUILD_BENCHMARKS=OFF \
      -DCRC32C_USE_GLOG=OFF \
      -H. -Bcmake-out/crc32c
RUN cmake --build cmake-out/crc32c --target install -- -j $(nproc)
RUN ldconfig

RUN dnf makecache && \
    dnf install -y ninja-build

COPY . /usr/ports/devel/google-cloud-cpp/work/google-cloud-cpp
WORKDIR /usr/ports/devel/google-cloud-cpp/work/.build
RUN /usr/bin/env HOME=/usr/ports/devel/google-cloud-cpp/work \
    cmake \
        -DGOOGLE_CLOUD_CPP_DEPENDENCY_PROVIDER=package \
        -DCMAKE_INSTALL_RPATH_USE_LINK_PATH=YES \
        -DBUILD_SHARED_LIBS:BOOL=ON \
        -DBUILD_TESTING:BOOL=OFF \
        -DCMAKE_INSTALL_PREFIX=/usr/ports/devel/google-cloud-cpp/work/stage \
    /usr/ports/devel/google-cloud-cpp/work/google-cloud-cpp

RUN cmake --build . -- -j $(nproc)

RUN dnf makecache && dnf install -y findutils

RUN tar -cf /tmp/pre-install.tar .

RUN find /usr/ports/devel/google-cloud-cpp/work/.build -name annotations.grpc.pb.h | xargs ls --full-time

RUN cmake --build . --target install -- -j $(nproc)
RUN tar -cf /tmp/pos-install.tar .

RUN find /usr/ports/devel/google-cloud-cpp/work/.build -name annotations.grpc.pb.h | xargs ls --full-time
RUN find /usr/ports/devel/google-cloud-cpp/work/stage -name annotations.grpc.pb.h | xargs ls --full-time

RUN /bin/bash -c "diff <(tar tvf /tmp/pre-install.tar | sort) <(tar tvf /tmp/pos-install.tar | sort)"
Step 24/24 : RUN /bin/bash -c "diff <(tar tvf /tmp/pre-install.tar | sort) <(tar tvf /tmp/pos-install.tar | sort)"
 ---> Running in db988f1afd9b
2050d2049
< -rw-r--r-- root/root     13614 2019-07-14 20:08 ./external/googleapis/src/googleapis_project-build/install_manifest.txt
2098a2098
> -rw-r--r-- root/root     16096 2019-07-14 20:30 ./external/googleapis/src/googleapis_project-build/install_manifest.txt
2266a2267
> -rw-r--r-- root/root     34349 2019-07-14 20:30 ./install_manifest.txt
2864d2864
< drwxr-xr-x root/root         0 2019-07-14 20:08 ./
2873,2876d2872
< drwxr-xr-x root/root         0 2019-07-14 20:08 ./external/
< drwxr-xr-x root/root         0 2019-07-14 20:08 ./external/googleapis/
< drwxr-xr-x root/root         0 2019-07-14 20:08 ./external/googleapis/src/
< drwxr-xr-x root/root         0 2019-07-14 20:08 ./external/googleapis/src/googleapis_project-build/
3041d3036
< drwxr-xr-x root/root         0 2019-07-14 20:09 ./CMakeFiles/
3076a3072,3077
> drwxr-xr-x root/root         0 2019-07-14 20:30 ./
> drwxr-xr-x root/root         0 2019-07-14 20:30 ./CMakeFiles/
> drwxr-xr-x root/root         0 2019-07-14 20:30 ./external/
> drwxr-xr-x root/root         0 2019-07-14 20:30 ./external/googleapis/
> drwxr-xr-x root/root         0 2019-07-14 20:30 ./external/googleapis/src/
> drwxr-xr-x root/root         0 2019-07-14 20:30 ./external/googleapis/src/googleapis_project-build/

@yoshi-automation yoshi-automation added the triage me I really want to be triaged. label Jul 15, 2019
@yurivict
Copy link
Author

Docker uses its own Docker file to install into a container, and I install into the file system using cmake/gmake combination, so Docker isn't a good comparison.

cmake/gmake install these files:

/usr/ports/devel/google-cloud-cpp/work/.build/external/lib/libgoogleapis_cpp_bigtable_protos.so.0.2.0
/usr/ports/devel/google-cloud-cpp/work/.build/external/lib/libgoogleapis_cpp_bigtable_protos.so.0
/usr/ports/devel/google-cloud-cpp/work/.build/external/lib/libgoogleapis_cpp_bigtable_protos.so
/usr/ports/devel/google-cloud-cpp/work/.build/external/lib/libgoogleapis_cpp_spanner_protos.so.0.2.0
/usr/ports/devel/google-cloud-cpp/work/.build/external/lib/libgoogleapis_cpp_spanner_protos.so.0
/usr/ports/devel/google-cloud-cpp/work/.build/external/lib/libgoogleapis_cpp_spanner_protos.so
/usr/ports/devel/google-cloud-cpp/work/.build/external/lib/libgoogleapis_cpp_longrunning_operations_protos.so.0.2.0
/usr/ports/devel/google-cloud-cpp/work/.build/external/lib/libgoogleapis_cpp_longrunning_operations_protos.so.0
/usr/ports/devel/google-cloud-cpp/work/.build/external/lib/libgoogleapis_cpp_longrunning_operations_protos.so
/usr/ports/devel/google-cloud-cpp/work/.build/external/lib/libgoogleapis_cpp_api_http_protos.so.0.2.0
/usr/ports/devel/google-cloud-cpp/work/.build/external/lib/libgoogleapis_cpp_api_http_protos.so.0
/usr/ports/devel/google-cloud-cpp/work/.build/external/lib/libgoogleapis_cpp_api_http_protos.so
/usr/ports/devel/google-cloud-cpp/work/.build/external/lib/libgoogleapis_cpp_api_annotations_protos.so.0.2.0
/usr/ports/devel/google-cloud-cpp/work/.build/external/lib/libgoogleapis_cpp_api_annotations_protos.so.0
/usr/ports/devel/google-cloud-cpp/work/.build/external/lib/libgoogleapis_cpp_api_annotations_protos.so
/usr/ports/devel/google-cloud-cpp/work/.build/external/lib/libgoogleapis_cpp_api_auth_protos.so.0.2.0
/usr/ports/devel/google-cloud-cpp/work/.build/external/lib/libgoogleapis_cpp_api_auth_protos.so.0
...

If installed, it would write back into the same place where the build was performed: /usr/ports/devel/google-cloud-cpp/work/.build/external/

@ghost
Copy link

ghost commented Jul 15, 2019

Can you please show us your port files (Makefile, pkg-plist, etc.) and the build/install commands you are running?

@yurivict
Copy link
Author

Here is my port: https://people.freebsd.org/~yuri/google-cloud-cpp.patch

To reproduce the problem:

  1. patch the ports tree with the above patch
  2. cd /usr/ports/devel/google-cloud-cpp
  3. make
  4. make check-plist

make check-plist complains about the odd files installed by the port.

@ghost
Copy link

ghost commented Jul 15, 2019

Thanks. I got FreeBSD up and running in KVM. Running make check-plist does indeed complain about orphaned files from external. However, when I ran make install, this is the list of files I saw:

Screenshot_2019-07-15_00-07-52

(Sorry about screenshot, I don't have copy/paste setup)

cat log | grep "build/external" doesn't turn up anything, suggesting that the install target does not install back into .build/external. As I stated earlier, the problem most likely is make check-plist thinks the intermediary files in external are meant to be installed. I would configure your tool to ignore the directory.

@yurivict
Copy link
Author

yurivict commented Jul 15, 2019

So it is wrong, it should never mention /usr/ports/devel ... during install. It installs things into the ports tree, this is why make check-plist complains. This should never happen.

@ghost
Copy link

ghost commented Jul 15, 2019

Huh? It's installing things in /usr/ports/devel/google-cloud-cpp/work/stage, which sounds like the correct behavior to me. From the wiki:

The stage directory support means that a port does not install directly into the destination directories, but instead into a separate directory (automake packages call this DESTDIR) from which the package is then built - in many cases, this does not require root privileges. If enabled for a port, the package is first built, installed into the STAGEDIR, packaged, and then installed from the package. If disabled, the traditional approach is used, which means to install directly into the destination and build the package from there.

Your claim is that the install target is creating files in work/.build/, which is not the behavior I observe. make build creates files in work/.build/ then make install creates files in work/stage/. Unless I'm missing something.

@yurivict
Copy link
Author

yurivict commented Jul 15, 2019

Huh? It's installing things in /usr/ports/devel/google-cloud-cpp/work/stage

No, you misunderstand it:
make in the port includes cmake's all and install targets. The install target installs into the local stage directory under work/stage. You should never see anything mentioning ports there. But ports is present for google-cloud-cpp.

make install installs from the stage directory into the final destination /usr/local. Again, it should never install anything under /usr/ports, but it does.

Your cmake's install target is faulty, it installs under work/stage/usr/ports/...

make check-plist is de-facto very reliable. When it complains - something is wrong.

You can for example compare with /usr/ports/net-im/qTox. It is also a cmake project, make check-plist doesn't complain there, because it installs only into /usr/local.

@ghost
Copy link

ghost commented Jul 15, 2019

Your cmake's install target is faulty, it installs under work/stage/usr/ports/...

Again, I must be missing something here. The files are installed to /usr/ports/devel/google-cloud-cpp/work/stage/usr/local/include (per the screenshot) not /usr/ports/devel/google-cloud-cpp/work/stage/usr/ports. Furthermore, when I look in /usr/local/include/google/api I can see the installed files. I have no ports directory under /usr/local/include.

@yurivict
Copy link
Author

Yes, it does install the correct files, but in addition it also installs wrong files.
find work/stage -name ports would find them.

@ghost
Copy link

ghost commented Jul 15, 2019

This is what I suspect is happening and @coryan can correct my math here. As googleapis is an external project, its files are built and installed to external/. We then later install the files to ${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_INCLUDEDIR}. Because the prefix for the external project is a relative path, it is probably expanding to usr/ports/devel/google-cloud-cpp/work/google-cloud-cpp-4dff795ca when put in the stage directory.

While I certainly agree that the external project intermediary files should not be installed to stage, our CMake install target does not install anything to external. That only happens when building the external projects.

I will defer to @coryan for what the correct action to take here is.

@coryan
Copy link
Contributor

coryan commented Jul 15, 2019

I am still confused about what is the problem here. As far as I can tell:

  • cmake --target install produce a complete set of files in $CMAKE_INSTALL_PREFIX (aka $DESTDIR)
  • cmake --target install only creates files in $CMAKE_BINARY_DIR or $CMAKE_INSTALL_PREFIX.

If that is what happens, then things are working as intended and I do not understand what your tooling is complaining about.

If that is not what is happening then we have a bug indeed, but I still do not understand what files are installed incorrectly and where.

@coryan coryan added type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. priority: p2 Moderately-important priority. Fix may not be included in next release. and removed triage me I really want to be triaged. labels Jul 15, 2019
@ghost
Copy link

ghost commented Jul 15, 2019

If that is not what is happening then we have a bug indeed, but I still do not understand what files are installed incorrectly and where.

In addition to the regular files being installed to stage/usr/include, the build artifacts from googleapis are also being installed to stage/usr/ports/. I think the install target for the external project is confusing the FreeBSD tooling and causing those files to be copied over when they shouldn't.

@yurivict
Copy link
Author

It dhouldn't install anything under stage/usr/ports, because everything under stage/ ends up being installed into the system, and it is clearly wrong when the files are installed into a temporary build directory.

@coryan
Copy link
Contributor

coryan commented Jul 15, 2019

If that is not what is happening then we have a bug indeed, but I still do not understand what files are installed incorrectly and where.

In addition to the regular files being installed to stage/usr/include, the build artifacts from googleapis are also being installed to stage/usr/ports/.

Thanks, now I get it. That sounds like a problem indeed.

I think the install target for the external project is confusing the FreeBSD tooling and causing those files to be copied over when they shouldn't.

Is there a way for us to extract the cmake and make commands executed by the FreeBSD tooling? When I execute the typical sequence: cmake -H... -B...; cmake --build ...; cmake --build ... --target install I cannot reproduce the problem, so I must be doing something different that the tooling, and it is not obvious to me what the difference is.

In any case, I expect that migrating to use the (new) googleapis/cpp-cmakefiles repository might help (or then again, maybe we will have the same problem there...)

@yurivict
Copy link
Author

Is there a way for us to extract the cmake and cmake commands executed by the FreeBSD tooling?

Yes.
For configure command, uncomment (remove @) in the do-configure target in /usr/ports/Mk/Uses/cmake.mk

For build command, uncomment the do-build commands in /usr/ports/Mk/bsd.port.mk

For install (into stage/) command, uncomment the do-install command in /usr/ports/Mk/bsd.port.mk

@ghost
Copy link

ghost commented Jul 15, 2019

Thanks. I've uploaded the logs for analysis.

configure log: http://sprunge.us/dgfZKR

build log: http://sprunge.us/IZwzmP

install log: http://sprunge.us/cvnReQ

@ghost
Copy link

ghost commented Jul 15, 2019

So as suspected, the problem occurs during the external project install target. During the make build stage, when we build/install the external project, it's installing the files to work/stage/usr/ports at step [87/142]. Here is the log for google-cloud-cpp-4dff795ca/external/googleapis/src/googleapis_project-stamp/googleapis_project-install-out.log.

I believe the problem is that for our purposes we actually expect the files to be installed to work/google-cloud-cpp-4dff795ca/external/lib, etc. But for some reason it's instead installing them to the stage directory.

@coryan
Copy link
Contributor

coryan commented Jul 16, 2019

DESTDIR is set as an environment variable even in the build phase (see below, from http://sprunge.us/IZwzmP)

That overrides the default settings for CMAKE_INSTALL_PREFIX in the ExternalProject_Add() for googleapis_project:

INSTALL_DIR "${CMAKE_BINARY_DIR}/external"

so instead of installing in ${CMAKE_BINARY_DIR}/external it installs in the staging area.

We could change the ExternalProject_add() to override DESTDIR again, but it seems simpler to just use the new repository for cpp-cmakefiles as we are planning to remove all the external projects anyway (see #2802 ).

===>  Building for google-cloud-cpp-0.11.0.g20190710
--- do-build ---
(cd /usr/ports/devel/google-cloud-cpp/work/google-cloud-cpp-4dff795ca; if ! /usr/bin/env OPENSSLBASE=/usr OPENSSLDIR=/etc/ssl OPENSSLINC=/usr/include OPENSSLLIB=/usr/lib XDG_DATA_HOME=/usr/ports/devel/google-cloud-cpp/work  XDG_CONFIG_HOME=/usr/ports/devel/google-cloud-cpp/work  HOME=/usr/ports/devel/google-cloud-cpp/work PATH=/usr/ports/devel/google-cloud-cpp/work/.bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin NO_PIE=yes MK_DEBUG_FILES=no MK_KERNEL_SYMBOLS=no SHELL=/bin/sh NO_LINT=YES DESTDIR=/usr/ports/devel/google-cloud-cpp/work/stage PREFIX=/usr/local  LOCALBASE=/usr/local  CC="cc" CFLAGS="-O2 -pipe  -I/usr/ports/devel/google-cloud-cpp/work/google-cloud-

@coryan coryan added the status: blocked Resolving the issue is dependent on other work. label Jul 16, 2019
@coryan
Copy link
Contributor

coryan commented Jul 16, 2019

Blocked by #2802

@coryan coryan removed the status: blocked Resolving the issue is dependent on other work. label Aug 15, 2019
@coryan
Copy link
Contributor

coryan commented Aug 15, 2019

AFAICT this is fixed. I ran a build with DESTDIR set to an empty directory and no files were created in $DESTDIR during the build phase.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

No branches or pull requests

3 participants