Matched problems dropped when running in container

[alexanderweiss]

Describe the bug
A problem matcher enabled using a 'normal' run step in a job running in a container drops all issues with "Dropping file value ... Path is not under the workflow repo." Removing the container option from the workflow file (such that it runs on ubuntu-latest) and leaving all configuration unchanged does not drop the issues and creates annotations as expected.

To Reproduce
Steps to reproduce the behavior:

1. Create a workflow with a configuration similar to (the example uses eslint):

name: PR Linting

on:
  pull_request:
    branches: [ develop ]

jobs:
  lint:
    name: Lint
    runs-on: ubuntu-latest
    container:
      image: node:14.13.1

    steps:
      - uses: actions/checkout@v2

      - name: Enable problem matcher
        run: echo "::add-matcher::.github/problem-matchers/eslint.json"

      - name: Run linter
        run: npm run lint

2. Add the actual problem matcher (I used the multiline example: https://github.com/actions/toolkit/blob/main/docs/problem-matchers.md)
3. Run it
4. Warnings / errors will be highlighted, but no annotations created.
5. Remove the container section from the definition
6. Warnings / errors will be highlighted and annotations will be created.

Expected behavior
Matched issues resulting in annotations when running in a container.

Runner Version and Platform

2.273.5 on GitHub.com

OS of the machine running the runner? OSX/Windows/Linux/...
Ubuntu 18.04.5

What's not working?

Job Log Output

See screenshot

[DaAwesomeP]

I'm also experiencing this issue. For me, it seems the problem matcher is not mounting the container volumes properly.

In a non-container job on ubuntu-latest:

• $GITHUB_WORKSPACE is /home/runner/work/ola/ola
• ${{ github.workspace }} is /home/runner/work/ola/ola
• The starting working directory is /home/runner/work/ola/ola
• Problem matcher works properly

In a job with container: debian:stable on ubuntu-latest:

• $GITHUB_WORKSPACE is /__w/ola/ola
• ${{ github.workspace }} is /home/runner/work/ola/ola
• The starting working directory is /__w/ola/ola
• Problem matcher prepends prepends /home/runner/work/ola/ola instead of /__w/ola/ola and fails to find files
• Running the checkout action does not change the working directory

In the container initialization I can see:

/usr/bin/docker create --name 2e276813af5c499883d5d5c8d44da4fd_debianstable_ae8621 --label 49859c --workdir /__w/ola/ola --network github_network_145999030d9d436eab6eb23e4e67c5cb  -e "HOME=/github/home" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work":"/__w" -v "/home/runner/runners/2.301.1/externals":"/__e":ro -v "/home/runner/work/_temp":"/__w/_temp" -v "/home/runner/work/_actions":"/__w/_actions" -v "/opt/hostedtoolcache":"/__t" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" --entrypoint "tail" debian:stable "-f" "/dev/null"

This specifically includes: -v "/home/runner/work":"/__w" and --workdir /__w/ola/ola. This would suggest that files destined for /__w/ola/ola in the container should end up back in /home/runner/work/ola/ola on the container host.

So, either:

1. The matcher is running within the container but is using ${{ github.workspace }} instead of $GITHUB_WORKSPACE, leading to the wrong path.
2. The matcher is running on the container host (outside the container) but the Docker volume is not working properly, so the files in the container are not ending up in /home/runner/work/ola/ola on the host.

For case 1), I tried:

- shell: bash
  run: |
    if [ "$GITHUB_WORKSPACE" != "${{ github.workspace }}" ]; then
      mkdir -p "$(dirname "${{ github.workspace }}")"
      ln -s "$GITHUB_WORKSPACE" "${{ github.workspace }}"
    fi
    ls -al "${{ github.workspace }}"

as well as (in case it doesn't follow symlinks):

- shell: bash
  run: |
    if [ "$GITHUB_WORKSPACE" != "${{ github.workspace }}" ]; then
      mkdir -p "$(dirname "${{ github.workspace }}")"
      cp -r "$GITHUB_WORKSPACE/." "${{ github.workspace }}/"
    fi
    ls -al "${{ github.workspace }}"

However, it seems that unfortunately the issue is 2). So this volume "/home/runner/work":"/__w" does not appear to be functioning properly.

One possibility is a UID/GID permissions mismatch between the host volume and the user that the container is running as. Normally the container runs as root (0:0), but I tried a workaround just in case maybe the host is on a rootless NFS share or something like that (although running mount on the host confirmed this was not the case). This would be quite annoying, as most out-of-the-box generic containers may need root access in one way or another (i.e. to run apt). Running ls -aln . shows that the folder is owned by UID:GID 1001:123 (runner:docker). So I tried:

container:
  image: debian:stable
  options: "--user 1001:123"

I confirmed the container was now running as that user with id and unfortunately the problem persists.

I think that is the length that I can go to troubleshoot without being able to run commands on the runner host after the container has run.

This whole issue is also peculiar because running echo "::add-matcher::.github/problem-matcher-lint-cpplint.json" works every time without issue, and it is also looking up a relative path in the repo.

cc @electronjoe, this does seem very related or the same issue as #1009.

[DaAwesomeP]

I should also note that in my case, I am matching a relative path. My workflow will print something like:

./relative/path/to/file.txt:1: There is an issue at line 1.

This file definitely exists from the repo root and from the current working directory, which is /__w/ola/ola.

[DaAwesomeP]

Variable mismatch related to #2058

[DaAwesomeP]

The answer is found from looking at this section of code:

runner/src/Runner.Worker/Handlers/OutputManager.cs

Lines 304 to 325 in aaf02ab

	// Check whether the file exists
	if (File.Exists(file))
	{
	// Check whether the file is under the workflow repository
	var repositoryPath = GetRepositoryPath(file);
	if (!string.IsNullOrEmpty(repositoryPath))
	{
	// Get the relative file path
	var relativePath = file.Substring(repositoryPath.Length).TrimStart(Path.DirectorySeparatorChar);
	// Prefer `/` on all platforms
	issue.Data["file"] = relativePath.Replace(Path.DirectorySeparatorChar, Path.AltDirectorySeparatorChar);
	}
	else
	{
	_executionContext.Debug($"Dropping file value '{file}'. Path is not under the workflow repo.");
	}
	}
	else
	{
	_executionContext.Debug($"Dropping file value '{file}'. Path does not exist");
	}

Because the error printed is Path is not under the workflow repo. and not Path does not exist, the issue appears to stem from GetRepositoryPath() searching for .git/config.

Puzzlingly, actions/checkout@v3 doesn't appear to create a .git folder in the debian:stable container because...well Git wasn't installed and it resorted to a REST request.

So my solution was:

 - name: Update package database
   run: apt-get update -y
 - name: Install Git
   run: apt-get -y install git
 - uses: actions/checkout@v3

For jobs that build on previous jobs using artifacts and do not run checkout themselves: I am now making sure that the .git folder is carried over.

I think this exposes a broader issue, which is that the build should provide a way to specify the base repo folder without requiring a .git folder.