The runner auto upgrade faild after download three times.

[ailncode]

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:
1.The runner auto update when my CI/CD job start
2.It's download package faild three times.(My runner in china server,the net speed is very slow when download package.)
3.The runner exit and my CI/CD job wait forever^_^.
Expected behavior
I think my CI/CD job is the more important than update.
I want close auto update for the runner.

Runner Version and Platform

v2.168.0
OS of the machine running the runner? OSX/Windows/Linux/...
Linux

What's not working?

Job Log Output

If applicable, include the relevant part of the job / step log output here. All sensitive information should already be masked out, but please double-check before pasting here.

Runner and Worker's Diagnostic Logs

[2020-04-22 06:40:32Z INFO RSAFileKeyManager] Loading RSA key parameters from file /home/github-runner/tssns.backend.runner/.credentials_rsaparams
[2020-04-22 06:40:33Z INFO GitHubActionsService] AAD Correlation ID for this token request: Unknown
[2020-04-22 06:43:55Z INFO MessageListener] No message retrieved from session 'f25a2900-b46b-4736-b564-f735c0569aef' within last 30 minutes.
[2020-04-22 06:45:22Z WARN SelfUpdater] Runner download has timed out after 900 seconds
[2020-04-22 06:45:22Z WARN SelfUpdater] Failed to get package '/home/github-runner/tssns.backend.runner/_work/_update/runner1.tar.gz' from 'https://github.com/actions/runner/releases/download/v2.169.0/actions-runner-linux-x64-2.169.0.tar.gz'. Exception System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
---> System.Net.Sockets.SocketException (125): Operation canceled
--- End of inner exception stack trace ---
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
at System.Net.Security.SslStream.g__InternalFillBufferAsync|215_0[TReadAdapter](TReadAdapter adap, ValueTask1 task, Int32 min, Int32 initial) at System.Net.Security.SslStream.ReadAsyncInternal[TReadAdapter](TReadAdapter adapter, Memory1 buffer)
at System.Net.Http.HttpConnection.FillAsync()
at System.Net.Http.HttpConnection.CopyToContentLengthAsync(Stream destination, UInt64 length, Int32 bufferSize, CancellationToken cancellationToken)
at System.Net.Http.HttpConnection.ContentLengthReadStream.CompleteCopyToAsync(Task copyTask, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.ContentLengthReadStream.CompleteCopyToAsync(Task copyTask, CancellationToken cancellationToken)
at GitHub.Runner.Listener.SelfUpdater.DownloadLatestRunner(CancellationToken token)
[2020-04-22 06:45:22Z INFO JobDispatcher] Shutting down JobDispatcher. Make sure all WorkerDispatcher has finished.
[2020-04-22 06:45:23Z ERR Terminal] WRITE ERROR: An error occurred: Runner package '/home/github-runner/tssns.backend.runner/_work/_update/runner1.tar.gz' failed after 3 download attempts
[2020-04-22 06:45:23Z ERR Listener] System.Threading.Tasks.TaskCanceledException: Runner package '/home/github-runner/tssns.backend.runner/_work/_update/runner1.tar.gz' failed after 3 download attempts
at GitHub.Runner.Listener.SelfUpdater.DownloadLatestRunner(CancellationToken token)
at GitHub.Runner.Listener.SelfUpdater.SelfUpdate(AgentRefreshMessage updateMessage, IJobDispatcher jobDispatcher, Boolean restartInteractiveRunner, CancellationToken token)
at GitHub.Runner.Listener.Runner.RunAsync(RunnerSettings settings, Boolean runOnce)
at GitHub.Runner.Listener.Runner.RunAsync(RunnerSettings settings, Boolean runOnce)
at GitHub.Runner.Listener.Runner.RunAsync(RunnerSettings settings, Boolean runOnce)
at GitHub.Runner.Listener.Runner.ExecuteCommand(CommandSettings command)
at GitHub.Runner.Listener.Program.MainAsync(IHostContext context, String[] args)

[jcleira]

Happened to me too
Fixed by manually updating, in my case:

cd actions-runner
curl -O -L https://github.com/actions/runner/releases/download/v2.169.1/actions-runner-linux-x64-2.169.1.tar.gz
tar xzf ./actions-runner-linux-x64-2.169.1.tar.gz

[ailncode]

CI/CD must be an automation process.
But, I need fixed it by manually every times when the runner need upgrade.
I think the CI/CD jobs is more important than upgrade.

[TingluoHuang]

@ailncode, @jcleira how long does it take for you to download the runner package using CURL or browser? the timeout on download is 15 minutes in the runner.

[ailncode]

@TingluoHuang I'm not sure. some times it's very fast(50kB/s) and some times it's very slow(0kB/s).
and GFW may be disconnect the download...

curl -O -L https://github.com/actions/runner/releases/download/v2.169.1/actions-runner-linux-arm64-2.169.1.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 654 100 654 0 0 213 0 0:00:03 0:00:03 --:--:-- 213
0 53.5M 0 509k 0 0 6845 0 2:16:38 0:01:16 2:15:22 6691

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 654 100 654 0 0 213 0 0:00:03 0:00:03 --:--:-- 213
1 53.5M 1 1087k 0 0 3973 0 3:55:25 0:04:40 3:50:45 0

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 654 100 654 0 0 213 0 0:00:03 0:00:03 --:--:-- 213
2 53.5M 2 1240k 0 0 1376 0 11:19:44 0:15:23 11:04:21 0

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 654 100 654 0 0 213 0 0:00:03 0:00:03 --:--:-- 213
2 53.5M 2 1240k 0 0 1050 0 14:50:46 0:20:08 14:30:38 0
curl: (56) GnuTLS recv error (-54): Error in the pull function.

What can i do?

[shoukoo]

@TingluoHuang I am also having an issue when the runner auto-upgrades from version 2.169.1 to 2.262.1.

ay 15 19:30:13 ip-172-27-42-xx runner.sh[2839]: Runner update in progress, do not shutdown runner.
May 15 19:30:13 ip-172-27-42-xx runner.sh[2839]: Downloading 2.262.1 runner
May 15 19:30:21 ip-172-27-42-xx runner.sh[2839]: Waiting for current job finish running.
May 15 19:30:21 ip-172-27-42-xx runner.sh[2839]: Generate and execute update script.
May 15 19:30:21 ip-172-27-42-xx runner.sh[2839]: Runner will exit shortly for update, should back online within 10 seconds.
May 15 19:30:21 ip-172-27-42-xx runner.sh[2839]: /home/runner/_work/_update.sh: line 31: nul: Permission denied
May 15 19:30:21 ip-172-27-42-xx runner.sh[2839]: /home/runner/_work/_update.sh: line 31: nul: Permission denied
May 15 19:30:21 ip-172-27-42-xx runner.sh[2839]: /home/runner/_work/_update.sh: line 31: nul: Permission denied
May 15 19:30:21 ip-172-27-42-xx runner.sh[2839]: /home/runner/_work/_update.sh: line 31: nul: Permission denied

This is in the _update.sh

# wait for runner process to exit.
date "+[%F %T-%4N] Waiting for $runnerprocessname ($runnerpid) to complete" >> "$logfile" 2>&1
while [ -e /proc/$runnerpid ]
do
    date "+[%F %T-%4N] Process $runnerpid still running" >> "$logfile" 2>&1
    ping -c 2 127.0.0.1 >nul
done
date "+[%F %T-%4N] Process $runnerpid finished running" >> "$logfile" 2>&1

# start re-organize folders
date "+[%F %T-%4N] Sleep 1 more second to make sure process exited" >> "$logfile" 2>&1
ping -c 2 127.0.0.1 >nul

Is there a flag to prevent the runner from auto-upgrading itself?

[TingluoHuang]

@shoukoo for your error nul: Permission denied, please check https://unix.stackexchange.com/a/146639

[pjmartorell]

@shoukoo for your error nul: Permission denied, please check https://unix.stackexchange.com/a/146639

This nul bug was fixed #647

[thboop]

Closing as this issue was fixed, if you see it again please comment in this issue and we can reopen.

Work to disable auto-update is being tracked here and in progress: #485