Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LauncherSpec: Check that all launched processes do exit #986

Merged
merged 5 commits into from
Nov 12, 2019

Conversation

rvl
Copy link
Contributor

@rvl rvl commented Nov 8, 2019

Relates to #703.

Overview

It appears that the jormungandr node backend process was not getting cleaned up on Windows and that actions which are supposed to run concurrently with the backend were not running.

These extra tests check that the withCreateProcess function takes care of terminating the process -- if it is given a chance (i.e. if the process is not killed with -9, or "End Task" on Windows).

It also checks that the concurrent actions run while the backend process is running, and that the backend process is terminated when the other action completes.

Finally, it implements a workaround for the unwanted behaviour of the process library on Windows where waitForProcess seems to block all concurrent async actions in the thread.

  • Adds a test to cardano-wallet-launcher:test:unit for process clean up.
  • Adjusts commands used so that the tests can be run under Wine.
  • Adds an assertion to check that the process is killed if the action does not complete.
  • Adds an assertion to check that the the process is killed promptly if the action completes.
  • Fixes async blocking issue on Windows.

Testing under Wine

Use something like this:

wine $(nix-build release.nix -A x86_64-pc-mingw32.tests.cardano-wallet-launcher.unit.x86_64-linux -o launcher-unit-windows)/cardano-wallet-launcher-2019.11.7/unit.exe --match "Backend process"

@rvl rvl self-assigned this Nov 8, 2019
@rvl rvl force-pushed the rvl/703/windows-process-cleanup branch 5 times, most recently from 7ad8e91 to 313ce1f Compare November 12, 2019 12:48
@rvl rvl marked this pull request as ready for review November 12, 2019 12:52
- Fix backend process concurrency issue on Windows

- Also get the LauncherSpec working on wine.
  It needs to detect Wine, then use commands which are available on wine.

- LauncherSpec: Add an assertion that withCreateProcess does not block
@rvl rvl force-pushed the rvl/703/windows-process-cleanup branch from 313ce1f to 6a05496 Compare November 12, 2019 13:15
-> IO ExitCode
interruptibleWaitForProcess tr' ph = do
status <- newEmptyMVar
void $ forkIO $ waitThread status `onException` continue status
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Duncan's variant used finally rather than onException here which has a slightly different meaning (finally will also execute the action in case of normal exits).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will likely be used only for starting the backend / server which should never actually exit normally, but we are creating a potential deadlock by doing this. Is it intentional @rvl ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I needed to change it to use onException because we are capturing exit status. Otherwise it would try to putMVar twice

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmmm. I see okay, there's actually a putMVar in waitThread itself which I guess is safe (provided the launcherLog doesn't throw. But I am ready to accept anything at this stage).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that because the putMVar is the last action in the block, either way there will be something put.

setupTrace_ cfg "tests"
after (tr, sb) = do
logDebug tr "Logging shutdown."
shutdown sb
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

after <- getCurrentTime
ph <- takeMVar mvar
assertProcessesExited [ph]
diffUTCTime after before `shouldSatisfy` (< 2)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not obvious that this 2 (seconds) actually refers to 1000000 (microseconds) above with a safe margin (which is what I assume).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, fixed with a comment.

@KtorZ
Copy link
Member

KtorZ commented Nov 12, 2019

bors r+

iohk-bors bot added a commit that referenced this pull request Nov 12, 2019
986: LauncherSpec: Check that all launched processes do exit r=KtorZ a=rvl

Relates to #703.

# Overview

It appears that the `jormungandr` node backend process was not getting cleaned up on Windows and that actions which are supposed to run concurrently with the backend were not running.

These extra tests check that the [`withCreateProcess`](http://hackage.haskell.org/package/process-1.6.6.0/docs/System-Process.html#v:withCreateProcess) function takes care of terminating the process -- if it is given a chance (i.e. if the process is not killed with -9, or "End Task" on Windows).

It also checks that the concurrent actions run while the backend process is running, and that the backend process is terminated when the other action completes.

Finally, it implements a workaround for the unwanted behaviour of the `process` library on Windows where `waitForProcess` seems to block all concurrent async actions in the thread.

- [x] Adds a test to `cardano-wallet-launcher:test:unit` for process clean up.
- [x] Adjusts commands used so that the tests can be run under Wine.
- [x] Adds an assertion to check that the process is killed if the action does not complete.
- [x] Adds an assertion to check that the the process is killed promptly if the action completes.
- [x] Fixes async blocking issue on Windows.

### Testing under Wine

Use something like this:
```
wine $(nix-build release.nix -A x86_64-pc-mingw32.tests.cardano-wallet-launcher.unit.x86_64-linux -o launcher-unit-windows)/cardano-wallet-launcher-2019.11.7/unit.exe --match "Backend process"
```


Co-authored-by: Rodney Lorrimar <[email protected]>
@iohk-bors
Copy link
Contributor

iohk-bors bot commented Nov 12, 2019

Build succeeded

@iohk-bors iohk-bors bot merged commit af47c2e into master Nov 12, 2019
@KtorZ KtorZ deleted the rvl/703/windows-process-cleanup branch November 12, 2019 19:19
@KtorZ KtorZ added this to the Usability & Compatibility milestone Nov 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants