Debugger frozen after first breakpoint

[jackmusick]

System Details

• Operating system name and version: macOS 10.12.6
• VS Code version: 1.14.2
• PowerShell extension version: 1.14.1
• Output from $PSVersionTable:

1.14.2
cb82febafda0c8c199b9201ad274e25d9a76874e
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]

Name                           Value                                           
----                           -----                                           
PSVersion                      6.0.0-beta                                      
PSEdition                      Core                                            
GitCommitId                    v6.0.0-beta.5                                   
OS                             Darwin 16.7.0 Darwin Kernel Version 16.7.0: T...
Platform                       Unix                                            
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}                         
PSRemotingProtocolVersion      2.3                                             
SerializationVersion           1.1.0.1                                         
WSManStackVersion              3.0   

Reproduce

1. Set breakpoint anywhere in the script.
2. Run 'PowerShell Launch Current File'
3. Once breakpoint has been hit, click Continue or Step In.

Issue Description

I cannot use the debugger in VS Code as expected. Once I try to step into a breakpoint or continue running a script, the debugging process appears to freeze. There are no visible errors and I can continue navigating and using VS Code. To continue and reproduce the issue, again, I have to stop the debugging process and restart the console.

Additionally, I've gone through and removed the extension, tried the latest insider build of VS Code and rolled back to PowerShell 6 Beta 3 and 4.

Attached Logs

logs.zip

[daviwil]

Thanks Jack, I'll take a look.

[jackmusick]

I try and troubleshoot this at least once a week. Something I discovered today is that if I hit enter in the integrated console window, the debugger will move to the next line of code. Here are my steps:

1. Set breakpoint
2. Start debugging
3. Breakpoint is hit, click the next button or the start button. The debugger freezes.
4. Hit enter in the Integrated Terminal window, debugger continues onto the next breakpoint or line, depends on if I hit next or the start button.
5. Hit next or start, the debugger continues.
6. Hit next or start again, I have to hit enter in the Integrated Terminal.

It seems like it's working every other time for whatever reason, without hitting enter at least.

DebugAdapter.log.zip

[devoncustard]

Version 1.16.1 suddenly stopped working. Have noticed that sometimes it will eventually (like up to 30 seconds) step to next line, but even as poor a coder as i am i dont need that long to read a line of code!

Reinstall worked on my work desktop which isnt Anniversay edition so im thinking it might be related to PS5.1?

[jackmusick]

Reinstall worked on my work desktop which isnt Anniversay edition so im thinking it might be related to PS5.1?

My issue is strictly with macOS and PowerShell 6, unfortunately.

[jackmusick]

I've just tested this with a fresh install of macOS and can confirm it's still a problem.

[rkeithhill]

Can you attach a zip of the EditorServices log file? If the debug adapter log file indicates a connection was made then we need to see the EditorServices log file.

BTW sorry for the delay on this. There's been a change in personnel working on this extension and the new folk(s) need to come up to speed - after they free up from the push to release PS Core 6.

[jackmusick]

I've attached the log. This was a fresh log after running the debugger and watching my breakpoints freeze. I can get them to continue if I hit the enter key in the terminal, but I have to do that every other thing.
logs.zip

[rkeithhill]

I think what you are seeing is a bug that exists on Linux and macOS where once a breakpoint has been hit, you have to press a key in the integrated console before you can resume debugging. Can you try this to see if that is indeed the case. It's not really a workaround as it's a major PITA to have to do this whenever a breakpoint has been hit.

[jackmusick]

Yes, that's the one. Any ETA on a fix? It's been like this for a few months, now. It makes PowerShell almost impossible to use in VS Code.

[rkeithhill]

@SeeminglyScience Are you working on a fix for the console ReadKey() bug on Linux/macOS?

[SeeminglyScience]

@rkeithhill Not currently, I'm working on getting PSReadLine working but that has the same issue unfortunately. I'll see if I can come up with something this weekend.

[SeeminglyScience]

I got a Linux environment set up to dig into this issue, and I think I found the cause.

1. The Unix System.Console implementation in corefx uses escape sequences for both input and cursor
   position, and when it's doing either of those it locks the internal StdInReader.

2. The way we "cancel" ReadKey in Windows is to just run it in another thread, and if it's cancelled
   we just ignore the output. It's still there waiting for a key, Windows just doesn't lock anything.

3. The default console host, our host implementation, and PSReadLine all use Console.CursorTop/Left
   when resetting the prompt after a pipeline finishes.

Add all that together and it looks like ReadKey is blocking the pipeline thread, but it's really the
host blocking it while getting the cursor position because ReadKey is blocking stdin. This seems to be an accepted drawback to the current System.Console implementation (assumption based on this comment in corefx)

Here's some easier repo steps that work in pwsh without PSES

$rs = [runspacefactory]::CreateRunspace($Host)
$rs.Open()
$ps = [powershell]::Create()
$ps.Runspace = $rs
$handle = $ps.AddScript('[Console]::ReadKey($true)').BeginInvoke()
Start-Sleep 1

If you debug pwsh on Linux, run the above, and pause the debugger it'll be stopped here waiting for cursor position.

A possible (but sort of dirty) solution is to write a query escape sequence like Device Attributes to
Console.Out. That would write a response escape sequence directly into stdin, bypassing the lock and
releasing ReadKey. The main issue with that is the response is more than one character, so we'd have
to also drain pending input.

Here's an example of what I mean using the repo steps above

$rs = [runspacefactory]::CreateRunspace($Host)
$rs.Open()
$ps = [powershell]::Create()
$ps.Runspace = $rs
$handle = $ps.AddScript('[Console]::ReadKey($true)').BeginInvoke()
Start-Sleep 1
[Console]::Write("`u{1b}[0c")
while ([Console]::KeyAvailable) {
    $null = [Console]::ReadKey($true)
}

@tylerl0706 @daviwil @rkeithhill Any concerns with this approach?

[rkeithhill]

One concern is the use of KeyAvailable and the fact that it echos chars to the screen on Linux/macOS - see https://github.com/dotnet/corefx/issues/16352.

My original attempt at fixing this was to use KeyAvailable in a cancellable loop before we called ReadKey() but that suffered the echoing chars problem. And backspace didn't work either. I wonder if ReadKey() was stealing position-related esc seqs from the input stream?

Since you are using pwsh there is a direct escape code for esc which is `e e.g. "`e[0c". I added that before the `u{} support. :-)

[SeeminglyScience]

@rkeithhill Hmm, it didn't echo when I was testing it, and I can't get it to echo with the example. But it does echo if I take away the initial call to ReadKey in a different runspace. I'll have to play around with it to see if I can figure out why it doesn't echo there. I think maybe the pending ReadKey makes it read the entire stream without echo and just pushes the remaining characters into a buffer, but that's a guess.

I should clarify though, KeyAvailable may not be required as long as the escape sequence emits a set number of characters. It's not being used to read user input, just to consume the rest of the escape sequence so it doesn't bleed into future ReadKey calls.

For example, in my Ubuntu vm the escape sequence emits this:

KeyChar    Key Modifiers
-------    --- ---------
      � Escape         0
      ?      0         0
      6     D6         0
      2     D2         0
      ;      0         0
      c      C         0

The escape releases the first ReadKey, the while loop consumes the rest.

It would be ideal if there was an escape sequence that emits a single character, but I don't know of one :\

Since you are using pwsh there is a direct escape code for esc which is `e e.g. "`e[0c". I added that before the `u{} support. :-)

Awesome! I didn't know about that one, that's really cool :)

[SeeminglyScience]

Tried that out, didn't work great unsurprisingly.

The only way I was able to fix this was by pulling out the little bit of native code corefx uses and working that into a implementation similar to yours @rkeithhill. It appears to work great, but a discussion needs to be had about how it should be integrated. I can think of three options.

1. Merge it as is. I don't really like this option because it complicates what is currently a very approachable build process. This would also probably add some extra manual steps to the release process, because the native code can only be built on Unix and the net452 assemblies can only be built on Windows. Not to mention adding native code to the scope of the project.

2. Create a separate project/package for the tiny bit of code to turn off input echo.

3. Wait for corefx to add a ReadKeyAsync method (or similar). Honestly, this is probably the best choice.

Here's the branch with the current progress.

[TylerLeonhardt]

@SeeminglyScience Looks like @rkeithhill opened an issue about your 3rd choice a little over a month ago https://github.com/dotnet/corefx/issues/25036 - maybe it's worth pinging on that issue to see what they say?

[daviwil]

The last time I asked them for this they didn't seem very interested, hopefully it's different this time.

I agree with Patricks approach #2, creating a small helper library that doesn't need to get recompiled with PSES. If we could make it work then we only need to compile/sign/ship it once and then use it in future PSES releases until corefx gains a ReadKeyAsync API.

Very nice work, @SeeminglyScience, you rock!

[rkeithhill]

@SeeminglyScience I like option 2 as well. Hopefully option 3 comes to pass but who knows how long that will take. We can't afford to wait - folks are getting pretty upset about this. And if option 3 does happen, then the changes to PSES would be minimized vs option 1.

[TylerLeonhardt]

+1 for option 2!