Debugging sort of works again when AD7Thread stops throwing exception for IDebugThread2.CanSetNextStatement

[ghost]

Hi

First of all very happy about the work! I know you guys have stopped looking at Visual Studio 2012 but I have still not yet upgraded. So hence my question. Not bug.

I noticed the debugging did not hit breakpoints anywhere when using latest with NodeJs. Switched to io.js and still nothing. Desperate to see if I could get this to work I managed to track it down to a NotImplementedException being thrown out of the AD7Thread's implementation of IDebugThread2.CanSetNextStatement.

When I traded out throw for return 0; the break points started getting hit again after I recompiled and re-installed the plugin locally.

Is the answer to upgade VS latest or is this something that could be accommodated going foward? Any info on what the future of the debugging interface would use would also be great to know.

[mousetraps]

Although it doesn't get as much testing... We actually do support 2012 for 1.0 so apologies for the gross oversight :-( We can look into this particular issue this week. What do you mean by "sort of works"?

Node.js's debugging and profiling APIs aren't stable yet, which has been one of the primary challenges with NTVS. There are people working on stabilizing them, but I'm not entirely sure what their progress is.

Re:2012 support, our next milestone will focus on VS2015 + bugfixes so we can include this fix there (as well as in a Dev build).

Going forward, we don't have any hard plans, but our core team is pretty small and it's going to be difficult to support three versions of VS, so it's definitely a discussion we'll need to have.

Just so we can get a sense... How frequently does your company typically upgrade?

[mousetraps]

Hmm we just tried this out on an express app in VS 2012, win7, and break points are indeed hit with v0.12.2 64-bit.

Could you give use a little more information about your machine and project? Do breakpoints work with a simple console app for you?

[ghost]

"Sort of works" - Equals can get it to hit break points on start up but cannot break on express routes like index or serving up "/".

[ghost]

Versions of VS?: Well this entirely depends on the institution I am working in. Mostly financial right now(which is way behind). I am a contractor in London who loves nodejs. Been hard to teach without the VS integration. Let me pose conjecture by saying at least 3 major versions will help me a lot :)

Where are you guys going with this? Joyent or break away(io.js)? Maybe I can help ...

[mousetraps]

Ultimately our goal isn't to take sides, but to support devs wherever they are. So along those lines, we are hoping to support both for as long as we can manage. Barring ES6 features (which we need to work on anyways), Node and io.js aren't too different from a tooling perspective (yet.) Here's the current status on io.js and ES6 support: https://nodejstools.codeplex.com/wikipage?title=io.js

Any help/contributions would be awesome! Like I said, our only concern re: VS2012 going forward is the engineering/testing work required to maintain compatibility. Granted, it's a bit early to tell... but it hasn't cost us much headache yet, so I'm not particularly worried. We'll see for sure after we really start digging into the VS2015 work and beyond.

We'll post roadmaps etc. soon, so we don't have to keep speaking so abstractly about the future. It might be that we don't have much to fret about in terms of maintainability, and there's no use fixing something if it ain't broken. :-)

[mousetraps]

Can you let me know some more details about the debugging issue since we still can't repro?

• Are you using VS2012 Update 4? We don't support anything below that.
• If you unset and reset the breakpoint on the express route, does it work?
• do you get a filled-in breakpoint or an "empty" breakpoint on express routes?
• could you include the snippet of code you are trying to set a breakpoint on? (please also indicate which line you are setting the breakpoint on)
• full machine config and node info (version, 32 or 64 bit?)

[ghost]

Sure,

1. Have tried various combinations of setting break point before and after
   with F5
   1.1) Break points before running display as missing source code
   locations (but get hit anyway, as long as they are not callbacks, like
   express route handlers)
   1.2) Break points set during the session have valid source code
   locations and get hit but with the same limitation as above.
2. Anything inside a callback yields "empty" breakpoints that do not get
   hit.
3. I generated an express app using the express generator using ejs.
   3.1) http://expressjs.com/starter/generator.html
   3.2) set breakpoints on; routes\index.js (res.render) does not get hit.
   anywhere in app.js does get hit. breakpoint on bin\www also seems to get
   hit.
4. Win 7 Ultimate (64 bit), VS 2012 (update 4+), io.js(v1.6.4) or (node
   v0.12.2)

On Tue, 7 Apr 2015 at 04:39 Sara Itani [email protected] wrote:

Can you let me know some more details about the debugging issue since we
still can't repro?

• If you unset and reset the breakpoint on the express route, does it
  work?
• do you get a filled-in breakpoint or an "empty" breakpoint on
  express routes?
• could you include the snippet of code you are trying to set a
  breakpoint on?
• full machine config and node info (version, 32 or 64 bit?)

—
Reply to this email directly or view it on GitHub
#27 (comment)
.

[ghost]

I have tried to run the tests but for some reason seem to be running into problems with the "VSTestHost" not being loaded. Busy wading through how this works, progress is slow ...

[ghost]

Anything that could help speed this up? I am sure the answer is in a failing test ...

[mousetraps]

Is VSTestHost installed?

Did you configure your test settings to use the .testsettings file in the Build directory?

[ghost]

Here is what I have tried ...

I am using this script to build the msi:

*** my bad 'build.bat' start ***

call "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\Tools\VsDevCmd.bat"
cd .\NodeJs
msbuild .\NodejsTools.sln /p:Configuation=Release
cd .\Setup
rem powershell .\BuildRelease.ps1 C:\NTVS_Out -name "dev" -vstarget "11.0" -internal -skiptests
powershell .\BuildRelease.ps1 C:\NTVS_Out -name "dev" -vstarget "11.0" -internal 

*** my bad 'build.bat' end ***

Once this is built and all is good. I install the msi from the output directory because I know this will drop the test adapter into the extensions folder. Kind of get this for free using this approach.

I then choose the v12 settings from the build directory from the test menu in visual studio and commence a test run.

Consequently, I get heaps of failing tests :/

Example of first failing test:

[TestMethod, Priority(0), TestCategory("Debugging")]
        public void Breaking_InFunctionPassedFewerThanTakenParms() {
            TestDebuggerSteps(
                "FunctionPassedFewerThanTakenParms.js",
                new[] {
                    new TestStep(action: TestAction.AddBreakpoint, targetBreakpoint: 1),

                    new TestStep(action: TestAction.ResumeThread, expectedEntryPointHit: 4),
                    new TestStep(action: TestAction.ResumeProcess, expectedBreakpointHit: 1),

                    new TestStep(action: TestAction.ResumeProcess, expectedExitCode: 0),
                }
            );
        }

Output:

Source: DebuggerTests.cs line 751

Assert.Fail failed. Failed to wait on event.

BaseDebuggerTests.cs

internal static void AssertWaited(EventWaitHandle eventObj) {
            if (!eventObj.WaitOne(10000)) {
                Assert.Fail("Failed to wait on event"); // <- Here!!!!
            }
        }


In total 609 tests fail and 646 pass. 

I am sure this is something toxic (or missing in my environment) ... what do you think?

[ghost]

How do I install "VSTestHost"?

[mousetraps]

There is a VSTestHost.msi in the Common/Prerequisites folder, so there is no need to build any installers to run the tests. https://github.com/Microsoft/nodejstools/blob/master/Common/Prerequisites/VSTestHost.msi

We have ~95% pass rate currently on the tests (we have many broken or flaky tests at the moment,) and we haven't gotten the chance to clean them up yet.

However, the 609/646 discovered tests is a little fishy - it's more on the order of ~1300 total tests.

The full set of build instructions is here. I just updated it because I realized the test-related stuff was outdated.
https://nodejstools.codeplex.com/wikipage?title=Build%20Instructions%20for%20NTVS

[mousetraps]

Also the Breaking_InFunctionPassedFewerThanTakenParms test is passing on my machine. If you want, you can add your failed tests to a playlist and send the file so we can verify it on our end.

[ghost]

Keep getting this.

The host type 'VSTestHost' cannot be loaded for the following reason: The key 'VSTestHost' cannot be found. Make sure that the appropriate host adapter is installed on the machine.

[ghost]

It is installed though ...

[ghost]

I am not sure how this works, all I can find in my program files directory is:

C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\CommonExtensions\Microsoft\VSTestHost\Microsoft.VisualStudioTools.VSTestHost.11.0.pkgdef

So went back to the msi. Extacted the 11 version and ran gacutil /i over it. Restarted visual studio. Now re-running the tests. Seems a bit better.

[ghost]

Nope still no luck. Still getting The host type 'VSTestHost' cannot be loaded for the following reason: The key 'VSTestHost' cannot be found. Make sure that the appropriate host adapter is installed on the machine.

Downgraded the test settings file to v11 no dice. What does the Common\Prerequisites\VSTestHost.msi actually do? Maybe I can just repeat manually?

[ghost]

http://stackoverflow.com/questions/13511135/unit-tests-fail-when-using-moles-0-94-and-vs2012-hosting-rules-specify-that-th

https://social.msdn.microsoft.com/Forums/vstudio/en-US/2c2f5e71-4787-4196-bc46-148057f71b68/the-host-type-aspnet-cannot-be-loaded-for-the-following-reason-the-key-aspnet-cannot-be?forum=vsunittest

No answers really as to how it works ...

[ghost]

Failed playlist imminent, just takes long because it keeps hitting timeouts. Need to leave it for an overnight run. Will post tomorrow.

[ghost]

https://gist.github.com/fir3pho3nixx/a8769f8e2af38420eb84

[ghost]

[mousetraps]

Yeah, definitely a config issue. A few comments/ideas

• Completely ignore the TCTestHostAdapter stuff. None of that should still be in our repo - we replaced it with VSTestHost a while ago, and I guess we just haven't merged changes from Python Tools for VS in a while :-/ The only installer you should worry about is Common\Prerequisites\VSTestHost.msi
• did you run Nodejs\Prerequisites\EnableSkipVerification.reg to disable strong name verification?
• Some of the tests have dependencies on the Azure Tools SDK. In theory you'll get a warning, but perhaps that did not happen. See here for downloads:
  • Azure Tools for VS 2013: http://go.microsoft.com/fwlink/p/?linkid=323510
  • Azure Tools for VS 2012: http://go.microsoft.com/fwlink/p/?linkid=323511
• Are there any messages in the test output window?
• VSTestHost allows us to control the visual studio process. There's a (pretty sparse) msdn article if you want to learn more. https://msdn.microsoft.com/en-US/library/bb286982%28v=vs.80%29.aspx
• fyi for some of the UI tests, you might have to leave your computer alone because it requires controlling the mouse (we try to avoid tests like this, but sometimes it's just unavoidable.) So yes, you can technically repeat the tests manually... but we automated them for a reason, so you probably don't want to :)
• try cleaning your directory and building your solution again. Sometimes things can get into a weird state.

Thanks for the patience, and hope this helps!

[ghost]

Yes I installed the azure sdk because without it, my solution would not build. The enable skip verification was also necessary and solved more problems. I am a resharper guy. So MSTest in VS looks pretty spartan by comparison. Will dig around more over the weekend. Something is telling me
that this is pathing issue.

Do you guys have the original source code + installer for the VSTestHost? I just need to know what the steps are that the installer does, so I can replicate and see which step is failing. Also do you think enabling fusion logs to try and find out where it is trying to resolve the test host from
is a good idea?

[mousetraps]

Hey, just looked into this a bit further... could you try entering the following node.exe arguments in project properties?
--debug-brk .\bin\www

You need to enter .\bin\www for the server to even start if you were running in the command line. And you shouldn't have to enter --debug-brk, but it looks like we have an issue with the way that our arguments are appended to one another, so because you are specifying a "startup file" in node.exe arguments, you'll have to specify --debug-brk to workaround the issue.
https://nodejstools.codeplex.com/workitem/1862

Normally you'd be able to right click and select "set as Node.js startup file, but we don't enable that context menu item on non-javascript files. (see #65)

I will try to find someone who knows about VSTestHost than I do so we can resolve the test issue.

[ghost]

This was one of the problems I had to solve to make sure it was all hanging together correctly.

I also did a few tests from the command line to node's debugger just to be sure. I had this set as a parameter and only after I made the code change did it start hitting break points.

Everyone who has used the debugging for this tool in the office so far has said it works. So I think I am in a minority here. Will dig deeper and feedback as soon as I know more.

[ghost]

I ran procmon over the test executing in Visual Studio. I can see it is looking for a registry key that is missing.

Date & Time:    15/04/2015 19:10:32
Event Class:    Registry
Operation:  RegOpenKey
Result: NAME NOT FOUND
Path:   HKLM\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\11.0\EnterpriseTools\QualityTools\HostAdapters\VSTestHost
TID:    7260
Duration:   0.0000071
Desired Access: Read

[ghost]

Added the following registry key

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\11.0\EnterpriseTools\QualityTools\HostAdapters\VSTestHost]
"Type"="Microsoft.VisualStudioTools.VSTestHost.11.0, Version=11.0.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"

and the "VSTestHost" missing failure changed to:

Test Name:  CopyFileFromFolderToLinkedFolderMouse
Test FullName:  Microsoft.VisualStudioTools.SharedProjectTests.DragDropCopyCutPaste.CopyFileFromFolderToLinkedFolderMouse
Test Source:    c:\Code\nodejstools\Common\Tests\SharedProjectTests\DragDropCopyCutPaste.cs : line 1619
Test Outcome:   Failed
Test Duration:  0:00:00

Result Message: Hosting rules specify that the test type 'Unit Test' cannot run in the host adapter 'VSTestHost'. To run this test in 'VSTestHost', change the hosting rules. To use the default test host for tests that cannot be run in the specified host adapter, change the test settings.

Surely I am config away from running the debugger tests. One issue we know about is the msi

Common\Prerequisites\VSTestHost.msi

Is definitely missing a registry installation step missing for Windows 7/VS 2012. Do you know how I could update the settings for the above hosting rules?

[mousetraps]

Hmm... could it be possible that you're running into this issue, where .testsettings isn't getting selected properly after reloading VS?
https://social.msdn.microsoft.com/Forums/vstudio/en-US/c5ccee4a-83d7-4f6d-bb0e-d27c591bc2ce/setting-default-runsettings-file?forum=vsunittest

[mousetraps]

Although I would expect a different error for that...

[mousetraps]

FYI the VSTestHost code is now here, along with a proper readme.

I believe the issue you may be running into is that VS2012 is a test target, but cannot be used to launch tests.

[ghost]

Appears so. Does this mean your developer workflow no longer supports VS2010 or VS2012 because of "VsTestHost" (master does does not load in VS2012)? Important to note on your "how to build guide".

Will upgrade my VS and do more digging.

[mousetraps]

You'll want to run side-by-side. The tests will still launch VS2012, but you just can't start from 2012.

Agreed. I'll update the docs shortly.

[mousetraps]

by the way, I'm getting the same errors as you are now, and it has nothing to do with VSTestHost :-/

[mousetraps]

Looked into it - many of the unit test errors seem to be due to #89 (if you try 0.10.33, many of those tests should pass). Also check out the latest dev build (5-13-2015), which fixed a race condition that caused similar symptoms.
https://github.com/Microsoft/nodejstools/releases

[ghost]

Hey! Thanks for this, have upgraded my VS and will give it another shot. Thanks for the update :)

[mousetraps]

Closing this since we haven't heard back in a while. Let us know if you're still running into issues.