Implement SourceLink for Windows PDBs #18539

tmat · 2017-04-07T23:26:05Z

Enables /sourcelink for Windows PDBs. Previously we only supported it for Portable PDBs.

Create package for Microsoft.DiaSymReader.Native, 1.6.0. The feature requires new diasymreader APIs.

tmat · 2017-04-08T00:04:50Z

@ctaggart FYI

tmat · 2017-04-08T00:07:13Z

@gregg-miskelly @caslan FYI

gregg-miskelly

Excellent.

ctaggart · 2017-04-08T00:44:37Z

@tmat Nice! I was wondering if ya'll were going to pursue this. Any ideas how to detect support when this ships? Currently I look for DebugType of portable or embedded. Should I look for the msbuild version, roslyn version, or something else?

<CompileDependsOn Condition="'$(SourceLinkCreate)' == 'true' and ($(DebugType) == 'portable' or $(DebugType) == 'embedded')">SourceLinkCreate;$(CompileDependsOn)</CompileDependsOn>

tmat · 2017-04-08T01:11:23Z

@ctaggart I don't think you need to check for anything. If one uses some version of SourceLink that doesn't have that check they should also use a compiler that supports it. If not an error will be reported by the compiler. So just mentioning it in SourceLink docs should be sufficient imo.

tmat · 2017-04-26T17:37:30Z

Requires changes in #18803.

tmat · 2017-04-26T18:40:46Z

@dotnet/roslyn-compiler Please review. Still need to update the dependencies to pass tests, but the code is ready for review (and tests pass on my box).

jaredpar · 2017-04-27T00:04:22Z

src/Compilers/CSharp/Test/CommandLine/CommandLineTests.cs

@@ -31,6 +31,8 @@

 using static Roslyn.Test.Utilities.SharedResourceHelpers;
 using static Microsoft.CodeAnalysis.CommonDiagnosticAnalyzers;
+using Roslyn.Test.PdbUtilities;
+using Microsoft.DiaSymReader;


Sort usings #Resolved

Fixed. #Resolved

jaredpar · 2017-04-27T00:08:44Z

src/Compilers/Core/CodeAnalysisTest/InternalUtilities/StreamExtensionsTests.cs

+            var stream = new TestStream(canSeek: false, backingStream: new MemoryStream(sourceArray));
+            stream.ReadByte();
+            Assert.Equal(new byte[] { 2, 3, 4 }, stream.ReadAllBytes());
+        }


Add a test verifying behavior when stream is at the end. #Resolved

Done. #Closed

jaredpar · 2017-04-27T00:19:09Z

src/Test/PdbUtilities/Pdb/SymReaderFactory.cs

@@ -23,7 +23,7 @@ public static class SymReaderFactory
        [DllImport("Microsoft.DiaSymReader.Native.amd64.dll", EntryPoint = "CreateSymReader")]
        private extern static void CreateSymReader64(ref Guid id, [MarshalAs(UnmanagedType.IUnknown)]out object symReader);

-        private static ISymUnmanagedReader3 CreateNativeSymReader(Stream pdbStream, object metadataImporter)
+        private static ISymUnmanagedReader5 CreateNativeSymReader(Stream pdbStream, object metadataImporter)


In the past there were cases where the compiler was installed without the specific version of native diasymreader that we needed. That meant it could be older. Is that the case anymore? If so do we have to worry about cases where ISymUnmanagedReader5 is not avialable? #Resolved

The compiler doesn't use ISymUnmanagedReader, only ISymUnmanagedWriter. This helper only loads SymReader from DSRN where it always is the latest version. #ByDesign

jaredpar · 2017-04-27T00:20:26Z

src/Compilers/Core/Portable/NativePdbWriter/PdbWriter.cs

+                fixed (byte* bufferPtr = buffer)
+                {
+                    byte b = 0;
+                    symWriter8.SetSourceLinkData((bufferPtr != null) ? bufferPtr : &b, buffer.Length);


In what case is bufferPtr going to be null? #Closed

When buffer is empty. #Closed

jaredpar · 2017-04-27T00:21:09Z

src/Compilers/Core/Portable/NativePdbWriter/PdbWriter.cs

+                return null;
+            }
+
+            var buffer = new StringBuilder(256);


Why 256? #Resolved

Arbitrary max path. I'll remove the commit from this PR - I accidentally added it to this branch while investigating why tests fail on Jenkins and not on my box. #Resolved

cston · 2017-04-27T03:19:42Z

src/Compilers/Core/CodeAnalysisTest/InternalUtilities/StreamExtensionsTests.cs

@@ -118,5 +118,33 @@ public void PrematureEndOfStream()
            var expected = new byte[] { 1, 2, 3, 4, 0, 0 };
            Assert.Equal(expected, destArray);
        }
+
+        [Fact]
+        public void ReadAllBytes_Seakable()


Typo: Seekable #Resolved

cston · 2017-04-27T03:29:11Z

src/Compilers/Core/CodeAnalysisTest/InternalUtilities/StreamExtensionsTests.cs

+            stream.ReadByte();
+            stream.ReadByte();
+            Assert.Equal(new byte[0], stream.ReadAllBytes());
+        }


Consider testing ReadAllBytes() when TestStream.Length > backingStream.Length (the Array.Resize case). #Resolved

cston · 2017-04-27T04:14:45Z

LGTM

jcouv · 2017-04-27T16:23:54Z

src/Compilers/CSharp/Test/Emit/PDB/PDBSourceLinkTests.cs

+";
+            var sourceLinkBlob = Encoding.UTF8.GetBytes(@"
+{
+  ""documents"": {


Nit: open brace on new line #WontFix

This is not C# ;) #WontFix

jcouv · 2017-04-27T16:32:30Z

src/Compilers/CSharp/Test/Emit/PDB/PDBSourceLinkTests.cs

+
+            var pdbStream = new MemoryStream();
+            c.EmitToArray(EmitOptions.Default.WithDebugInformationFormat(format), pdbStream: pdbStream, sourceLinkStream: new MemoryStream(sourceLinkBlob));
+            pdbStream.Position = 0;


Is this position reset necessary? Maybe GetSourceLinkData should handle the reset, if it doesn't already. #Resolved

Moved to GetSourceLinkData.

In reply to: 113743925 [](ancestors = 113743925)

jcouv · 2017-04-27T16:37:05Z

src/Compilers/CSharp/Test/Emit/PDB/PDBSourceLinkTests.cs

+    }
+}
+";
+            var sourceLinkStream = new TestStream(canRead: true, readFunc: (_, __, ___) => { throw new Exception("Error!"); });


Hopefully, discards will be supported in lambdas soon ;-) #Resolved

+1

In reply to: 113744881 [](ancestors = 113744881)

jcouv · 2017-04-27T16:45:23Z

src/Compilers/CSharp/Portable/CommandLine/CSharpCommandLineParser.cs

@@ -1193,12 +1193,9 @@ public new CSharpCommandLineArguments Parse(IEnumerable<string> args, string bas
                keyFileSetting = ParseGenericPathToFile(keyFileSetting, diagnostics, baseDirectory);
            }

-            if (sourceLink != null)
+            if (sourceLink != null && !emitPdb)


Should the VB command-line parser have a similar change? #Resolved

Yes, forgot about that.

In reply to: 113746766 [](ancestors = 113746766)

jcouv

You'll also want to update the command-line documentation file: https://github.com/dotnet/roslyn/blob/master/docs/compilers/CSharp/CommandLine.md

(and its VB counter-part if applicable)
Thanks

tmat · 2017-04-27T21:52:45Z

@jcouv Updated documentation of the command line args for both VB and C#. Please check if it looks good.

jcouv · 2017-04-27T21:59:32Z

src/Compilers/CSharp/csc/project.json

@@ -1,6 +1,6 @@
 {
  "dependencies": {
-    "Microsoft.DiaSymReader.Native": "1.5.0"
+    "Microsoft.DiaSymReader.Native": "1.6.0-beta2-25219"


I'm assuming that we will not ship dev15.3 with a reference to beta package. Can you file a follow-up issue to fix this reference once a release package is available?
It's probably also good to think about timeline to make this happen.

We produce RTM packages once the final VS RTM build is available. I don't think a work item is needed as this is a common step taken during Roslyn nuget package publishing.

I should probably clarify. The flow for this package is documented here: https://github.com/dotnet/roslyn-debug/tree/master/src/Microsoft.DiaSymReader.Native. To get RTM binaries we need to wait until VS RTM build is produced by the lab.

Ok. Sounds like this step is already covered by standard release process.

jcouv

LGTM

jaredpar · 2017-05-02T04:31:45Z

@tmat looks like we have two sign offs but unfortunately this ended up conflicting with my SDK change. Can we rebase this and get it merged soon?

Bar for compiler layer starts going up next week. Want to get this in while it still meets the bar.

tmat · 2017-05-02T04:33:18Z

@jaredpar Waiting for a fixed build of DSRN. Should be available on Wed.

tmat · 2017-05-09T03:00:51Z

test windows_release_vs-integration_prtest please

kzu · 2017-06-01T03:05:11Z

Is there a tentative milestone where this will ship? 15.[something]?

jaredpar · 2017-06-01T05:06:20Z

@kzu this will ship in 15.3

kzu · 2017-06-01T19:53:21Z

Should it be in the current preview then? or will it make it to preview 2?

dnfclas added the cla-already-signed label Apr 7, 2017

tmat added the Interactive-Debugging label Apr 7, 2017

tmat requested a review from nguerrera April 7, 2017 23:26

gregg-miskelly approved these changes Apr 8, 2017

View reviewed changes

This was referenced Apr 8, 2017

enable source link for Windows PDBs too by default ctaggart/SourceLink#194

Closed

Enable source link support aspnet/Mvc#5905

Closed

Pilchie added the Area-Interactive label Apr 13, 2017

tmat force-pushed the DSRN16 branch 3 times, most recently from 4d5dc67 to b50fbcd Compare April 26, 2017 17:04

tmat requested review from jcouv and agocke April 26, 2017 18:40

tmat added the Area-Compilers label Apr 26, 2017

jaredpar reviewed Apr 27, 2017

View reviewed changes

tmat force-pushed the DSRN16 branch from 168b96a to 8841f63 Compare April 27, 2017 01:01

cston reviewed Apr 27, 2017

View reviewed changes

jcouv reviewed Apr 27, 2017

View reviewed changes

tmat force-pushed the DSRN16 branch from 8841f63 to 5393fab Compare April 27, 2017 21:50

jcouv reviewed Apr 27, 2017

View reviewed changes

jcouv approved these changes Apr 27, 2017

View reviewed changes

jaredpar added the Approved to merge label May 2, 2017

tmat force-pushed the DSRN16 branch 3 times, most recently from d2d3a92 to 6fa0bc7 Compare May 9, 2017 00:26

Implement SourceLink for Windows PDBs

135bc47

tmat force-pushed the DSRN16 branch from 6fa0bc7 to 135bc47 Compare May 9, 2017 16:33

tmat merged commit 3986cdb into dotnet:master May 9, 2017

tmat deleted the DSRN16 branch May 9, 2017 17:39

rainersigwald mentioned this pull request May 22, 2017

Pdb git and source link dotnet/msbuild#2107

Merged

tmat added this to the 15.3 milestone Jun 1, 2017

zvirja mentioned this pull request Jun 30, 2017

Implement SourceLink approach to enable simpler source code stepping AutoFixture/AutoFixture#770

Closed

stakx mentioned this pull request Jul 30, 2017

Issue with Moq.pdb in version 4.7.99: "Symbol indexes could not be retrieved" devlooped/moq#428

Closed

Implement SourceLink for Windows PDBs #18539

Implement SourceLink for Windows PDBs #18539

Conversation

tmat commented Apr 7, 2017 • edited Loading

tmat commented Apr 8, 2017

tmat commented Apr 8, 2017

gregg-miskelly left a comment

Choose a reason for hiding this comment

ctaggart commented Apr 8, 2017 • edited Loading

tmat commented Apr 8, 2017 • edited Loading

tmat commented Apr 26, 2017

tmat commented Apr 26, 2017

jaredpar Apr 27, 2017 • edited by tmat Loading

Choose a reason for hiding this comment

tmat Apr 27, 2017 • edited Loading

Choose a reason for hiding this comment

jaredpar Apr 27, 2017 • edited by tmat Loading

Choose a reason for hiding this comment

tmat Apr 27, 2017 • edited Loading

Choose a reason for hiding this comment

jaredpar Apr 27, 2017 • edited by tmat Loading

Choose a reason for hiding this comment

tmat Apr 27, 2017 • edited Loading

Choose a reason for hiding this comment

jaredpar Apr 27, 2017 • edited by tmat Loading

Choose a reason for hiding this comment

tmat Apr 27, 2017 • edited Loading

Choose a reason for hiding this comment

jaredpar Apr 27, 2017 • edited by tmat Loading

Choose a reason for hiding this comment

tmat Apr 27, 2017 • edited Loading

Choose a reason for hiding this comment

cston Apr 27, 2017 • edited by tmat Loading

Choose a reason for hiding this comment

cston Apr 27, 2017 • edited by tmat Loading

Choose a reason for hiding this comment

cston commented Apr 27, 2017

jcouv Apr 27, 2017 • edited by tmat Loading

Choose a reason for hiding this comment

tmat Apr 27, 2017 • edited Loading

Choose a reason for hiding this comment

jcouv Apr 27, 2017 • edited by tmat Loading

Choose a reason for hiding this comment

tmat Apr 27, 2017

Choose a reason for hiding this comment

jcouv Apr 27, 2017 • edited by tmat Loading

Choose a reason for hiding this comment

tmat Apr 27, 2017

Choose a reason for hiding this comment

jcouv Apr 27, 2017 • edited by tmat Loading

Choose a reason for hiding this comment

tmat Apr 27, 2017

Choose a reason for hiding this comment

jcouv left a comment

Choose a reason for hiding this comment

tmat commented Apr 27, 2017

jcouv Apr 27, 2017

Choose a reason for hiding this comment

tmat Apr 27, 2017 • edited Loading

Choose a reason for hiding this comment

tmat Apr 27, 2017 • edited Loading

Choose a reason for hiding this comment

jcouv Apr 28, 2017

Choose a reason for hiding this comment

jcouv left a comment

Choose a reason for hiding this comment

jaredpar commented May 2, 2017

tmat commented May 2, 2017

tmat commented May 9, 2017

kzu commented Jun 1, 2017

jaredpar commented Jun 1, 2017

kzu commented Jun 1, 2017

tmat commented Apr 7, 2017 •

edited

Loading

ctaggart commented Apr 8, 2017 •

edited

Loading

tmat commented Apr 8, 2017 •

edited

Loading

jaredpar Apr 27, 2017 •

edited by tmat

Loading

tmat Apr 27, 2017 •

edited

Loading

jaredpar Apr 27, 2017 •

edited by tmat

Loading

tmat Apr 27, 2017 •

edited

Loading

jaredpar Apr 27, 2017 •

edited by tmat

Loading

tmat Apr 27, 2017 •

edited

Loading

jaredpar Apr 27, 2017 •

edited by tmat

Loading

tmat Apr 27, 2017 •

edited

Loading

jaredpar Apr 27, 2017 •

edited by tmat

Loading

tmat Apr 27, 2017 •

edited

Loading

cston Apr 27, 2017 •

edited by tmat

Loading

cston Apr 27, 2017 •

edited by tmat

Loading

jcouv Apr 27, 2017 •

edited by tmat

Loading

tmat Apr 27, 2017 •

edited

Loading

jcouv Apr 27, 2017 •

edited by tmat

Loading

jcouv Apr 27, 2017 •

edited by tmat

Loading

jcouv Apr 27, 2017 •

edited by tmat

Loading

tmat Apr 27, 2017 •

edited

Loading

tmat Apr 27, 2017 •

edited

Loading