New public APIs for UTF8 String Literals

[AlekseyTs]

namespace Microsoft.CodeAnalysis
{
    public enum OperationKind
    {
+        /// <summary>Indicates an <see cref="IUTF8StringOperation"/>.</summary>
+        UTF8String = 0x7c,
    }
}

namespace Microsoft.CodeAnalysis.Operations
{
+    /// <summary>
+    /// Represents a UTF8 encoded byte representation of a string.
+    /// <para>
+    ///   Current usage:
+    ///   (1) C# UTF8 string literal expression.
+    /// </para>
+    /// </summary>
+    /// <remarks>
+    /// <para>This node is associated with the following operation kinds:</para>
+    /// <list type="bullet">
+    /// <item><description><see cref="OperationKind.UTF8String"/></description></item>
+    /// </list>
+    /// <para>This interface is reserved for implementation by its associated APIs. We reserve the right to
+    /// change it in the future.</para>
+    /// </remarks>
+    public interface IUTF8StringOperation : IOperation
+    {
+        /// <summary>
+        /// The underlying string value.
+        /// </summary>
+        string Value { get; }
+    }

    public abstract partial class OperationVisitor
    {
+        public virtual void VisitUTF8String(IUTF8StringOperation operation) => DefaultVisit(operation);
    }

    public abstract partial class OperationVisitor<TArgument, TResult>
    {
+        public virtual TResult? VisitUTF8String(IUTF8StringOperation operation, TArgument argument) => DefaultVisit(operation, argument);
    }
}

+Microsoft.CodeAnalysis.CSharp.SyntaxKind.UTF8StringLiteralExpression = 8756 -> Microsoft.CodeAnalysis.CSharp.SyntaxKind
+Microsoft.CodeAnalysis.CSharp.SyntaxKind.UTF8StringLiteralToken = 8520 -> Microsoft.CodeAnalysis.CSharp.SyntaxKind
+Microsoft.CodeAnalysis.CSharp.SyntaxKind.UTF8SingleLineRawStringLiteralToken = 8521 -> Microsoft.CodeAnalysis.CSharp.SyntaxKind
+Microsoft.CodeAnalysis.CSharp.SyntaxKind.UTF8MultiLineRawStringLiteralToken = 8522 -> Microsoft.CodeAnalysis.CSharp.SyntaxKind

[Youssef1313]

Minor naming thing that you may consider discussing is UTF8 vs Utf8. The runtime used Utf8 for recent type (https://docs.microsoft.com/en-us/dotnet/api/system.text.json.utf8jsonwriter?view=net-6.0)

I'm aware that some types in .NET uses UTF all capitalized, so I referred to the more recent type.

From https://github.com/dotnet/runtime/blob/main/docs/coding-guidelines/framework-design-guidelines-digest.md#naming-guidelines

✓ DO use PascalCasing or camelCasing for any acronyms over two
characters long. For example, use HtmlButton rather than HTMLButton, but
System.IO instead of System.Io.

[333fred]

API Review

• User-facing documentation should use UTF-8
• Could it be ILiteralOperation?
  • It's not a constant, like other literals
  • Couldn't expose the string value from the operation
• Need to sync with the Framework Design folks on the proper spelling of Utf8 in type names (BCL is inconsistent here).

Conclusion: Approved, modulo naming of the types

[AlekseyTs]

From Framework Design folks:

From a framework naming perspective, “Utf8” is preferred, because it’s a 3 letter abbreviation (for Unicode Transformation Format), and 3+ letter abbreviations should be PascalCased, like Html and Uri.

From a documentation/conceptual perspective, the preferred spelling of the Unicode Consortium seems to be “UTF-8” (all caps, with a hyphen). To wit: https://www.unicode.org/faq/utf_bom.html