Prejit support on FunctionsNetHost #2711

kshyju · 2024-09-13T15:34:59Z

This PR adds support for pre-jitting framework methods during placeholder mode. Currently this is enabled only for net8 onwards. This behavior can be explicitly disabled by setting AZURE_FUNCTIONS_FUNCTIONSNETHOST_DISABLE_PREJIT environment variable value to 0.

See internal wiki page for more context.

~~I will have a separate follow up PR for fixing the build pipeline & nuget packaging needed for this change.~~ Also included the build pipeline changes to include the new artifacts in the final package.

Did a minor version bump.

Pull request checklist

My changes do not require documentation changes
- Otherwise: Documentation issue linked to PR
My changes should not be added to the release notes for the next release
- Otherwise: I've added my notes to release_notes.md
My changes do not need to be backported to a previous version
- Otherwise: Backport tracked by issue/PR #issue_or_pr
I have added all required tests (Unit tests, E2E tests)

kshyju · 2024-09-13T15:37:12Z

@steveisok @ivdiazsa FYI.

host/src/FunctionsNetHost/PreJit/PreJitManager.cs

kshyju · 2024-09-13T16:38:41Z

@khkh-ms FYI. I used a previously generated trace for this PR. Once your changes are done, we could replace this file with that. Let's merge both traces to single file (like what we do for host)

host/src/FunctionsNetHost/Grpc/IncomingGrpcMessageHandler.cs

…t package. Minor version bumped.

kshyju · 2024-09-18T00:06:27Z

@steveisok @ivdiazsa Appreciate if you could do review the changes.

liliankasem

Can you added unit tests where possible?

host/src/FunctionsNetHost.Shared/EnvironmentVariables.cs

host/src/FunctionsNetHost/Environment/EnvironmentVariables.cs

host/src/FunctionsNetHost/PreJit/PreJitManager.cs

host/src/PlaceholderApp/JitTrace/JitTraceRuntime.cs

host/src/PlaceholderApp/StartupHook.cs

host/src/global.json

host/src/FunctionsNetHost/PreJit/PreJitManager.cs

host/src/PlaceholderApp/StartupHook.cs

ivdiazsa · 2024-09-18T21:14:04Z

I don't have further comments other than addressing the ones that have been pointed out so far.

host/src/FunctionsNetHost.Shared/EnvironmentVariables.cs

…ONS_NETHOST" for env variables.

kshyju · 2024-09-19T20:51:50Z

Can you add unit tests where possible?

We don't have any tests as of today as what we want to validate is the native output. There has been discussion about adding an E2E test which simulates the linux scenarios as part of build pipeline (potentially a nightly build). Never got a chance to create a work item until now. #2724

I think that should be the priority. We can certainly revisit as we have more cycles to work on this.

steveisok · 2024-09-19T21:08:41Z

Looks good to me! Glad you were able to work this in 👍

safihamid

…er app so that the pubish output (portable) works for linux use cases.

kshyju · 2024-09-23T02:05:57Z

eng/ci/templates/official/jobs/build-host-prelaunch-artifacts.yml

@@ -25,5 +25,5 @@ jobs:
        publishWebProjects: false
        zipAfterPublish: false
        modifyOutputPath: false
-        arguments: -c Release -o $(Build.ArtifactStagingDirectory)/_preLaunchAppPackages/${{ version }} -f ${{ version }} -p:UseAppHost=false
+        arguments: -c Release -o $(Build.ArtifactStagingDirectory)/_preLaunchAppPackages/${{ replace(version, 'net', '') }} -f ${{ version }} -p:UseAppHost=false


Fixing an existing error in output path value, which was introduced during the pipeline migration.

host/src/PlaceholderApp/StartupHook.cs

…k. Named eventwait handle does not work in unix,

…-functions-dotnet-worker into shkr/2637_fnh_prejit

kshyju · 2024-09-27T14:38:37Z

host/src/PlaceholderApp/JitTrace/coldstart.jittrace

@@ -0,0 +1,331 @@
+[S.P.CoreLib]System.Array.Empty<Candidate>()~System.Array,System.Private.CoreLib~0~1:Microsoft.AspNetCore.Routing.Matching.Candidate,Microsoft.AspNetCore.Routing~Empty


@eliaslopezgt please replace coldstart.jittrace and linux.coldstart.jittrace files.

kshyju · 2024-09-27T16:00:42Z

E2E test results

Windows

Validated E2E in private stamp. Everything works as expected.

Linux

Specialization is working as expected, but the http trigger invocation fails with below error.

Function.HelloHttp[3]
2024-09-27 06:50:31 Executed 'Functions.HelloHttp' (Failed, Id=98960abe-dec0-4665-bd88-d0e1a93d8092, Duration=5ms)
2024-09-27 06:50:31 Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: Functions.HelloHttp
2024-09-27 06:50:31 ---> Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcException: Result: Failure
2024-09-27 06:50:31 Exception: System.IO.FileNotFoundException: Could not load file or assembly '/tmp/functions\standby\wwwroot/v4-cold-aotisolated9.dll'. The system cannot find the file specified.
2024-09-27 06:50:31
2024-09-27 06:50:31 File name: '/tmp/functions\standby\wwwroot/v4-cold-aotisolated9.dll'
2024-09-27 06:50:31 at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyPath(String assemblyPath)
2024-09-27 06:50:31 at Microsoft.Azure.Functions.Worker.Invocation.DefaultMethodInfoLocator.GetMethod(String pathToAssembly, String entryPoint) in D:\a_work\1\s\src\DotNetWorker.Core\Invocation\DefaultMethodInfoLocator.cs:line 25
2024-09-27 06:50:31 at Microsoft.Azure.Functions.Worker.Definition.GrpcFunctionDefinition..ctor(FunctionLoadRequest loadRequest, IMethodInfoLocator methodInfoLocator) in D:\a_work\1\s\src\DotNetWorker.Grpc\Definition\GrpcFunctionDefinition.cs:line 53
2024-09-27 06:50:31 at Microsoft.Azure.Functions.Worker.GrpcWorker.FunctionLoadRequestHandler(FunctionLoadRequest request, IFunctionsApplication application, IMethodInfoLocator methodInfoLocator) in D:\a_work\1\s\src\DotNetWorker.Grpc\GrpcWorker.cs:line 268
2024-09-27 06:50:31 Stack: at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyPath(String assemblyPath)
2024-09-27 06:50:31 at Microsoft.Azure.Functions.Worker.Invocation.DefaultMethodInfoLocator.GetMethod(String pathToAssembly, String entryPoint) in D:\a_work\1\s\src\DotNetWorker.Core\Invocation\DefaultMethodInfoLocator.cs:line 25
2024-09-27 06:50:31 at Microsoft.Azure.Functions.Worker.Definition.GrpcFunctionDefinition..ctor(FunctionLoadRequest loadRequest, IMethodInfoLocator methodInfoLocator) in D:\a_work\1\s\src\DotNetWorker.Grpc\Definition\GrpcFunctionDefinition.cs:line 53
2024-09-27 06:50:31 at Microsoft.Azure.Functions.Worker.GrpcWorker.FunctionLoadRequestHandler(FunctionLoadRequest request, IFunctionsApplication application, IMethodInfoLocator methodInfoLocator) in D:\a_work\1\s\src\DotNetWorker.Grpc\GrpcWorker.cs:line 268
2024-09-27 06:50:31 at Microsoft.Azure.WebJobs.Script.Description.WorkerFunctionInvoker.InvokeCore(Object[] parameters, FunctionInvocationContext context) in /src/azure-functions-host/src/WebJobs.Script/Description/Workers/WorkerFunctionInvoker.cs:line 101
2024-09-27 06:50:31 at Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase.Invoke(Object[] parameters) in /src/azure-functions-host/src/WebJobs.Script/Description/FunctionInvokerBase.cs:line 82
2024-09-27 06:50:31 at Microsoft.Azure.WebJobs.Script.Description.FunctionGenerator.Coerce[T](Task1 src) in /src/azure-functions-host/src/WebJobs.Script/Description/FunctionGenerator.cs:line 225 2024-09-27 06:50:31 at Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker2.InvokeAsync(Object instance, Object[] arguments) in D:\a_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionInvoker.cs:line 53
2024-09-27 06:50:31 at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.InvokeWithTimeoutAsync(IFunctionInvoker invoker, ParameterHelper parameterHelper, CancellationTokenSource timeoutTokenSource, CancellationTokenSource functionCancellationTokenSource, Boolean throwOnTimeout, TimeSpan timerInterval, IFunctionInstance instance) in D:\a_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 581

The GrpcFunctionDefinition code relies on FUNCTIONS_APPLICATION_DIRECTORY and FUNCTIONS_WORKER_DIRECTORY (as fallback) environment variable values to build the path used to load the assembly. Need to investigate further why this value is coming as incorrect.

azure-functions-dotnet-worker/src/DotNetWorker.Grpc/Definition/GrpcFunctionDefinition.cs

Lines 31 to 43 in 24cd553

    
           string? scriptRoot = Environment.GetEnvironmentVariable(FunctionsApplicationDirectoryKey) ?? Environment.GetEnvironmentVariable(FunctionsWorkerDirectoryKey); 
        
           if (string.IsNullOrWhiteSpace(scriptRoot)) 
        
           { 
        
               throw new InvalidOperationException($"The '{FunctionsApplicationDirectoryKey}' environment variable value is not defined. This is a required environment variable that is automatically set by the Azure Functions runtime."); 
        
           } 
        
           if (string.IsNullOrWhiteSpace(loadRequest.Metadata.ScriptFile)) 
        
           { 
        
               throw new InvalidOperationException($"Metadata for function '{loadRequest.Metadata.Name} ({loadRequest.Metadata.FunctionId})' does not specify a 'ScriptFile'."); 
        
           } 
        
           string scriptFile = Path.Combine(scriptRoot, loadRequest.Metadata.ScriptFile); 
        
           PathToAssembly = Path.GetFullPath(scriptFile);

fabiocav · 2024-10-19T02:12:37Z

host/src/PlaceholderApp/StartupHook.cs

+            Log("Waiting for cold start request.");
+
+            // When specialization request arrives, FNH will connect to this named server stream and send the assembly path.
+            using (var pipeServer = new NamedPipeServerStream(Constants.NetHostWaitHandleName, PipeDirection.In))


Can we instead make an interop call from the native host following the pattern used against the worker?

fabiocav · 2024-10-19T02:18:01Z

src/DotNetWorker.Core/StartupHook.cs

@@ -57,12 +57,12 @@ static bool WaitOnDebugger(int cycle)

        if (string.Equals(jsonOutputEnabled, bool.TrueString, StringComparison.OrdinalIgnoreCase))
        {
-            Console.WriteLine($"azfuncjsonlog:{{ \"name\":\"dotnet-worker-startup\", \"workerProcessId\" : { processId } }}");


Revert whitespace only changes unrelated to the PR.

fabiocav · 2024-10-19T02:24:50Z

host/src/PlaceholderApp/FunctionsNetHost.PlaceholderApp.csproj

+    <TargetFrameworks>net8.0;net9.0</TargetFrameworks>
+    <DebugSymbols>False</DebugSymbols>
+    <DebugType>None</DebugType>
+    <TieredCompilation>false</TieredCompilation>


Set all appropriate assembly properties here.

This assembly will be injected and visible in the default context. It should be signed, properly versioned and named

fabiocav · 2024-10-19T02:49:26Z

host/src/PlaceholderApp/StartupHook.cs

+        var taEx = new System.Threading.ThreadInterruptedException();
+
+        // Same with ASP.NET Core types.
+        var provider = new Microsoft.Extensions.Configuration.EnvironmentVariables.EnvironmentVariablesConfigurationProvider();


Are this part of our safe list of dependencies that won't ship out-of-band?

fabiocav · 2024-10-19T02:56:17Z

host/src/PlaceholderApp/StartupHook.cs

+        Assembly entryAssembly = Assembly.GetEntryAssembly();
+        string version = System.Diagnostics.FileVersionInfo.GetVersionInfo(entryAssembly.Location).FileVersion;
+
+        var jsonPath = Path.Combine(Path.GetDirectoryName(entryAssembly.Location), Path.GetFileNameWithoutExtension(entryAssembly.Location) + ".deps.json");


It's unclear whether this is simply trying to exercise these paths or if there's something missing here. Can you comment?

fabiocav · 2024-10-19T03:06:31Z

host/src/PlaceholderApp/FunctionsNetHost.PlaceholderApp.csproj

+    <TieredCompilation>false</TieredCompilation>
+  </PropertyGroup>
+  <ItemGroup>
+    <FrameworkReference Include="Microsoft.AspNetCore.App" />


These are not part of our safe list of dependencies to bring in. At least in this iteration, we don't want to have this present.

kshyju · 2024-11-01T14:51:51Z

This pull request has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 7 days. It will be closed if no further activity occurs within 7 days of this comment.

Domo.

jviau · 2024-11-04T18:03:38Z

eng/ci/templates/official/jobs/build-host-prejit-artifacts.yml

+    outputs:
+    - output: pipelineArtifact
+      displayName: Publish Pre-jit placeholder app artifacts
+      path: $(Build.ArtifactStagingDirectory)/_preJitAppPackages


consider making this a variable

jviau · 2024-11-04T18:04:04Z

eng/ci/templates/official/jobs/build-host-prejit-artifacts.yml

+  steps:
+  - template: /eng/ci/templates/steps/install-dotnet.yml@self
+
+  - ${{ each version in split(variables.dotnetVersions, ',') }}:


small nit: use framework terminology instead of version.

jviau · 2024-11-04T18:16:45Z

eng/ci/templates/official/jobs/pack-host-artifacts.yml

-      path: $(Build.SourcesDirectory)/host/dist/portable
+      path: $(Build.SourcesDirectory)/host/dist/portable/prelaunchapps
+
+  - task: DownloadPipelineArtifact@2


Are we supposed to use DownloadPipelineArtifact? 1ES has its own syntax for artifact inputs:

https://eng.ms/docs/cloud-ai-platform/devdiv/one-engineering-system-1es/1es-docs/1es-pipeline-templates/features/inputs/pipeline-artifact

jviau · 2024-11-04T18:20:09Z

host/src/PlaceholderApp/FunctionsNetHost.PlaceholderApp.sln

Instead of a separate solution file, can we not use an existing one?

FNH-WIP for prejit support

fb4bd15

kshyju added 3 commits September 13, 2024 08:54

Install net9 rc1

450b956

Switching back to .NET9 preview 6

7068487

fix YAML to install preview6

1833452

safihamid reviewed Sep 13, 2024

View reviewed changes

host/src/FunctionsNetHost/PreJit/PreJitManager.cs Outdated Show resolved Hide resolved

safihamid reviewed Sep 13, 2024

View reviewed changes

host/src/FunctionsNetHost/Grpc/IncomingGrpcMessageHandler.cs Outdated Show resolved Hide resolved

kshyju added 2 commits September 16, 2024 08:13

Changed a trace level log to Info level.

8f02352

Updating build YAML file to include the pre-jit artifacts in the nuge…

4c51c4e

…t package. Minor version bumped.

liliankasem reviewed Sep 18, 2024

View reviewed changes

PR review fixes

76651aa

liliankasem reviewed Sep 19, 2024

View reviewed changes

host/src/FunctionsNetHost.Shared/EnvironmentVariables.cs Outdated Show resolved Hide resolved

Replaced "AZURE_FUNCTIONS_FUNCTIONSNETHOST" prefix with "AZURE_FUNCTI…

4d8d4a4

…ONS_NETHOST" for env variables.

liliankasem approved these changes Sep 19, 2024

View reviewed changes

safihamid approved these changes Sep 20, 2024

View reviewed changes

kshyju added 2 commits September 22, 2024 18:58

Added linux.coldstart jittrace and removed removed R2R from placehold…

2dd4703

…er app so that the pubish output (portable) works for linux use cases.

Fixed incorrect output path for prelaunch apps

e884965

kshyju commented Sep 23, 2024

View reviewed changes

kshyju requested a review from fabiocav September 23, 2024 02:06

kshyju commented Sep 23, 2024

View reviewed changes

host/src/PlaceholderApp/StartupHook.cs Show resolved Hide resolved

kshyju added 2 commits September 23, 2024 18:39

Switching to named pipes for communication between FNH and StartupHoo…

c1a449c

…k. Named eventwait handle does not work in unix,

Merge branch 'shkr/2637_fnh_prejit' of https://github.com/Azure/azure…

1e58d2f

…-functions-dotnet-worker into shkr/2637_fnh_prejit

kshyju commented Sep 27, 2024

View reviewed changes

Merge branch 'main' into shkr/2637_fnh_prejit

7377750

fabiocav requested changes Oct 24, 2024

View reviewed changes

microsoft-github-policy-service bot added the Needs: Author Feedback label Oct 24, 2024

microsoft-github-policy-service bot added no-recent-activity no recent activity labels Oct 31, 2024

microsoft-github-policy-service bot added Needs: Attention 👋 and removed Needs: Author Feedback no-recent-activity no recent activity labels Nov 1, 2024

jviau reviewed Nov 4, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Prejit support on FunctionsNetHost #2711

Prejit support on FunctionsNetHost #2711

kshyju commented Sep 13, 2024 •

edited

Loading

kshyju commented Sep 13, 2024

kshyju commented Sep 13, 2024

kshyju commented Sep 18, 2024

liliankasem left a comment

ivdiazsa commented Sep 18, 2024

kshyju commented Sep 19, 2024

steveisok commented Sep 19, 2024

safihamid left a comment

kshyju Sep 23, 2024

kshyju Sep 27, 2024

kshyju commented Sep 27, 2024

fabiocav Oct 19, 2024

fabiocav Oct 19, 2024

fabiocav Oct 19, 2024

fabiocav Oct 19, 2024

fabiocav Oct 19, 2024

fabiocav Oct 19, 2024

kshyju commented Nov 1, 2024

jviau Nov 4, 2024

jviau Nov 4, 2024

jviau Nov 4, 2024

jviau Nov 4, 2024

		@@ -0,0 +1,331 @@
		[S.P.CoreLib]System.Array.Empty<Candidate>()~System.Array,System.Private.CoreLib~0~1:Microsoft.AspNetCore.Routing.Matching.Candidate,Microsoft.AspNetCore.Routing~Empty

Prejit support on FunctionsNetHost #2711

Are you sure you want to change the base?

Prejit support on FunctionsNetHost #2711

Conversation

kshyju commented Sep 13, 2024 • edited Loading

Pull request checklist

kshyju commented Sep 13, 2024

kshyju commented Sep 13, 2024

kshyju commented Sep 18, 2024

liliankasem left a comment

Choose a reason for hiding this comment

ivdiazsa commented Sep 18, 2024

kshyju commented Sep 19, 2024

steveisok commented Sep 19, 2024

safihamid left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kshyju commented Sep 27, 2024

E2E test results

Windows

Linux

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kshyju commented Nov 1, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kshyju commented Sep 13, 2024 •

edited

Loading