Ensure JIT/AOT code is not invalidated post-restore under `-XX:+DebugOnRestore` #20047

dsouzai · 2024-08-22T21:08:47Z

#19754 adds VM Support for Debug On Restore. As the code stands, this will result in the JIT invalidating all code post-restore because it is unable to distinguish between FSD mode caused by the VM or the user.

This PR fixes this by adding the following changes:

Cache the status of certain runtime events
Return early from isFSDNeeded under -XX:+DebugOnRestore

Depends on #19754 (specifically the update to the isDebugOnRestoreEnabled API).

JasonFengJ9 · 2024-09-18T22:27:59Z

@dsouzai would you mind enabling the following failure conditions that were disabled in #19754? I verified manually this PR passed the test with the failure conditions were reestablished.

openj9/test/functional/cmdLineTests/criu/criu_jitPostRestore.xml

Lines 75 to 77 in ebe8718

    
           		<!-- Following two failure conditions to be re-enabled with https://github.com/eclipse-openj9/openj9/pull/20047 --> 
        
           		<!-- output type="failure" caseSensitive="yes" regex="no">Some or all compiled code in the code cache invalidated post restore.</output> 
        
           		<output type="failure" caseSensitive="yes" regex="no">JIT compilation disabled post restore.</output -->

openj9/test/functional/cmdLineTests/criu/criu_jitPostRestore.xml

Lines 94 to 96 in ebe8718

    
           		<!-- Following two failure conditions to be re-enabled with https://github.com/eclipse-openj9/openj9/pull/20047 --> 
        
           		<!-- output type="failure" caseSensitive="yes" regex="no">JIT compilation disabled post restore.</output> 
        
           		<output type="failure" caseSensitive="yes" regex="no">AOT load and compilation disabled post restore.</output -->

dsouzai · 2024-09-19T14:21:57Z

@ymanton could you please review/merge?

dsouzai · 2024-09-20T19:12:29Z

@TobiAjila fyi this also needs to get into 0.48

ymanton

LGTM.

ymanton · 2024-09-20T19:40:02Z

Jenkins test sanity.functional xlinux,plinux,zlinux jdk11,jdk17

keithc-ca · 2024-09-20T21:31:44Z

runtime/compiler/runtime/CRRuntime.hpp

+    *        as well as whether method trace and FSD were enabled
+    *        pre-checkpoint.
+    */
+   cacheEventsStatus();


This should have an explicit return type (i.e. void).

Ah damn, thanks for pointing out. Surprising this ever built locally.

Cache the status of JVMTI events such as exception throw/catch as well as whether method trace and FSD were enabled pre-checkpoint. Signed-off-by: Irwin D'Souza <[email protected]>

Under -XX:+DebugOnRestore, the VM will enable some capabilities that will be reset on restore (if the user does not want debug on restore). However, if the JIT detects these capabilities via the J9HookDisable call, there is no way for the JIT to know whether the capability was set by the VM or a user pre-checkpoint. With the recent change to the isDebugOnRestoreEnabled to check if a uesr specified an JDWP agent pre-checkpoint, the JIT can simply return early from isFSDNeeded. Under -XX:+DebugOnRestore, the JIT already generates FSD code so there is on need to check the caps that were set by the VM pre-checkpoint. isFSDNeeded is called on restore, so at that point it is appropriate for the JIT to execute the rest of the method at that point. Signed-off-by: Irwin D'Souza <[email protected]>

Signed-off-by: Irwin D'Souza <[email protected]>

dsouzai · 2024-09-20T23:04:12Z

Jenkins test sanity.functional xlinux,plinux,zlinux jdk11,jdk17

dsouzai · 2024-09-23T13:00:07Z

The ppc64le test failure are all the Test -Xjit:exclude={*} without -XX:+DebugOnRestore:

[2024-09-21T00:23:24.079Z] Output from test:
[2024-09-21T00:23:24.079Z]  [OUT] start running script
[2024-09-21T00:23:24.079Z]  [OUT] export GLIBC_TUNABLES=glibc.cpu.hwcaps=-XSAVEC,-XSAVE,-AVX2,-ERMS,-AVX,-AVX_Fast_Unaligned_Load
[2024-09-21T00:23:24.079Z]  [OUT] export LD_BIND_NOT=on
[2024-09-21T00:23:24.079Z]  [OUT] /home/jenkins/workspace/Test_openjdk11_j9_sanity.functional_ppc64le_linux_Personal_testList_0/jdkbinary/j2sdk-image/bin/java -XX:+EnableCRIUSupport  -Xjit  -cp /home/jenkins/workspace/Test_openjdk11_j9_sanity.functional_ppc64le_linux_Personal_testList_0/aqa-tests/TKG/../../jvmtest/functional/cmdLineTests/criu/criu.jar org.openj9.criu.OptionsFileTest JitOptionsTest -Xjit:exclude={*} 1
[2024-09-21T00:23:24.079Z]  [OUT] Pre-checkpoint
[2024-09-21T00:23:24.079Z]  [OUT] main: Sat Sep 21 00:23:18 UTC 2024, Performing CRIUSupport.checkpointJVM(), System.currentTimeMillis(): 1726878198957, System.nanoTime(): 1363144591107177
[2024-09-21T00:23:24.079Z]  [OUT] JVMJITM043W AOT load and compilation disabled post restore.
[2024-09-21T00:23:24.079Z]  [OUT] JVMJITM044W Some or all compiled code in the code cache invalidated post restore.
[2024-09-21T00:23:24.079Z]  [OUT] Post-checkpoint
[2024-09-21T00:23:24.079Z]  [OUT] initiate restore
[2024-09-21T00:23:24.079Z]  [OUT] Removed test output files
[2024-09-21T00:23:24.079Z]  [OUT] finished script
[2024-09-21T00:23:24.079Z]  [ERR] /home/jenkins/workspace/Test_openjdk11_j9_sanity.functional_ppc64le_linux_Personal_testList_0/aqa-tests/TKG/../../jvmtest/functional/cmdLineTests/criu/criuScript.sh: line 41: 2543286 Killed                  $2 -XX:+EnableCRIUSupport $3 -cp "$1/criu.jar" $4 $5 $6 > testOutput 2>&1
[2024-09-21T00:23:24.079Z] >> Success condition was found: [Output match: Killed]
[2024-09-21T00:23:24.079Z] >> Required condition was found: [Output match: Pre-checkpoint]
[2024-09-21T00:23:24.079Z] >> Success condition was found: [Output match: Post-checkpoint]
[2024-09-21T00:23:24.079Z] >> Failure condition was not found: [Output match: CRIU is not enabled]
[2024-09-21T00:23:24.080Z] >> Failure condition was not found: [Output match: Operation not permitted]
[2024-09-21T00:23:24.080Z] >> Success condition was found: [Output match: Some or all compiled code in the code cache invalidated post restore.]
[2024-09-21T00:23:24.080Z] >> Failure condition was not found: [Output match: JIT compilation disabled post restore.]
[2024-09-21T00:23:24.080Z] >> Failure condition was found: [Output match: AOT load and compilation disabled post restore.]
[2024-09-21T00:23:24.080Z] >> Success condition was not found: [Output match: Thread pid mismatch]
[2024-09-21T00:23:24.080Z] >> Success condition was not found: [Output match: do not match expected]
[2024-09-21T00:23:24.080Z] >> Success condition was not found: [Output match: Unable to create a thread:]
[2024-09-21T00:23:24.080Z] >> Failure condition was not found: [Output match: User requested Java dump using]

Because AOT load and compilation disabled post restore. gets printed out before Some or all compiled code in the code cache invalidated post restore., I believe this is due to AOT being disabled because of the SCC or similar, rather than because of something like FSD. Essentially, I believe it gets printed out here

openj9/runtime/compiler/control/OptionsPostRestore.cpp

Line 862 in 0213b04

disableAOTCompilation();

rather than here

openj9/runtime/compiler/control/OptionsPostRestore.cpp

Line 752 in 0213b04

disableAOTCompilation();

@ymanton I believe you've also seen this issue before right?

ymanton · 2024-09-23T13:08:37Z

@ymanton I believe you've also seen this issue before right?

Yeah that's a pre-existing issue.

I'll merge shortly if there are no further comments and requested changes are confirmed.

dsouzai · 2024-09-24T15:04:23Z

@ymanton could this be merged?

dsouzai added comp:jit criu Used to track CRIU snapshot related work labels Aug 22, 2024

This was referenced Aug 22, 2024

CRIU supports Java debugger via the restore option file #19754

Merged

Compiler Support for Debug On Restore #18866

Open

dsouzai force-pushed the fsdPostRestore branch from 10291b2 to 967540d Compare September 12, 2024 16:15

dsouzai force-pushed the fsdPostRestore branch from 967540d to ff8812d Compare September 19, 2024 14:21

dsouzai mentioned this pull request Sep 20, 2024

Refactor a few CRIU APIs to take J9JavaVM* instead of J9VMThread* #20205

Merged

ymanton approved these changes Sep 20, 2024

View reviewed changes

JasonFengJ9 mentioned this pull request Sep 20, 2024

-agentpath: libraries require no path/name decoration #20202

Merged

pshipton added this to the Release 0.48 (Java 8, 11, 17, 21, 23) October refresh milestone Sep 20, 2024

keithc-ca requested changes Sep 20, 2024

View reviewed changes

dsouzai added 3 commits September 20, 2024 18:51

Cache the status of certain runtime events

89ea4a2

Cache the status of JVMTI events such as exception throw/catch as well as whether method trace and FSD were enabled pre-checkpoint. Signed-off-by: Irwin D'Souza <[email protected]>

Renable failure conditions disabled in eclipse-openj9#19754

d3a7474

Signed-off-by: Irwin D'Souza <[email protected]>

dsouzai force-pushed the fsdPostRestore branch from ff8812d to d3a7474 Compare September 20, 2024 22:52

ymanton merged commit 8cdfd73 into eclipse-openj9:master Sep 24, 2024
11 of 13 checks passed

dsouzai mentioned this pull request Sep 24, 2024

(0.48) Ensure JIT/AOT code is not invalidated post-restore under -XX:+DebugOnRestore #20219

Merged

dsouzai removed this from the Release 0.48 (Java 8, 11, 17, 21, 23) October refresh milestone Sep 24, 2024

hangshao0 mentioned this pull request Sep 25, 2024

cmdLineTester_criu_jitPostRestore Testing: Test -Xjit:exclude={*} AOT load and compilation disabled post restore #20233

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ensure JIT/AOT code is not invalidated post-restore under `-XX:+DebugOnRestore` #20047

Ensure JIT/AOT code is not invalidated post-restore under `-XX:+DebugOnRestore` #20047

dsouzai commented Aug 22, 2024 •

edited

Loading

JasonFengJ9 commented Sep 18, 2024

dsouzai commented Sep 19, 2024

dsouzai commented Sep 20, 2024

ymanton left a comment

ymanton commented Sep 20, 2024

keithc-ca Sep 20, 2024

dsouzai Sep 20, 2024

dsouzai commented Sep 20, 2024

dsouzai commented Sep 23, 2024

ymanton commented Sep 23, 2024

dsouzai commented Sep 24, 2024

Ensure JIT/AOT code is not invalidated post-restore under -XX:+DebugOnRestore #20047

Ensure JIT/AOT code is not invalidated post-restore under -XX:+DebugOnRestore #20047

Conversation

dsouzai commented Aug 22, 2024 • edited Loading

JasonFengJ9 commented Sep 18, 2024

dsouzai commented Sep 19, 2024

dsouzai commented Sep 20, 2024

ymanton left a comment

Choose a reason for hiding this comment

ymanton commented Sep 20, 2024

keithc-ca Sep 20, 2024

Choose a reason for hiding this comment

dsouzai Sep 20, 2024

Choose a reason for hiding this comment

dsouzai commented Sep 20, 2024

dsouzai commented Sep 23, 2024

ymanton commented Sep 23, 2024

dsouzai commented Sep 24, 2024

Ensure JIT/AOT code is not invalidated post-restore under `-XX:+DebugOnRestore` #20047

Ensure JIT/AOT code is not invalidated post-restore under `-XX:+DebugOnRestore` #20047

dsouzai commented Aug 22, 2024 •

edited

Loading