[BUG] Azure batch sdk not working with java 17.

[vipintrnal]

Describe the bug
azure batch sdk not working with java 17.

Exception or Stack Trace

java.lang.reflect.InaccessibleObjectException: Unable to make field private static final long java.util.UUID.serialVersionUID accessible: module java.base does not "opens java.util" to unnamed module @45afc369
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
	at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)
	at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)
	at com.microsoft.rest.Validator.validateClass(Validator.java:77)
	at com.microsoft.rest.Validator.validate(Validator.java:64)
	at com.microsoft.rest.Validator.validateClass(Validator.java:115)
	at com.microsoft.rest.Validator.validate(Validator.java:64)
	at com.microsoft.azure.batch.protocol.implementation.TasksImpl.deleteWithServiceResponseAsync(TasksImpl.java:1050)
	at com.microsoft.azure.batch.protocol.implementation.TasksImpl.delete(TasksImpl.java:990)
	at com.microsoft.azure.batch.TaskOperations.deleteTask(TaskOperations.java:579)
	at com.microsoft.azure.batch.TaskOperations.deleteTask(TaskOperations.java:552)

To Reproduce
Steps to reproduce the behavior:
Use azure batch sdk with java 17.

Setup (please complete the following information):

• OS: macOs monetery, centos 7
• IDE: IntelliJ
• Library/Libraries: com.microsoft.azure:azure-batch:10.0.0
• Java version: 17

Additional context
I suspect the issue is due to below code:

Problem due to accessing the private fields using reflection:

package com.microsoft.rest;
class: Validator.java 

Reflective access not permitted with java 17

Information Checklist

• Bug Description Added
• Repro Steps Added
• Setup information Added

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @mksuni, @bgklein, @mscurrell, @cRui861, @paterasMSFT, @gingi, @dpwatrous.

Issue Details

---

Describe the bug
azure batch sdk not working with java 17.

Exception or Stack Trace

java.lang.reflect.InaccessibleObjectException: Unable to make field private static final long java.util.UUID.serialVersionUID accessible: module java.base does not "opens java.util" to unnamed module @45afc369
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
	at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)
	at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)
	at com.microsoft.rest.Validator.validateClass(Validator.java:77)
	at com.microsoft.rest.Validator.validate(Validator.java:64)
	at com.microsoft.rest.Validator.validateClass(Validator.java:115)
	at com.microsoft.rest.Validator.validate(Validator.java:64)
	at com.microsoft.azure.batch.protocol.implementation.TasksImpl.deleteWithServiceResponseAsync(TasksImpl.java:1050)
	at com.microsoft.azure.batch.protocol.implementation.TasksImpl.delete(TasksImpl.java:990)
	at com.microsoft.azure.batch.TaskOperations.deleteTask(TaskOperations.java:579)
	at com.microsoft.azure.batch.TaskOperations.deleteTask(TaskOperations.java:552)

To Reproduce
Steps to reproduce the behavior:
Use azure batch sdk with java 17.

Setup (please complete the following information):

• OS: macOs monetery, centos 7
• IDE: IntelliJ
• Library/Libraries: com.microsoft.azure:azure-batch:10.0.0
• Java version: 17

Additional context
I suspect the issue is due to below code:

Problem due to accessing the private fields using reflection:

package com.microsoft.rest;
class: Validator.java 

Reflective access not permitted with java 17

Information Checklist

• Bug Description Added
• Repro Steps Added
• Setup information Added

Author:	vipintrnal
Assignees:	-
Labels:	question, Batch, Service Attention, Client, customer-reported, needs-triage, Track 1
Milestone:	-

[joshfree]

@srnagar can you please follow up from the autorest / codegen side?

[loicmathieu]

Hi, we hit this issue recently after migrating to Java 17, I saw this issue is one year old is there any news about it ?

[maroony]

I'm also facing this bug. I hope, that this will be fixed soon. I didn't expect this because of:

All libraries are baselined on Java 8, with testing and forward support up until the latest Java long-term support release (currently Java 17).

See: Prerequisites

[coder-msl]

Hello,

as I am also facing this issue and it blocks my team with migration to Java 17 and it's been a while since it's reported, I've decided to debug the code and try to fix it for the community. I've tried to push new branch and create a PullRequest (in repository https://github.com/Azure/autorest-clientruntime-for-java ) however I have no sufficient rights to do it.

Issue is located in this file https://github.com/Azure/autorest-clientruntime-for-java/blob/master/client-runtime/src/main/java/com/microsoft/rest/Validator.java and it throws runtime exception for a field private static final long java.util.UUID.serialVersionUID accessible in class JobGetOptions. Please note that this field is both final and static.

The root cause of this issue is order of methods calls. In reflection it's not necessary to make a field accessible before getting both modifiers: static and/or final. Since Java17 it's not possible to make accessible fields which are part of java.util without opening modules etc, that's why this exception happens.

Below you can find a diff-patch which fixed the issue for me.

index 84ac58aea..6ab110f81 100644
--- a/client-runtime/src/main/java/com/microsoft/rest/Validator.java
+++ b/client-runtime/src/main/java/com/microsoft/rest/Validator.java
@@ -88,12 +88,12 @@ public final class Validator {
             return;
         }
         for (Field field : c.getDeclaredFields()) {
-            field.setAccessible(true);
             int mod = field.getModifiers();
             // Skip static fields since we don't have any, skip final fields since users can't modify them
             if (Modifier.isFinal(mod) || Modifier.isStatic(mod)) {
                 continue;
             }
+            field.setAccessible(true);
             JsonProperty annotation = field.getAnnotation(JsonProperty.class);
             // Skip read-only properties (WRITE_ONLY)
             if (annotation != null && annotation.access().equals(JsonProperty.Access.WRITE_ONLY)) {

Final code should look like:

        for (Field field : c.getDeclaredFields()) {
            int mod = field.getModifiers();
            // Skip static fields since we don't have any, skip final fields since users can't modify them
            if (Modifier.isFinal(mod) || Modifier.isStatic(mod)) {
                continue;
            }
            field.setAccessible(true);

           // more code here
      }

I hope it helps to fix this issue for the community. I didn't test it on all versions of Java between 8 - 16 but it should work as there were no changes in getModifiers() method as far as I know.

@loicmathieu @vipintrnal @maroony If you wait for this fix, you can apply above patch on your own and deploy adjusted version to internal artifactory and override dependency in pom.xml file of your project like below:

<dependency> 
    <groupId>com.microsoft.azure</groupId>
    <artifactId>azure-client-runtime</artifactId>
    <vertsion>1.7.15-SNAPSHOT</version>
</dependency>

It will also help to test on different scenarios on configuration setup.

If you need any further support, please do not hesitate to contact me.

[maroony]

At the moment, I don't have any issues with Java 17. Maybe it's because I'm using azure batch sdk 11.0.0? I don't know.

[coder-msl]

At the moment, I don't have any issues with Java 17. Maybe it's because I'm using azure batch sdk 11.0.0? I don't know.

Would you mind sharing your pom dependency? In my project it looks like below
:

<dependency>
    <groupId>com.microsoft.azure</groupId>
    <artifactId>azure-batch</artifactId>
    <version>11.0.0</version>
</dependency>

And I'm still facing this issue. Maybe you applied symilar workaround for the VM to open all modules like here kestra-io/kestra#1669 ?

I'm using OpenJDK 17.0.8 by Azul Zulu.

[maroony]

Ah... Found it! You are right. I'm using the "open workaround".

[coder-msl]

Ah... Found it! You are right. I'm using the "open workaround".

Ok. So it looks that this issue still exists and we must wait for @joshfree and/or @srnagar for code review process and release of new version.

[joshfree]

@mario-guerra could you help get this track1 batch issue discussed in the Batch team, and/or provide any updates on Batch release timelines?

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @bgklein @cRui861 @dpwatrous @gingi @mksuni @mscurrell @paterasMSFT.

[mario-guerra]

@dpwatrous for comment.

[dpwatrous]

Just a small update from the Batch team: @skapur12 was able to reproduce this issue and confirmed that the problem appears to be with the azure-client-runtime package. It looks like this is the issue we're hitting: Azure/autorest-clientruntime-for-java#709

If this is fixed in a new version of azure-client-runtime we will happily update our dependency to resolve this, but I don't see a way we can fix this inside the Batch SDK.

Also, we are actively working on shipping a preview of a new package based on the more modern azure-core SDK libraries instead of azure-client-runtime, so if the workaround mentioned in this PR gets you past this error for now, that may be a better long-term option.

[coder-msl]

@dpwatrous this workaround (opening modules widely to reflection) reduces security of the applications. It is there for a reason IMHO.

[dpwatrous]

Agreed the workaround isn't ideal. We'll update the Batch SDK's dependency as soon as there is a fix available in the client-runtime package.

[srnagar]

I am looking at this issue and will have an update on the best path forward for this issue.

[srnagar]

We are currently working on a track 2 version of the Azure Batch SDK which is expected to preview in early 2024. The newer version of this library will support Java 17 and will also follow the track 2 design guidelines for Azure SDKs. We recommend migrating from the existing Azure Batch library to the new version as the older versions of the library will be deprecated and will only have patch releases for security issues or critical bug fixes.

cc: @mario-guerra

[github-actions]

Hi @vipintrnal, we deeply appreciate your input into this project. Regrettably, this issue has remained inactive for over 2 years, leading us to the decision to close it. We've implemented this policy to maintain the relevance of our issue queue and facilitate easier navigation for new contributors. If you still believe this topic requires attention, please feel free to create a new issue, referencing this one. Thank you for your understanding and ongoing support.