Geonetwork on Spring 6 codesprint June 2023

Participants

Jose
Francois

Agenda

The main goal of this sprint is to evaluate the update of GeoNetwork to Java 11, Java 17, identify issues, and try to limit the number of critical security vulnerabilities that can be fixed by library updates.

Java 11 runtime tests
Java 17 runtime tests
Spring 5 and hibernate 5 update
Spring 6 and hibernate 6 evaluation

Java 11 runtime tests

Based on https://github.com/geonetwork/core-geonetwork/pull/6276, the application is starting up with:

mvn jetty:run
from the release build
from Intellij

Issues

Failed to startup in Intellij while using maven 3.8.1. Fixed with maven 3.8.7.

One runtime issue identified:

Metrics getOpenFileDescriptorCount is failing on API call http://localhost:8080/geonetwork/monitor/metrics?pretty=true

HTTP ERROR 500 java.lang.reflect.InaccessibleObjectException: 
Unable to make public long
 com.sun.management.internal.OperatingSystemImpl.getOpenFileDescriptorCount() accessible: 
module jdk.management does not "opens com.sun.management.internal" 
to unnamed module @4f7f59dd

Fixed by

--add-opens jdk.management/com.sun.management.internal=ALL-UNNAMED

Conclusion

GeoNetwork on Java 11 looks to be usable. Question:

Do we keep the possibility to build with Java 8 ? or we just move to Java 11 and add a warning if Java >11.
Raise an error if > Java11 ?

Java 17 or 20 runtime tests

Issues on startup

Java package access

This type of issue is related to the fact that JDK17 does not provide illegal-access option and the default is like JDK11 --illegal-access=deny

java.lang

module java.base does not "opens java.lang" to unnamed module

fixed by

--add-opens java.base/java.lang=ALL-UNNAMED

javax.annotation

javax.annotation

Error creating bean with name 'translationPackBuilder' defined in file 
[classes/org/fao/geonet/api/tools/i18n/TranslationPackBuilder.class]: 
Post-processing of merged bean definition failed; nested exception is 
java.lang.NoSuchMethodError: 'java.lang.String javax.annotation.Resource.lookup()'

Moving to jakarta EE9+ as done by Spring 6+

Tested with https://docs.openrewrite.org/recipes/java/migrate/jakarta/javaxannotationmigrationtojakartaannotation:

mvn -U org.openrewrite.maven:rewrite-maven-plugin:run \
  -Drewrite.recipeArtifactCoordinates=org.openrewrite.recipe:rewrite-migrate-java:RELEASE  \
  -Drewrite.activeRecipes=org.openrewrite.java.migrate.jakarta.JavaxAnnotationMigrationToJakartaAnnotation

org.codehaus.groovy.vmplugin.v7.Java7

From the release

java.lang.NoClassDefFoundError: Could not initialize class org.codehaus.groovy.vmplugin.v7.Java7

?

Conclusion

In order to run on Java 17+, Spring 6 migration is required (and Spring 6 requires Java 17 and Jakarta EE9).

So it probably means that GeoNetwork will then require at least (to be confirmed):

maven 3.8.7+
Java 17
Tomcat 10
Jetty 11 and also require that all dependencies used also moved to Jakarta EE9, Spring 6

Spring 5 update

Update to latest version:

Spring, Hibernate 5
Jetty 9

Branch https://github.com/geonetwork/core-geonetwork/tree/lib-spring5

Questions:

GN 4.4.0 ?
Java 8 and 11 or only 11 support ?
Need for warning message when running Java >11 ?
Doc update (JVM config, max version of Jetty/Tomcat)

Spring 6 migration evaluation?

Evaluate Spring 6 migration.

Startup or build warning

[WARNING] The POM for com.sun.xml.bind:jaxb-impl:jar:2.2.11 is invalid, transitive dependencies (if any) will not be available, enable debug logging for more details

Security vulnerability

One goal is to also update libraries which have CRITICIAL vulnerabilities reported.

Check github security alerts
Trivy

Scan the application with trivy

export GNPATH=/core-geonetwork
cd $GNPATH
docker pull aquasec/trivy:latest
  
docker run -v $PWD:$GNPATH aquasec/trivy fs --cache-dir $GNPATH/.trivycache/  --download-java-db-only
docker run -v $PWD:$GNPATH aquasec/trivy fs --cache-dir $GNPATH/.trivycache/  --download-db-only
docker run -v $PWD:$GNPATH aquasec/trivy \
  rootfs --cache-dir $GNPATH/.trivycache/ \
  --vuln-type library --skip-db-update --scanners vuln --severity CRITICAL \
  $GNPATH/web/target/geonetwork


# Scanning docker image
docker run -v $PWD:/core-geonetwork aquasec/trivy image --cache-dir /core-geonetwork/.trivycache/   --vuln-type library --skip-db-update --scanners vuln --severity HIGH,CRITICAL geonetwork:4.2.4

Upgrade snakeyml

Some libraries like JCloud and ES High level client depend on snakeyml version 1.2.6 that has vulnerabilities.

snakeyaml
org.springdoc:springdoc-openapi-common 
https://mvnrepository.com/artifact/org.springdoc/springdoc-openapi-common/1.7.0

Forcing the dependency to version 2.0, it builds and apparently it doesn't cause issues. But in the changes list, are indicated some backwards incompatible changes, that are hard to say if the related code is used: https://bitbucket.org/snakeyaml/snakeyaml/wiki/Changes

      <dependency>
        <groupId>org.yaml</groupId>
        <artifactId>snakeyaml</artifactId>
        <version>2.0</version>
      </dependency>

Apparently, new versions of ES doesn't depend on that library, but doesn't provide the ES High Level Client. The code requires to be changed to use the elasticsearch-rest-client.

For JCloud data storage should be checked if an upgrade is possible that depends on snakeyaml version 2.0 or to move it to a custom module, that is build on demand. As most GeoNetwork instalations doesn't require this feature (same strategy can be applied for CMIS and S3 storages).

Upgrade mapfish print

org.mapfish.print:print-lib (print-lib-2.2.0.jar)

Vulnerability report was wrongly targeting version 2. The vulnerability only affects version 3. Vulnerability report https://github.com/mapfish/mapfish-print/security/advisories/GHSA-vjv6-gq77-3mjw updated with the minimum version affected. Trivy java db not yet updated ?

Spring web

org.springframework:spring-web (spring-web-5.3.27.jar)

See

Does not apply to GeoNetwork configuration.

Remaining vulnerabilities:

org.jyaml:jyaml (jyaml-1.3.jar)
https://mvnrepository.com/artifact/org.jyaml/jyaml 
= No more recent version available

org.apache.cxf:cxf-core (cxf-core-3.0.12.jar) 
OpenCMIS https://mvnrepository.com/artifact/org.apache.chemistry.opencmis/chemistry-opencmis-client-api
= No more recent version available
= Add a module to exclude CMIS from build?