Merge branch 'main' into main.metadata.history.access.level

geonetwork · Jan 15, 2024 · 3d172b8 · 3d172b8
2 parents d908561 + 5d85e8c
commit 3d172b8
Show file tree

Hide file tree

Showing 231 changed files with 3,375 additions and 1,985 deletions.
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
@@ -6,15 +6,22 @@ Ask in a comment if you have troubles with any of them. -->
 
 # Checklist
 
-- [ ] I have read the [contribution guidelines](https://github.com/geonetwork/core-geonetwork/blob/main/CONTRIBUTING.md
+- [ ] I have read the [contribution guidelines](https://github.com/geonetwork/core-geonetwork/blob/main/CONTRIBUTING.md)
 - [ ] *Pull request* provided for `main` branch, backports managed with label
 - [ ] *Good housekeeping* of code, cleaning up comments, tests, and documentation
 - [ ] *Clean commit history* broken into understandable chucks, avoiding big commits with hundreds of files, cautious of reformatting and whitespace changes
 - [ ] *Clean commit message*s, longer verbose messages are encouraged
 - [ ] *API Changes* are identified in commit messages
-- [ ] *Testing* provided for features or enhancements using [automatic tests](https://github.com/geonetwork/core-geonetwork/blob/main/software_development/TESTING.md))
-- [ ] *User documentation* provided for new features or enhancements in [mannual](https://github.com/geonetwork/core-geonetwork/tree/main/docs/manual)
+- [ ] *Testing* provided for features or enhancements using [automatic tests](https://github.com/geonetwork/core-geonetwork/blob/main/software_development/TESTING.md)
+- [ ] *User documentation* provided for new features or enhancements in [manual](https://github.com/geonetwork/core-geonetwork/tree/main/docs/manual)
 - [ ] *Build documentation* provided for development instructions in `README.md` files
 - [ ] *Library management* using `pom.xml` dependency management. Update build documentation with intended library use and library tutorials or documentation
 
 <!--Submitting the PR does not require you to check all items, but by the time it gets merged, they should be either satisfied or not applicable.-->
+
+<!-- If you can, it's better to give credits to organisation supporting this work:
+- `Funded by NAME`
+- `Funded by URL`
+- `Funded by NAME URL`
+-->
+
diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml
@@ -22,7 +22,7 @@ jobs:
         with:
           fetch-depth: 0
       - name: Install Python
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: 3.x
       - name: mkdocs install

diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml
@@ -28,7 +28,7 @@ jobs:
         java-version: ${{ matrix.jdk }}
         cache: 'maven'
     - name: Setup Python
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v5
       with:
         python-version: 3.x
     - name: mkdocs install

diff --git a/.gitignore b/.gitignore
@@ -80,7 +80,6 @@ web/src/main/webapp/WEB-INF/data/wro4j*.db
 web/src/main/webapp/WEB-INF/data/wro4j-cache*
 web/src/main/webapp/WEB-INF/data_*
 web/src/main/webapp/WEB-INF/metadata_subversion/
-web/src/main/webapp/WEB-INF/server.prop
 web/src/main/webapp/WEB-INF/prebuilt
 web/src/main/webapp/data/
 web/src/main/webapp/doc/en

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -19,13 +19,13 @@ Thank you for contributing to GeoNetwork:
 
 * Good housekeeping: Anytime you commit, try and clean the code around it to latest style guide. If you improve a function without comments: add comments. If you modify functionality that does not have tests: write a test. If you fix functionality without documentation: add documentation.
 
-* History: Clean commit messages and history: avoid big commits with hundreds of files, break commits up into understandable chunks, longer verbose commit messages are encouraged. Beware of reformatting and needless whitespace changes.
+* History: Clean commit messages and history: avoid big commits with hundreds of files, break commits up into understandable chunks, longer verbose commit messages are encouraged. Avoid reformatting and needless whitespace changes.
 
 * Draft: Use pull request *Draft** (or even the text "WIP") to identify work in progress.
 
 * Rebase / Squash and merge: No merge commits with current branch, use Rebase or Squash and merge!
 
-* API Change: Please identify any API change or behavior changes in commit messages.
+* API Change: Please identify any API change or behavior changes in commit messages. Also make sure that a [process for deprecation](PROCESS_FOR_DEPRECATION.md) of a feature is carefully dealt with.
 
 * Review: Review is required by another person, or more than one! Don't be shy asking for help or reviewing.
 

diff --git a/PROCESS_FOR_DEPRECATION.md b/PROCESS_FOR_DEPRECATION.md
@@ -0,0 +1,6 @@
+* Proposal for a Process for Deprecation (draft)
+
+# Feature deprecation process
+
+This page describes considerations and steps to take when removing a product feature. This is the process of depreciation or 'deprecating a feature'.
+
diff --git a/README.md b/README.md
@@ -1,4 +1,4 @@
-# GeoNetwork Open-source
+# GeoNetwork opensource
 
 ## Build Health
 
@@ -11,13 +11,13 @@
 * An interactive Web Map Viewer to combine Web Map Services from distributed servers around the world
 * Online editing of metadata with a powerful template system
 * Scheduled harvesting and synchronization of metadata between distributed catalogs
-* Support for OGC-CSW 2.0.2 ISO Profile, OAI-PMH, SRU protocols
+* Support for OGC-CSW 2.0.2, ISO 1911x and DCAT-AP metadata profiles, OAI-PMH, SRU protocols
 * Fine-grained access control with group and user management
 * Multi-lingual user interface
 
 ## Documentation
 
-The Geonetwork Manual and Online Help are included in the `docs/manual` folder. This content is compiled into html pages during a release for a publishing on docs.geonetwork-opensource.org website.
+The GeoNetwork Manual and Online Help are included in the `docs/manual` folder. This content is compiled into html pages during a release for a publishing on docs.geonetwork-opensource.org website.
 
 * [docs.geonetwork-opensource.org](https://docs.geonetwork-opensource.org)
 
@@ -28,5 +28,5 @@ The online help is compiled into html pages during a release and is included in
 Developer documentation located in ``README.md`` files in the code-base:
 
 * General documentation for the project as a whole is in this [README.md](README.md)
-* [Software Development Documentation](/software_development/) provides instructions for setting up a development environment, building Geonetwork, compiling user documentation, and making a releases.
+* [Software Development Documentation](/software_development/) provides instructions for setting up a development environment, building GeoNetwork, compiling user documentation, and making a releases.
 * Module specific documentation can be found in each module:
diff --git a/SECURITY.md b/SECURITY.md
@@ -2,20 +2,20 @@
 
 The GeoNetwork community takes the security of the software and all services based on the software product seriously. On this page you can find the versions for which the community provides security patches. 
 
-If you believe you have found a security vulnerability in the software or an implementation of the software, please report it to [email protected] as described below. Do not publish the vulnerability in any public forums (such as twitter, email list or issue tracker).
+If you believe you have found a security vulnerability in the software or an implementation of the software, please report it to [email protected] as described below. Do not publish the vulnerability in any public forums (such as Twitter/X, email list or issue tracker).
 
 ## Supported Versions
 
 Each GeoNetwork release is supported with bug fixes for a limited period, with patch releases made approximately every three to six  months. 
 
-- We recommend to update to latest incremental release as soon as possible to address security vulnarabilities.
+- We recommend to update to latest incremental release as soon as possible to address security vulnerabilities.
 - Some overlap is provided when major versions are announced with both a current version and a maintenance version being made available to provide time for organizations to upgrade.
 
-| Version | Supported          | Comment                 |
-|---------| ------------------ |------------------------ |
-| 4.4.x   | :white_check_mark: | Current version         |
-| 4.2.x   | :white_check_mark: | Maintenance version     |
-| 3.12.x  | :white_check_mark: | Maintenance version     |
+| Version | Supported          | Comment             |
+|---------|--------------------|---------------------|
+| 4.4.x   | :white_check_mark: | Latest version      |
+| 4.2.x   | :white_check_mark: | Stable version      |
+| 3.12.x  | :white_check_mark: | Maintenance version |
 
 If your organisation is making use of a GeoNetwork version that is no longer in use by the community all is not lost. You can volunteer on the developer list to make additional releases, or engage with one of our [Commercial Support](https://www.osgeo.org/service-providers/?p=geonetwork) providers. 
 

diff --git a/core/pom.xml b/core/pom.xml
@@ -495,6 +495,12 @@
       <artifactId>gn-schema-iso19139</artifactId>
       <version>${project.version}</version>
     </dependency>
+    <dependency>
+      <groupId>org.geonetwork-opensource.schemas</groupId>
+      <artifactId>gn-schema-iso19115-3.2018</artifactId>
+      <version>${project.version}</version>
+      <scope>test</scope>
+    </dependency>
     <dependency> <!-- dummy API for ARC SDE stuff -->
       <groupId>${project.groupId}</groupId>
       <artifactId>gn-dummy-api</artifactId>

diff --git a/core/src/main/java/org/fao/geonet/api/records/attachments/AbstractStore.java b/core/src/main/java/org/fao/geonet/api/records/attachments/AbstractStore.java
@@ -97,7 +97,7 @@ public final ResourceHolder getResource(ServiceContext context, String metadataU
     }
 
     protected static AccessManager getAccessManager(final ServiceContext context) {
-        return context.getBean(AccessManager.class);
+        return ApplicationContextHolder.get().getBean(AccessManager.class);
     }
 
     public static int getAndCheckMetadataId(String metadataUuid, Boolean approved) throws Exception {

diff --git a/core/src/main/java/org/fao/geonet/constants/Edit.java b/core/src/main/java/org/fao/geonet/constants/Edit.java
@@ -74,6 +74,7 @@ public static final class Elem {
             public static final String TITLE = "title";
             public static final String IS_HARVESTED = "isHarvested";
             public static final String HARVEST_INFO = "harvestInfo";
+            public static final String OWNERID = "ownerId";
             public static final String OWNERNAME = "ownername";
             public static final String GROUPOWNERNAME = "groupOwnerName";
             public static final String POPULARITY = "popularity";

diff --git a/core/src/main/java/org/fao/geonet/kernel/datamanager/base/BaseMetadataManager.java b/core/src/main/java/org/fao/geonet/kernel/datamanager/base/BaseMetadataManager.java
@@ -867,6 +867,7 @@ private Element buildInfoElem(ServiceContext context, String id, String version)
         // add owner name
         java.util.Optional<User> user = userRepository.findById(Integer.parseInt(owner));
         if (user.isPresent()) {
+            addElement(info, Edit.Info.Elem.OWNERID, user.get().getId());
             String ownerName = user.get().getName();
             addElement(info, Edit.Info.Elem.OWNERNAME, ownerName);
         }

diff --git a/core/src/main/java/org/fao/geonet/kernel/search/EsSearchManager.java b/core/src/main/java/org/fao/geonet/kernel/search/EsSearchManager.java
@@ -35,6 +35,7 @@
 import jeeves.server.context.ServiceContext;
 import org.apache.commons.io.IOUtils;
 import org.apache.commons.lang.StringUtils;
+import org.elasticsearch.ElasticsearchParseException;
 import org.elasticsearch.action.ActionListener;
 import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
 import org.elasticsearch.action.bulk.BulkRequest;
@@ -47,7 +48,7 @@
 import org.elasticsearch.client.indices.CreateIndexRequest;
 import org.elasticsearch.client.indices.CreateIndexResponse;
 import org.elasticsearch.client.indices.GetIndexRequest;
-import org.elasticsearch.common.xcontent.XContentType;
+import org.elasticsearch.xcontent.XContentType;
 import org.elasticsearch.index.query.QueryStringQueryBuilder;
 import org.elasticsearch.index.reindex.DeleteByQueryRequest;
 import org.elasticsearch.script.Script;
@@ -271,11 +272,11 @@ private void createIndex(String indexId, String indexName, boolean dropIndexFirs
                 DeleteIndexRequest request = new DeleteIndexRequest(indexName);
                 AcknowledgedResponse deleteIndexResponse = client.getClient().indices().delete(request, RequestOptions.DEFAULT);
                 if (deleteIndexResponse.isAcknowledged()) {
-                    LOGGER.debug("Index '{}' removed.", new Object[]{indexName});
+                    LOGGER.debug("Index '{}' removed.", indexName);
                 }
             } catch (Exception e) {
                 // index does not exist ?
-                LOGGER.debug("Error during index '{}' removal. Error is: {}", new Object[]{indexName, e.getMessage()});
+                LOGGER.debug("Error during index '{}' removal. Error is: {}", indexName, e.getMessage());
             }
         }
 
@@ -304,7 +305,7 @@ private void createIndex(String indexId, String indexName, boolean dropIndexFirs
                     CreateIndexResponse createIndexResponse = client.getClient().indices().create(createIndexRequest, RequestOptions.DEFAULT);
 
                     if (createIndexResponse.isAcknowledged()) {
-                        LOGGER.debug("Index '{}' created", new Object[]{indexName});
+                        LOGGER.debug("Index '{}' created", indexName);
                     } else {
                         final String message = String.format("Index '%s' was not created. Error is: %s", indexName, createIndexResponse.toString());
                         LOGGER.error(message);
@@ -317,10 +318,13 @@ private void createIndex(String indexId, String indexName, boolean dropIndexFirs
                         indexName));
                 }
             }
+        } catch (ElasticsearchParseException ex) {
+            LOGGER.error(ex.getMessage(), ex);
+            throw new IOException(ex.getMessage());
         } catch (Exception cnce) {
             final String message = String.format("Could not connect to index '%s'. Error is %s. Is the index server  up and running?",
                 defaultIndex, cnce.getMessage());
-            LOGGER.error(message);
+            LOGGER.error(message, cnce);
             throw new IOException(message);
         }
     }
@@ -444,7 +448,7 @@ private void sendDocumentsToIndex() {
             } catch (Exception e) {
                 LOGGER.error(
                     "An error occurred while indexing {} documents in current indexing list. Error is {}.",
-                    new Object[]{listOfDocumentsToIndex.size(), e.getMessage()});
+                        listOfDocumentsToIndex.size(), e.getMessage());
             } finally {
                 // TODO: Trigger this async ?
                 documents.keySet().forEach(uuid -> overviewFieldUpdater.process(uuid));
@@ -489,14 +493,14 @@ private void checkIndexResponse(BulkResponse bulkItemResponses,
                     // TODO: Report the JSON which was causing the error ?
 
                     LOGGER.error("Document with error #{}: {}.",
-                        new Object[]{e.getId(), e.getFailureMessage()});
+                            e.getId(), e.getFailureMessage());
                     LOGGER.error(failureDoc);
 
                     try {
                         listErrorOfDocumentsToIndex.put(e.getId(), mapper.writeValueAsString(docWithErrorInfo));
                     } catch (JsonProcessingException e1) {
                         LOGGER.error("Generated document for the index is not properly formatted. Check document #{}: {}.",
-                            new Object[]{e.getId(), e1.getMessage()});
+                                e.getId(), e1.getMessage());
                     }
                 }
             });
@@ -505,7 +509,7 @@ private void checkIndexResponse(BulkResponse bulkItemResponses,
                 BulkResponse response = client.bulkRequest(defaultIndex, listErrorOfDocumentsToIndex);
                 if (response.status().getStatus() != 201) {
                     LOGGER.error("Failed to save error documents {}.",
-                        new Object[]{Arrays.toString(errorDocumentIds.toArray())});
+                            Arrays.toString(errorDocumentIds.toArray()));
                 }
             }
         }
@@ -638,7 +642,7 @@ public ObjectNode documentToJson(Element xml) {
                                 mapper.readTree(node.getText()));
                         } catch (IOException e) {
                             LOGGER.error("Parsing invalid JSON node {} for property {}. Error is: {}",
-                                new Object[]{node.getTextNormalize(), propertyName, e.getMessage()});
+                                    node.getTextNormalize(), propertyName, e.getMessage());
                         }
                     } else {
                         arrayNode.add(
@@ -657,7 +661,7 @@ public ObjectNode documentToJson(Element xml) {
                     doc.set("geom", mapper.readTree(nodeElements.get(0).getTextNormalize()));
                 } catch (IOException e) {
                     LOGGER.error("Parsing invalid geometry for JSON node {}. Error is: {}",
-                        new Object[]{nodeElements.get(0).getTextNormalize(), e.getMessage()});
+                            nodeElements.get(0).getTextNormalize(), e.getMessage());
                 }
                 continue;
             }
@@ -670,7 +674,7 @@ public ObjectNode documentToJson(Element xml) {
                         ));
                 } catch (IOException e) {
                     LOGGER.error("Parsing invalid JSON node {} for property {}. Error is: {}",
-                        new Object[]{nodeElements.get(0).getTextNormalize(), propertyName, e.getMessage()});
+                            nodeElements.get(0).getTextNormalize(), propertyName, e.getMessage());
                 }
             } else {
                 doc.put(propertyName,

diff --git a/...src/main/java/org/fao/geonet/kernel/security/openidconnect/GeonetworkOidcUserService.java b/...src/main/java/org/fao/geonet/kernel/security/openidconnect/GeonetworkOidcUserService.java
@@ -22,9 +22,11 @@
  */
 package org.fao.geonet.kernel.security.openidconnect;
 
+import org.fao.geonet.kernel.security.GeonetworkAuthenticationProvider;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.security.access.hierarchicalroles.RoleHierarchy;
 import org.springframework.security.core.GrantedAuthority;
+import org.springframework.security.core.userdetails.UserDetails;
 import org.springframework.security.oauth2.client.oidc.userinfo.OidcUserRequest;
 import org.springframework.security.oauth2.client.oidc.userinfo.OidcUserService;
 import org.springframework.security.oauth2.core.OAuth2AuthenticationException;
@@ -55,12 +57,33 @@ public class GeonetworkOidcUserService extends OidcUserService {
     @Autowired
     RoleHierarchy roleHierarchy;
 
+    @Autowired
+    GeonetworkAuthenticationProvider geonetworkAuthenticationProvider;
+
+    @Autowired
+    protected SimpleOidcUserFactory simpleOidcUserFactory;
+
     @Override
     public OidcUser loadUser(OidcUserRequest userRequest) throws OAuth2AuthenticationException {
         OidcUser user = super.loadUser(userRequest);
 
+        Collection<? extends GrantedAuthority> authorities;
+
+        if (!oidcConfiguration.isUpdateProfile()) {
+            // Retrieve the authorities from the local user
+            try {
+                SimpleOidcUser simpleUser = simpleOidcUserFactory.create(user.getAttributes());
+                UserDetails userDetails = geonetworkAuthenticationProvider.loadUserByUsername(simpleUser.getUsername());
+
+                authorities = userDetails.getAuthorities();
+            } catch (Exception ex) {
+                authorities = createAuthorities(user);
+            }
+        } else {
+            authorities = createAuthorities(user);
+        }
+
         OidcUserInfo userInfo = user.getUserInfo();
-        Collection<? extends GrantedAuthority> authorities = createAuthorities(user);
 
         //get the user name from a specific attribute (if specified) or use default.
         String userNameAttributeName = userRequest.getClientRegistration()