Cannot use default metadata configuration after having datasets with custom metadata blocks #8471

j-n-c · 2022-03-08T15:07:39Z

What steps does it take to reproduce the issue?

When does this issue occur?

Setup a custom metadata block based on the default Citation Metadada block (basically the same as the default but adding a new field to the Author section)
On root dataverse create a new dataverse collection (DV2) unselecting Use metadata fields from Root and selecting the custom metadata block
On the child dataverse (DV2) create a new dataset (DS1) and fill the custom field
On DV2 unselect custom metadata block and selecting Use metadata fields from Root (basically returning configuration to default settings)
On DV2 create a new dataset
1. On DV2 unselect Use metadata fields from Root and select custom metadata block

Which page(s) does it occurs on?
<dataverse_host>:8080/dataset.xhtml?ownerId=2
What happens?

Although both Citation Metadata (Required) and Custom Citation Metadata are selected, only Custom Citation Metadata is available when creating the dataset
An error is returned:

[2022-03-08T12:47:42.030+0000] [Payara 5.2021.5] [WARNING] [] [javax.enterprise.resource.webcontainer.jsf.lifecycle] [tid: _ThreadID=97 _ThreadName=http-thread-pool::jk-connector(1)] [timeMillis: 1646743662030] [levelValue: 900] [[
  #{DatasetPage.init}: java.lang.NullPointerException
javax.faces.FacesException: #{DatasetPage.init}: java.lang.NullPointerException
	at com.sun.faces.application.ActionListenerImpl.getNavigationOutcome(ActionListenerImpl.java:96)
	at com.sun.faces.application.ActionListenerImpl.processAction(ActionListenerImpl.java:71)
	at javax.faces.component.UIViewAction.broadcast(UIViewAction.java:548)
	at javax.faces.component.UIViewRoot.broadcastEvents(UIViewRoot.java:847)
	at javax.faces.component.UIViewRoot.processApplication(UIViewRoot.java:1396)
	at com.sun.faces.lifecycle.InvokeApplicationPhase.execute(InvokeApplicationPhase.java:58)
	at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:76)
	at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:177)
	at javax.faces.webapp.FacesServlet.executeLifecyle(FacesServlet.java:707)
	at javax.faces.webapp.FacesServlet.service(FacesServlet.java:451)
	at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1636)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:331)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
	at org.glassfish.tyrus.servlet.TyrusServletFilter.doFilter(TyrusServletFilter.java:282)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:253)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
	at org.ocpsoft.rewrite.servlet.RewriteFilter.doFilter(RewriteFilter.java:226)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:253)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:257)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
	at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:757)
	at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:577)
	at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:158)
	at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:371)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:238)
	at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:520)
	at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:217)
	at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:182)
	at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:156)
	at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:218)
	at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
	at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
	at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:524)
	at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:89)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:94)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:33)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:114)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: javax.faces.el.EvaluationException: java.lang.NullPointerException
	at com.sun.faces.application.MethodBindingMethodExpressionAdapter.invoke(MethodBindingMethodExpressionAdapter.java:76)
	at com.sun.faces.application.ActionListenerImpl.getNavigationOutcome(ActionListenerImpl.java:82)
	... 45 more
Caused by: java.lang.NullPointerException
	at edu.harvard.iq.dataverse.DatasetPage.updateDatasetFieldInputLevels(DatasetPage.java:1698)
	at edu.harvard.iq.dataverse.DatasetPage.init(DatasetPage.java:2014)
	at edu.harvard.iq.dataverse.DatasetPage.init(DatasetPage.java:1760)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.sun.el.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:163)
	at com.sun.el.parser.AstValue.invoke(AstValue.java:261)
	at com.sun.el.MethodExpressionImpl.invoke(MethodExpressionImpl.java:237)
	at org.jboss.weld.module.web.util.el.ForwardingMethodExpression.invoke(ForwardingMethodExpression.java:40)
	at org.jboss.weld.module.web.el.WeldMethodExpression.invoke(WeldMethodExpression.java:50)
	at com.sun.faces.facelets.el.TagMethodExpression.invoke(TagMethodExpression.java:65)
	at com.sun.faces.application.MethodBindingMethodExpressionAdapter.invoke(MethodBindingMethodExpressionAdapter.java:66)
	... 46 more
]]

[2022-03-08T12:47:42.030+0000] [Payara 5.2021.5] [WARNING] [] [javax.enterprise.web] [tid: _ThreadID=97 _ThreadName=http-thread-pool::jk-connector(1)] [timeMillis: 1646743662030] [levelValue: 900] [[
  StandardWrapperValve[Faces Servlet]: Servlet.service() for servlet Faces Servlet threw exception
java.lang.NullPointerException
	at edu.harvard.iq.dataverse.DatasetPage.updateDatasetFieldInputLevels(DatasetPage.java:1698)
	at edu.harvard.iq.dataverse.DatasetPage.init(DatasetPage.java:2014)
	at edu.harvard.iq.dataverse.DatasetPage.init(DatasetPage.java:1760)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.sun.el.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:163)
	at com.sun.el.parser.AstValue.invoke(AstValue.java:261)
	at com.sun.el.MethodExpressionImpl.invoke(MethodExpressionImpl.java:237)
	at org.jboss.weld.module.web.util.el.ForwardingMethodExpression.invoke(ForwardingMethodExpression.java:40)
	at org.jboss.weld.module.web.el.WeldMethodExpression.invoke(WeldMethodExpression.java:50)
	at com.sun.faces.facelets.el.TagMethodExpression.invoke(TagMethodExpression.java:65)
	at com.sun.faces.application.MethodBindingMethodExpressionAdapter.invoke(MethodBindingMethodExpressionAdapter.java:66)
	at com.sun.faces.application.ActionListenerImpl.getNavigationOutcome(ActionListenerImpl.java:82)
	at com.sun.faces.application.ActionListenerImpl.processAction(ActionListenerImpl.java:71)
	at javax.faces.component.UIViewAction.broadcast(UIViewAction.java:548)
	at javax.faces.component.UIViewRoot.broadcastEvents(UIViewRoot.java:847)
	at javax.faces.component.UIViewRoot.processApplication(UIViewRoot.java:1396)
	at com.sun.faces.lifecycle.InvokeApplicationPhase.execute(InvokeApplicationPhase.java:58)
	at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:76)
	at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:177)
	at javax.faces.webapp.FacesServlet.executeLifecyle(FacesServlet.java:707)
	at javax.faces.webapp.FacesServlet.service(FacesServlet.java:451)
	at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1636)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:331)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
	at org.glassfish.tyrus.servlet.TyrusServletFilter.doFilter(TyrusServletFilter.java:282)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:253)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
	at org.ocpsoft.rewrite.servlet.RewriteFilter.doFilter(RewriteFilter.java:226)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:253)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:257)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
	at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:757)
	at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:577)
	at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:158)
	at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:371)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:238)
	at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:520)
	at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:217)
	at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:182)
	at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:156)
	at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:218)
	at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
	at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
	at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
	at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:524)
	at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:89)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:94)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:33)
	at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:114)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)
	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)
	at java.base/java.lang.Thread.run(Thread.java:829)
]]

It is now possible to create a dataset

To whom does it occur (all users, curators, superusers)?
Superuser
What did you expect to happen?

Default Citation metadata and custom metadata blocks would be displayed
A new dataset would be created with the default citation metadata fields available

Which version of Dataverse are you using?
v5.8

Any related open or closed issues to this bug report?
Not that I coud find

Screenshots:

No matter the issue, screenshots are always welcome.

To add a screenshot, please use one of the following formats and/or methods described here:

https://help.github.com/en/articles/file-attachments-on-issues-and-pull-requests

The text was updated successfully, but these errors were encountered:

pdurbin · 2022-03-08T16:03:26Z

DatasetPage.updateDatasetFieldInputLevels(DatasetPage.java:1698)

For Dataverse 5.8 that line is a comment so I don't see how a NPE could be thrown: https://github.com/IQSS/dataverse/blob/v5.8/src/main/java/edu/harvard/iq/dataverse/DatasetPage.java#L1698

Are you running a fork?

Thanks for participating in the community call just now. I wanted to give you this quick feedback but I'll try to circle back to this soon.

j-n-c · 2022-03-08T16:25:51Z

I tested this on our Dataverse v5.8 installation and on a clean fork of the IQSS v5.9 repo using vagrant. The behaviour is the same.

Below are the steps I performed to implement the custom metadata blocks. Perhaps this could give a hint for debugging or highlight an error in our deployment (I aggregated data from IQSS/dataverse/releases/tag/v5.8 > Upgrade Instructions and https://guides.dataverse.org/en/latest/admin/metadatacustomization.html:

Edit/add metadata block .tsv files and corresponding Bundle properties files
Build .war file (used IntelliJ IDEA)
Copy .war, edited/added .tsv and update-fields.sh files to dataverse host
Undeploy currently deployed .war file

/usr/local/payara5/bin/asadmin list-applications
/usr/local/payara5/bin/asadmin undeploy dataverse<-version>

Stop Payara and remove the generated directory

service payara stop
rm -rf /usr/local/payara5/glassfish/domains/domain1/generated

Start Payara
service payara start
Deploy new version (.war):
/usr/local/payara5/bin/asadmin deploy <path to war_file>/dataverse-5.8.war

NOTE: Some org.postgresql.util.PSQLException messages related to foreign keys already existing may be displayed

Restart payara

service payara stop
service payara start

Load TSV files into a Dataverse Installation
curl http://localhost:8080/api/admin/datasetfield/load -H "Content-type: text/tab-separated-values" -X POST --upload-file <path to .tsv file>/new-metadata-block.tsv
Update the Solr Schema

curl "http://localhost:8080/api/admin/index/solr/schema" -o custom_solr_schema.xml
sudo ./update-fields.sh /usr/local/solr/solr-8.8.1/server/solr/collection1/conf/schema.xml <path_to_custom_solar_schema>/custom_solr_schema.xml

Reload Solr schema:
curl "http://localhost:8983/solr/admin/cores?action=RELOAD&core=collection1"
Restart Solr

service solr stop
service solr start

pdurbin · 2022-03-28T14:36:50Z

@j-n-c it's been on my list to test this and I'm sorry I haven't gotten to it yet. Thanks for testing 5.9. Would you be able to test the "develop" branch? Pull request #8454 was merged recently and it makes changes to the citation.tsv block. Basically, I'm simply wondering if you're seeing the same behavior or not. If you're able to test on develop, please mention the commit you're on when you reply. Thanks!

pdurbin · 2022-04-15T18:46:13Z

@j-n-c I'm only now realizing that I should have asked you for your tsv file, the one that shows as "Custom Citation Metadata" in your screenshot? Can you please attach it in a comment? You'll probably have to rename it to end with .txt to upload it.

Also, have you had a chance to try this on the "develop" branch?

j-n-c · 2022-05-10T16:09:19Z

@pdurbin I have made more tests and have not had an issue adding new metadata fields or inserting fields into exiting ones. I think that the error I got was because I copied the citation.tsv file and added new fields with may have caused some issues due to duplicated identifiers for fields metadataBlock and datasetField.

If you agree I will close this issue and open a new one if I ever come across any issues in the future.

pdurbin · 2022-05-10T19:12:01Z

@j-n-c sounds great. I'm glad you're all set. Please do feel free to close this issue.

I'm not surprised if you have trouble with two citation metadata blocks. You should probably change the new one enough to be a completely different block.

j-n-c · 2022-05-11T10:39:37Z

@pdurbin both adding new fields to citation.tsv (making sure there are no repeated identifiers and that the display order field values are consiste) and creating a new metadata block worked for me.
Thank's again for the help

pdurbin · 2022-05-11T20:51:32Z

@j-n-c ok. Better to create a new block than to edit citation.tsv because we reserve the right to update the later. (We'll mention in release notes when to reload citation.tsv.)

j-n-c closed this as completed May 11, 2022

mreekie added the pm.sprint.2022_05_11 label May 16, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot use default metadata configuration after having datasets with custom metadata blocks #8471

Cannot use default metadata configuration after having datasets with custom metadata blocks #8471

j-n-c commented Mar 8, 2022

pdurbin commented Mar 8, 2022

j-n-c commented Mar 8, 2022 •

edited

Loading

pdurbin commented Mar 28, 2022

pdurbin commented Apr 15, 2022

j-n-c commented May 10, 2022

pdurbin commented May 10, 2022

j-n-c commented May 11, 2022

pdurbin commented May 11, 2022

Cannot use default metadata configuration after having datasets with custom metadata blocks #8471

Cannot use default metadata configuration after having datasets with custom metadata blocks #8471

Comments

j-n-c commented Mar 8, 2022

pdurbin commented Mar 8, 2022

j-n-c commented Mar 8, 2022 • edited Loading

pdurbin commented Mar 28, 2022

pdurbin commented Apr 15, 2022

j-n-c commented May 10, 2022

pdurbin commented May 10, 2022

j-n-c commented May 11, 2022

pdurbin commented May 11, 2022

j-n-c commented Mar 8, 2022 •

edited

Loading