Investigate error when trying to create dataset links for Dean Karlan's datasets

[jggautier]

I used the Dataverse API to try to add links of datasets on Harvard Dataverse authored by Dean Karlan into two collections and the API returned a 403 error for a number of them.

3 dataset links not added to deankarlan collection
In mid-April 2024, I used the Dataverse API to try to add 54 dataset links into the collection at https://dataverse.harvard.edu/dataverse/deankarlan. Links for 51 of those dataset were created.

I can't add links for three of those datasets. When I try, the API returns a 403 error. In the datasetlinkingdataverse table in the Harvard Dataverse database, I can see the IDs of all 54 datasets, including the three that don't appear as links in the UI. Maybe that's related to the 403 error?

The three datasets that I couldn't add as links into the collection are:

• doi:10.7910/DVN/EJTYHC (link)
• doi:10.7910/DVN/NHIXNT (link)
• doi:10.7910/DVN/QT7IXR (link)

12 dataset links not added to DFEEP collection
In early June 2024, I used the Dataverse API to try to add 12 dataset links to the collection at https://dataverse.harvard.edu/dataverse/DFEEP. The API returned 403 errors and the links were not added.

Here are the DOIs for those 12 datasets:

• doi:10.7910/DVN/EX9FU1 (link)
• doi:10.7910/DVN/TUXWCQ (link)
• doi:10.7910/DVN/QT7IXR (link)
• doi:10.7910/DVN/BTZXQX (link)
• doi:10.7910/DVN/A2KSIV (link)
• doi:10.7910/DVN/RNGHDV (link)
• doi:10.7910/DVN/MA31ZM (link)
• doi:10.7910/DVN/L0I9HW (link)
• doi:10.7910/DVN/5NM88Z (link)
• doi:10.7910/DVN/XXT0GR (link)
• doi:10.7910/DVN/ULI7IX (link)
• doi:10.3886/ICPSR02526.v1 (link)

More context
In the email thread at https://help.hmdc.harvard.edu/Ticket/Display.html?id=359122, Dean Karlan and I discussed how to make sure that the datasets he's an author of are linked into his collection and linked into the DFEEP collection, including the 54 datasets published so far and any datasets published in the future.

I also used the Saved Search feature to add links of any datasets he's an author of that are published in the future (see #275 and #277).

[jggautier]

I was able to use the Saved Search feature so that all Dean Karlan-authored datasets are added as links into his collection and the DFEEP collections. @sbarbosadataverse , @scolapasta and I wondered if Dataverse would then add links for the datasets I listed above. I checked today and the links have not been added.

[cmbz]

@scolapasta will investigate several cases to see if reindexing helps, then will perform additional troubleshooting if needed.

[jggautier]

As of this writing, links for all but one of the datasets that I listed in this issue's first comment are in the two collections. I'm not sure how or when this happened. @scolapasta wrote in a Slack message that it might have happened during some reindexing.

The unpublished dataset doi:10.7910/DVN/QT7IXR (link) is the only dataset that doesn't have a link in the deankarlan collection and in the DFEEP collection.

It's the only unpublished dataset that we needed to create links for, as of this writing. And I'm not able to create links for unpublished datasets when I try on Demo Dataverse. When I use the API to try, I get a 403 error. As far as I can tell, the User Guides, such as https://guides.dataverse.org/en/6.2/user/dataverse-management.html#dataset-linking, don't mention that links can't be created for unpublished datasets.