Uploading updated master lists

[helendallas]

• Initial master lists were harvested from GBIF.
• These were then download in Taxon Management as CSV files.
• Attributes were added to the Taxon Group.
• These attributes as well as national conservation status were added to the upload file.

These were then uploaded to staging.

Issues:

Initially upload hung at record 40 of 632 - this is the first SubSpecies record

Email sent to Dimas to check if this is the issue:
Captured here as well

I am checking upload functionality for the Master lists that the sanparks team have edited with attributes.

When we uploaded to FBIS we kept columns as follows, with Species column only the species part of the name, and SubSpecies only the subspecies part of the name.

When we download a csv from Taxon Management, the names come down as follows, with the full name for species and subspecies.

I have now tried to upload the edited csv file for reptiles and it hangs on 40 of of 632 - this is the first SubSpecies record

Is this the reason that the Upload is not working or is it something else. We are battling to test anything on sanparks as there an issues that prevent the testing.

Please can you just respond if this is the problem or not. I will make a ticket as well to address in case it needs to hold over until Monday as well.

Upload file attached

BIMS_reptiles test upload_19July2024 No CS categories.xlsx

I cannot do any further testing as upload not working beyond 40

[dimasciput]

@helendallas Now you can upload the subspecies/species either by part of the name or by the full name

[helendallas]

I am getting an error message

Whether I upload master list with Conservation status global blank or using what came down in the csv file.

Here is the file

BIMS_reptiles test upload_19July2024.xlsx

[dimasciput]

@helendallas please test it again.

[helendallas]

Yah. @dimasciput Success ! uploaded. Thanks

[helendallas]

I uploaded the master list and received a success file. But when I download the csv none of the attributes re downloaded.

Here is the success file - attributes have been uploaded.

success_BIMS_reptiles_test_upload_29July2024.xlsx

Here is the csv downloaded with no attributes .

New Reptiles-2024-7-29-8-4807187372850922156.csv

Any chance you cna check on this before our Wednesday meeting so I can check and report.

[dimasciput]

@helendallas sure, I will check tomorrow

[dimasciput]

@helendallas, if there is an issue with the ticket, could you please move it back to the TODO column? This way, I can keep track of the pending tickets. Thanks!

[helendallas]

Okay sure. will do.

Here is the bias taxa record for a species that has all three attributes

{"TOPS": "TOPS", "Class": "Squamata", "Genus": "Smaug", "Order": "", "Taxon": "Smaug giganteus", "Family": "Cordylidae", "Origin": "Native", "Phylum": "Chordata", "Kingdom": "Animalia", "On gbif": "Yes", "Species": "Smaug giganteus", "Endemism": "South African endemic", "Gbif link": "https://www.gbif.org/species/8151308", "Sensitive": "Sensitive", "SubSpecies": "", "Taxon Rank": "Species", "Common Name": "Gant spinytail lizard", "CITES listing": "II", "Conservation status global": "Vulnerable", "Conservation status national": "Vulnerable", "Scientific name and authority": "Smaug giganteus (Smith, 1844)"}

[dimasciput]

Reptiles-2024-7-30-8--1039012285491978311.csv

@helendallas I just downloaded the reptiles, and it's showing both TOPS and Sensitive. There might be a conflict with the CITES listing because we have the CITES listing field in taxonomy table and it's coming from the CITES site. Do you want this to be overwritten if there's a value from the CSV?

[dimasciput]

@helendallas but all the data are stored in database, so I think we can close this ticket? And create a new one for cites?

[helendallas]

IF is is easy to pull in Cites data then we don't need to add it to the master lists. This would be better long term I reckon. I assume we would still have a CITES listing column as an attribute but it would pull data from CITES site?

[helendallas]

Waiting for clarification from Dimas re CITES

Question emailed to him on 31 July 2024.

Please clarify what you are suggesting for the CITES.

Should the team add as an attribute for a taxon for relevant biodiversity modules?

Or should we not add as an attribute but instead it gets pulled down from CITES site like IUCN categories? The NB point is that CITES listing is a populated column in the taxon cab download.

[helendallas]

From Dimas:
It's better to pull down the data, but it will be a little bit slow and sometimes we don't get any data.

So my suggestion is to keep uploading it for now, but you don't need to add additional attributes on the taxa management page because we already have a field called "cites listing."

I will update the download later. If there is no data from the CITES table, then just use the uploaded data for CITES listing.

[helendallas]

I have extracted CITES issue to a separate tickets #4121

Closing this ticket now as seems to be working. SANparks are also testing

[helendallas]

I am opening this ticket again as now the Cites listing, Tops and `sensitive attributes are not being populated.

Here is the upload file

BIMS_reptiles test upload_29July2024.xlsx

here is the download csv after upload

Reptiles-2024-8-8-8-2791889857809966798.csv

[dimasciput]

@helendallas it's just the CITES listing or also TOPS and Sensitive attributes? I can see tops and sensitive on the csv.

[helendallas]

Correct TOPS and Sensitivity have pulled through.

[helendallas]

Closing this as dealt with in #4121