Getting 2 prediction files out of a 2-video SLP file with all prior predictions for analysis in SIMBA

[rfkova]

Hello!

I have an SLP file with tons of predictions (not 100% due to some possible bugs), enough to probably interpolate intermediate poses with some of the built-in SIMBA tools. Problem is, SIMBA wants single predictions files per video and I have numerous predictions files covering both videos from each time I ran inference - thus I need to find a way to separate predictions from each video and wrap all the inference predictions into two files representing all predictions from each video.

Cheers!
Ryan

[talmo]

Hi @rfkova,

Here's a quick snippet that should do what you want:

import sleap

# Load multi-video labels file.
labels = sleap.load_file("labels.v000.slp")

# Save labels for each video separately.
for i, video in enumerate(labels.videos):
    # Create a new Labels object containing only labels from the current video.
    labels_i = sleap.Labels(labels[video])

    # Save.
    labels_i.save(f"labels.v000.video{i}.slp")

Let me know if that works for you!

Talmo

[talmo]

Hi @rfkova,

Sorry, been a little swamped. I'm not sure why SIMBA wouldn't accept it the way we've generated them above -- the predictions files are identical to the labeling files, except that they contain only predicted instances.

Is there a particular error you're running into when you try to upload the video-individual SLP files generated as above?

Talmo

[rfkova]

Hi Talmo,

As far as I'm concerned you are a hero for this level of support so no worries about delay!

That's very confusing but valuable to learn. Yes I get an error - let me explain in detail but the error is "cannot set a row with mismatched columns":

Scenario 1:
I run the code above and it generates two .slp files from my original project file with two videos. SimBA wants .slp files named after videos verbatim (minus extension) so I rename the file in file explorer. I make a new project in SimBA, load up my custom skeleton, import my video with matching name, try to import the split .slp file of the corresponding video -> Erorr: cannot set a row with mismatched columns.

Scenario 2:
I predict all frames which generates very weird predictions but does fill every frame with something. That file gets a prediction file in the prediction folder with a name like LBNJa_3_2021-12-04_19-46-14c.mp4.220522_173842.predictions.slp. If I rename that file LBNJa_3_2021-12-04_19-46-14c.slp so it matches the video I can load it into SimBA just fine... but the predictions are terrible.

My goal was to import the project file from Scenario 1 which has 50% of frames labeled (from iterating 1000 predictions at a time) and use the interpolate tool from SimBA to estimate intervening predictions but the import fails. I can also use the prediction files from any of the 1000 frame inferences as in Scenario 2, but with 100k+ frames in the video interpolation isn't really enough to fill the gaps.

Scenario 1:

Scenario 2:

[talmo]

Hmmmm, @sronilsson might have some ideas here since I'm not quite certain what would cause the import error in scenario 1.

(I'm tagging him here, but I don't know if he'll get the notification across repositories, so you may want to open an issue in the SimBA repo and link back to this discussion.

We're still troubleshooting scenario 2 though and hope to have something sorted soon.

[sronilsson]

Spotted! @rfkova if you open an issue on the simba github repo, and share the the two files: lables.v000.slp and labels.v000.220525_122200.predictions.slp (through a drive link with [email protected]) I can take a look why it's not working.

cannot set a row with mismatched columns is a generic error: it tells you that the number of body-parts tracked in the slp file, and the number of body-part names you have defined in the project, don't match up, e.g., simba tries to assign N header names to N+1 columns. There is an option do drop body-parts from your tracking data before import in gui, but this might not be relevant here - https://github.com/sgoldenlab/simba/blob/master/docs/Tutorial_tools.md#remove-body-parts-from-tracking-data

[rfkova]

Lovely! Will do! I'm just rerunning things to be extra sure which files to send you (because of all the renaming on the fly).

I'll post that shortly and circle back to close this thread as answered if/when we figure it out with an explanation !

Thank you for your help!