Averaging multiple 2d histograms from folder of videos

[bruno-f-cruz]

The following question was asked in the Bonsai Users - Google Groups
Link to original post

---

Original post
Hello,

I have developed a pipeline (shown here) that tracks an animals movement and creates a 2d density plot of their location. I'm having trouble, however, identifying how to do this across multiple videos and then averaging the density plots to get 1 for the experimental condition. Any help would be greatly appreciated!

Best,
Brandon

---

[bruno-f-cruz]

Hi Brandon,

I will split the answer in a few self-contained points. Let me know if something is not clear.

1. How to load multiple videos?
I am assuming you want to load several videos and analyze them offline. The easist way to achieve this is using GetFiles which returns all files that match a specific search pattern inside a folder. I am also going to assume you want to load ALL video files inside a folder. This means we can simply load all files with an ".avi" extension (SearchPattern = "*.avi). The output of this node will be an array of strings which contains all the files matching the search pattern. I will also take note of the length of the array which may be handy to have around latter (BehaviorSubject(nVideos))

2. Apply the same pipeline to multiple videos
Once we have the path of all the files we want to analyze, we can push them through the same pipeline. The first thing we need to do is to turn the string array (a single event) into multiple events, each containing a single string that corresponds to an individual file. We can achieve this by applying the Merge operator to the array. Next, for each video path, we will create a new analysis pipeline inside CreateObservable.

This pipeline can be anything you want, I just coded something that outputs an Histogram2D for each video, similar to your example. Since you want to only take the final 2D histogram for each video for averaging, we add a Last node at the end of the analysis pipeline. This node only allows the final "frame" from the histogram to be propagated.

3. Combine the video outputs
CreateObservable should output a single Histogram2D image per video. However, we still need to define how these videos are loaded/combined. In these situations, most of the time, you are probably looking at either a Concat or a Merge node. Concat will load files sequentially, i.e., analyize a new file once the previous one has finished. Merge will load them in parallel but it will not preserve any order that might otherwise be important for your analysis. I left the Concat in, but I encourage you to try both. In your case I suspect Merge should actually make it run faster.
Finally, the only thing left to do is to combine these final Histogram2D frames. In your specific case, a single node will likely do the trick, and you can get away with simply using IncrementalMean, which computes a running average of incoming frames. Alternatively you could also Accumulate incoming frames and use the nVideos subject, we created earlier, to divide the sum by the total number of videos.

I am also attaching an example workflow.

Hope it works!
Bruno

centroidHist.zip