-
Notifications
You must be signed in to change notification settings - Fork 249
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
read_spikegadgets broken for neuropixel recordings #1517
Comments
Hey @RobertoDF, thanks for the report. I don't use spikegadgets but one quick question: why is your path We recently made the change because the channel IDs were not accurate (according to a user). So I may need your dataset in order to troubleshoot this (since I don't know the format by itself). spikegadgets passed our tests so if you could make a small file (<10MB) with your machine and send it to us then we can add it to our tests so we never break your workflow again :) |
Hey @RobertoDF, based on all our test data it looks like the file shouldn't be nested like that |
Just tested it and i get the same error:
I ll try to send a smaller .rec file soon. Thanks for quick response! |
I cant produce a small file for now (technical problem). Do you know if the testing file is with one or two probes? I am using two probes. Can you maybe share the testing file? Thanks!!! Btw before being totally broken, the function (read_spikegadgets) was returning a probe with gain scaling always set to 1. I did not pinpoint which commit was the cause of this though. This is all relative to .rec files with 2 neuropixels: |
I don't know if it the test files are multi-probe for sure. I didn't help with the gain, but the recent PR referred to 4 shanks I believe. So by two probe you mean 8 shanks or 2 shanks total? Our test files are all located here: https://gin.g-node.org/NeuralEnsemble/ephy_testing_data/src/master/spikegadgets You can also share the full sized file here so that I can work on a fix. But we won't merge the fix until we have the test file so that might allow me to at least create a branch you can install from. |
all files i have access to right now are huge. smaller I found was 8 GB. The file I can´t load is with two NPX 1.0 probes. one shank each. Ill check the test file now. |
oh! I can´t open any of the test files with read_spikegadgets, it just hangs forever. I tried also to update neo to latest version. |
8GB I can work with. (Not ideal), but if this is urgent for your analysis I can see if it is a quick fix. The gain change was here: 7d0e575. The idea being that if the header doesn't contain explicit gain information we must set it to 1. Do you know of another way to get the gain information?
g-node can be a bit tricky. Do you want to make sure you downloaded the actual file and not a stub of the file :) |
ahah! thanks, i try again |
The gain information was correctly identified here 83a84b2 . The test recording does include 2 probes and is loaded smoothly so I am really confused now. I get the same error with this smaller file (https://www.dropbox.com/scl/fi/grt4dn52zhaolnfdu8bvu/20231210_160309.rec?rlkey=nc71uycmp09f803223hlo3num&dl=0) |
Weird. We haven't changed the gain since then (that I know of).... What is your gain saying? You said you have neuropixels. Since I don't know spikegadgets does that come with a headstage or is part of the neuropixel system? The recent change was to account for the way intan-headstages would need to be counted. So we basically fixed the number of channels based on the standard intan headstages. Maybe because neuropixels can record from different channel numbers this means we need to have flexibility instead. Since we didn't have the fixed count before it might have read yours, but now it can't. I'll look at your file soon (I'm already running some analyses on my computer and I'm resource limited on this station :( ) |
Sure, thanks for looking into this!! My gain with the pinned neo version is scaled at 0.018311. With a scaling of 1, all channels are detected as noise by spikeinterface (detect_bad_channels func). That´s how I realized something has changed. Than I updated every spikeinterface related package and the func was broken. Yes, there is a special neuropixel headstage |
Then if that is the case I think that our test data must just use the intan headstage. So if (when technically difficulties allow) you can send us the small file that will definitely help us make sure this doesn't break in the future. My guess is that the neuropixel headstage provides a different gain header that I will need find, and hopefully spikegadgets will provide some header info that will allow me to distinguish the different headstages so that we can check for that and not use fixed channel numbers. |
The test file "SpikeGadgets_test_data_2xNpix1.0_20240318_173658.rec" comes from the same recording system, a colleague of mine here sent it. That´s why I am really confused that it can be loaded smoothly and some other files instead don´t work. I checked just before that it includes 2 NPX probes. |
Then my guess would be:
|
Hi,
The read_spikegadgets from spikeinterface was working with neuropixel probes with this committ 83a84b2 but now it is broken.
Environment:
The text was updated successfully, but these errors were encountered: