Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

On-board data storage solution #4

Open
christiankuttke opened this issue Jun 28, 2024 · 13 comments
Open

On-board data storage solution #4

christiankuttke opened this issue Jun 28, 2024 · 13 comments

Comments

@christiankuttke
Copy link
Collaborator

Make a decision by specifying:

  • How much data needs to be stored
  • What environment will it be placed in
  • How much space and ventilation will our enclosure have
@beniroquai
Copy link
Contributor

beniroquai commented Jun 28, 2024

We had several schemes for data storing. E.g. a RAW image every minute, JPEGs every second or even capture H.264-encoded videos. I can imagine that there is a lot of static background that could be removed.

Let's say we produce 100MB/minute and we let the microscope run 24h10days60minutes*100MB. This would be ~2TB.

Vibration is a thing. I would prefer SSDs. But has to be compatible with Raspi. We will probably use a Raspi4, so SSD needs to interface via USB.

@christiankuttke
Copy link
Collaborator Author

Raspberry Pi 4 has PCIe x1 lane compatible with nvme SSDs: https://www.raspberrypi.com/products/m2-hat-plus/

@beniroquai
Copy link
Contributor

beniroquai commented Jun 30, 2024 via email

@christiankuttke
Copy link
Collaborator Author

christiankuttke commented Jul 18, 2024

Design-locking this to a 2 TB SSD suitable for Raspberry Pi 5: A NVMe drive in M.2 M key, in 2230 or 2242 form factor (aka length).

Additional hardware needed is a board which accepts the SSD and adapts the M.2 to ribbon cable to go into the Raspberry Pi 5 PCIe FFC connector.

Example: Raspberry Pi 5 M.2 HAT https://core-electronics.com.au/raspberry-pi-m2-hat.html

@beniroquai
Copy link
Contributor

Question: Should we really go for a PCI solution? My fear is that, once we have to pull down the data from the device we have to rely on network. I would prefer an USB3 SSD, what do you think @ranranking @christiankuttke ?

@christiankuttke
Copy link
Collaborator Author

When there is the built-in nvme M.2 SSD, the images can be recorded quickly. This leaves all USB and the Ethernet port free.

USB and Ethernet can be routed to face the user when the box is open. This way, users can connect a USB hard drive and a mouse to start transfer. Or, they can use the ethernet to plug it in to their network (which I don't think will happen because of university networking rules) and then use mouse/keyboard via usb to start the transfer.

For putting USB and ethernet to the inside surface of the box from wherever the Raspberry Pi is mounted, short passive extension cables or even a USB hub can be used.

@ranranking
Copy link
Contributor

for the upper raspberry pi part, should it also be waterproof? if yes, then i would prefer USB3 ssd, as there are some waterproof ones, and also it can be easily swaped. If we at somepoint want to change raspberry to some kind of windows tablet, then usb3 is still compatibel with it

@beniroquai
Copy link
Contributor

I personally wouldn't stress the water-proofiness so much. AS long as some smaller droplets won't make it into the box I think we're good!

@beniroquai
Copy link
Contributor

In the End we had to take an SD-Card adapter and a large SD card. @christiankuttke is this any good? Are there solutions where micro sd cards with 2TB become reliable?
https://www.amazon.de/2TB-Micro-SD-Karte-Computer-Zubeh%C3%B6r/s?k=2TB+Micro+SD-Karte&rh=n%3A340843031

@christiankuttke
Copy link
Collaborator Author

Unfortunately no. Next best thing vs. SD cards are emmc chips. SATA or M.2 SSDs are the best. Enterprise SSDs have supercaps inside to be able to write cached sectors to non-volatile memory at unexpected power loss, but those are probably outside our price range.

@christiankuttke
Copy link
Collaborator Author

Also, cheap generic SD-card adapters are the worst. The one we had yesterday had an unreliable contact, and if that prevents the Raspi from booting or causes data loss in high-vibration environments, then that's a design fault in our product.

@beniroquai
Copy link
Contributor

Agree. So we have to chase the power lack for the pi or find a way to power it externally somehow. My suspicion is that the official raspi power supply will bypass the current limitation so that more current can be drawn from the USB powers. An alternative could be an externally powered USB hub for the external devices. Are there any of these devices as "Gehäusedurchgangslösung" or IP6x standard?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants