Grabber input_mu function ROI limits

[lelroy1]

One-Line Summary

Is there anyway to get more than 16 ROIs from the grabber per EMCCD camera frame?

Issue Details

We have a system that requires us to average around 200 ROIs per camera frame with real time calculation. In order to do this we hoped to be able to use the grabber's functions that are shipped with the hardware. We have hit a wall because it seems that when we use more than 16 ROIs it gives a artiq.coredevice.grabber.OutOfSyncException. Anyone have any advice on how to proceed?

from artiq.experiment import *
import numpy as np

class RTIOAnalyzerGrabber(EnvExperiment):
def build(self):
self.setattr_device("core")
self.setattr_device("grabber0")

@kernel
def grabber_capture(self, card_dev, rois):
    self.core.break_realtime()
    delay(100*us)
    mask = 0
    for i in range(len(rois)):
        i = rois[i][0]
        x0 = rois[i][1]
        y0 = rois[i][2]
        x1 = rois[i][3]
        y1 = rois[i][4]
        mask |= 1 << i
        card_dev.setup_roi(i, x0, y0, x1, y1)
    card_dev.gate_roi(mask)
    n = [0]*len(rois)
    card_dev.input_mu(n)
    self.core.break_realtime()
    card_dev.gate_roi(0)
    print("ROI sums:", n)

@kernel
def run(self):
    self.core.reset()
    k = 0 
    start = self.core.mu_to_seconds(now_mu())
    # When adding a 17th list to this list it will give the outofsync
    roi = [[0,0,0,41,41],[1,0,0,41,41],[2,0,0,41,41],[3,0,0,41,41],[4,0,0,41,41],[5,0,0,41,41],[6,0,0,41,41],[7,0,0,41,41],[8,0,0,41,41],[9,0,0,41,41],[10,0,0,41,41],[11,0,0,41,41],[12,0,0,41,41],[13,0,0,41,41],[14,0,0,41,41],[15,0,0,41,41],[16,0,0,41,41]]
    self.grabber_capture(self.grabber0, roi)
    print(self.core.mu_to_seconds(now_mu()) - start)

[marmeladapk]

You'd have to get Kasli gateware with non-standard number of ROI engines. I'm not sure if 200 wouldn't be problematic though in terms of fpga resources and timing.

artiq/artiq/gateware/rtio/phy/grabber.py

Lines 61 to 62 in 692572a

	class Grabber(Module):
	def __init__(self, pins, roi_engine_count=16, res_width=12, count_shift=0):

[jkellerPTB]

You'd have to get Kasli gateware with non-standard number of ROI engines. I'm not sure if 200 wouldn't be problematic though in terms of fpga resources and timing.

artiq/artiq/gateware/rtio/phy/grabber.py

Lines 61 to 62 in 692572a

	class Grabber(Module):
	def __init__(self, pins, roi_engine_count=16, res_width=12, count_shift=0):

Is this really the only adjustment needed?

I tried increasing roi_engine_count to 32 (in release-6, 9a3f6f85a48b81cbaebb315f00aa82782617df0c). It compiles and works as before for the first 16 ROI engines, but if I define more and try to read out their data, the following call to rtio_input_data does not return on the 17th iteration:

artiq/artiq/coredevice/grabber.py

Line 102 in 9a3f6f8

roi_output = rtio_input_data(channel)

I'm afraid I didn't have the time to dig any deeper yet, but if there is some obvious place for me to look, I'd appreciate a hint.