Question: Add License Plate Model to Frigate NVR

[inspired27]

Hi,

Your model works really well for plate recognition day and night and OCR in BlueIris, would it be possible to incorporate your license plate model in Frigate NVR:

https://docs.frigate.video/configuration/advanced/#model

https://github.com/blakeblackshear/frigate/discussions?discussions_q=is%3Aopen++custom

https://coral.ai/docs/edgetpu/retrain-detection/

[MikeLud]

I will look into it.

[inspired27]

Okay thanks, I have Frigate installed in a Proxmox LXC container and a Coral TPU let me know if you want me try anything out, test etc. Looks like the model needs to be in a particular format size etc for Coral and CPU version.

[inspired27]

I will look into it.

The information on how to add custom model into Frigate is a bit thin, not sure if this guys post helps:

blakeblackshear/frigate#1043 (reply in thread)

I have started a discussion thread on Frigate site to see if someone has done this already, to help.

blakeblackshear/frigate#6822 (comment)

[MikeLud]

@inspired27 I just add the TensorFlow Edge TPU Plate model. The model only has one label Plate. Let me know if it works,

https://github.com/MikeLud/CodeProject.AI-Custom-IPcam-Models/tree/main/TensorFlow%20Edge%20TPU%20Models

[inspired27]

@inspired27 I just add the TensorFlow Edge TPU Plate model. The model only has one label Plate. Let me know if it works,

https://github.com/MikeLud/CodeProject.AI-Custom-IPcam-Models/tree/main/TensorFlow%20Edge%20TPU%20Models

Hi,

Thanks I have added to Frigate and I am getting an error "IndexError: list index out of range" and keeps restarting detection process. The config file and error in log are below. I renamed the file edgetpu.tflite as you can see in my Frigate config.yml. I am not sure what to put for width and height just gone with 640. Also to help I have included a link to a couple of articles.

Extract of My Frigate config.yml:

detectors:
  coral:
    type: edgetpu
    device: usb
model:
  path: /custom/edgetpu.tflite
  labelmap_path: /custom/labelmap.txt
  width: 640
  height: 640
  labelmap:
    0: Plate
    1: plate

My Frigate Log:
2023-06-19 22:36:58.152156375 [2023-06-19 22:36:55] frigate.detectors.plugins.edgetpu_tfl INFO : Attempting to load TPU as usb
2023-06-19 22:36:58.154139630 [2023-06-19 22:36:58] frigate.detectors.plugins.edgetpu_tfl INFO : TPU found
2023-06-19 22:36:58.250559085 Process detector:coral:
2023-06-19 22:36:58.251091030 Traceback (most recent call last):
2023-06-19 22:36:58.251092992 File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2023-06-19 22:36:58.251094102 self.run()
2023-06-19 22:36:58.251095312 File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2023-06-19 22:36:58.251096396 self._target(*self._args, **self._kwargs)
2023-06-19 22:36:58.251097595 File "/opt/frigate/frigate/object_detection.py", line 121, in run_detector
2023-06-19 22:36:58.251107965 detections = object_detector.detect_raw(input_frame)
2023-06-19 22:36:58.251109693 File "/opt/frigate/frigate/object_detection.py", line 71, in detect_raw
2023-06-19 22:36:58.251110969 return self.detect_api.detect_raw(tensor_input=tensor_input)
2023-06-19 22:36:58.251112118 File "/opt/frigate/frigate/detectors/plugins/edgetpu_tfl.py", line 59, in detect_raw
2023-06-19 22:36:58.251119983 class_ids = self.interpreter.tensor(self.tensor_output_details[1]["index"])()[0]
2023-06-19 22:36:58.251136250 IndexError: list index out of range
2023-06-19 22:37:00.766964663 192.168.1.106 - - [19/Jun/2023:22:37:00 +1000] "GET / HTTP/1.1" 200 10522 "-" "HomeAssistantSupervisor/2023.06.2 aiohttp/3.8.4 Python/3.11" "172.30.32.2"
2023-06-19 22:37:04.642953000 [INFO] Starting go2rtc healthcheck service...
2023-06-19 22:37:15.618593077 [2023-06-19 22:37:15] frigate.watchdog INFO : Detection appears to be stuck. Restarting detection process...

Frigate example config.yml for when adding custom model: (I did not add optionals for colour space and tensor model format unless you can tell me)

# Optional: model modifications
model:
  # Optional: path to the model (default: automatic based on detector)
  path: /edgetpu_model.tflite

  # Optional: path to the labelmap (default: shown below)
  labelmap_path: /labelmap.txt

  # Required: Object detection model input width (default: shown below)
  width: 320

  # Required: Object detection model input height (default: shown below)
  height: 320

  # Optional: Object detection model input colorspace
  # Valid values are rgb, bgr, or yuv. (default: shown below)
  input_pixel_format: rgb

  # Optional: Object detection model input tensor format
  # Valid values are nhwc or nchw (default: shown below)
  input_tensor: nhwc

  # Optional: Object detection model type, currently only used with the OpenVINO detector
  # Valid values are ssd, yolox, yolov5, or yolov8 (default: shown below)
  model_type: ssd

  # Optional: Label name modifications. These are merged into the standard labelmap.
  labelmap:
    0: plate
    1: Plate

[Support]: Yolov5 as TFLITE model works? #2726:
[Support]: Yolov5 as TFLITE model works? #2726

Info on custom model
https://docs.frigate.video/configuration/advanced#model

[MikeLud]

You might need a labelmap.txt file, it should be the below The model was trained using 640 x 640 resolution.

Plate
Plate

Also the below should be should all be Plate

[inspired27]

You might need a labelmap.txt file, it should be the below

Plate
Plate

Thanks, still getting error, could it be something to do with the way the model has been converted as per some of the links? or is my config still wrong?

2023-06-20 00:05:08.168342382 Process detector:coral:
2023-06-20 00:05:08.168906954 Traceback (most recent call last):
2023-06-20 00:05:08.168944019 File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2023-06-20 00:05:08.168945955 self.run()
2023-06-20 00:05:08.168947481 File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2023-06-20 00:05:08.168948819 self._target(*self._args, **self._kwargs)
2023-06-20 00:05:08.168950943 File "/opt/frigate/frigate/object_detection.py", line 121, in run_detector
2023-06-20 00:05:08.168961545 detections = object_detector.detect_raw(input_frame)
2023-06-20 00:05:08.169056705 File "/opt/frigate/frigate/object_detection.py", line 71, in detect_raw
2023-06-20 00:05:08.169058482 return self.detect_api.detect_raw(tensor_input=tensor_input)
2023-06-20 00:05:08.169059846 File "/opt/frigate/frigate/detectors/plugins/edgetpu_tfl.py", line 59, in detect_raw
2023-06-20 00:05:08.169061199 class_ids = self.interpreter.tensor(self.tensor_output_details[1]["index"])()[0]
2023-06-20 00:05:08.169150010 IndexError: list index out of range

My Config Now:

My labelmap.txt has gone from:
0 plate
1 Plate

To:

And my frigate config.yml is now:

detectors:
  coral:
    type: edgetpu
    device: usb
    
model:
  path: /custom/edgetpu.tflite
  labelmap_path: /custom/labelmap.txt
  width: 640
  height: 640
  labelmap:
    0: Plate
    1: Plate

It is 12:15am here in Australia now I will check-in in the morning to see what next to try.

[Gie92]

Any luck on getting Frigate to work with custom tflite models? I generated a model with Teachable Machine and have not been able to get it to work, I was getting the same error.

[Porencephaly]

Anyone have any luck with this?