Segmentation fault during CoreML NMS #752
Comments
|
@dlawrences thanks for reporting this. We are looking into this. |
|
@dlawrences can you share new |
Sure. Could you please provide a private e-mail address so I can send it over? I wouldn't be able to share it publicly tho for now. Cheers |
Hello @dlawrences, could you file a radar and attach the model there? Thank you. |
|
Hi I'll try to file the radar in the coming days. In terms of the code, this is what I am doing: builder.add_nms(
name="nms",
input_names=["raw_coordinates", "raw_confidence"],
output_names=["coordinates", "confidence", "indices", "num_boxes"],
iou_threshold=0.5,
score_threshold=0.0,
max_boxes=10,
per_class_suppression=True
)With the settings above, I am actually getting the following error: objc[12134]: Attempted to unregister unknown __weak variable at 0x7f83c98913d0. This is probably incorrect use of objc_storeWeak() and objc_loadWeak(). Break on objc_weak_error to debug.
python3(12134,0x10efc8dc0) malloc: *** error for object 0x7f83c98912b0: pointer being freed was not allocated
python3(12134,0x10efc8dc0) malloc: *** set a breakpoint in malloc_error_break to debugThe description of the spec is the following: input {
name: "raw_results1"
type {
multiArrayType {
shape: 1
shape: 3
shape: 18
shape: 18
shape: 8
dataType: FLOAT32
}
}
}
input {
name: "raw_results2"
type {
multiArrayType {
shape: 1
shape: 3
shape: 36
shape: 36
shape: 8
dataType: FLOAT32
}
}
}
input {
name: "raw_results3"
type {
multiArrayType {
shape: 1
shape: 3
shape: 72
shape: 72
shape: 8
dataType: FLOAT32
}
}
}
output {
name: "coordinates"
type {
multiArrayType {
dataType: DOUBLE
}
}
}
output {
name: "confidence"
type {
multiArrayType {
dataType: DOUBLE
}
}
}
output {
name: "indices"
type {
multiArrayType {
dataType: DOUBLE
}
}
}
output {
name: "num_boxes"
type {
multiArrayType {
dataType: DOUBLE
}
}
}The issue is sometimes I actually get segmentation fault as in the original post. It is really weird and inconsistent from a run to another. I tried passing data as both np.float32 and np.double, no real differences. Any thoughts? Thanks |
|
One thing to be specified that's missing from the documentation. I am pretty sure the layer |
|
As described above, the NMS layer is the last step in the pipeline. Everything else is working as expected... I am providing for additional debugging purposes the actual data that enters the NMS layer. The attached zip archive contains two files:
|
|
Hi @dlawrences, thank you for the additional information. Unfortunately I have not been able to recreate the issue. We would like to have a radar with these attachments:
(*) You can take sysdiagnose with the following command on the terminal. It creates a file something like |
|
@dlawrences I ran into this as well. The issue in my case (and very likely in your case) was that Yolo does not use a softmax but predicts probabilities for all the classes independently. This means that the probabilities for multiple classes can sum up to more than 1. CoreML's NMS network expects the probabilities to sum to at most 1. When the sum exceeds 1 you get a crash (in my case when running on the Neural Engine on iOS). PS edit: In what I wrote above I was maybe a bit overly confident that this is the same issue you are seeing. It very well might not be. But try limiting the predicted confidence values to sum to <= 1 before feeding them into the NMS layer and see if it fixes the crashes for you. |
|
Thanks @pocketpixels. I moved away from this and ended up using a pipeline type of model, where the CNN is just a model in the pipeline feeding off results to another "model" that is based on the NMS implementation from CoreML. That seems to work as expected. For reference, here's an example: https://github.com/hollance/coreml-survival-guide/blob/4dfcbb97c065726a3da240c55d90b2075959801d/MobileNetV2%2BSSDLite/ssdlite.py#L333 |
|
@dlawrences Actually, that's how I am using the CoreML NMS also, as a model in a pipeline. (Sorry, I clearly didn't read your original description closely enough). |
|
@dlawrences Any chance you'd still be willing to share the mlmodel you put together to decode YOLOv5 output? I'm trying to do the same using the builder but running into an "Error computing NN outputs" I can't get past. I'm sure it's something I'm doing wrong but I haven't been able to figure it out. |
|
@gomer-noah You can try my CoreML export script |
I'll take a look. Thank you so much! |
|
@dlawrences - did you ever submit the requested information to https://developer.apple.com/bug-reporting/ ? If so, what was the id value you received? |
TobyRoseman
added
the
awaiting response
|
@TobyRoseman nope, I never went ahead with the radar report to Apple. |
Hi team
I have created a model using the coremltools API. This takes the detections done by a YOLOv5 CNN (https://github.com/ultralytics/yolov5) converted from PyTorch to CoreML and does the following operations:
If I am showing the output of the tensors feeding into NMS ("slice_xywh_output" and "multiply_obj_conf_output"), these look like the following:
However, sometimes, feeding these two tensors in a NMS layer like the following
triggers segmentation fault and the result looks wrong:
This has been tested on a 2020 MacBook Pro (16 GB RAM, 2.0 Ghz processor).
Config
Any thoughts on this?
Thanks
The text was updated successfully, but these errors were encountered: