关于SAM2中ID一致性的问题

[Morgott-The-Omen-King]

Search before asking

• I have searched the X-AnyLabeling Docs and issues and found no similar questions.

Question

您好~

我在使用SAM2进行视频目标标注，我遇到其中一种情况不知道怎么解决：

1. 假设目标A在1~10帧出现了，我将其标记为object0
2. 在11帧的实现目标A消失了，目标B出现了，我重置了Tracker，同时我将目标B标记为object1
   上面这种是做法似乎会导致目标B的object id变为object0，请问有没有什么解决方法？
   非常感谢~~

Additional

No response

[CVHub520]

Hello, @Morgott-The-Omen-King,

Thank you for your inquiry about SAM2 video annotation. The scenario you've described is indeed a common challenge in object tracking across frames.

For this situation, may be we could consider implementing a sequential accumulator for object IDs. By adding the previous accumulated value to the ID after each reset, you can maintain a unique identifier for each object throughout the video sequence. For instance:

• Frames 1-10: Objects A and B are initialized with labels object0 and object1.
• Frame 11: Object A disappears, and a new object C appears. The labels for objects B and C would then be object2 and object3, respectively.
  This way, you can avoid the confusion that arises from reassigning the same ID to a different object.

After the annotation process, you can easily relabel object0 as A, object1 and object2 as B, and object3 as C via label_manager.
Certainly, if you have any alternative strategies or ideas that could improve this process; I would be more than happy to hear them. Your feedback is valuable to us as we continue to refine our tools for better user experience.

Best regards,
X-AnyLabeling Maintainer

[Morgott-The-Omen-King]

感谢您的回复，我觉得这是一个很好的方式，但我认为主要的原因是

X-AnyLabeling/anylabeling/services/auto_labeling/segment_anything_2_video.py

Lines 354 to 360 in 5f13574

	for i in range(0, len(out_obj_ids)):
	masks = out_mask_logits[i].cpu().numpy()
	if len(masks.shape) == 4:
	masks = masks[0][0]
	else:
	masks = masks[0]
	shapes.extend(self.post_process(masks, label=f"object{i}"))

我们在外面设置的label会被上面的代码覆盖，label的text只会受到sam2中tracklet顺序的影响，就如你所说的，为什么不复用我们完成目标时设置的label呢

[Morgott-The-Omen-King]

Hello, @Morgott-The-Omen-King,

Thank you for your inquiry about SAM2 video annotation. The scenario you've described is indeed a common challenge in object tracking across frames.

For this situation, may be we could consider implementing a sequential accumulator for object IDs. By adding the previous accumulated value to the ID after each reset, you can maintain a unique identifier for each object throughout the video sequence. For instance:

• Frames 1-10: Objects A and B are initialized with labels object0 and object1.
• Frame 11: Object A disappears, and a new object C appears. The labels for objects B and C would then be object2 and object3, respectively.
  This way, you can avoid the confusion that arises from reassigning the same ID to a different object.

After the annotation process, you can easily relabel object0 as A, object1 and object2 as B, and object3 as C via label_manager. Certainly, if you have any alternative strategies or ideas that could improve this process; I would be more than happy to hear them. Your feedback is valuable to us as we continue to refine our tools for better user experience.

Best regards, X-AnyLabeling Maintainer

请问一下，有没有交流群

[CVHub520]

Yes, first of all, thank you very much for your feedback and suggestions.

Regarding the issue you mentioned about the code overriding externally set labels, it is indeed due to the current framework design. Currently, X-AnyLabeling does not provide a direct and simple mechanism to obtain or pass the label information set by users externally, which has led to these pieces of information being overwritten during the "internal processing."

We will optimize this issue when we have time in the future. If you wish to join the communication group, you can add the WeChat ID: ww10874, with a note "X-AnyLabeling".

[CVHub520]

@Morgott-The-Omen-King, Hello there!

I'm thrilled to let you know that the most recent update has introduced support for custom labels and track IDs. Make sure to perform a git pull to get the latest source code and give it a try!

Should you require any additional help, don't hesitate to get in touch.

Enjoy the new features! 😄

111.mp4