Bug linking properties across cycles when UBID is matching criteria

[perryr16]

Issue

Step 6 linking function does not account for the UBID Jaccard Index (as it does in the step 4 matching function).

This can lead to two error scenarios if properties have matching pm_property_ids (or other matching criteria), but different UBIDs. Consider the following setup.

P1: { cycle: 1, pm_property_id: 1, ubid: 1 }
P2: { cycle: 2, pm_property_id: 1, ubid: 2 }

1. P1 is existing in Cycle 1 and P2 is uploaded to Cycle 2. These properties will be linked across cycles even though the UBIDs do not match.

P1: { cycle: 1, pm_property_id: 1, ubid: 1 }
P2: { cycle: 2, pm_property_id: 1, ubid: 2 }
P3: { cycle: 1, pm_property_id: 1, ubid: 2 }

2. If P1 and P3 are uploaded to Cycle 1, they will appear as distinct properties as step 4 accounts for UBID. When P2 is uploaded to Cycle 2 we end up with an integrity error and the app hangs

seed_celery    | django.db.utils.IntegrityError: duplicate key value violates unique constraint "seed_propertyview_property_id_cycle_id_f8bdf6c2_uniq"
seed_celery    | DETAIL:  Key (property_id, cycle_id)=(31, 4) already exists.

Proposed solution:

Add the jaccard checking logic from step 4 to step 6.

• Add a check_jaccard boolean argument to link_states
• Insert jaccard checking logic into the for loop within function link_states

for idx, state in enumerate(states):
   existing_state_matches = match_lookup.get(tuple(getattr(state, c) for c in tuple_values), [])
   
   if check_jaccard:
         existing_state_matches = [
             existing_state
             for existing_state in existing_state_matches
             if existing_state.get("ubid")
             and check_jaccard_match(matching_criteria.get("ubid"), existing_state.get("ubid"), org.ubid_threshold)
         ]

   existing_views_matches = [view_lookup[x["id"]] for x in existing_state_matches]
   ...