feat: support table joins on key format mismatch

[agavra]

Description

fixes confluentinc#6229 and also remove the code around "required formats". This means that we will always chose the first "preferred" format using the algoirthm outlined in confluentinc#6393 but removing the restriction that tables are not re-partitioned. This still doesn't fully address what confluentinc#6017 (klip-33) outlines because it prefers the left instead of the right source to maintain some compatibility with previous code. It would be backwards compatible to change this as this is done at logical plan and not the physical plan.

Testing done

QTT testing

Reviewer checklist

• Ensure docs are updated if necessary. (eg. if a user visible feature is being added or changed).
• Ensure relevant issues are linked (description should include text like "Fixes #")