Do not override default in spark hook with default in operator

[r-richmond]

This re-implements the same change as #6680

My goal here is to get this change in the next 1.10.* release

Please let me know if there is a better way to do this

---

Make sure to mark the boxes below before creating PR: [x]

• Description above provides context of the change
• Unit tests coverage for changes (not needed for documentation changes)
• Commits follow "How to write a good git commit message"
• Relevant documentation is updated including usage instructions.
• I will engage committers as explained in Contribution Workflow Example.

---

In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.
Read the Pull Request Guidelines for more information.

[Spasnof]

To add a bit of background @r-richmond and I ran into this issue today upgrading from 1.10.2 to 1.10.10 and relying on a spark2-submit that was set in our connection to set the behavior for 150+ operator calls in our environment.

The "normal" order of operations for most operators (pun not intended) appears to be:

1. Operator arg values (local to dag)
2. Connection arg values (local to environment)
3. Global parameter values (fallback)

In this case we have the connection_id and spark_binary args competing for the same value. And the operator will always win if set and it is set by default. So we are faced with a couple unpleasant options:

• Thinly wrapping the implementation in our own operator
• Setting the operator value in the 150+ occurrences.
• Monkey patching and linking back to this pr.

I hope this offers some background into the dilemma of having connection vars overwritten by operator defaults.

[r-richmond]

Failure looks like a dockerfile lint failure and isn't related to this change

[potiuk]

Please rebase to latest master. As announced on the devlist/slack we fixed the problem with docker lint (resulted from release of hadolint with new tests). It's been fixed and pinned so that it won't happen again, but you need to rebase to latest master to get it fixed.

[r-richmond]

This change is already present in master; it just wasn’t present on this branch or in the latest release. (Note this file has moved in master as well)

So I don’t think it makes sense to rebase on master; right?