Update spark-core, spark-mllib, spark-sql to 3.3.0 #637
Conversation
![typelevel-steward[bot]](https://avatars.githubusercontent.com/u/3731824?s=60&v=4){kind=small}
pomadchin
force-pushed
the
update/spark-core-3.3.0
branch
from
9806ce5
to
e9f91e6
Compare
Codecov Report
@@ Coverage Diff @@
## master #637 +/- ##
==========================================
- Coverage 95.20% 95.11% -0.09%
==========================================
Files 65 65
Lines 1168 1168
Branches 8 7 -1
==========================================
- Hits 1112 1111 -1
- Misses 56 57 +1
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
pomadchin
force-pushed
the
update/spark-core-3.3.0
branch
from
622b106
to
b2c6297
Compare
pomadchin
force-pushed
the
update/spark-core-3.3.0
branch
from
b2c6297
to
3fe486b
Compare
|
This PR drops Spark 3.0, cc @cchantep do you still need the old Spark relese? We don't have any significant changes anyway and we can cut 0.12.1 if you need before merging this PR. |
Ok if we keep compat down to 3.1 |
| lazy val spark32Settings = Seq( | ||
| mimaPreviousArtifacts := Set.empty | ||
| ) |
There was a problem hiding this comment.
As a rule-of-thumb, you should never set mimaPreviousArtifacts := Set.empty, this is easy to forget and break binary-compatibiliy in a future release by mistake.
There are basically two situations:
- You are not publishing the module, so use
.enablePlugins(NoPublishPlugin)to disable MiMa. - It is a new module that you have not published yet. So use this setting to indicate which version it will be introduced, and MiMa will automatically start from that version.
tlVersionIntroduced := Map("2.12" -> "0.13.0", "2.13" -> "0.13.0")
There was a problem hiding this comment.
@armanbilge thanks! Is this intel documented anywhere?
What will be the setting for the artifact we publish for the first time and it has not been published before?
To be clear: 3.2 is the new artifact and we don't need to compare it with the previous version.
There was a problem hiding this comment.
is this intel documented anywhere?
Sort of. This particular setting is documented here:
https://typelevel.org/sbt-typelevel/faq.html#how-do-i-indicate-the-first-version-that-i-published-scala-3-artifacts-for
Although perhaps it can use some other examples like for your situation.
What will be the setting for the artifact we publish (first time) and it has not been published before?
What is the first version you plan to publish it in?
There was a problem hiding this comment.
@armanbilge o i c, 0.13 is the first version, so we need to add tlVersionIntroduced := Map("2.12" -> "0.13.0", "2.13" -> "0.13.0") as an indication that the first artifact was published as the 0.13 version for scala 2.12 and 2.13.
Makes sense now, thanks!
There was a problem hiding this comment.
Right, exactly, you got it! :) And this way you don't need to remember to change it again after publishing 0.13.0, MiMa will start working automatically.
Updates
from 3.2.1 to 3.3.0.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a
.scala-steward.conffile.Have a fantastic day writing Scala!
Files still referring to the old version number
The following files still refer to the old version number (3.2.1).
You might want to review and update them manually.
Ignore future updates
Add this to your
.scala-steward.conffile to ignore future updates of this dependency:labels: library-update, early-semver-minor, semver-spec-minor, old-version-remains, commit-count:1