Merge pull request #3740 from Kotlin/version-1.7.0

Version 1.7.0

CHANGES.md

@@ -1,5 +1,78 @@
 # Change log for kotlinx.coroutines
 
+## Version 1.7.0
+
+### Core API significant improvements
+
+* New `Channel` implementation with significant performance improvements across the API (#3621).
+* New `select` operator implementation: faster, more lightweight, and more robust (#3020).
+* `Mutex` and `Semaphore` now share the same underlying data structure (#3020).
+* `Dispatchers.IO` is added to K/N (#3205)
+  * `newFixedThreadPool` and `Dispatchers.Default` implementations on K/N were wholly rewritten to support graceful growth under load (#3595).
+* `kotlinx-coroutines-test` rework:
+  - Add the `timeout` parameter to `runTest` for the whole-test timeout, 10 seconds by default (#3270). This replaces the configuration of quiescence timeouts, which is now deprecated (#3603).
+  - The `withTimeout` exception messages indicate if the timeout used the virtual time (#3588).
+  - `TestCoroutineScheduler`, `runTest`, and `TestScope` API are promoted to stable (#3622).
+  - `runTest` now also fails if there were uncaught exceptions in coroutines not inherited from the test coroutine (#1205).
+
+### Breaking changes
+
+* Old K/N memory model is no longer supported (#3375).
+* New generic upper bounds were added to reactive integration API where the language since 1.8.0 dictates (#3393).
+* `kotlinx-coroutines-core` and `kotlinx-coroutines-jdk8` artifacts were merged into a single artifact (#3268).
+* Artificial stackframes in stacktrace recovery no longer contain the `\b` symbol and are now navigable in IDE and supplied with proper documentation (#2291).
+* `CoroutineContext.isActive` returns `true` for contexts without any job in them (#3300).
+
+### Bug fixes and improvements
+
+* Kotlin version is updated to 1.8.20
+* Atomicfu version is updated to 0.20.2.
+* `JavaFx` version is updated to 17.0.2 in `kotlinx-coroutines-javafx` (#3671)..
+* JPMS is supported (#2237). Thanks @lion7!
+* `BroadcastChannel` and all the corresponding API are deprecated (#2680).
+* Added all supported K/N targets (#3601, #812, #855).
+* K/N `Dispatchers.Default` is backed by the number of threads equal to the number of available cores (#3366).
+* Fixed an issue where some coroutines' internal exceptions were not properly serializable (#3328).
+* Introduced `Job.parent` API (#3201).
+* Fixed a bug when `TestScheduler` leaked cancelled jobs (#3398).
+* `TestScope.timeSource` now provides comparable time marks (#3617). Thanks @hfhbd!
+* Fixed an issue when cancelled `withTimeout` handles were preserved in JS runtime (#3440).
+* Ensure `awaitFrame` only awaits a single frame when used from the main looper (#3432). Thanks @pablobaxter!
+* Obsolete `Class-Path` attribute was removed from `kotlinx-coroutines-debug.jar` manifest (#3361).
+* Fixed a bug when `updateThreadContext` operated on the parent context (#3411).
+* Added new `Flow.filterIsInstance` extension (#3240).
+* `Dispatchers.Default` thread name prefixes are now configurable with system property (#3231).
+* Added `Flow.timeout` operator as `@FlowPreview` (#2624). Thanks @pablobaxter!
+* Improved the performance of the `future` builder in case of exceptions (#3475). Thanks @He-Pin!
+* `Mono.awaitSingleOrNull` now waits for the `onComplete` signal (#3487).
+* `Channel.isClosedForSend` and `Channel.isClosedForReceive` are promoted from experimental to delicate (#3448).
+* Fixed a data race in native `EventLoop` (#3547).
+* `Dispatchers.IO.limitedParallelism(valueLargerThanIOSize)` no longer creates an additional wrapper (#3442). Thanks @dovchinnikov!
+* Various `@FlowPreview` and `@ExperimentalCoroutinesApi` are promoted to experimental and stable respectively (#3542, #3097, #3548).
+* Performance improvements in `Dispatchers.Default` and `Dispatchers.IO` (#3416, #3418).
+* Fixed a bug when internal `suspendCancellableCoroutineReusable` might have hanged (#3613).
+* Introduced internal API to process events in the current system dispatcher (#3439).
+* Global `CoroutineExceptionHandler` is no longer invoked in case of unprocessed `future` failure (#3452).
+* Performance improvements and reduced thread-local pressure for the `withContext` operator (#3592).
+* Improved performance of `DebugProbes` (#3527).
+* Fixed a bug when the coroutine debugger might have detected the state of a coroutine incorrectly (#3193).
+* `CoroutineDispatcher.asExecutor()` runs tasks without dispatching if the dispatcher is unconfined (#3683). Thanks @odedniv!
+* `SharedFlow.toMutableList` and `SharedFlow.toSet` lints are introduced (#3706).
+* `Channel.invokeOnClose` is promoted to stable API (#3358).
+* Improved lock contention in `Dispatchers.Default` and `Dispatchers.IO` during the startup phase (#3652).
+* Fixed a bug that led to threads oversubscription in `Dispatchers.Default` (#3642).
+* Fixed a bug that allowed `limitedParallelism` to perform dispatches even after the underlying dispatcher was closed (#3672).
+* Fixed a bug that prevented stacktrace recovery when the exception's constructor from `cause` was selected (#3714).
+* Improved sanitizing of stracktrace-recovered traces (#3714).
+* Introduced an internal flag to disable uncaught exceptions reporting in tests as a temporary migration mechanism (#3736).
+* Various documentation improvements and fixes.
+
+### Changelog relative to version 1.7.0-RC
+
+* Fixed a bug that prevented stacktrace recovery when the exception's constructor from `cause` was selected (#3714).
+* Improved sanitizing of stracktrace-recovered traces (#3714).
+* Introduced an internal flag to disable uncaught exceptions reporting in tests as a temporary migration mechanism (#3736).
+
 ## Version 1.7.0-RC
 
 * Kotlin version is updated to 1.8.20.

README.md

@@ -3,7 +3,7 @@
 [![Kotlin Stable](https://kotl.in/badges/stable.svg)](https://kotlinlang.org/docs/components-stability.html)
 [![JetBrains official project](https://jb.gg/badges/official.svg)](https://confluence.jetbrains.com/display/ALL/JetBrains+on+GitHub)
 [![GitHub license](https://img.shields.io/badge/license-Apache%20License%202.0-blue.svg?style=flat)](https://www.apache.org/licenses/LICENSE-2.0)
-[![Download](https://img.shields.io/maven-central/v/org.jetbrains.kotlinx/kotlinx-coroutines-core/1.7.0-RC)](https://central.sonatype.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core/1.7.0-RC)
+[![Download](https://img.shields.io/maven-central/v/org.jetbrains.kotlinx/kotlinx-coroutines-core/1.7.0)](https://central.sonatype.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core/1.7.0)
 [![Kotlin](https://img.shields.io/badge/kotlin-1.8.20-blue.svg?logo=kotlin)](http://kotlinlang.org)
 [![Slack channel](https://img.shields.io/badge/chat-slack-green.svg?logo=slack)](https://kotlinlang.slack.com/messages/coroutines/)
 
@@ -85,7 +85,7 @@ Add dependencies (you can also add other modules that you need):
 <dependency>
     <groupId>org.jetbrains.kotlinx</groupId>
     <artifactId>kotlinx-coroutines-core</artifactId>
-    <version>1.7.0-RC</version>
+    <version>1.7.0</version>
 </dependency>
 ```
 
@@ -103,7 +103,7 @@ Add dependencies (you can also add other modules that you need):
 
 ```kotlin
 dependencies {
-    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.7.0-RC")
+    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.7.0")
 }
 ```
 
@@ -133,7 +133,7 @@ Add [`kotlinx-coroutines-android`](ui/kotlinx-coroutines-android)
 module as a dependency when using `kotlinx.coroutines` on Android:
 
 ```kotlin
-implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.7.0-RC")
+implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.7.0")
 ```
 
 This gives you access to the Android [Dispatchers.Main]
@@ -168,7 +168,7 @@ In common code that should get compiled for different platforms, you can add a d
 ```kotlin
 commonMain {
     dependencies {
-        implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.7.0-RC")
+        implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.7.0")
     }
 }
 ```
@@ -180,7 +180,7 @@ Platform-specific dependencies are recommended to be used only for non-multiplat
 #### JS
 
 Kotlin/JS version of `kotlinx.coroutines` is published as 
-[`kotlinx-coroutines-core-js`](https://central.sonatype.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core-js/1.7.0-RC)
+[`kotlinx-coroutines-core-js`](https://central.sonatype.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core-js/1.7.0)
 (follow the link to get the dependency declaration snippet) and as [`kotlinx-coroutines-core`](https://www.npmjs.com/package/kotlinx-coroutines-core) NPM package. 
 
 #### Native

benchmarks/build.gradle.kts

@@ -24,7 +24,7 @@ java {
 tasks.named<KotlinCompile>("compileJmhKotlin") {
     kotlinOptions {
         jvmTarget = "1.8"
-        freeCompilerArgs += "-Xjvm-default=enable"
+        freeCompilerArgs += "-Xjvm-default=all"
     }
 }
 

benchmarks/src/jmh/kotlin/benchmarks/flow/scrabble/ShakespearePlaysScrabble.kt

@@ -34,14 +34,12 @@ abstract class ShakespearePlaysScrabble {
     public interface LongWrapper {
         fun get(): Long
 
-        @JvmDefault
         fun incAndSet(): LongWrapper {
             return object : LongWrapper {
                 override fun get(): Long = this@LongWrapper.get() + 1L
             }
         }
 
-        @JvmDefault
         fun add(other: LongWrapper): LongWrapper {
             return object : LongWrapper {
                 override fun get(): Long = this@LongWrapper.get() + other.get()

gradle.properties

@@ -3,7 +3,7 @@
 #
 
 # Kotlin
-version=1.7.0-RC-SNAPSHOT
+version=1.7.0-SNAPSHOT
 group=org.jetbrains.kotlinx
 kotlin_version=1.8.20
 
@@ -13,7 +13,7 @@ junit5_version=5.7.0
 atomicfu_version=0.20.2
 knit_version=0.4.0
 html_version=0.7.2
-lincheck_version=2.16
+lincheck_version=2.17
 dokka_version=1.8.10
 byte_buddy_version=1.10.9
 reactor_version=3.4.1

integration-testing/gradle.properties

@@ -1,5 +1,5 @@
 kotlin_version=1.8.20
-coroutines_version=1.7.0-RC-SNAPSHOT
+coroutines_version=1.7.0-SNAPSHOT
 asm_version=9.3
 
 kotlin.code.style=official

kotlinx-coroutines-core/common/src/channels/BufferedChannel.kt

@@ -1220,9 +1220,9 @@ internal open class BufferedChannel<E>(
                 incCompletedExpandBufferAttempts()
                 return
             }
-            // Is `bufferEndSegment` outdated?
+            // Is `bufferEndSegment` outdated or is the segment with the required id already removed?
             // Find the required segment, creating new ones if needed.
-            if (segment.id < id) {
+            if (segment.id != id) {
                 segment = findSegmentBufferEnd(id, segment, b)
                     // Restart if the required segment is removed, or
                     // the linked list of segments is already closed,

kotlinx-coroutines-core/jvm/src/internal/ExceptionsConstructor.kt

@@ -32,42 +32,48 @@ internal fun <E : Throwable> tryCopyException(exception: E): E? {
 
 private fun <E : Throwable> createConstructor(clz: Class<E>): Ctor {
     val nullResult: Ctor = { null } // Pre-cache class
-    // Skip reflective copy if an exception has additional fields (that are usually populated in user-defined constructors)
+    // Skip reflective copy if an exception has additional fields (that are typically populated in user-defined constructors)
     if (throwableFields != clz.fieldsCountOrDefault(0)) return nullResult
     /*
-    * Try to reflectively find constructor(), constructor(message, cause), constructor(cause) or constructor(message).
-    * Exceptions are shared among coroutines, so we should copy exception before recovering current stacktrace.
-    */
-    val constructors = clz.constructors.sortedByDescending { it.parameterTypes.size }
-    for (constructor in constructors) {
-        val result = createSafeConstructor(constructor)
-        if (result != null) return result
-    }
-    return nullResult
-}
-
-private fun createSafeConstructor(constructor: Constructor<*>): Ctor? {
-    val p = constructor.parameterTypes
-    return when (p.size) {
-        2 -> when {
-            p[0] == String::class.java && p[1] == Throwable::class.java ->
-                safeCtor { e -> constructor.newInstance(e.message, e) as Throwable }
-            else -> null
+     * Try to reflectively find constructor(message, cause), constructor(message), constructor(cause), or constructor(),
+     * in that order of priority.
+     * Exceptions are shared among coroutines, so we should copy exception before recovering current stacktrace.
+     *
+     * By default, Java's reflection iterates over ctors in the source-code order and the sorting is stable, so we can
+     * not rely on the order of iteration. Instead, we assign a unique priority to each ctor type.
+     */
+    return clz.constructors.map { constructor ->
+        val p = constructor.parameterTypes
+        when (p.size) {
+            2 -> when {
+                p[0] == String::class.java && p[1] == Throwable::class.java ->
+                    safeCtor { e -> constructor.newInstance(e.message, e) as Throwable } to 3
+                else -> null to -1
+            }
+            1 -> when (p[0]) {
+                String::class.java ->
+                    safeCtor { e -> (constructor.newInstance(e.message) as Throwable).also { it.initCause(e) } } to 2
+                Throwable::class.java ->
+                    safeCtor { e -> constructor.newInstance(e) as Throwable } to 1
+                else -> null to -1
+            }
+            0 -> safeCtor { e -> (constructor.newInstance() as Throwable).also { it.initCause(e) } } to 0
+            else -> null to -1
         }
-        1 -> when (p[0]) {
-            Throwable::class.java ->
-                safeCtor { e -> constructor.newInstance(e) as Throwable }
-            String::class.java ->
-                safeCtor { e -> (constructor.newInstance(e.message) as Throwable).also { it.initCause(e) } }
-            else -> null
-        }
-        0 -> safeCtor { e -> (constructor.newInstance() as Throwable).also { it.initCause(e) } }
-        else -> null
-    }
+    }.maxByOrNull(Pair<*, Int>::second)?.first ?: nullResult
 }
 
-private inline fun safeCtor(crossinline block: (Throwable) -> Throwable): Ctor =
-    { e -> runCatching { block(e) }.getOrNull() }
+private fun safeCtor(block: (Throwable) -> Throwable): Ctor = { e ->
+    runCatching {
+        val result = block(e)
+        /*
+         * Verify that the new exception has the same message as the original one (bail out if not, see #1631)
+         * or if the new message complies the contract from `Throwable(cause).message` contract.
+         */
+        if (e.message != result.message && result.message != e.toString()) null
+        else result
+    }.getOrNull()
+}
 
 private fun Class<*>.fieldsCountOrDefault(defaultValue: Int) =
     kotlin.runCatching { fieldsCount() }.getOrDefault(defaultValue)

kotlinx-coroutines-core/jvm/src/internal/StackTraceRecovery.kt

@@ -33,16 +33,16 @@ private val stackTraceRecoveryClassName = runCatching {
 internal actual fun <E : Throwable> recoverStackTrace(exception: E): E {
     if (!RECOVER_STACK_TRACES) return exception
     // No unwrapping on continuation-less path: exception is not reported multiple times via slow paths
-    val copy = tryCopyAndVerify(exception) ?: return exception
+    val copy = tryCopyException(exception) ?: return exception
     return copy.sanitizeStackTrace()
 }
 
 private fun <E : Throwable> E.sanitizeStackTrace(): E {
     val stackTrace = stackTrace
     val size = stackTrace.size
-    val lastIntrinsic = stackTrace.frameIndex(stackTraceRecoveryClassName)
+    val lastIntrinsic = stackTrace.indexOfLast { stackTraceRecoveryClassName == it.className }
     val startIndex = lastIntrinsic + 1
-    val endIndex = stackTrace.frameIndex(baseContinuationImplClassName)
+    val endIndex = stackTrace.firstFrameIndex(baseContinuationImplClassName)
     val adjustment = if (endIndex == -1) 0 else size - endIndex
     val trace = Array(size - lastIntrinsic - adjustment) {
         if (it == 0) {
@@ -70,7 +70,7 @@ private fun <E : Throwable> recoverFromStackFrame(exception: E, continuation: Co
     val (cause, recoveredStacktrace) = exception.causeAndStacktrace()
 
     // Try to create an exception of the same type and get stacktrace from continuation
-    val newException = tryCopyAndVerify(cause) ?: return exception
+    val newException = tryCopyException(cause) ?: return exception
     // Update stacktrace
     val stacktrace = createStackTrace(continuation)
     if (stacktrace.isEmpty()) return exception
@@ -82,14 +82,6 @@ private fun <E : Throwable> recoverFromStackFrame(exception: E, continuation: Co
     return createFinalException(cause, newException, stacktrace)
 }
 
-private fun <E : Throwable> tryCopyAndVerify(exception: E): E? {
-    val newException = tryCopyException(exception) ?: return null
-    // Verify that the new exception has the same message as the original one (bail out if not, see #1631)
-    // CopyableThrowable has control over its message and thus can modify it the way it wants
-    if (exception !is CopyableThrowable<*> && newException.message != exception.message) return null
-    return newException
-}
-
 /*
  * Here we partially copy original exception stackTrace to make current one much prettier.
  * E.g. for
@@ -109,7 +101,7 @@ private fun <E : Throwable> tryCopyAndVerify(exception: E): E? {
 private fun <E : Throwable> createFinalException(cause: E, result: E, resultStackTrace: ArrayDeque<StackTraceElement>): E {
     resultStackTrace.addFirst(ARTIFICIAL_FRAME)
     val causeTrace = cause.stackTrace
-    val size = causeTrace.frameIndex(baseContinuationImplClassName)
+    val size = causeTrace.firstFrameIndex(baseContinuationImplClassName)
     if (size == -1) {
         result.stackTrace = resultStackTrace.toTypedArray()
         return result
@@ -157,7 +149,6 @@ private fun mergeRecoveredTraces(recoveredStacktrace: Array<StackTraceElement>,
     }
 }
 
-@Suppress("NOTHING_TO_INLINE")
 internal actual suspend inline fun recoverAndThrow(exception: Throwable): Nothing {
     if (!RECOVER_STACK_TRACES) throw exception
     suspendCoroutineUninterceptedOrReturn<Nothing> {
@@ -198,7 +189,7 @@ private fun createStackTrace(continuation: CoroutineStackFrame): ArrayDeque<Stac
 }
 
 internal fun StackTraceElement.isArtificial() = className.startsWith(ARTIFICIAL_FRAME_PACKAGE_NAME)
-private fun Array<StackTraceElement>.frameIndex(methodName: String) = indexOfFirst { methodName == it.className }
+private fun Array<StackTraceElement>.firstFrameIndex(methodName: String) = indexOfFirst { methodName == it.className }
 
 private fun StackTraceElement.elementWiseEquals(e: StackTraceElement): Boolean {
     /*

kotlinx-coroutines-core/jvm/test-resources/stacktraces/channels/testReceiveFromChannel.txt

@@ -1,10 +1,9 @@
 kotlinx.coroutines.RecoverableTestException
-	at kotlinx.coroutines.internal.StackTraceRecoveryKt.recoverStackTrace(StackTraceRecovery.kt)
 	at kotlinx.coroutines.channels.BufferedChannel.receive$suspendImpl(BufferedChannel.kt)
 	at kotlinx.coroutines.channels.BufferedChannel.receive(BufferedChannel.kt)
 	at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.channelReceive(StackTraceRecoveryChannelsTest.kt)
 	at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest.access$channelReceive(StackTraceRecoveryChannelsTest.kt)
 	at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$channelReceive$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt)
 Caused by: kotlinx.coroutines.RecoverableTestException
 	at kotlinx.coroutines.exceptions.StackTraceRecoveryChannelsTest$testReceiveFromChannel$1$job$1.invokeSuspend(StackTraceRecoveryChannelsTest.kt)
-	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt)
+	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt)