airframe-http: Deprecate the legacy HttpClient[F[_], Req, Resp] interface

[xerial]

This is a proposal to deprecate the current http client interface HttpClient[F[_], Req, Resp] that uses get[Resp], getResource[Resp], post[Resp], etc. Instead, adding a Scala-3 friendly simplified interface SyncClient/AsyncClient that consolidate these legacy methods into a simple one. For example, you can send a http request send(request: HttpMessage.Request): HttpMessage.Response method.

You can test the new interface like this:

import wvlet.airframe.http.Http

// Using Java's standard http client
val client: SyncClient = Http
  .client
  .withRequestFilter(...)
  .withConnectTimeout(...)
  .newSyncClient(baseUrl)

// Simple request and receives a raw http response
val r: Response = client.send(Http.GET("/path").withHeader(....))

// Send POST request with JSON body and read the response as an MyObj object
client.readAs[MyObj](Http.POST("/path2").withJson(...))

// Send an object data as the request body, and receive the response as ResponseType object
// JSON/MessagePack data will be transformed internally
val resp: ResponseType = client.call[RequestType, ResponseType](Http.POST("/path3"), requestDataObj) 

// When you need to use OkHttp client 
import wvlet.airframe.http.okhttp.OkHttp

// The same http interface will be used (WARNING! This is not yet implemented)
val okHttpClient: SyncClient = OkHttp
   .client
   .withXXX(...) // OkHttp specific configuration
   .newSyncClient(baseUrl)

What will be changed

• All http clients will use a new SyncClient/AsyncClient interface https://github.com/wvlet/airframe/blob/master/airframe-http/src/main/scala/wvlet/airframe/http/client/HttpClients.scala. Mapping to Scala case classes from HttpResponse are already supported.
  • Differences of client backend (e.g., Java11 http client, okhttp, Finagle, Netty) will be consumed in HttpChannel interface https://github.com/wvlet/airframe/blob/master/airframe-http/src/main/scala/wvlet/airframe/http/client/HttpChannel.scala
• airframe-finagle, okhttp, recorder interfaces will take the standard HttpMessage.Request/Response. airframe-http: Use HttpMessage.Request/Response as the standard interface #2167 The existing code that use these client implementations need to be adjusted to the new interface, or migrate to the standard Java client, which can be created with Http.client.newSyncClient(...) method.
• airframe-okhttp, recorder will support Scala 3.
• airframe-finagle might be able to support Scala 3 by referencing Scala 2.13 version

What will not be changed

• Server side interface. Finagle and gRPC servers can be used without any change

The current usage of HttpClient[F[_], Req, Resp] interface

• airframe-okhttp (TD's conductor, td-scala-util, etc.)
• airframe-http-finagle FinagleClient (it's frequently used in test cases)
• airframe-http-recorder (Several test usages)

Hopefully, changing the client initialization and the request/response interface to HttpMessage.Request/Response will work for the migratin.

And unit tests that uses Finagle.client, OkHttpClient, http recorder, etc.

Background: Why does it need to be changed?

Difficulty of Scala 3 Support

The current interface HttpClient[F[_], Req, Resp] has many methods that takes TypeTag, which is no longer available in Scala 3:

• https://github.com/wvlet/airframe/blob/master/airframe-http/src/main/scala/wvlet/airframe/http/HttpClient.scala
• https://github.com/wvlet/airframe/blob/master/airframe-http/src/main/scala-2/wvlet/airframe/http/HttpClientBase.scala
  As a workaround to support Scala 3, HttpClientBase class was added to provide different implementations for Scala 2 and Scala 3.

airframe-http-recorder implementation is tied to Finagle because it uses Finagle as the backend. But, Finagle will not be migrated to Scala 3 in near future, so we need to find an alternative solution (e.g., implementing Netty4-based server backend)

Supporting multiple HTTP client backends becomes less important

HttpClient[F[_], Res, Resp] design was introduced to support various types of Future types (Scala Future or Twitter Future), http request/response types (Finagle's http Request/Response, OkHttp's Request/Response). Now that airframe-http has the standard HttpMessage.Request/Response classes, supporting different request/response types becomes less important. If you use RPC, directly manipulating http request and response is unnecessary other than for implementing HTTP request filters.

Since Java 9, you can use a standard Java client, which is already supported in https://github.com/wvlet/airframe/blob/master/airframe-http/.jvm/src/main/scala/wvlet/airframe/http/client/JavaClientChannel.scala
This client requires no extra dependency other than Java 9 or later.

Finagle Request are mutable

Finagle's request object is mutable. The new HttpMessage.Request is an immutable object, which is better to avoid unexpected side-effect.

Future Work

AsyncClient is currently using Scala's Future[X] interface, but it's a bit inconvenient that we always need to provide ExecutionContext for Future objects. Instead, having an Rx-based http client interface #2300, which can support Scala.js and gRPC would be ideal.

[xerial]

This is not a blocker for Scala 3 support. Removed from Scala 3 milestone Actually, it's better to have this change in an earlier phase so that we can migrate okhttp client to Scala 3 #2670