Goroutine leak when usign /api/v1/changefeed to create kafka changefeed

[overvenus]

What did you do?

• Create changefeeds using /api/v1/changefeed.
• TiCDC fails to create changefeed, because downstream kafka rejects unauthorized TiCDC.

[2022/04/26 14:12:26.682 +00:00] [INFO] [http_router.go:148] [/api/v1/changefeeds] [status=500] [method=POST] [path=/api/v1/changefeeds] [query=] [ip=[172.30.104.117](http://172.30.104.117/)] [user-agent=Go-http-client/1.1] [error="[CDC:ErrKafkaNewSaramaProducer]new sarama producer: [CDC:ErrKafkaInvalidConfig]because TiCDC Kafka producer's `request.required.acks` defaults to -1, TiCDC cannot deliver messages when the `replication-factor` is less than `min.insync.replicas`: kafka server: The client is not authorized to send this request type."] [errorVerbose="[CDC:ErrKafkaNewSaramaProducer]new sarama producer: [CDC:ErrKafkaInvalidConfig]because TiCDC Kafka producer's `request.required.acks` defaults to -1, TiCDC cannot deliver messages when the `replication-factor` is less than `min.insync.replicas`: kafka server: The client is not authorized to send this request type.\[ngithub.com/pingcap/errors.AddStack\n\tgithub.com/pingcap/[email protected]/errors.go:174\ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs\n\tgithub.com/pingcap/[email protected]/normalize.go:164\ngithub.com/pingcap/tiflow/pkg/errors.WrapError\n\tgithub.com/pingcap/tiflow/pkg/errors/helper.go:30\ngithub.com/pingcap/tiflow/cdc/sink/producer/kafka.NewKafkaSaramaProducer\n\tgithub.com/pingcap/tiflow/cdc/sink/producer/kafka/kafka.go:284\ngithub.com/pingcap/tiflow/cdc/sink.newKafkaSaramaSink\n\tgithub.com/pingcap/tiflow/cdc/sink/mq.go:398\ngithub.com/pingcap/tiflow/cdc/sink.init.1.func3\n\tgithub.com/pingcap/tiflow/cdc/sink/sink.go:103\ngithub.com/pingcap/tiflow/cdc/sink.New\n\tgithub.com/pingcap/tiflow/cdc/sink/sink.go:123\ngithub.com/pingcap/tiflow/cdc/sink.Validate\n\tgithub.com/pingcap/tiflow/cdc/sink/sink.go:136\ngithub.com/pingcap/tiflow/cdc/capture.verifyCreateChangefeedConfig\n\tgithub.com/pingcap/tiflow/cdc/capture/http_validator.go:145\ngithub.com/pingcap/tiflow/cdc/capture.(*HTTPHandler).CreateChangefeed\n\tgithub.com/pingcap/tiflow/cdc/capture/http_handler.go:216\ngithub.com/gin-gonic/gin.(*Context).Next\n\tgithub.com/gin-gonic/[email protected]/context.go:165\ngithub.com/pingcap/tiflow/cdc.errorHandleMiddleware.func1\n\tgithub.com/pingcap/tiflow/cdc/http_router.go:163\ngithub.com/gin-gonic/gin.(*Context).Next\n\tgithub.com/gin-gonic/[email protected]/context.go:165\ngithub.com/pingcap/tiflow/cdc.timeoutMiddleware.func1\n\tgithub.com/pingcap/tiflow/cdc/http_router.go:126\ngithub.com/gin-gonic/gin.(*Context).Next\n\tgithub.com/gin-gonic/[email protected]/context.go:165\ngithub.com/pingcap/tiflow/cdc.logMiddleware.func1\n\tgithub.com/pingcap/tiflow/cdc/http_router.go:135\ngithub.com/gin-gonic/gin.(*Context).Next\n\tgithub.com/gin-gonic/[email protected]/context.go:165\ngithub.com/gin-gonic/gin.(*Engine).handleHTTPRequest\n\tgithub.com/gin-gonic/[email protected]/gin.go:489\ngithub.com/gin-gonic/gin.(*Engine).ServeHTTP\n\tgithub.com/gin-gonic/[email protected]/gin.go:445\nnet/http.serverHandler.ServeHTTP\n\tnet/http/server.go:2887\nnet/http.(*conn).serve\n\tnet/http/server.go:1952\nruntime.goexit\n\truntime/asm_amd64.s:1371](http://ngithub.com/pingcap/errors.AddStack/n/tgithub.com/pingcap/[email protected]/errors.go:174/ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs/n/tgithub.com/pingcap/[email protected]/normalize.go:164/ngithub.com/pingcap/tiflow/pkg/errors.WrapError/n/tgithub.com/pingcap/tiflow/pkg/errors/helper.go:30/ngithub.com/pingcap/tiflow/cdc/sink/producer/kafka.NewKafkaSaramaProducer/n/tgithub.com/pingcap/tiflow/cdc/sink/producer/kafka/kafka.go:284/ngithub.com/pingcap/tiflow/cdc/sink.newKafkaSaramaSink/n/tgithub.com/pingcap/tiflow/cdc/sink/mq.go:398/ngithub.com/pingcap/tiflow/cdc/sink.init.1.func3/n/tgithub.com/pingcap/tiflow/cdc/sink/sink.go:103/ngithub.com/pingcap/tiflow/cdc/sink.New/n/tgithub.com/pingcap/tiflow/cdc/sink/sink.go:123/ngithub.com/pingcap/tiflow/cdc/sink.Validate/n/tgithub.com/pingcap/tiflow/cdc/sink/sink.go:136/ngithub.com/pingcap/tiflow/cdc/capture.verifyCreateChangefeedConfig/n/tgithub.com/pingcap/tiflow/cdc/capture/http_validator.go:145/ngithub.com/pingcap/tiflow/cdc/capture.(*HTTPHandler).CreateChangefeed/n/tgithub.com/pingcap/tiflow/cdc/capture/http_handler.go:216/ngithub.com/gin-gonic/gin.(*Context).Next/n/tgithub.com/gin-gonic/[email protected]/context.go:165/ngithub.com/pingcap/tiflow/cdc.errorHandleMiddleware.func1/n/tgithub.com/pingcap/tiflow/cdc/http_router.go:163/ngithub.com/gin-gonic/gin.(*Context).Next/n/tgithub.com/gin-gonic/[email protected]/context.go:165/ngithub.com/pingcap/tiflow/cdc.timeoutMiddleware.func1/n/tgithub.com/pingcap/tiflow/cdc/http_router.go:126/ngithub.com/gin-gonic/gin.(*Context).Next/n/tgithub.com/gin-gonic/[email protected]/context.go:165/ngithub.com/pingcap/tiflow/cdc.logMiddleware.func1/n/tgithub.com/pingcap/tiflow/cdc/http_router.go:135/ngithub.com/gin-gonic/gin.(*Context).Next/n/tgithub.com/gin-gonic/[email protected]/context.go:165/ngithub.com/gin-gonic/gin.(*Engine).handleHTTPRequest/n/tgithub.com/gin-gonic/[email protected]/gin.go:489/ngithub.com/gin-gonic/gin.(*Engine).ServeHTTP/n/tgithub.com/gin-gonic/[email protected]/gin.go:445/nnet/http.serverHandler.ServeHTTP/n/tnet/http/server.go:2887/nnet/http.(*conn).serve/n/tnet/http/server.go:1952/nruntime.goexit/n/truntime/asm_amd64.s:1371)"] [cost=812.845849ms]

goroutine-leak.log.zip

What did you expect to see?

No goroutine leak.

What did you see instead?

Goroutine leak.

Versions of the cluster

TiCDC version (execute cdc version):

v5.4.0