Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix][broker] Fix heartbeat namespace create transaction internal topic #21348

Conversation

TakaHiR07
Copy link
Contributor

@TakaHiR07 TakaHiR07 commented Oct 12, 2023

Motivation

If we set transactionCoordinatorEnabled=true, heartbeat namespace create __transaction_buffer_snapshot topic.

Actually heartbeat namespace should not create transaction internal topic.

2023-10-12T11:24:44,738+0800 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO  org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryImpl - [pulsar/localhost:59381/persistent/healthcheck] Successfully initialize managed ledger
2023-10-12T11:24:44,809+0800 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO  org.apache.pulsar.broker.systopic.NamespaceEventsSystemTopicFactory - Create transaction buffer snapshot client, topicName : persistent://pulsar/localhost:59381/__transaction_buffer_snapshot
2023-10-12T11:24:44,839+0800 [broker-topic-workers-OrderedExecutor-7-0] INFO  org.apache.pulsar.broker.service.BrokerService - Created topic persistent://pulsar/localhost:59381/healthcheck - dedup is disabled
2023-10-12T11:24:44,843+0800 [pulsar-io-16-3] INFO  org.apache.pulsar.client.impl.ConnectionPool - [[id: 0x50bf7bc3, L:/127.0.0.1:59501 - R:localhost/127.0.0.1:59338]] Connected to server
2023-10-12T11:24:44,870+0800 [pulsar-io-16-4] INFO  org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:59501] connected with clientVersion=Pulsar-Java-v3.2.0-SNAPSHOT, clientProtocolVersion=21, proxyVersion=null
2023-10-12T11:24:44,899+0800 [configuration-metadata-store-4-1] INFO  org.apache.pulsar.broker.service.BrokerService - partitioned metadata successfully created for persistent://pulsar/localhost:59381/__transaction_buffer_snapshot
2023-10-12T11:24:44,965+0800 [pulsar-io-16-3] INFO  org.apache.pulsar.client.impl.ProducerImpl - [persistent://pulsar/localhost:59381/healthcheck] [null] Creating producer on cnx [id: 0x50bf7bc3, L:/127.0.0.1:59501 - R:localhost/127.0.0.1:59338]
2023-10-12T11:24:44,971+0800 [pulsar-io-16-3] INFO  org.apache.pulsar.client.impl.ProducerImpl - [persistent://pulsar/localhost:59381/__transaction_buffer_snapshot-partition-0] [null] Creating producer on cnx [id: 0x50bf7bc3, L:/127.0.0.1:59501 - R:localhost/127.0.0.1:59338]
2023-10-12T11:24:44,972+0800 [pulsar-io-16-3] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [persistent://pulsar/localhost:59381/__transaction_buffer_snapshot-partition-0][multiTopicsReader-0fc4fb7bcc] Subscribing to topic on cnx [id: 0x50bf7bc3, L:/127.0.0.1:59501 - R:localhost/127.0.0.1:59338], consumerId 0
2023-10-12T11:24:44,975+0800 [pulsar-io-16-3] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [persistent://pulsar/localhost:59381/__transaction_buffer_snapshot-partition-1][multiTopicsReader-0fc4fb7bcc] Subscribing to topic on cnx [id: 0x50bf7bc3, L:/127.0.0.1:59501 - R:localhost/127.0.0.1:59338], consumerId 1
2023-10-12T11:24:44,976+0800 [pulsar-io-16-3] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [persistent://pulsar/localhost:59381/__transaction_buffer_snapshot-partition-2][multiTopicsReader-0fc4fb7bcc] Subscribing to topic on cnx [id: 0x50bf7bc3, L:/127.0.0.1:59501 - R:localhost/127.0.0.1:59338], consumerId 2
2023-10-12T11:24:44,976+0800 [pulsar-io-16-3] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [persistent://pulsar/localhost:59381/__transaction_buffer_snapshot-partition-3][multiTopicsReader-0fc4fb7bcc] Subscribing to topic on cnx [id: 0x50bf7bc3, L:/127.0.0.1:59501 - R:localhost/127.0.0.1:59338], consumerId 3
2023-10-12T11:24:44,977+0800 [pulsar-io-16-3] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [persistent://pulsar/localhost:59381/__transaction_buffer_snapshot-partition-4][multiTopicsReader-0fc4fb7bcc] Subscribing to topic on cnx [id: 0x50bf7bc3, L:/127.0.0.1:59501 - R:localhost/127.0.0.1:59338], consumerId 4
2023-10-12T11:24:44,991+0800 [pulsar-io-16-4] INFO  org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Opening managed ledger pulsar/localhost:59381/persistent/__transaction_buffer_snapshot-partition-0

Modifications

do not construct TransactionBuffer if topic belongs to heartbeat namespace

Verifying this change

  • Make sure that the change passes the CI checks.

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository: TakaHiR07#15

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Oct 12, 2023
@Technoboy- Technoboy- added this to the 3.2.0 milestone Oct 18, 2023
@Technoboy- Technoboy- requested review from codelipenghui and Technoboy- and removed request for codelipenghui October 18, 2023 13:07
@Technoboy- Technoboy- merged commit c8a2f49 into apache:master Oct 19, 2023
48 of 50 checks passed
shibd pushed a commit to shibd/pulsar that referenced this pull request Oct 22, 2023
shibd pushed a commit to shibd/pulsar that referenced this pull request Oct 23, 2023
shibd pushed a commit to shibd/pulsar that referenced this pull request Oct 23, 2023
shibd pushed a commit to shibd/pulsar that referenced this pull request Oct 24, 2023
poorbarcode pushed a commit that referenced this pull request Oct 24, 2023
nikhil-ctds pushed a commit to datastax/pulsar that referenced this pull request Dec 20, 2023
srinath-ctds pushed a commit to datastax/pulsar that referenced this pull request Dec 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants