Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Strange error after topic deletion, needs investigation #25

Open
ivanyu opened this issue Jun 5, 2023 · 0 comments
Open

Strange error after topic deletion, needs investigation #25

ivanyu opened this issue Jun 5, 2023 · 0 comments
Labels
bug Stale tiered storage Issues related to Tiered Storage

Comments

@ivanyu
Copy link
Member

ivanyu commented Jun 5, 2023

Sorry for the unclear issue, this needs looking deeper, filing to not forget.

The setup:

  1. 3 node cluster + tiered storage (S3).
  2. A topic was successfully created, filled, read from the remote.
  3. The topic was deleted and the leader node started logging this continuously:
[2023-06-05 04:57:23,491] ERROR Error encountered while writing committed offsets to a local file (org.apache.kafka.server.log.remote.metadata.storage.PrimaryConsumerTask:222)
java.nio.file.NoSuchFileException: /kafka/log/topic1-0/remote_log_snapshot.tmp
        at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
        at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:181)
        at java.base/java.nio.channels.FileChannel.open(FileChannel.java:298)
        at java.base/java.nio.channels.FileChannel.open(FileChannel.java:357)
        at org.apache.kafka.server.log.remote.metadata.storage.RemoteLogMetadataSnapshotFile.write(RemoteLogMetadataSnapshotFile.java:92)
        at org.apache.kafka.server.log.remote.metadata.storage.FileBasedRemoteLogMetadataCache.flushToFile(FileBasedRemoteLogMetadataCache.java:107)
        at org.apache.kafka.server.log.remote.metadata.storage.RemotePartitionMetadataStore.syncLogMetadataSnapshot(RemotePartitionMetadataStore.java:124)
        at org.apache.kafka.server.log.remote.metadata.storage.PrimaryConsumerTask.syncCommittedDataAndOffsets(PrimaryConsumerTask.java:215)
        at org.apache.kafka.server.log.remote.metadata.storage.PrimaryConsumerTask.consumeFromPrimaryConsumer(PrimaryConsumerTask.java:193)
        at org.apache.kafka.server.log.remote.metadata.storage.PrimaryConsumerTask.run(PrimaryConsumerTask.java:172)
        at java.base/java.lang.Thread.run(Thread.java:833)

Without looking deeper into the code I can't tell if it's just cosmetics that needs to be suppressed or it's an indicator of some real issue.

@ivanyu ivanyu added tiered storage Issues related to Tiered Storage bug labels Jun 5, 2023
@github-actions github-actions bot added the Stale label Sep 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Stale tiered storage Issues related to Tiered Storage
Projects
None yet
Development

No branches or pull requests

1 participant