Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix empty block location #18385

Merged
merged 1 commit into from
Nov 9, 2023
Merged

Fix empty block location #18385

merged 1 commit into from
Nov 9, 2023

Conversation

dbw9580
Copy link
Contributor

@dbw9580 dbw9580 commented Nov 7, 2023

What changes are proposed in this pull request?

Fix a bug caused by empty block location strings. The worker sends block locations which contain empty medium type strings and block master sync is therefore broken.

Why are the changes needed?

2023-11-06 10:18:26,714 ERROR BlockMasterSync - Failed to receive master heartbeat command. worker id 1812723380431920939
alluxio.exception.status.InternalException: MediumType must be one of {MEM, HDD and SSD} but got
        at alluxio.exception.status.AlluxioStatusException.from(AlluxioStatusException.java:159)
        at alluxio.exception.status.AlluxioStatusException.fromStatusRuntimeException(AlluxioStatusException.java:215)
        at alluxio.AbstractClient.retryRPCInternal(AbstractClient.java:486)
        at alluxio.AbstractClient.retryRPC(AbstractClient.java:450)
        at alluxio.AbstractClient.retryRPC(AbstractClient.java:439)
        at alluxio.worker.block.BlockMasterClient.heartbeat(BlockMasterClient.java:238)
        at alluxio.worker.block.BlockMasterSyncHelper.heartbeat(BlockMasterSyncHelper.java:140)
        at alluxio.worker.block.SpecificMasterBlockSync.heartbeat(SpecificMasterBlockSync.java:207)
        at alluxio.heartbeat.HeartbeatThread.run(HeartbeatThread.java:128)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: io.grpc.StatusRuntimeException: INTERNAL: MediumType must be one of {MEM, HDD and SSD} but got
        at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:262)
        at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:243)
        at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:156)
        at alluxio.grpc.BlockMasterWorkerServiceGrpc$BlockMasterWorkerServiceBlockingStub.blockHeartbeat(BlockMasterWorkerServiceGrpc.java:611)
        at alluxio.worker.block.BlockMasterClient.lambda$heartbeat$4(BlockMasterClient.java:240)
        at alluxio.AbstractClient.retryRPCInternal(AbstractClient.java:484)
        ... 11 more

Does this PR introduce any user facing changes?

No.

dbw9580
dbw9580 commented Nov 7, 2023
View reviewed changes
core/server/worker/src/main/java/alluxio/worker/page/PagedBlockStore.java
@@ -389,15 +389,7 @@ public CacheManager.State getCacheManagerState() {
public void moveBlock(long sessionId, long blockId, AllocateOptions moveOptions)
throws IOException {
// TODO(bowen): implement actual move and replace placeholder values
int dirIndex = getDirIndexOfBlock(blockId);
BlockStoreLocation srcLocation = new BlockStoreLocation(DEFAULT_TIER, dirIndex);
BlockStoreLocation destLocation = moveOptions.getLocation();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the destination location is empty alluxio.worker.block.BlockStoreLocation#ANY_MEDIUM, and is used by the moveBlock operation as a special place holder. It's not supposed to be the actual block location.

@dbw9580 dbw9580 requested a review from beinan November 7, 2023 06:43
@dbw9580 dbw9580 added the type-bug This issue is about a bug label Nov 7, 2023
beinan
beinan approved these changes Nov 9, 2023
View reviewed changes
Copy link
Contributor

@beinan beinan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@beinan beinan merged commit b258340 into Alluxio:master-2.x Nov 9, 2023
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@beinan beinan beinan approved these changes

Assignees
No one assigned
Labels
type-bug This issue is about a bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dbw9580 @beinan