Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix row/map block builder for hive thrift reader #119

Merged

Conversation

Yaliang
Copy link
Collaborator

@Yaliang Yaliang commented Oct 31, 2017

Fix the thrift reader row/map block builder issue

SELECT * FROM datapipeline.logs_mediaservices_media_upload LIMIT 2;

ava.lang.UnsupportedOperationException: com.facebook.presto.spi.block.RowBlockBuilder
	at com.facebook.presto.spi.block.BlockBuilder.writeBytes(BlockBuilder.java:58)
	at com.facebook.presto.spi.type.VarcharType.writeSlice(VarcharType.java:166)
	at com.facebook.presto.spi.type.VarcharType.writeSlice(VarcharType.java:160)
	at com.facebook.presto.twitter.hive.thrift.ThriftHiveRecordCursor.serializePrimitive(ThriftHiveRecordCursor.java:691)
	at com.facebook.presto.twitter.hive.thrift.ThriftHiveRecordCursor.serializeObject(ThriftHiveRecordCursor.java:553)
	at com.facebook.presto.twitter.hive.thrift.ThriftHiveRecordCursor.serializeStruct(ThriftHiveRecordCursor.java:650)
	at com.facebook.presto.twitter.hive.thrift.ThriftHiveRecordCursor.serializeObject(ThriftHiveRecordCursor.java:563)
	at com.facebook.presto.twitter.hive.thrift.ThriftHiveRecordCursor.getBlockObject(ThriftHiveRecordCursor.java:543)
	at com.facebook.presto.twitter.hive.thrift.ThriftHiveRecordCursor.parseObjectColumn(ThriftHiveRecordCursor.java:451)
	at com.facebook.presto.twitter.hive.thrift.ThriftHiveRecordCursor.parseColumn(ThriftHiveRecordCursor.java:498)
	at com.facebook.presto.twitter.hive.thrift.ThriftHiveRecordCursor.isNull(ThriftHiveRecordCursor.java:462)
	at com.facebook.presto.hive.HiveCoercionRecordCursor.isNull(HiveCoercionRecordCursor.java:143)
	at com.facebook.presto.hive.HiveRecordCursor.isNull(HiveRecordCursor.java:231)
	at com.facebook.presto.spi.RecordPageSource.getNextPage(RecordPageSource.java:101)
	at com.facebook.presto.operator.TableScanOperator.getOutput(TableScanOperator.java:262)
	at com.facebook.presto.operator.Driver.processInternal(Driver.java:337)
	at com.facebook.presto.operator.Driver.lambda$processFor$6(Driver.java:241)
	at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:614)
	at com.facebook.presto.operator.Driver.processFor(Driver.java:235)
	at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:622)
	at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:163)
	at com.facebook.presto.execution.executor.LegacyPrioritizedSplitRunner.process(LegacyPrioritizedSplitRunner.java:23)
	at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:485)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
SELECT inputfeatures FROM datapipeline.logs_spam_botmaker LIMIT 10;

java.lang.UnsupportedOperationException: com.facebook.presto.spi.block.MapBlockBuilder
	at com.facebook.presto.spi.block.BlockBuilder.writeBytes(BlockBuilder.java:58)
	at com.facebook.presto.spi.type.VarcharType.writeSlice(VarcharType.java:166)
	at com.facebook.presto.spi.type.VarcharType.writeSlice(VarcharType.java:160)
	at com.facebook.presto.twitter.hive.thrift.ThriftHiveRecordCursor.serializePrimitive(ThriftHiveRecordCursor.java:691)
	at com.facebook.presto.twitter.hive.thrift.ThriftHiveRecordCursor.serializeObject(ThriftHiveRecordCursor.java:553)
	at com.facebook.presto.twitter.hive.thrift.ThriftHiveRecordCursor.serializeMap(ThriftHiveRecordCursor.java:617)
	at com.facebook.presto.twitter.hive.thrift.ThriftHiveRecordCursor.serializeObject(ThriftHiveRecordCursor.java:560)
	at com.facebook.presto.twitter.hive.thrift.ThriftHiveRecordCursor.getBlockObject(ThriftHiveRecordCursor.java:543)
	at com.facebook.presto.twitter.hive.thrift.ThriftHiveRecordCursor.parseObjectColumn(ThriftHiveRecordCursor.java:451)
	at com.facebook.presto.twitter.hive.thrift.ThriftHiveRecordCursor.parseColumn(ThriftHiveRecordCursor.java:498)
	at com.facebook.presto.twitter.hive.thrift.ThriftHiveRecordCursor.isNull(ThriftHiveRecordCursor.java:462)
	at com.facebook.presto.hive.HiveCoercionRecordCursor.isNull(HiveCoercionRecordCursor.java:143)
	at com.facebook.presto.hive.HiveRecordCursor.isNull(HiveRecordCursor.java:231)
	at com.facebook.presto.spi.RecordPageSource.getNextPage(RecordPageSource.java:101)
	at com.facebook.presto.operator.TableScanOperator.getOutput(TableScanOperator.java:262)
	at com.facebook.presto.operator.Driver.processInternal(Driver.java:337)
	at com.facebook.presto.operator.Driver.lambda$processFor$6(Driver.java:241)
	at com.facebook.presto.operator.Driver.tryWithLock(Driver.java:614)
	at com.facebook.presto.operator.Driver.processFor(Driver.java:235)
	at com.facebook.presto.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:622)
	at com.facebook.presto.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:163)
	at com.facebook.presto.execution.executor.LegacyPrioritizedSplitRunner.process(LegacyPrioritizedSplitRunner.java:23)
	at com.facebook.presto.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:485)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)


@Yaliang Yaliang requested a review from dabaitu October 31, 2017 22:16
@Yaliang Yaliang merged commit 611a7ba into twitter-forks:twitter-master Oct 31, 2017
@xiaoyao1991
Copy link

noice! this saves my day!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants