Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HiveStreamingDataSource format writes string data in binary form in Hive Table whereas CLI writes string #270

Open
sinhashreesh opened this issue Jun 7, 2019 · 0 comments

Comments

@sinhashreesh
Copy link

sinhashreesh commented Jun 7, 2019

I am trying to write from Kafka Source to Hive Target.
I am using "com.hortonworks.spark.sql.hive.llap.streaming.HiveStreamingDataSource" format to write data into Hive table.
Table is ORC format and fully transactional. I am using HDP 3.1 cluster.

The column data type is binary.
When I write from CLI, the data can be seen in String.
While doing the same using "com.hortonworks.spark.sql.hive.llap.streaming.HiveStreamingDataSource" format, the data is being inserted in binary form in the Hive Table.

+----------------------+
| hive_binary.co |
+----------------------+
| ShreeshData1 | -->when inserted from CLI
| [B@1ebd3260 | -->when writing from spark using "com.hortonworks.spark.sql.hive.llap.streaming.HiveStreamingDataSource" format
+----------------------+

Is it the bug or is it the expected behavior?

Attaching Standalone spark program
standAlone.txt

Create Table Command : create table hive_binary (co binary);

Table Properties :
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant