Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#84] Fix pinpoint-hbase connection configurations #85

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

cwJohnPark
Copy link

Fix pinpoint-hbase/docker-compose.yml and pinpoint-hbase/hbase-site.xml

The pinpoint-hbase container network should be on the pinpoint-zookeeper container bridge network.

hbase.regionserver.hostname should be localhost, hbase.regionserver.ipc.address and hbase.master.ipc.address should be 0.0.0.0 to connect region server in the docker container.

close #84
close #83

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


cwJohnPark seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

Copy link
Contributor

@RoySRose RoySRose left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please review my comments. Let me know what you think.

I've not yet decided if this should be separated into another folder for automated upload to DockerHub.
I need to check a few more things for this. I'll let you know.

@@ -15,20 +15,28 @@ services:
- /home/pinpoint/hbase
- /home/pinpoint/zookeeper
expose:
# # zookeeper
# - "2181"
# HBase Master API port
- "60000"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about also changing 60000 to 16000 and all the code below that is related (https://hbase.apache.org/book.html#hbase_default_configurations)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I think that is more related to the others.

Comment on lines +6 to +9
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if it's stand-alone mode. this should be removed

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is necessary, otherwise region server ipc port will be randomized whenever the hbase container runs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why does region server ipc port not be randomized?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pinpoint collector should connect to hbase region server ipc.
I found a similar problem on stackoverflow.

https://stackoverflow.com/questions/43524733/how-to-specify-rpc-server-port-for-hbase

Comment on lines +14 to +17
<property>
<name>hbase.zookeeper.quorum</name>
<value>zoo1,zoo2,zoo3</value>
</property>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if it's stand-alone mode. Is this even necessary? At least, zoo1 is not the zookeeper hbase needs to connect. It should be the internal one.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be obsolete, I just want to use the same yaml file in pinpoint-zookeeper directory.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are misunderstading the architecture of stand-alone hbase.

Comment on lines +30 to +41
<property>
<name>hbase.regionserver.hostname</name>
<value>localhost</value>
</property>
<property>
<name>hbase.regionserver.ipc.address</name>
<value>0.0.0.0</value>
</property>
<property>
<name>hbase.master.ipc.address</name>
<value>0.0.0.0</value>
</property>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary when the hbase is in stand-alone mode?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is.
In order to pinpoint collector and web module that run in host need to know ipc address to connect hbase in docker container.

I had a wrong choice on the word 'stand-alone'.
It is more related to docker container not the hbase itself.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you are using not referring to stand-alone hbase. What is this docker file for?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is for the one runs pinpoint collector and web module in IDE or host environment, and want to run hbase as container. because running with hbase binary is too troublesome.

Copy link
Contributor

@RoySRose RoySRose Sep 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cwJohnPark

I've understood your intentions. But with the below configuration, this will ONLY work IF HBase is on localhost.

<property>
        <name>hbase.regionserver.hostname</name>
        <value>localhost</value>
</property>

Actually there is a solution for below

This is for the one runs pinpoint collector and web module in IDE or host environment
should be taken care of by adding the hbase containers IP address in the host file.

I know adding a line in the host file may be annoying. But since the reason for this is due to the networking architecture of docker containers. I couldn't find any appropriate solution for this yet.
Please let me know what you think about this, and welcome if there is any better solution.

@RoySRose
Copy link
Contributor

RoySRose commented Sep 15, 2020

@cwJohnPark any updates? or explanation of what the new image is trying to achieve?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants