Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metrics id resolver should be able to resolve metrics in bulk. #1

Open
octo47 opened this issue Apr 16, 2013 · 0 comments
Open

Metrics id resolver should be able to resolve metrics in bulk. #1

octo47 opened this issue Apr 16, 2013 · 0 comments

Comments

@octo47
Copy link
Owner

octo47 commented Apr 16, 2013

We are migrating our old system and due of its architecture we have a substantial rate of metrics variations. After daemon started cache filled with gets to 'tsdb-uid' table with followed join() on Deferred. This leads to syncronous requests to hbase. In case of our rate (tens of thousands) we doing slow cache filling.
Cache in opentsdb should be able to resolve ids in a bulk.

All OpenTSDBSink threads looks like:

Thread 12136: (state = BLOCKED)
 - java.lang.Object.wait(long) @bci=0 (Interpreted frame)
 - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame)
 - com.stumbleupon.async.Deferred.doJoin(boolean, long) @bci=77, line=1122 (Interpreted frame)
 - com.stumbleupon.async.Deferred.joinUninterruptibly() @bci=3, line=1048 (Interpreted frame)
 - net.opentsdb.uid.UniqueId.hbaseGet(byte[], byte[], org.hbase.async.RowLock) @bci=48, line=585 (Interpreted frame)
 - net.opentsdb.uid.UniqueId.hbaseGet(byte[], byte[]) @bci=4, line=573 (Interpreted frame)
 - net.opentsdb.uid.UniqueId.getIdFromHBase(java.lang.String) @bci=8, line=209 (Interpreted frame)
 - net.opentsdb.uid.UniqueId.getId(java.lang.String) @bci=35, line=189 (Interpreted frame)
 - net.opentsdb.uid.UniqueId.getOrCreateId(java.lang.String) @bci=15, line=234 (Interpreted frame)
 - net.opentsdb.core.IncomingDataPoints.rowKeyTemplate(net.opentsdb.core.TSDB, java.lang.String, java.util.Map) @bci=76, line=124 (Interpreted frame)
 - net.opentsdb.core.IncomingDataPoints.setSeries(java.lang.String, java.util.Map) @bci=12, line=139 (Interpreted frame)
 - ru.yandex.opentsdb.flume.OpenTSDBSink$State.writeDataPoints(java.util.List) @bci=42, line=103 (Interpreted frame)
 - ru.yandex.opentsdb.flume.OpenTSDBSink$State.access$700(ru.yandex.opentsdb.flume.OpenTSDBSink$State, java.util.List) @bci=2, line=73 (Interpreted frame)
 - ru.yandex.opentsdb.flume.OpenTSDBSink$1.call() @bci=238, line=368 (Interpreted frame)
 - ru.yandex.opentsdb.flume.OpenTSDBSink$1.call() @bci=1, line=332 (Interpreted frame)
 - java.util.concurrent.FutureTask$Sync.innerRun() @bci=30, line=303 (Interpreted frame)
 - java.util.concurrent.FutureTask.run() @bci=4, line=138 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.runTask(java.lang.Runnable) @bci=59, line=886 (Interpreted frame)
 - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=28, line=908 (Interpreted frame)
 - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant