Skip to content

Commit

Permalink
Merge branch 'unstable' into fix_bitmap_cache
Browse files Browse the repository at this point in the history
  • Loading branch information
@chejinge
chejinge authored Jan 23, 2024
2 parents de921cd + 1d71542 commit 95ec2fb
Show file tree
Hide file tree
Showing 9 changed files with 178 additions and 166 deletions.
4 changes: 2 additions & 2 deletions README_CN.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

## Pika简介 [English](https://github.com/Qihoo360/pika/blob/master/README.md)

Pika 是一个以 RocksDB 为存储引擎的的大容量、高性能、多租户、数据可持久化的弹性 KV 数据存储系统，完全兼容 Redis 协议，支持其常用的数据结构，如 string/hash/list/zset/set/geo/hyperloglog/pubsub/bitmap/stream 等 [Redis 接口](https://github.com/OpenAtomFoundation/pika/wiki/pika-%E6%94%AF%E6%8C%81%E7%9A%84redis%E6%8E%A5%E5%8F%A3%E5%8F%8A%E5%85%BC%E5%AE%B9%E6%83%85%E5%86%B5)。
Pika 是一个以 RocksDB 为存储引擎的的大容量、高性能、多租户、数据可持久化的弹性 KV 数据存储系统，完全兼容 Redis 协议，支持其常用的数据结构，如 string/hash/list/zset/set/geo/hyperloglog/pubsub/bitmap/stream 等 [Redis 接口](https://github.com/OpenAtomFoundation/pika/wiki/pika-%E6%94%AF%E6%8C%81%E7%9A%84redis%E6%8E%A5%E5%8F%A3%E5%8F%8A%E5%85%BC%E5%AE%B9%E6%83%85%E5%86%B5)。

Redis 的内存使用了超过 16GiB 时，会面临内存容量有限、单线程阻塞、启动恢复时间长、内存硬件成本贵、缓冲区容易写满、一主多从故障时切换代价大等问题。Pika 的出现并不是为了替代 Redis, 而是 Redis 补充。Pika 力求在完全兼容Redis 协议、继承 Redis 便捷运维设计的前提下，通过持久化存储的方式解决了 Redis 一旦存储数据量巨大就会出现内存容量不足的瓶颈问题，并且可以像 Redis 一样，支持使用 slaveof 命令实现主从模式，还支持数据的全量同步和增量同步。
Redis 的内存使用量超过一定阈值【如 16GiB 】时，会面临内存容量有限、单线程阻塞、启动恢复时间长、内存硬件成本贵、缓冲区容易写满、一主多从故障时切换代价大等问题。Pika 的出现并不是为了替代 Redis, 而是 Redis 补充。Pika 力求在完全兼容Redis 协议、继承 Redis 便捷运维设计的前提下，通过持久化存储的方式解决了 Redis 一旦存储数据量巨大就会出现内存容量不足的瓶颈问题，并且可以像 Redis 一样，支持使用 slaveof 命令实现主从模式，还支持数据的全量同步和增量同步。

还可以通过 twemproxy or [Codis](https://github.com/OpenAtomFoundation/pika/tree/unstable/cluster) 以静态数据分片方式实现 Pika 集群。

Expand Down
12 changes: 5 additions & 7 deletions codis/cmd/fe/assets/dashboard-fe.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -456,7 +456,7 @@ function isValidInput(text) {
function processGroupStats(codis_stats) {
var group_array = codis_stats.group.models;
var group_stats = codis_stats.group.stats;
var keys = 0, memory = 0;
var keys = 0, memory = 0, pika_memory = 0;
var dbkeyRegexp = /db\d+/
for (var i = 0; i < group_array.length; i++) {
var g = group_array[i];
Expand Down Expand Up @@ -504,7 +504,7 @@ function processGroupStats(codis_stats) {
if (s.stats["used_memory"]) {
var v = parseInt(s.stats["used_memory"], 10);
if (j == 0) {
memory += v;
pika_memory += v;
}
x.memory = humanSize(v);
}
Expand Down Expand Up @@ -552,7 +552,7 @@ function processGroupStats(codis_stats) {
x.server_text = x.server;
}
}
return {group_array: group_array, keys: keys, memory: memory};
return {group_array: group_array, keys: keys, pika_memory: pika_memory};
}

dashboard.config(['$interpolateProvider',
Expand Down Expand Up @@ -588,8 +588,7 @@ dashboard.controller('MainCodisCtrl', ['$scope', '$http', '$uibModal', '$timeout
$scope.codis_coord_addr = "NA";
$scope.codis_qps = "NA";
$scope.codis_sessions = "NA";
$scope.redis_mem = "NA";
$scope.redis_keys = "NA";
$scope.pika_mem = "NA";
$scope.slots_array = [];
$scope.proxy_array = [];
$scope.group_array = [];
Expand Down Expand Up @@ -662,8 +661,7 @@ dashboard.controller('MainCodisCtrl', ['$scope', '$http', '$uibModal', '$timeout

$scope.codis_qps = proxy_stats.qps;
$scope.codis_sessions = proxy_stats.sessions;
$scope.redis_mem = humanSize(group_stats.memory);
$scope.redis_keys = group_stats.keys.toString().replace(/\B(?=(\d{3})+(?!\d))/g, ",");
$scope.pika_mem = humanSize(group_stats.pika_memory);
$scope.slots_array = merge($scope.slots_array, codis_stats.slots);
$scope.proxy_array = merge($scope.proxy_array, proxy_stats.proxy_array);
$scope.group_array = merge($scope.group_array, group_stats.group_array);
Expand Down
8 changes: 2 additions & 6 deletions codis/cmd/fe/assets/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -63,12 +63,8 @@ <h4 style="padding-left:20px;"><b>Overview</b></h4>
<td>[[codis_sessions]]</td>
</tr>
<tr>
<td>Redis Memory</td>
<td>[[redis_mem]]</td>
</tr>
<tr>
<td>Redis Keys</td>
<td>[[redis_keys]]</td>
<td>Pika Memory</td>
<td>[[pika_mem]]</td>
</tr>
<tr>
<td>Slot Nums</td>
Expand Down
1 change: 1 addition & 0 deletions codis/config/proxy.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -128,3 +128,4 @@ metrics_report_statsd_prefix = ""

# Maximum delay statistical time interval.(This value must be greater than 0.)
max_delay_refresh_time_interval = "15s"

2 changes: 1 addition & 1 deletion include/pika_admin.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -342,7 +342,7 @@ class ConfigCmd : public Cmd {
std::vector<std::string> config_args_v_;
void DoInitial() override;
void ConfigGet(std::string& ret);
void ConfigSet(std::string& ret, std::shared_ptr<DB> db);
void ConfigSet(std::shared_ptr<DB> db);
void ConfigRewrite(std::string& ret);
void ConfigResetstat(std::string& ret);
void ConfigRewriteReplicationID(std::string& ret);
Expand Down
2 changes: 1 addition & 1 deletion include/pika_command.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -441,7 +441,7 @@ class CmdRes {

void AppendStringVector(const std::vector<std::string>& strArray) {
if (strArray.empty()) {
AppendArrayLen(-1);
AppendArrayLen(0);
return;
}
AppendArrayLen(strArray.size());
Expand Down
Loading

0 comments on commit 95ec2fb

Please sign in to comment.