Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: dlopen error, install udf dir and doc #3326

Merged
merged 7 commits into from
Jun 27, 2023
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -328,3 +328,4 @@ install(
DESTINATION tools
PERMISSIONS OWNER_EXECUTE OWNER_WRITE OWNER_READ GROUP_EXECUTE GROUP_READ WORLD_EXECUTE WORLD_READ
)
install(DIRECTORY DESTINATION udf)
12 changes: 11 additions & 1 deletion docs/zh/deploy/install_deploy.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,17 @@

### 操作系统

发行的预编译包支持:CentOS 7.x, Ubuntu 20.04, SUSE 12 SP3, macOS 12。对于其他操作系统发行版本,预编译包未做充分测试,无法保证其完全兼容。你可以尝试 [从源码编译](compile.md),来支持其他的操作系统。
发行的预编译包支持:CentOS 7.x, Ubuntu 20.04, SUSE 12 SP3, macOS 12。因此Linux要求glibc version >= 2.17。对于其他操作系统发行版本,预编译包未做充分测试,无法保证其完全兼容。你可以尝试 [从源码编译](compile.md),来支持其他的操作系统。

```{note}
Linux可通过以下命令来检查系统的支持情况。
````shell
cat /etc/redhat-release # redhat only
vagetablechicken marked this conversation as resolved.
Show resolved Hide resolved
ldd --version
strings /lib64/libc.so.6 | grep ^GLIBC_
````
通常ldd版本>=2.17,libc.so.6中也会有`GLIBC_2.17`,也就是该系统支持glibc 2.17的程序/动态库运行。如果系统的glibc版本低于2.17,则需要尝试从源码编译。
```

### 第三方组件依赖

Expand Down
6 changes: 5 additions & 1 deletion docs/zh/openmldb_sql/ddl/CREATE_FUNCTION.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,4 +53,8 @@ CREATE AGGREGATE FUNCTION special_sum(x BIGINT) RETURNS BIGINT OPTIONS (FILE = '
创建一个输入参数是bigint并且支持null, 返回类型是bigint并且支持返回null聚合函数,动态库文件为libtest_udf.so
```sql
CREATE AGGREGATE FUNCTION count_null(x BIGINT) RETURNS BIGINT OPTIONS (FILE = 'libtest_udf.so', ARG_NULLABLE=true, RETURN_NULLABLE=true);
```
```

```{warning}
如果集群版本<=0.8.0,TabletServer部署目录中默认没有`udf`目录,启动TabletServer后再创建目录并拷贝udf动态库,将无法生效(环境变量问题)。需保证TabletServer启动前存在udf目录,如果已启动,需要将所有TabletServer重启使环境变量生效。
```
2 changes: 1 addition & 1 deletion docs/zh/openmldb_sql/udf_develop_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ g++ -shared -o libtest_udf.so examples/test_udf.cc -I /work/OpenMLDB/include -st
```

### 2.3 拷贝动态库
编译过的动态库需要被拷贝到 TaskManager 和 tablets中。如果 TaskManager 和 tablets中不存在`udf`目录,请先创建
编译过的动态库需要被拷贝到 TaskManager 和 tablets中。如果 TaskManager 和 tablets中不存在`udf`目录,请先创建并重启这些进程(保证环境变量生效)
- tablet的UDF目录是 `path_to_tablet/udf`。
- TaskManager的UDF目录是 `path_to_taskmanager/taskmanager/bin/udf`。

Expand Down
10 changes: 5 additions & 5 deletions hybridse/src/udf/dynamic_lib_manager.cc
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ DynamicLibManager::~DynamicLibManager() {
handle_map_.clear();
}

base::Status DynamicLibManager::ExtractFunction(const std::string& name, bool is_aggregate,
const std::string& file, std::vector<void*>* funs) {
base::Status DynamicLibManager::ExtractFunction(const std::string& name, bool is_aggregate, const std::string& file,
std::vector<void*>* funs) {
CHECK_TRUE(funs != nullptr, common::kExternalUDFError, "funs is nullptr")
std::shared_ptr<DynamicLibHandle> so_handle;
{
Expand All @@ -45,7 +45,8 @@ base::Status DynamicLibManager::ExtractFunction(const std::string& name, bool is
}
if (!so_handle) {
void* handle = dlopen(file.c_str(), RTLD_LAZY);
CHECK_TRUE(handle != nullptr, common::kExternalUDFError, "can not open the dynamic library: " + file)
CHECK_TRUE(handle != nullptr, common::kExternalUDFError,
"can not open the dynamic library: " + file + ", error: " + dlerror())
so_handle = std::make_shared<DynamicLibHandle>(handle);
std::lock_guard<std::mutex> lock(mu_);
handle_map_.emplace(file, so_handle);
Expand Down Expand Up @@ -96,8 +97,7 @@ base::Status DynamicLibManager::RemoveHandler(const std::string& file) {
}
}
if (so_handle) {
CHECK_TRUE(dlclose(so_handle->handle) == 0, common::kExternalUDFError,
"dlclose run error. file is " + file)
CHECK_TRUE(dlclose(so_handle->handle) == 0, common::kExternalUDFError, "dlclose run error. file is " + file)
}
return {};
}
Expand Down