Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix model locale issue and improve model R/W performance. #3405

Merged
merged 33 commits into from
Dec 8, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
4bf22a9
Fix LightGBM models locale sensitivity and improve R/W performance.
AlbertoEAF Sep 11, 2020
538e102
Align CommonC namespace
AlbertoEAF Sep 24, 2020
bfcdb89
Add new external_libs/ to python setup
AlbertoEAF Sep 24, 2020
30ae0c0
Try fast_double_parser fix #1
AlbertoEAF Sep 29, 2020
3a97aba
CMake: Attempt to link fmt without explicit PUBLIC tag
AlbertoEAF Sep 30, 2020
447050e
Exclude external_libs from linting
AlbertoEAF Sep 30, 2020
37c0cd4
Add exernal_libs to MANIFEST.in
AlbertoEAF Oct 2, 2020
6dcc5d0
Set dynamic linking option for fmt.
AlbertoEAF Oct 2, 2020
c7c1977
linting issues
AlbertoEAF Oct 2, 2020
a206715
Try to fix lint includes
AlbertoEAF Oct 2, 2020
f8556ab
Try to pass fPIC with static fmt lib
AlbertoEAF Oct 2, 2020
18c6773
Try CMake P_I_C option with fmt library
AlbertoEAF Oct 2, 2020
8d6ec54
[R-package] Add CMake support for R and CRAN
jameslamb Oct 4, 2020
859013a
Cleanup CMakeLists
AlbertoEAF Oct 6, 2020
01a07a4
Try fmt hack to remove stdout
AlbertoEAF Oct 6, 2020
9ed5a39
Switch to header-only mode
AlbertoEAF Oct 17, 2020
658b00b
Add PRIVATE argument to target_link_libraries
AlbertoEAF Oct 18, 2020
91dd2d8
use fmt in header-only mode
jameslamb Nov 1, 2020
f90fa3b
Remove CMakeLists comment
AlbertoEAF Nov 1, 2020
d062736
Change OpenMP to PUBLIC linking in Mac
AlbertoEAF Nov 1, 2020
e3235fc
Update fmt submodule to 7.1.2
AlbertoEAF Nov 7, 2020
e4997bd
Use fmt in header-only-mode
AlbertoEAF Nov 7, 2020
5c16913
Remove fmt from CMakeLists.txt
AlbertoEAF Nov 7, 2020
03bd3b2
Upgrade fast_double_parser to v0.2.0
AlbertoEAF Nov 7, 2020
51843f7
Revert "Add PRIVATE argument to target_link_libraries"
AlbertoEAF Nov 10, 2020
90f8bd8
Address James Lamb's comments
AlbertoEAF Nov 14, 2020
eda4759
Update R-package/.Rbuildignore
AlbertoEAF Nov 14, 2020
082c1d4
Upgrade to fast_double_parser v0.3.0 - Solaris support
AlbertoEAF Nov 14, 2020
cc19daf
Use legacy code only in Solaris
AlbertoEAF Nov 16, 2020
69d0488
Fix lint issues
AlbertoEAF Nov 21, 2020
1411133
Fix comment
AlbertoEAF Nov 21, 2020
ba28c0f
Address StrikerRUS's comments (solaris ifdef).
AlbertoEAF Nov 23, 2020
7e17729
Change header guards
AlbertoEAF Nov 24, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .ci/test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,8 @@ if [[ $TRAVIS == "true" ]] && [[ $TASK == "lint" ]]; then
"r-lintr>=2.0"
pip install --user cpplint
echo "Linting Python code"
pycodestyle --ignore=E501,W503 --exclude=./compute,./.nuget . || exit -1
pydocstyle --convention=numpy --add-ignore=D105 --match-dir="^(?!^compute|test|example).*" --match="(?!^test_|setup).*\.py" . || exit -1
pycodestyle --ignore=E501,W503 --exclude=./compute,./.nuget,./external_libs . || exit -1
pydocstyle --convention=numpy --add-ignore=D105 --match-dir="^(?!^compute|external_libs|test|example).*" --match="(?!^test_|setup).*\.py" . || exit -1
echo "Linting R code"
Rscript ${BUILD_DIRECTORY}/.ci/lint_r_code.R ${BUILD_DIRECTORY} || exit -1
echo "Linting C++ code"
Expand Down
6 changes: 6 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
[submodule "include/boost/compute"]
path = compute
url = https://github.com/boostorg/compute
[submodule "external_libs/fmt"]
path = external_libs/fmt
url = https://github.com/fmtlib/fmt.git
[submodule "external_libs/fast_double_parser"]
path = external_libs/fast_double_parser
url = https://github.com/lemire/fast_double_parser.git
guolinke marked this conversation as resolved.
Show resolved Hide resolved
23 changes: 23 additions & 0 deletions R-package/.Rbuildignore
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
\.appveyor\.yml
AUTOCONF_UBUNTU_VERSION
^autom4te.cache/.*$
^.*\.bin
^build_r.R$
\.clang-format
^cran-comments\.md$
^docs$
^.*\.dll
\.drone\.yml
\.git
\.gitkeep$
^.*\.history
^Makefile$
Expand All @@ -24,3 +28,22 @@ AUTOCONF_UBUNTU_VERSION
^src/compute/.gitignore$
^src/compute/CONTRIBUTING.md$
^src/compute/README.md$
src/external_libs/fast_double_parser/benchmarks
src/external_libs/fast_double_parser/Makefile
src/external_libs/fast_double_parser/.*\.md
src/external_libs/fast_double_parser/tests
AlbertoEAF marked this conversation as resolved.
Show resolved Hide resolved
src/external_libs/fast_double_parser/.*\.yaml
src/external_libs/fast_double_parser/.*\.yml
src/external_libs/fmt/.*\.md
src/external_libs/fmt/doc
src/external_libs/fmt/support/Android\.mk
src/external_libs/fmt/support/.*\.gradle
src/external_libs/fmt/support/.*\.pro
src/external_libs/fmt/support/.*\.py
src/external_libs/fmt/support/rtd
src/external_libs/fmt/support/.*sublime-syntax
src/external_libs/fmt/support/Vagrantfile
src/external_libs/fmt/support/.*\.xml
src/external_libs/fmt/support/.*\.yml
src/external_libs/fmt/test
\.travis\.yml
3 changes: 3 additions & 0 deletions R-package/DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ Authors@R: c(
person("Jay", "Loden", role = c("cph")),
person("Dave", "Daeschler", role = c("cph")),
person("Giampaolo", "Rodola", role = c("cph")),
person("Alberto", "Ferreira", role = c("ctb")),
person("Daniel", "Lemire", role = c("ctb")),
person("Victor", "Zverovich", role = c("cph")),
person("IBM Corporation", role = c("ctb"))
)
Description: Tree based algorithms can be improved by introducing boosting frameworks.
Expand Down
1 change: 1 addition & 0 deletions R-package/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,7 @@ For more information on this approach, see ["Writing R Extensions"](https://cran
From the root of the repository, run the following.

```shell
git submodule update --init --recursive
sh build-cran-package.sh
```

Expand Down
19 changes: 19 additions & 0 deletions build-cran-package.sh
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,15 @@ cp -R R-package/* ${TEMP_R_DIR}
cp -R include ${TEMP_R_DIR}/src/
cp -R src/* ${TEMP_R_DIR}/src/

cp \
external_libs/fast_double_parser/include/fast_double_parser.h \
${TEMP_R_DIR}/src/include/LightGBM

mkdir -p ${TEMP_R_DIR}/src/include/LightGBM/fmt
cp \
external_libs/fmt/include/fmt/*.h \
${TEMP_R_DIR}/src/include/LightGBM/fmt/

cd ${TEMP_R_DIR}

# Remove files not needed for CRAN
Expand Down Expand Up @@ -67,6 +76,16 @@ cd ${TEMP_R_DIR}
done
find . -name '*.h.bak' -o -name '*.hpp.bak' -o -name '*.cpp.bak' -exec rm {} \;

sed \
-i.bak \
-e 's/\.\..*fmt\/format\.h/LightGBM\/fmt\/format\.h/' \
src/include/LightGBM/utils/common.h

sed \
-i.bak \
-e 's/\.\..*fast_double_parser\.h/LightGBM\/fast_double_parser\.h/' \
src/include/LightGBM/utils/common.h

# When building an R package with 'configure', it seems
# you're guaranteed to get a shared library called
# <packagename>.so/dll. The package source code expects
Expand Down
11 changes: 11 additions & 0 deletions build_r.R
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,17 @@ result <- file.remove(
)
.handle_result(result)

#------------#
# submodules #
#------------#
result <- file.copy(
from = "external_libs/"
, to = sprintf("%s/", TEMP_SOURCE_DIR)
, recursive = TRUE
, overwrite = TRUE
)
.handle_result(result)

# copy files into the place CMake expects
for (src_file in c("lightgbm_R.cpp", "lightgbm_R.h", "R_object_helper.h")) {
result <- file.copy(
Expand Down
1 change: 1 addition & 0 deletions external_libs/fast_double_parser
Submodule fast_double_parser added at ace606
1 change: 1 addition & 0 deletions external_libs/fmt
Submodule fmt added at cc09f1
Loading