Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set dynamic memory limit in pdfalto_server #1038

Merged
merged 2 commits into from
Aug 8, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,8 @@ public File pdfalto(Integer timeout, boolean force, int startPage,
if (GrobidProperties.isContextExecutionServer()) {
cmd.add("--timeout");
cmd.add(String.valueOf(GrobidProperties.getPdfaltoTimeoutS()));
cmd.add("--ulimit");
cmd.add(String.valueOf(GrobidProperties.getPdfaltoMemoryLimitMb() * 1024));
tmpPathXML = processPdfaltoServerMode(pdfPath, tmpPathXML, cmd);
} else {
if (!SystemUtils.IS_OS_WINDOWS && !SystemUtils.IS_OS_MAC) {
Expand Down
14 changes: 12 additions & 2 deletions grobid-home/pdfalto/lin-64/pdfalto_server
Original file line number Diff line number Diff line change
Expand Up @@ -6,26 +6,36 @@ timeout=20 # 20 seconds
interval=1
# Delay between posting the SIGTERM signal and destroying the process by SIGKILL.
delay=0
# Default limit for the memory usage by this bash script
memory_limit=6242304

command=${0:0:${#0}-7}
args=("$@")
pdfalto_params=()

for ((n=0; n<="$#";n++)); do
case ${args[n]} in
case ${args[n]} in
--timeout)
timeout=${args[n+1]}
((n++))
;;
--ulimit)
memory_limit=${args[n+1]}
((n++))
;;
*)
pdfalto_params+=" ${args[n]}"
;;
esac
esac
done

#echo timeout $timeout
#echo pdfalto commands: $command $pdfalto_params

#echo memory_limit $memory_limit
# Limit the memory usage by this bash script
ulimit -Sv $memory_limit

# kill -0 pid Exit code indicates if a signal may be sent to $pid process.
(
((t = timeout))
Expand Down
11 changes: 10 additions & 1 deletion grobid-home/pdfalto/mac-64/pdfalto_server
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,10 @@ timeout=20 # 20 seconds
interval=1
# Delay between posting the SIGTERM signal and destroying the process by SIGKILL.
delay=0
command=${0:0:${#0}-7}
# Default limit for the memory usage by this bash script
memory_limit=6242304

command=${0:0:${#0}-7}
args=("$@")
pdfalto_params=()

Expand All @@ -17,12 +19,19 @@ for ((n=0; n<="$#";n++)); do
timeout=${args[n+1]}
((n++))
;;
--ulimit)
memory_limit=${args[n+1]}
((n++))
;;
*)
pdfalto_params+=" ${args[n]}"
;;
esac
done

# Limit the memory usage by this bash script
ulimit -Sv $memory_limit

# kill -0 pid Exit code indicates if a signal may be sent to $pid process.
(
((t = timeout))
Expand Down
Loading