Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement ES daemon-mode in process launcher #701

Merged
merged 29 commits into from
Jun 20, 2019
Merged

Implement ES daemon-mode in process launcher #701

Changes from 4 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
a1d139a
WIP - Implement ES daemon-mode in process launcher
May 30, 2019
d924f65
Minor cleanup and code review fixes.
Jun 3, 2019
5b6ba4c
Replace logfile watcher with pidfile watcher
Jun 4, 2019
545fd42
Remove old StartupWatcher / subproc path
Jun 6, 2019
16f8555
Drop unneeded pyyaml dependency.
Jun 10, 2019
3bc1400
Bump up pidfile wait to 60s
Jun 10, 2019
05b761e
Remove unused _get_pid_from_file, wait_for_pidfile call
Jun 10, 2019
53ab793
Fix _start_process call.
Jun 11, 2019
3febdd8
Add env back in for now.
Jun 11, 2019
e2e936c
Add ,
Jun 11, 2019
e6a90a1
Use keyword-arg for env.
Jun 11, 2019
1d71917
Mock out process launcher unit test.
Jun 12, 2019
95f7127
Fix sysstat telemetry in mocked tests.
Jun 12, 2019
874dc16
Merge branch 'master' into es-as-daemon
Jun 12, 2019
d8d5b7d
Remove unused 'daemon' option.
Jun 12, 2019
72f30a0
Remove duplicate log line.
Jun 13, 2019
14dc6bd
Re-arrange conditions for cpu_model.
Jun 13, 2019
69cd57f
Add .j2 to docker-compose.yml, add healthcheck.
Jun 14, 2019
f08fc00
Remove unused _parse_log_ts.
Jun 14, 2019
0d98e4a
Get container ID, wait for it to show up in docker container list.
Jun 14, 2019
8a25645
Implement docker wait.
Jun 18, 2019
1e4561a
Use docker cli instead of SDK, use LaunchError instead of TimeoutError.
Jun 18, 2019
eb91a37
Change stop() to use _get_docker_compose_cmd.
Jun 18, 2019
58c9fcd
Change TimeoutError to LaunchError.
Jun 19, 2019
d500662
Remove self.node_name, replace with node.node_name.
Jun 20, 2019
6b84bf8
Re-order time import.
Jun 20, 2019
6c803ca
Comment to explain discrepancy between returncode and wait() return i…
Jun 20, 2019
d0457b8
Remove unused unit testing helper functions.
Jun 20, 2019
f03e8f4
Remove some more unused leftovers.
Jun 20, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 4 additions & 9 deletions esrally/mechanic/launcher.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -166,8 +166,7 @@ def start(self, node_configurations):

def _start_process(self, cmd):
subprocess.Popen(shlex.split(cmd), close_fds=True)
drawlerr marked this conversation as resolved.
Show resolved Hide resolved
#TODO: wait_for_docker_pidfile?
return wait_for_pidfile("./pid")
return 0

def stop(self, nodes):
if self.keep_running:
Expand Down Expand Up @@ -221,11 +220,6 @@ def stop(self, nodes):
pass


def _get_pid_from_file(filename):
with open(filename, "r") as f:
return int(f.read())


def wait_for_pidfile(pidfilename, timeout=60):
endtime = _time() + timeout
while _time() < endtime:
Expand Down Expand Up @@ -290,7 +284,7 @@ def _start_node(self, node_configuration, node_count_on_host):
t = telemetry.Telemetry(enabled_devices, devices=node_telemetry)
env = self._prepare_env(car, node_name, java_home, t)
t.on_pre_node_start(node_name)
node_pid = self._start_process(env, node_name, binary_path)
node_pid = self._start_process(binary_path, env)
node = cluster.Node(node_pid, host_name, node_name, t)
self.logger.info("Attaching telemetry devices to node [%s].", node_name)
t.attach_to_node(node)
Expand Down Expand Up @@ -327,13 +321,14 @@ def _set_env(self, env, k, v, separator=' '):
env[k] = v + separator + env[k]

@staticmethod
def _start_process(binary_path):
def _start_process(binary_path, env):
if os.geteuid() == 0:
raise exceptions.LaunchError("Cannot launch Elasticsearch as root. Please run Rally as a non-root user.")
os.chdir(binary_path)
cmd = ["bin/elasticsearch"]
cmd.extend(["-d", "-p", "pid"])
proc = subprocess.Popen(cmd,
drawlerr marked this conversation as resolved.
Show resolved Hide resolved
env,
drawlerr marked this conversation as resolved.
Show resolved Hide resolved
close_fds=True)
ret = proc.wait()
if ret != 0:
Expand Down