feat: Improve shutdown Delay default

When a baseplate server is shutting down in k8s * baseplate calls func (srv *Server) Shutdown(ctx context.Context) error which does the following: * Shutdown works by first closing all open listeners * k8s is attempting to remove the endpoint for the endpoints group and thus make (new) incoming traffic to the pod 0 This is a race condition because some callers are still trying to connect. We have empirically seen that the 5 second default is good enough for most services. See also reddit/baseplate.go#615 Signed-off-by: Sotiris Nanopoulos <[email protected]>
davinci26 · May 5, 2023 · f44e463 · f44e463
1 parent a278040
commit f44e463
Showing 1 changed file with 5 additions and 2 deletions.
diff --git a/baseplate/server/__init__.py b/baseplate/server/__init__.py
@@ -297,9 +297,12 @@ def load_app_and_run_server() -> None:
         SERVER_STATE.state = ServerLifecycle.SHUTTING_DOWN
 
         cfg = parse_config(config.server, {"drain_time": OptionalConfig(Timespan)})
+        # Default drain time across all baseplate apps
+        drain_time_seconds = 5
         if cfg.drain_time:
-            logger.debug("Draining inbound requests...")
-            time.sleep(cfg.drain_time.total_seconds())
+            drain_time_seconds = cfg.drain_time.total_seconds()
+        logger.debug("Draining inbound requests...")
+        time.sleep(drain_time_seconds)
     finally:
         logger.debug("Gracefully shutting down...")
         server.stop()