fix: backtracking on adding docker/.env-local for docker-compose

As reported here #28026, in #27953 I added support for a supplemental `docker/.env-local` override file, not thinking it was going to be required upon starting docker-compose. I thought I had tested that while switching branches but somehow didn't test that docker-compose would simply boot up in the absence of that file. Since this is confusing to most, and we don't really have a script I can latch onto to `touch docker/.env-local` for now, i decided to backtrack and simply remove the convenience "feature". People should find an alternate way to alter their environment somehow before starting the service. I added a bit of documentation to point to an existing mechanism around providing a `docker/pythonpath_dev/superset_config_docker.py` file, which existed in the logic but wasn't clearly documented. This solves for my use case where I was working a PR adding features around Slack, where I wanted to provide an API key configuration without risks of commiting/pushing it back to the repo.
apache · Apr 15, 2024 · 9b56f6f · 9b56f6f
1 parent 2a62c40
commit 9b56f6f
Show file tree

Hide file tree

Showing 5 changed files with 12 additions and 56 deletions.
diff --git a/.gitignore b/.gitignore
@@ -110,7 +110,6 @@ release.json
 messages.mo
 
 docker/requirements-local.txt
-docker/.env-local
 
 cache/
 docker/*local*
diff --git a/docker-compose-image-tag.yml b/docker-compose-image-tag.yml
@@ -34,10 +34,7 @@ services:
 
   db:
     env_file:
-      # defaults
       - docker/.env
-      # gitignored overrides
-      - docker/.env-local
     image: postgres:15
     container_name: superset_db
     restart: unless-stopped
@@ -47,10 +44,7 @@ services:
 
   superset:
     env_file:
-      # defaults
       - docker/.env
-      # gitignored overrides
-      - docker/.env-local
     image: *superset-image
     container_name: superset_app
     command: ["/app/docker/docker-bootstrap.sh", "app-gunicorn"]
@@ -66,10 +60,7 @@ services:
     container_name: superset_init
     command: ["/app/docker/docker-init.sh"]
     env_file:
-      # defaults
       - docker/.env
-      # gitignored overrides
-      - docker/.env-local
     depends_on: *superset-depends-on
     user: "root"
     volumes: *superset-volumes
@@ -81,10 +72,7 @@ services:
     container_name: superset_worker
     command: ["/app/docker/docker-bootstrap.sh", "worker"]
     env_file:
-      # defaults
       - docker/.env
-      # gitignored overrides
-      - docker/.env-local
     restart: unless-stopped
     depends_on: *superset-depends-on
     user: "root"
@@ -101,10 +89,7 @@ services:
     container_name: superset_worker_beat
     command: ["/app/docker/docker-bootstrap.sh", "beat"]
     env_file:
-      # defaults
       - docker/.env
-      # gitignored overrides
-      - docker/.env-local
     restart: unless-stopped
     depends_on: *superset-depends-on
     user: "root"

diff --git a/docker-compose-non-dev.yml b/docker-compose-non-dev.yml
@@ -39,10 +39,7 @@ services:
 
   db:
     env_file:
-      # defaults
       - docker/.env
-      # gitignored overrides
-      - docker/.env-local
     image: postgres:15
     container_name: superset_db
     restart: unless-stopped
@@ -52,10 +49,7 @@ services:
 
   superset:
     env_file:
-      # defaults
       - docker/.env
-      # gitignored overrides
-      - docker/.env-local
     build:
       <<: *common-build
     container_name: superset_app
@@ -73,10 +67,7 @@ services:
       <<: *common-build
     command: ["/app/docker/docker-init.sh"]
     env_file:
-      # defaults
       - docker/.env
-      # gitignored overrides
-      - docker/.env-local
     depends_on: *superset-depends-on
     user: "root"
     volumes: *superset-volumes
@@ -89,10 +80,7 @@ services:
     container_name: superset_worker
     command: ["/app/docker/docker-bootstrap.sh", "worker"]
     env_file:
-      # defaults
       - docker/.env
-      # gitignored overrides
-      - docker/.env-local
     restart: unless-stopped
     depends_on: *superset-depends-on
     user: "root"
@@ -110,10 +98,7 @@ services:
     container_name: superset_worker_beat
     command: ["/app/docker/docker-bootstrap.sh", "beat"]
     env_file:
-      # defaults
       - docker/.env
-      # gitignored overrides
-      - docker/.env-local
     restart: unless-stopped
     depends_on: *superset-depends-on
     user: "root"

diff --git a/docker-compose.yml b/docker-compose.yml
@@ -55,10 +55,7 @@ services:
 
   db:
     env_file:
-      # defaults
       - docker/.env
-      # gitignored overrides
-      - docker/.env-local
     image: postgres:15
     container_name: superset_db
     restart: unless-stopped
@@ -70,10 +67,7 @@ services:
 
   superset:
     env_file:
-      # defaults
       - docker/.env
-      # gitignored overrides
-      - docker/.env-local
     build:
       <<: *common-build
     container_name: superset_app
@@ -125,10 +119,7 @@ services:
     container_name: superset_init
     command: ["/app/docker/docker-init.sh"]
     env_file:
-      # defaults
       - docker/.env
-      # gitignored overrides
-      - docker/.env-local
     depends_on: *superset-depends-on
     user: *superset-user
     volumes: *superset-volumes
@@ -146,10 +137,7 @@ services:
     container_name: superset_node
     command: ["/app/docker/docker-frontend.sh"]
     env_file:
-      # defaults
       - docker/.env
-      # gitignored overrides
-      - docker/.env-local
     depends_on: *superset-depends-on
     volumes: *superset-volumes
 
@@ -159,10 +147,7 @@ services:
     container_name: superset_worker
     command: ["/app/docker/docker-bootstrap.sh", "worker"]
     env_file:
-      # defaults
       - docker/.env
-      # gitignored overrides
-      - docker/.env-local
     restart: unless-stopped
     depends_on: *superset-depends-on
     user: *superset-user
@@ -181,10 +166,7 @@ services:
     container_name: superset_worker_beat
     command: ["/app/docker/docker-bootstrap.sh", "beat"]
     env_file:
-      # defaults
       - docker/.env
-      # gitignored overrides
-      - docker/.env-local
     restart: unless-stopped
     depends_on: *superset-depends-on
     user: *superset-user
@@ -198,10 +180,7 @@ services:
     container_name: superset_tests_worker
     command: ["/app/docker/docker-bootstrap.sh", "worker"]
     env_file:
-      # defaults
       - docker/.env
-      # gitignored overrides
-      - docker/.env-local
     environment:
       DATABASE_HOST: localhost
       DATABASE_DB: test

diff --git a/docs/docs/installation/installing-superset-using-docker-compose.mdx b/docs/docs/installation/installing-superset-using-docker-compose.mdx
@@ -112,6 +112,7 @@ with docker compose will store that data in a PostgreSQL database contained in a
 [volume](https://docs.docker.com/storage/volumes/), which is not backed up.
 
 Again **DO NOT USE THIS FOR PRODUCTION**
+
 :::
 
 You should see a wall of logging output from the containers being launched on your machine. Once
@@ -126,17 +127,24 @@ can skip to the next section.
 You can install additional python packages and apply config overrides by following the steps
 mentioned in [docker/README.md](https://github.com/apache/superset/tree/master/docker#configuration)
 
-Note that `docker/.env` sets the default environment variables for all the docker images
-used by `docker-compose`, and that `docker/.env-local` can be used to override those defaults.
-Also note that `docker/.env-local` is referenced in our `.gitignore`,
-preventing developers from risking committing potentially sensitive configuration to the repository.
+You can configure the Docker Compose environment variables for dev and non-dev mode with
+`docker/.env`. This environment file sets the environment
+for most containers in the Docker Compose setup, and some variables affect multiple containers and
+others only single ones.
 
 One important variable is `SUPERSET_LOAD_EXAMPLES` which determines whether the `superset_init`
 container will populate example data and visualizations into the metadata database. These examples
 are helpful for learning and testing out Superset but unnecessary for experienced users and
 production deployments. The loading process can sometimes take a few minutes and a good amount of
 CPU, so you may want to disable it on a resource-constrained device.
 
+For more advanced or dynamic configurations that are typically managed in a `superset_config.py` file
+located in your `PYTHONPATH`, note that it can be done by providing a
+`docker/pythonpath_dev/superset_config_docker.py` that will be ignored by git
+(preventing you to commit/push your local configuration back to the repository).
+The mechanics of this are in `docker/pythonpath_dev/superset_config.py` where you can see
+that the logic runs a `from superset_config_docker import *`
+
 
 :::note
 Users often want to connect to other databases from Superset. Currently, the easiest way to