Hosting behind external reverse proxy

[jsboige]

Hi, I'm hosting web app containers on several machines behind a common external reverse proxy that deals with domains and SSL certificates and maps to the corresponding individual IPs and docker ports in unencrypted http.
This has proved a bit challenging but eventually it did work with Oobabooga's, Dify (which also has an additional nginx reverse proxy container), Open-webui and Whisper-webui, mostly by tinkering with the front facing web proxy params (e.g providing additional headers such as host, which gradio correctly processes on startup).
However, this doesn't seem to work with your stack.
I suspect Caddy's the culprit, and I tried to play with the configuration files, removing auth, adding cors headers etc. but I keep getting 401 unauthorized responses and some raw port-based local IPs urls client-side, pretty much the way I did when hosts headers were not properly propagated to gradio with some other apps.
Quick tunnel works though, so I'll go with that for now, but I'd like to expose the app directly with gradio auth.
I've given up for now after trying very hard, but it would be great if you can investigate and support such a scenario.

[robballantyne]

Hi, this should work out of the box with WEB_ENABLE_AUTH=false

Some brief info on how this is set up might help:

Forge is launched on port 17860 bound only to container localhost and caddy reverse proxies that port on external 7860, so if you cannot make this work through caddy you might consider trying to modify the startup routine to have Forge bind on 0.0.0.0 and you can interact with that directly

My plan is to break up all of the additional functionality of this monolith into it's component parts. The current build system focuses on cloud environments that can only handle a single docker image, but my main target environment will support multi image soon so we can have a more sane set-up

[jsboige]

Hi, this should work out of the box with WEB_ENABLE_AUTH=false

Thanks for you quick answer and guidance ! Unfortunately, this is the first thing I tried and it did not work, even after customizing the Caddy files to remove auth all together.

However I just found out about your "SERVERLESS" param, which looks like it could do the trick as I can live without the additional services. I'll try it and report back.

[jsboige]

@robballantyne

I tried the SERVERLESS solution in a new container side by side with the running one with all services and I was able to get it working. I was planning to have 2 instances running, one with a XL turbo model and one with flux, so I figured that was the occasion to test that.

• I had to use the default 17860 port though, could it be that the "- ${FORGE_PORT_HOST:-7860}:${FORGE_PORT_HOST:-7860}" line in your compose file is faulty (looks to me like it should be - ${FORGE_PORT_HOST:-7860}:${FORGE_PORT_LOCAL:-17860})?

• I still had to set distinct ports for most services in my dedicated second .env file even though the services were not started (I can understand this is more of a minor docker issue and once you break the monolith I won't have to).

• Also, I have 2 gpus on that machine (one 3080 and one 3090), and so far I did not figure out how to get the second "turbo" container to use the 3080 instead of the 3090. It keeps using the 3090 and it seems neither the compose file "deploy:" section nore the gradio's "--device-id" start command arg in FORGE_ARGS are having any effect.

I should also mention I mapped the 2 workspaces on the same WSL directory if you think it could related or just bad practice, and while I'm here, I found it weird that the WORKSPACE param would stand for the inner volume rather that the leftmost mapped directory, since appart from the nvidia config, this is the only param I wasn't able to set in the env files.

Anyway, it's working great in https on a dedicated sub-domain, so if I can get the gpu split, I'll be very happy with those 2 containers running side by side.

Thanks in advance for your lights and sorry for the many questions...