Proxy Exception List for Outbound HTTP Connectors

[shaarmann]

Is your feature request related to a problem? Please describe.

May companies have private and use public APIs. Private APIs must be accessed via a proxy while public ones must not. The HTTP Connectors have not option to turn the proxy on/off via configuration nor can a list of exceptions be provided.

Describe the solution you'd like

Option 1:
Add a flag to the HTTP Connectors element templates. The request will be routed to proxy if and only if the flag is set to true.

Option 2:
The Proxy can be configured via an environment variable (HTTP_PROXY_URL). Another Environment variable may be used to specify exceptions, similiar to Java's http.nonProxyHosts.

Describe alternatives you've considered

Use the hybrid mode of the connectors to run two instances of the HTTP Connectors: One with configured Proxy, one without.

Additional context

• Slack Discussion: https://camunda.slack.com/archives/C02JLRNQQ05/p1725282139313859

[sbuettner]

Decision: Go with Option 2 for now to now confuse SaaS users with this flag (as they will always go through a proxy.)

[shaarmann]

As mentioned in the ticket's description, there are JVM/system properties for configuring a proxy, i.e.,

http[s].proxyHost
http[s].proxyPort
http[s].nonProxyHosts
http[s].proxyUser
http[s].proxyPassword

Customers familiar with Java are used to these properties.
Therefore, I believe it is worth evaluating whether it is possible to use these configuration options instead of custom environment variables. It has the additional benefit that functionality, such as nonProxyHosts and authentication are supported out-of-the-box.

Since version 8.6, the HTTP connector is based on Apache HTTP Client. Since version 5, the HTTPClientBuilder uses the system properties described above if no other proxy configuration is set.
The connector, however, always sets a proxy configuration even if it is null. Thereby, it overwrites the default configuration. If we change the code to only set the proxy if it is configured via the environment variable above, it will fallback to the system configuration which supports the desired feature.

[johnBgood]

QA information

Test Environment

• Self-Managed only
• Connectors version: camunda/connectors-bundle:SNAPSHOT
• Beeceptor:
  • use the Camunda domain for the test
  • use another domain as a proxy that forward requests to the Camunda domain.

Test Scope

• Docs: https://preview.docs.camunda.cloud/pr-4469/docs/next/components/connectors/protocol/rest/#proxy-configuration
• Test that:
  • requests that should use a proxy are sent to the proxy domain
  • requests with a host contained in the proxy exception list go don't go through the proxy domain

Test Data

You will need to provide the JAVA_OPTS env var:

services:
  connectors:
    environment:
      - JAVA_OPTS=-Dhttp.proxyHost=<PROXY> -Dhttp.proxyPort=<PORT> -Dhttps.proxyHost=<PROXY> -Dhttps.proxyPort=<PORT> -Dhttp.nonProxyHosts=<OTHER_DOMAIN>

Replace the variables accordingly.

[slolatte]

@Szik and I tested this and everything looks good to us!