Extra config import fetched through cloud config server property isn't injected during app start #2271

okulbida · 2023-05-25T13:35:38Z

Library versions

Spring Boot 2.7.12
Spring Cloud 2021.0.6
Spring Cloud AWS 2.4.1

I have a Spring Cloud Config setup with a Git backend. Generally, property files fetching function well. My aim is store secrets in AWS and reference them from property files, so I added additional spring.config.import: "aws-secretsmanager:/test/springconfig" directive within the properties file for the Spring Cloud Config Server.

This option only works when I add such a configuration to an application property itself as follows:

spring:
  application:
    name: my-java-client
  config:
    import: 
    - "configserver:"
    - "aws-secretsmanager:/test/springconfig"

My intention is to house both configurations and sensitive data (in the form of AWS references like "aws-secretsmanager:/test/springconfig") in the Git backend. So, the properties file in Git backend would look like

server.datasource.url: jdbc:postgresql://dev:5432/example
spring.config.import: "aws-secretsmanager:/test/springconfig"
some.dummy: value

This is what I get now:

{
"name": "configserver:[email protected]:xxx/spring-configs.git/configs-location/java-client/java-client-dev.yaml",
"properties": {
"server.datasource.url": {
"value": "jdbc:postgresql://dev:5432/example",
"origin": "Config Server [email protected]:xxx/spring-configs.git/configs-location/java-client/java-client-dev.yaml:1:24"
},
"some.dummy": {
"value": "value",
"origin": "Config Server [email protected]:xxx/spring-configs.git/configs-location/java-client/java-client-dev.yaml:2:13"
},
"aws.secretsmanager.region": {
"value": "us-east-1",
"origin": "Config Server [email protected]:xxx/spring-configs.git/configs-location/java-client/java-client-dev.yaml:5:13"
},
"management.endpoints.web.exposure.include": {
"value": "env",
"origin": "Config Server [email protected]:xxx/spring-configs.git/configs-location/java-client/java-client-dev.yaml:7:44"
},
"management.endpoint.env.enabled": {
"value": true,
"origin": "Config Server [email protected]:xxx/spring-configs.git/configs-location/java-client/java-client-dev.yaml:8:34"
},
"spring.config.import": {
"value": "aws-secretsmanager:/test/springconfig",
"origin": "Config Server [email protected]:xxx/spring-configs.git/configs-location/java-client/java-client-dev.yaml:10:23"
}
}
},

The problem is that I don't get retrieved values from aws secret while storing spring.config.import: "aws-secretsmanager:/test/springconfig" in Git backend for Cloud Config server.
I would appreciate any guidance on potential options to achieve this result. I have read that this can be done using Spring Cloud Vault, but it seems excessive for our needs as we would like to use AWS facilities for this purpose.

Thanks,
Oleksandr

The text was updated successfully, but these errors were encountered:

ryanjbaxter · 2023-05-25T18:54:26Z

Yes you cannot put spring.config.import in configuration files coming from external sources. It needs to be placed in the application yaml locally of the client.

The config server itself does support AWS Secrets Manager as a backend though
https://docs.spring.io/spring-cloud-config/docs/4.0.2/reference/html/#_aws_secrets_manager

okulbida · 2023-05-26T06:45:22Z

Thanks @ryanjbaxter
I'd like to manage both configurations and references to AWS Secrets Manager secrets within property files. For example:

management.endpoints.web.exposure.include: env
management.endpoint.env.enabled: true
aws-secretsmanager:/test/springconfig

I am seeking a way to store all application configuration references, including secrets, in one location. I am interested in the most straightforward and efficient method to achieve this. Could you please suggest an appropriate solution that minimizes complexity?

ryanjbaxter · 2023-05-26T14:18:24Z

What version of Spring Cloud are you using?

okulbida · 2023-05-26T14:25:47Z

Currently I have this

<spring-cloud.version>2021.0.6</spring-cloud.version>
<spring-cloud-aws.version>2.4.4</spring-cloud-aws.version>

ryanjbaxter · 2023-05-26T14:29:25Z

Can you try your original project with Spring Cloud 2022.0.3?

okulbida · 2023-06-01T18:07:07Z

@ryanjbaxter
Is it possible to have multiple backends for config server ? Option to have both git and something like this would help a lot to manage secrets in proper way https://docs.spring.io/spring-cloud-config/docs/current/reference/html/#_aws_secrets_manager_backend

ryanjbaxter · 2023-06-01T19:17:27Z

Yes you can use a composite https://docs.spring.io/spring-cloud-config/docs/current/reference/html/#composite-environment-repositories

okulbida · 2023-06-02T16:50:36Z

Thanks @ryanjbaxter
I tested with Spring Cloud 2022.0.3 and it didn't help. For some reason I also cannot run awsparamstore or aws-secretsmanager composite type (multiple git, native types work).

Here is my current config

application.yaml:

server.port: 8888

management.endpoints.web.exposure.include: env
management.endpoint.env.enabled: true

aws.secretsmanager.region: us-east-1
aws.paramstore.region: us-east-1

credentials:
  profile:
    name: default
 
spring:
  profiles:
    include: composite
  # config:
  #   import:
  #   - 'aws-secretsmanager:/test/springconfig'
  #   - 'aws-parameterstore:/spring-cloud/server/privatekey'
  cloud:
    config:
      server:
        composite:
        - type: awsparamstore
          region: us-east-1
          prefix: /dev 
          profile-separator: '/'    
        - type: git
          uri: [email protected]:repo/configs.git
          basedir: configs-location
          clone-on-start: true
          default-label: main
          timeout: 5
          refresh-rate: 1
          ignoreLocalSshSettings: true
          search-paths:
            - "configs-location/shared"
            - "configs-location/{application}"

pom:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>3.1.0</version>
    <relativePath/> <!-- lookup parent from repository -->
  </parent>
  <groupId>io.example</groupId>
  <artifactId>configserver</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <name>spring-cloud-config-server</name>
  <description>Spring Config server providing config information</description>
  <properties>
    <java.version>17</java.version>
    <spring-cloud.version>2022.0.3</spring-cloud.version>
    <spring-cloud-aws.version>2.4.4</spring-cloud-aws.version>
    <amazon-awssdk-ssm.version>2.20.69</amazon-awssdk-ssm.version>
  </properties>
  <dependencies>
    <dependency>
      <groupId>org.springframework.cloud</groupId>
      <artifactId>spring-cloud-config-server</artifactId>
    </dependency>
    <dependency>
      <groupId>org.springframework.cloud</groupId>
      <artifactId>spring-cloud-starter-config</artifactId>
    </dependency>
        <dependency>
            <groupId>io.awspring.cloud</groupId>
            <artifactId>spring-cloud-starter-aws-secrets-manager-config</artifactId>
            <version>${spring-cloud-aws.version}</version>
        </dependency>
        <dependency>
            <groupId>io.awspring.cloud</groupId>
            <artifactId>spring-cloud-starter-aws-parameter-store-config</artifactId>
            <version>${spring-cloud-aws.version}</version>
        </dependency>
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-aws-parameter-store-config</artifactId>
        <version>2.2.6.RELEASE</version>
    </dependency>
        <dependency>
       <groupId>org.springframework.boot</groupId>
       <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>software.amazon.awssdk</groupId>
        <artifactId>ssm</artifactId>
      <version>${amazon-awssdk-ssm.version}</version>
    </dependency>

  </dependencies>
  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-dependencies</artifactId>
        <version>${spring-cloud.version}</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>

  <build>
    <plugins>
      <plugin>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-maven-plugin</artifactId>
      </plugin>
    </plugins>
  </build>

</project>

ERROR:

config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616244539+03:00 2023-06-02T16:44:58.609Z ERROR 1 --- [           main] o.s.boot.SpringApplication               : Application run failed
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616293659+03:00
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616304029+03:00 java.lang.IllegalStateException: Error processing condition on org.springframework.cloud.config.server.config.CompositeRepositoryConfiguration.searchPathCompositeEnvironmentRepository
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616310859+03:00   at org.springframework.boot.autoconfigure.condition.SpringBootCondition.matches(SpringBootCondition.java:60) ~[spring-boot-autoconfigure-3.1.0.jar!/:3.1.0]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616317126+03:00   at org.springframework.context.annotation.ConditionEvaluator.shouldSkip(ConditionEvaluator.java:108) ~[spring-context-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616323933+03:00   at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitionsForBeanMethod(ConfigurationClassBeanDefinitionReader.java:183) ~[spring-context-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616329747+03:00   at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitionsForConfigurationClass(ConfigurationClassBeanDefinitionReader.java:144) ~[spring-context-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616335625+03:00   at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitions(ConfigurationClassBeanDefinitionReader.java:120) ~[spring-context-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616342099+03:00   at org.springframework.context.annotation.ConfigurationClassPostProcessor.processConfigBeanDefinitions(ConfigurationClassPostProcessor.java:427) ~[spring-context-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616348214+03:00   at org.springframework.context.annotation.ConfigurationClassPostProcessor.postProcessBeanDefinitionRegistry(ConfigurationClassPostProcessor.java:287) ~[spring-context-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616354789+03:00   at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanDefinitionRegistryPostProcessors(PostProcessorRegistrationDelegate.java:344) ~[spring-context-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616360413+03:00   at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:115) ~[spring-context-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616366608+03:00   at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:771) ~[spring-context-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616372949+03:00   at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:589) ~[spring-context-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616379074+03:00   at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:146) ~[spring-boot-3.1.0.jar!/:3.1.0]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616401782+03:00   at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:733) ~[spring-boot-3.1.0.jar!/:3.1.0]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616408682+03:00   at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:435) ~[spring-boot-3.1.0.jar!/:3.1.0]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616414915+03:00   at org.springframework.boot.SpringApplication.run(SpringApplication.java:311) ~[spring-boot-3.1.0.jar!/:3.1.0]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616420954+03:00   at org.springframework.boot.SpringApplication.run(SpringApplication.java:1305) ~[spring-boot-3.1.0.jar!/:3.1.0]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616426666+03:00   at org.springframework.boot.SpringApplication.run(SpringApplication.java:1294) ~[spring-boot-3.1.0.jar!/:3.1.0]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616432147+03:00   at io.example.configserver.ConfigServerApplication.main(ConfigServerApplication.java:13) ~[classes!/:0.0.1-SNAPSHOT]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616438333+03:00   at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616443910+03:00   at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616449824+03:00   at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616455529+03:00   at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616461058+03:00   at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) ~[app.jar:0.0.1-SNAPSHOT]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616467082+03:00   at org.springframework.boot.loader.Launcher.launch(Launcher.java:95) ~[app.jar:0.0.1-SNAPSHOT]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616472374+03:00   at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) ~[app.jar:0.0.1-SNAPSHOT]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616478058+03:00   at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65) ~[app.jar:0.0.1-SNAPSHOT]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616483715+03:00 Caused by: java.lang.NullPointerException: Cannot invoke "Object.hashCode()" because "key" is null
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616508433+03:00   at java.base/java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936) ~[na:na]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616515734+03:00   at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanDefinition(DefaultListableBeanFactory.java:888) ~[spring-beans-6.0.9.jar!/:6.0.9]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616521508+03:00   at org.springframework.cloud.config.server.composite.CompositeUtils.getEnvironmentRepositoryFactoryTypeParams(CompositeUtils.java:77) ~[spring-cloud-config-server-4.0.3.jar!/:4.0.3]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616530630+03:00   at org.springframework.cloud.config.server.composite.OnSearchPathLocatorPresent.getMatchOutcome(OnSearchPathLocatorPresent.java:46) ~[spring-cloud-config-server-4.0.3.jar!/:4.0.3]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616536604+03:00   at org.springframework.boot.autoconfigure.condition.SpringBootCondition.matches(SpringBootCondition.java:47) ~[spring-boot-autoconfigure-3.1.0.jar!/:3.1.0]
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616542537+03:00   ... 25 common frames omitted
config-server-deployment-54d557ff99-blrn8 config-server 2023-06-02T19:44:58.616547774+03:00

ryanjbaxter · 2023-10-12T20:09:59Z

Can you change type: awsparamstore to type: awsParameterStore and try again?

okulbida · 2023-10-13T07:43:24Z

thanks for your reply, this was solved in different way

okulbida added the waiting-for-triage label May 25, 2023

ryanjbaxter added question waiting for feedback and removed waiting-for-triage labels May 25, 2023

spring-cloud-issues added feedback-provided and removed waiting for feedback labels May 30, 2023

ryanjbaxter added waiting for feedback and removed feedback-provided labels Jun 1, 2023

spring-cloud-issues added feedback-provided and removed waiting for feedback labels Jun 2, 2023

ryanjbaxter added waiting for feedback and removed feedback-provided labels Oct 12, 2023

okulbida closed this as completed Oct 13, 2023

ryanjbaxter mentioned this issue Oct 13, 2023

Align AWS Parameter Store Factory bean name with profile name #2336

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Extra config import fetched through cloud config server property isn't injected during app start #2271

Extra config import fetched through cloud config server property isn't injected during app start #2271

okulbida commented May 25, 2023 •

edited

Loading

ryanjbaxter commented May 25, 2023

okulbida commented May 26, 2023

ryanjbaxter commented May 26, 2023

okulbida commented May 26, 2023

ryanjbaxter commented May 26, 2023

okulbida commented Jun 1, 2023

ryanjbaxter commented Jun 1, 2023

okulbida commented Jun 2, 2023 •

edited

Loading

ryanjbaxter commented Oct 12, 2023

okulbida commented Oct 13, 2023

Extra config import fetched through cloud config server property isn't injected during app start #2271

Extra config import fetched through cloud config server property isn't injected during app start #2271

Comments

okulbida commented May 25, 2023 • edited Loading

ryanjbaxter commented May 25, 2023

okulbida commented May 26, 2023

ryanjbaxter commented May 26, 2023

okulbida commented May 26, 2023

ryanjbaxter commented May 26, 2023

okulbida commented Jun 1, 2023

ryanjbaxter commented Jun 1, 2023

okulbida commented Jun 2, 2023 • edited Loading

ryanjbaxter commented Oct 12, 2023

okulbida commented Oct 13, 2023

okulbida commented May 25, 2023 •

edited

Loading

okulbida commented Jun 2, 2023 •

edited

Loading