@aws-cdk/aws-s3-deployment: broken cross stack reference when used in Source.jsonData

[smnrd]

Describe the bug

I'm trying to use S3Deployment with a Source.jsonData source to deploy a json configuration file with values resolved at deploy time (eg cognito userpool id). These values, properties of a construct from another nested stack passed as StackProp to the S3Deployment nested stack, doesn't get resolved properly.

Expected Behavior

When I use values from another stack construct in a Source.jsonData source with S3Deployment, they are properly resolved at deploy time.

Current Behavior

A 'unresolved resource dependencies' error is thrown.

1:20:17 PM | CREATE_FAILED        | AWS::CloudFormation::Stack | DeploymentNestedSt...ckResource3980C4F4
Template format error: Unresolved resource dependencies [UserPool6BA7E5F2] in the Resources block of the template

        new NestedStack (nested-stack.ts:133:21)
        \_ new DeploymentNestedStack (nested-stack-ref-bug-repro-lowing resource(s) failed to create: [ResourceNestedStackNestedStackResourceNestedStackNestedStackResourceCDDDB2BF, DeploymentNestedStackNestedStackDeploymentNestedStackNested
stack-bug-repro/app.ts:19:5)
        \_ new MainStack nest-stack-ref-bug-repro/app.ts:33:5)
        \_ Object.<anonymous> (nested-stack-b
ug-repro/app.ts:38:1)
        \_ Module._compile (internal/modules/cjs/loader.js:1072:14)
        \_ Object.Module._extensions..js (internal/modules/cjs/loader.js:1101:10)
        \_ Module.load (internal/modules/cjs/loader.js:937:32)
        \_ Function.Module._load (internal/modules/cjs/loader.js:778:12)
        \_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:76:12)
        \_ internal/main/run_main_module.js:17:47

Reproduction Steps

import { App, Construct, Stack, NestedStack, StackProps, NestedStackProps } from '@aws-cdk/core'
import { UserPool, IUserPool } from '@aws-cdk/aws-cognito'
import { BucketDeployment, Source } from '@aws-cdk/aws-s3-deployment'
import { Bucket } from '@aws-cdk/aws-s3'

class ResourceNestedStack extends NestedStack {
  userPool: UserPool
  constructor (scope: Construct, id: string, props: NestedStackProps = {}){
    super(scope, id, props)
    this.userPool = new UserPool(this, 'UserPool')
  }
}

interface DeploymentNestedStackProps extends NestedStackProps {
  userPool: IUserPool
}
class DeploymentNestedStack extends NestedStack {
  constructor (scope: Construct, id: string, props: DeploymentNestedStackProps){
    super(scope, id, props)
    const bucket = new Bucket(this, 'Bucket')
    new BucketDeployment(this, 'Deployment', {
      destinationBucket: bucket,
      sources          : [
        Source.jsonData('appconfig.json', { userPoolId: props.userPool.userPoolId })
      ]
    })
  }
}
class MainStack extends Stack {
  constructor (scope: Construct, id: string, props: StackProps = {}){
    super(scope, id, props)
    const resourceNestedStack = new ResourceNestedStack(this, 'ResourceNestedStack')
    new DeploymentNestedStack(this, 'DeploymentNestedStack', { userPool: resourceNestedStack.userPool })
  }
}

const app = new App()
new MainStack(app, 'MainStack', {
  env: { account, region }
})

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

1.180.0

Framework Version

No response

Node.js Version

14.19.0

OS

Debian GNU/Linux 10 (buster)

Language

Typescript

Language Version

4.4.2

Other information

No response

[peterwoodworth]

You're right @smnrd, I can reproduce this. Thanks for the easily copy+pastable code!

Looking at the template, it's because it's trying to directly reference another resource as if it's in the same template. In other words, it's not making use of parameters which are how we handle cross-stack references.

    "SourceMarkers": [
     {
      "<<marker:0xbaba:0>>": {
       "Ref": "UserPool6BA7E5F2"
      }
     }
    ],

I believe the cause of this is due to this line of code where it makes the Stack.toJsonString() call:

aws-cdk/packages/@aws-cdk/aws-s3-deployment/lib/source.ts

Line 163 in 4c11af6

return Source.data(objectKey, Stack.of(scope).toJsonString(obj)).bind(scope, context);

[cdanhowell]

Hey! I wanted to check on this issue. I see the PR may be stalled out a bit. While waiting on that to resolve, is there a workaround that can be used? I attempted to use Fn.importValue and similar constructs, but wasn't having too much luck.

Thanks!

[AKoetsier]

@cdanhowell see #19257 (comment) for a workaround.

[tbhanson96]

Multiple issues have been closed related to this issue, all of which reference this issue and to use the workaround. Is the root cause going to be addressed? this seems like a pretty common use case for Source.jsonData...

[joannanowakowska]

I observed the same issue when adding source to BucketDeployment using the s3deploy.Source.data directly. I'm trying to overwrite an env var in a js file for a react app based on api gateway api url. The api gateway is in a different stack. I also tried adding a temporary output to the stack as part of which I want to deploy the file but using the output doesn't work either.

[mikevoets]

It looks like this is still an issue and I was wondering if there's any updates regarding a fix?