Can't decode data in provider

[raphink]

In camptocamp/terraform-provider-pass#6, we're hitting a situation where Terraform fails to decode a structured data to store it into a parameter.

This seems to be linked to the way https://github.com/mitchellh/mapstructure/blob/master/mapstructure.go works, but I'm not sure if it's our provider code that is wrong or if it's a bug.

[apparentlymart]

Hi @raphink! Sorry for this confusing behavior.

It looks like the data attribute in question here is defined as schema.TypeMap and no explicit Elem is provided, so the default element type of schema.TypeString is being selected.

In many cases Terraform is able to automatically convert values to the target type, but I think it's failing in this case because the integer value is coming from the provider code rather than from the configuration, and so the automatic type conversion behavior in the configuration language doesn't apply.

With the helper/schema API as currently implemented it's necessary for the caller to guarantee the correct types, and so in this case all of the map values should be converted to strings.

The underlying implementation of the ResourceData methods will be changing in the near future to make use of the value and type facilities of the new configuration language interpreter, so this may mean that conversions here could happen "for free" as part of that, but we won't know that until we get further along with the implementation... it might require some further work before we could make conversions automatic here.

[raphink]

Thank you for your answer @apparentlymart. Let me see if I got that right. In order for it to work ATM, I would have to either:

• use the same type for all values (string by default, unless I specify the Elem type)
• specify the type of each value by its key

Is that right?

[apparentlymart]

Hi @raphink,

The helper/schema type system gives you two options here:

• An attribute of TypeMap is a map where all of the values have the same type.
• An attribute of TypeList or TypeSet that has a schema.Resource as its Elem which describes exactly which keys and values are expected.

I think I am just repeating back to you what you said in more specific terms here, but I just wanted to write it out to be sure.

There is a practical reason for these constraints: the state and plan formats don't currently allow dynamic type information to be retained, so helper/schema relies on the schema in order to understand whether a string like "true" is actually a string containing those letters or a boolean value true.

This constraint may be relaxed in future, since we plan to change the state and file formats to be able to retain dynamic type information for situations like this.

In the mean time, the usual way providers have achieved a dynamic "arbitrary map of anything" sort of structure is to have a string attribute called something_json where the user provides the data structure in JSON format. That isn't sufficient for your use-case here because this is a Computed attribute intended for interpolation elsewhere.

My suggestion for the immediate term would be to convert all of the values to strings -- which is what Terraform is going to do in order to write them to the state anyway -- and then users can rely on Terraform's automatic type conversions to pass numeric strings into attributes that expect integers.

In the next major release we will have a jsondecode function, which offers another interim solution where you could expect a data_json string attribute containing a JSON-serialized version of that structure, which can then be decoded to use it.

It is a longer-term plan to support dynamically-typed attributes in the provider schema system, but that work is blocked by other work we are currently doing and so we've not yet planned exactly how that will work. Eventually though we do want to support an attribute whose type is decided dynamically by what is assigned to it, rather than hard-coded into the schema. At that point your data attribute could make use of that to get the behavior I think you really want here. We'll investigate this further once current work is completed (which includes updates to the state and plan models and the provider plugin protocol so that such values can be saved).

I'm going to leave this labelled as a provider-sdk enhancement so we can find it again once we start that work, so we can include this use-case in the design effort.

[paultyng]

Merging this with #62 for richer map support.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.