CIP-32: Inline datums

[michaelpj]

This CIP proposes adding "inline datums" which allow attaching a datum itself rather than a datum hash to an output.

[L-as]

Does it make sense to also allow inline scripts at the same time?

[michaelpj]

Does it make sense to also allow inline scripts at the same time?

Inline scripts are logical, but less pressing, since scripts change less often. Plus they have the added complication that they make the "address" field much more complicated (and, generally, a lot bigger), and many places make assumptions that addresses are roughly hash-sized. So given that there isn't really compelling benefit, I'm not proposing those right now.

[SebastienGllmt]

The main use for creating this is that the Rust codebase and a few others are automatically generated from the CDDL definition so if you create a datum-hash-or-datum definition, it means this will be a common struct that represents this for all implementations

[SebastienGllmt]

For data-availability uses cases being able to enforce the usage of inline datums is crucial so I would definitely make some way to enforce it (be it through its presence in the script context or some other way)

[WhatisRT]

This only makes a difference off-chain. IICR validators can already enforce datums for new outputs to be present in a transaction, so the only real difference between inline or not is whether to look it up in the transaction history or the UTXO. Making the inlining status visible to validators would simplify this lookup, at the cost of adding complexity to the on-chain code for everyone.

[michaelpj]

is whether to look it up in the transaction history or the UTXO

Which is a big deal! So I think Sebastian's point is relevant: people will want to be able to enforce this.

[theganant]

@WhatisRT Can you explain how to find datums trustlessly offchain? I can't seem to find any answers online, this issue comes closest

[WhatisRT]

Since the hash is published on-chain, there's no requirement for trustlessness here

[SebastienGllmt]

Agreed. To add to this, probably we should have a follow-up CIP to this that changes the tx fee function so that inline datums contribute less towards tx fee than other fields (since unlike other fields, it's just data that don't require extra computation to simply inline into the tx)

[WhatisRT]

They do contribute to block size though, which is also a very limited resource. Maybe the correct way of dealing with this is to split up txfee into a memory and a time component, and have two constants that can be used to adjust them independently.

[michaelpj]

There are two things here:

• UTXO size deposits. This is mostly driven by memory constraints, and as such there is no reason to treat inline datums differently, we just need a single parameter based on size.
• Transaction size fee. This is really about processing time, and there there arguably is a reason to pay more or less for different parts of the transaction. But that would be a fairly major change.

[michaelpj]

I think this CIP is ready to be merged as Draft. I will revisit it and update to Active once the implementation has progressed and the CDDL is nailed down.

[WhatisRT]

During a meeting today, we've discussed how inline datums should be provided to the transaction context. There are multiple options (including the one currently mentioned in this CIP) with different kinds of advantages and drawbacks. First, here are some potential requirements:

• Low ledger overhead
• Low overhead for scripts
• The ability to distinguish whether a datum is inlined
• Backwards compatibility

Having a representation that is as close to the actual ledger representation as possible reduces the probability of getting bitten by something unexpected down the line. Since scripts should access datums via an API that would allow the same kind of access for datums whether they are inlined or not, the overhead should always be pretty low. Of course there would be some runtime costs that would be reduced if the ledger provided uniform access, but that's probably a small issue. In the case we did do a uniform access approach in the ledger, we could provide a boolean that tells us whether a datum is inlined or not.

However, the big issue seems to be backwards compatibility, i.e. should inline datums be provided to scripts of previous (i.e. current) versions. If not, a user could send funds to a script address with an inline datum and that datum would still be passed to the script that's spending it, but not to other scripts, which breaks the assumption that scripts can always see the datum of other scripts that spend outputs, so we can't do that. This forces us to either provide inline datums to old scripts, or to make them incompatible by force, for example by disallowing inline datums with previous Plutus versions.

Disallowing inline datums and scripts of previous Plutus versions in the same transaction has an other issue though, which is that there'd be a subtle way of making an output unspendable, which is to send it to such a script but with an inline datum. If inline datums become popular, having them might become the default which sounds dangerous. We have no way of distinguishing what language version a script address belongs to (in general, i.e. without knowing the script), so we can't prevent users from making this mistake in the ledger. Wallets could potentially do that, if they know the script.

Providing inline datums to previous scripts is also at odds with having low overhead for the ledger, since they would have to be provided in the same way as datums currently are.

All of this suggests the following options:

1. Staying as close to the actual representation in the ledger as possible, i.e. providing inline datums as part of the outputs (currently in this CIP), and disable inline datums with older scripts. Potentially dangerous for users.
2. Same as above, but with inserting inline datums into old script contexts. This has the worst kind of ledger overhead, since the ledger needs to do different things with inline datums depending on the language version. But it could solve the problem now, while possibly introducing less (or more!) future problems.
3. A uniform approach for all datums, inserting inline datums in the same map where datums live right now, with an additional boolean for newer versions of Plutus to distinguish the type of datum. Not great for the ledger, but should have less runtime overhead, and has backwards compatibility.

Maybe there's something that we've missed when we discussed it, or maybe having more types of unspendable outputs is fine since almost anyone will send money to scripts only though applications that generate the transaction for them, which should really know what kind of input to use. Or maybe it's really really bad for some reason I can't think of right now.

[michaelpj]

Thank you for the excellent summary. I'll incorporate this discussion into the CIP text.

For the moment, my inclination is to pick option 1. The reason is:

1. I do think we want scripts to be able to enforce the use of inline datums. If an application developer wants to use inline datums to make things convenient for their users, then it would be quite annoying if someone who interacts with the application can just choose not to do that.
2. Unless there is a pressing reason, I think it's very good to keep the representation of transactions in the script context as close as possible to the representation of transactions in the ledger. This avoids surprises and conceptual overhead for users, which can really hurt us if it builds up.
3. Adding a new way to make outputs unspendable is bad. I think we should avoid these wherever possible. However, I think that in this instance breakage is likely to be rare because: a) it should easily be noticed in testing, b) the client will typically know what kind of script they are using and can avoid this, c) hopefully most users will migrate to Plutus V2 quickly since it will support a number of desirable features.

[michaelpj]

I've updated the text.

[L-as]

This approach seems fine to me.

[L-as]

Even if it did include this information, it would still not be enough. The script might be V2, but to consume it you might need another input that uses a V1 validator, or you might need to mint/burn with a V1 minting policy, which wouldn't work because those scripts can't be run.

[L-as]

This could be because e.g. the datum of the UTXO in question contains a reference to a minting policy hash, and the script asserts that a token of that minting policy is minted. In general, this problem is only solvable by the client knowing the contents of the scripts in question, so I don't think there's any way around this.

[michaelpj]

Right, that would be a more elaborate way of getting yourself stuck in this position indeed.

[crptmppt]

This was discussed at the Editor meeting 38 (see notes)- it is assumed on hold until further development (do flag if ready to review again)

[michaelpj]

It's ready for review.

[KtorZ]

Fine to go as draft while the implementation is being worked on 👍

[KtorZ]

(cc @rphair @SebastienGllmt)

[rphair]

Fine to go as draft while the implementation is being worked on

agreed 👍