Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inner groups #3009

Merged
merged 10 commits into from
Oct 7, 2021
Merged

Inner groups #3009

merged 10 commits into from
Oct 7, 2021

Conversation

jannotti
Copy link
Contributor

@jannotti jannotti commented Oct 6, 2021

Introduces itxn_next (feature flagged behind LogicVersion 6). This allows contracts to create inner transaction groups, not just singleton transactions. This is not of much value on its own, but is needed to make groups for inner app invocations. Until then, the only difference between running two inner txns consecutively, and running two as part of a group is that fee pooling allows the second txn to pay for a shortfall from the first.

@codecov-commenter
Copy link

codecov-commenter commented Oct 6, 2021

Codecov Report

Merging #3009 (ea5867a) into master (f17f73e) will decrease coverage by 17.75%.
The diff coverage is 79.03%.

Impacted file tree graph

@@             Coverage Diff             @@
##           master    #3009       +/-   ##
===========================================
- Coverage   47.63%   29.88%   -17.76%     
===========================================
  Files         359      357        -2     
  Lines       57964    57966        +2     
===========================================
- Hits        27611    17321    -10290     
- Misses      27231    38138    +10907     
+ Partials     3122     2507      -615     
Impacted Files Coverage Δ
data/transactions/logic/doc.go 25.80% <ø> (-51.62%) ⬇️
data/transactions/logic/opcodes.go 96.22% <ø> (-3.78%) ⬇️
data/transactions/logic/fields.go 71.79% <25.00%> (-9.79%) ⬇️
data/transactions/logic/eval.go 52.71% <82.75%> (-32.73%) ⬇️
util/condvar/timedwait.go 0.00% <0.00%> (-100.00%) ⬇️
data/transactions/payset.go 0.00% <0.00%> (-100.00%) ⬇️
crypto/merklearray/worker.go 0.00% <0.00%> (-100.00%) ⬇️
tools/network/dnssec/sort.go 0.00% <0.00%> (-100.00%) ⬇️
crypto/compactcert/structs.go 0.00% <0.00%> (-100.00%) ⬇️
tools/network/dnssec/config.go 0.00% <0.00%> (-100.00%) ⬇️
... and 216 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f17f73e...ea5867a. Read the comment docs.

@@ -1318,7 +1318,7 @@ bitlen interprets arrays as big-endian integers, unlike setbit/getbit
- Opcode: 0xb3
- Pops: _None_
- Pushes: _None_
- execute the current inner transaction. Fail if 16 inner transactions have already been executed, or if the transaction itself fails.
- execute the current inner transaction group. Fail if executing this group would exceed 16 total inner transactions, or if any transaction in the group fails.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you consider changing the name of "itxn_submit", since "itxn" in the other opcodes is referring to a single inner transaction, whereas here it's referring to the whole group of them?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I originally thought I'd be explicit about the whole thing, wrap a bunch of matched itxn_begin and itxn_end pairs with itxg_begin, itxg_submit. But I couldn't justify all those opcodes once I started trying to write apps with them.

This seemed like the minimal change possible, and you can use itxn_submit to submit a single transaction, so I didn't want to waste an opcode on the distinction. But it's not crazy if people think we should go that way.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I personally like it the way you did it, I think the additional opcodes aren't worth the minimal increase in explicitness/clarity.

data/transactions/logic/eval.go Show resolved Hide resolved
data/transactions/logic/evalAppTxn_test.go Outdated Show resolved Hide resolved
@@ -169,9 +169,10 @@ var opDocByName = map[string]string{
"b~": "X with all bits inverted",

"log": "write bytes to log state of the current application",
"itxn_begin": "begin preparation of a new inner transaction",
"itxn_begin": "begin preparation of a new inner transaction in a new transaction group",
"itxn_next": "begin preparation of a new inner transaction in the same transaction group",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An expanded description for this opcode would be helpful to explain when it's useful.

I.e., all inner transactions are atomic in the sense that if one fails, the entire top-level transaction fails. However, explicitly putting transactions into groups enables fee pooling and setting up the environment for inner transaction application calls.

test/scripts/e2e_subs/app-group.py Outdated Show resolved Hide resolved
@jasonpaulos
Copy link
Contributor

jasonpaulos commented Oct 6, 2021

Also, I noticed you didn't implement a gitxn opcode, so all transactions in a submitted group are hidden from TEAL except for the final one.

@jannotti
Copy link
Contributor Author

jannotti commented Oct 6, 2021

Also, I noticed you didn't implement a gitxn opcode, so all transactions in a submitted group are hidden from TEAL except for the final one.

Yes, that's true at the moment. I'm going to do all the txn like opcodes at once. I also need to make gtxn see the new "effects" fields (logs, created ids).

@jasonpaulos
Copy link
Contributor

Additionally, the group ID of the inner txns is never set. I agree this isn't necessary from a technical perspective, but setting the ID for each submitted group txn would make it possible to distinguish between normal txns and group txns. This is probably useful for observers of the chain.

@jannotti
Copy link
Contributor Author

jannotti commented Oct 6, 2021

Additionally, the group ID of the inner txns is never set. I agree this isn't necessary from a technical perspective, but setting the ID for each submitted group txn would make it possible to distinguish between normal txns and group txns. This is probably useful for observers of the chain.

I've been thinking about this a lot, and I don't have a perfect answer yet. There are two interesting "fields" that appear in two "contexts". The fields are TxId and GroupID. The contexts are "as seen by AVM code" and "as seen on chain".

TxID does not appear on chain, but there is a well-defined way to determine it from on-chain data (the fields of the transaction). GroupID does appear on chain (for top-level transactions) and it had better match the hashing scheme that defines it for the group members. Both are accessible from AVM.

So, first for on-chain visibility, I can see some value in chain observers looking at a txn's list of inner-txns and being able to say "The first 3 were part of a group, then the next 2, and the last was a single transaction". I don't think there's a ton of such value, but it seems reasonable. That can be done in two ways. 1) Give each inner group a sequential group id. or 2) Give the group its "real" GroupID, based on the existing hashing rule. The first is cheaper, the second seems more correct. But notice that the second is still a little strange - you can get identical group IDs, even for adjacent groups, since there's no rule against duplicate transactions. This would actually mean chain observers could have a hard time differentiating consecutive identical groups.

Now txid. Since it never appears on chain, we don't have to worry too much about that. But there have been calls for some sort of identifier for inner transactions. I have so far been against that, but we need to decide what global TxID does in an inner txn anyway. If introduced, it would have to involve the parent txid, and the index of the inner transaction, so the resulting txid would be unique. This is a departure from the current definition. If that definition were exposed to AVM code, is there any code that might get confused because it tried to calculate the hash explicitly? Or because it intended to authenticate a transaction by looking at its txid?

So here is my current thinking.

In an inner transaction, with teal <= 5, using global TxID or global GroupID panics. (In fact, there is some thought that calling teal <= 5 apps as inner transactions should be disallowed entirely.)

Starting from teal 6, TxID is defined to include the parent txid and index in the hash. GroupID is defined similarly. This makes them unique, but never exposes this new definition to old code. Thoughts?

I've had some discussion of this with @fabrice102 and Shai. Further input welcome.

For THIS PR, I'm not concerned yet, since itxn_next is behind Teal 6.

@jasonpaulos
Copy link
Contributor

In an inner transaction, with teal <= 5, using global TxID or global GroupID panics. (In fact, there is some thought that calling teal <= 5 apps as inner transactions should be disallowed entirely.)

I believe teal <= 3 should not be allowed in inner txns, since the resource access model is more relaxed in those versions (any asset or app that an account holds can be accessed using asset_holding_get and app_local_get_ex, regardless of whether it's in a foreign array or not). If we did allow programs with this relaxed access, then they could be used as a (somewhat expensive) way to circumvent the current restrictions.

Seeing as that's the case, I can understand extending the restriction to teal <= 5 as well. My primary security concern for inner app txns is that a program designed before they were allowed (or detectable) might somehow be abused by being invoked as an inner txn, so requiring a minimum teal version of 6 would certainly solve that. I'm not yet convinced that this is a real problem though.

Starting from teal 6, TxID is defined to include the parent txid and index in the hash. GroupID is defined similarly. This makes them unique, but never exposes this new definition to old code. Thoughts?

That certainly works, but does the AVM-exposed txID really need to be unique? If not, I can imagine either the top-level txn ID being reused, or a txID of all 0s potentially working. I'm still thinking through the implications of these options.

// here, because they might never itxn_submit, or they might
// change the fee. Do it in itxn_submit.
fee = basics.SubSaturate(fee, *cx.FeeCredit)
if len(cx.InnerTxns)+len(cx.subtxns) >= cx.Proto.MaxInnerTransactions ||
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once apps can be inner transactions, this code should ideally also count any child inner transactions created by txns in cx.InnerTxns too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I'll have to propagate down a sort of "tx budget" saying how many txns have been consumed already.

Copy link
Contributor

@jasonpaulos jasonpaulos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are still some decisions to be made about txIDs and group IDs, but if you want to merge this and address them in a separate PR, that sounds fine to me.

@jannotti jannotti merged commit c579bf6 into algorand:master Oct 7, 2021
@jannotti jannotti deleted the inner-groups branch October 7, 2021 15:48
cce pushed a commit to cce/go-algorand that referenced this pull request Oct 28, 2021
itxn_next to give us inner groups

Questions remain around how TxID and GroupID should be defined for inners.
@egieseke egieseke mentioned this pull request Nov 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants