Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARC56: Extended App Description #258

Open
wants to merge 57 commits into
base: main
Choose a base branch
from
Open
Changes from 24 commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
ef5f338
extended app description draft
joe-p Nov 15, 2023
6beb9f9
apply rob's suggestions
joe-p Nov 15, 2023
a18a968
feedback based on rob's suggestions
joe-p Nov 15, 2023
1265709
add comments to types
joe-p Nov 15, 2023
1b19b76
Assign number + fix linting
SudoWeezy Nov 16, 2023
45a4f6a
add errors key
joe-p Nov 21, 2023
96bf578
add defaultValue, fix typos
joe-p Mar 29, 2024
8c983a1
ARC28, formatting
joe-p Mar 29, 2024
a4c523e
add sourceInfo
joe-p Mar 29, 2024
f6a62f0
reorg spec
joe-p Mar 29, 2024
358a65b
add arcs key
joe-p Mar 29, 2024
d9046e1
network names
joe-p Mar 29, 2024
9a5e445
fix arc format
joe-p Mar 29, 2024
ef47b89
require ARC28
joe-p Apr 24, 2024
8e3499b
add Method.recommendations, Contract.templateVariables
joe-p Apr 24, 2024
b02c3c1
add template vars to spec
joe-p Apr 24, 2024
2a5eb6f
requires order
joe-p Apr 24, 2024
cebfc7d
minor updates
joe-p Apr 26, 2024
e6115f0
pushint/pushbyte opcdes, formatting
joe-p Apr 29, 2024
fd8fe2a
add @robdmoore as author
joe-p Apr 29, 2024
89a6c0c
error -> errorMessage
joe-p Apr 29, 2024
35fd3fc
Storage interface update
joe-p May 6, 2024
8572596
use names as keys for template variables and state keys/maps
joe-p May 8, 2024
ac0a9b1
make readonly optional
joe-p May 8, 2024
05cd0d5
add scratchVariables
joe-p May 20, 2024
ab956a7
add value? to templateVariables
joe-p May 20, 2024
22372b2
add btyeCode
joe-p May 20, 2024
8ee7866
make teal optional
joe-p May 20, 2024
f82a2fa
make scratchVariables optional
joe-p May 20, 2024
ad5d033
add compilerVersion
joe-p May 21, 2024
e540d92
compilerInfo
joe-p May 21, 2024
7b3982b
add disassembledTeal to Sourceinfo
joe-p Jul 28, 2024
f30b5f6
base64 prefix
joe-p Aug 6, 2024
59edfa8
add approval/clear keys to sourceInfo
joe-p Sep 4, 2024
7f5816a
commit -> commitHash
joe-p Sep 11, 2024
9908f86
pushbytes for TMPL
joe-p Sep 11, 2024
faf5f4b
move networks tsdoc
joe-p Sep 18, 2024
506acbd
state default values
joe-p Sep 18, 2024
d8d4c46
type/struct tsdoc
joe-p Sep 18, 2024
6bc830c
StructFields -> StructField[]
joe-p Sep 18, 2024
da85edf
AVMString
joe-p Sep 18, 2024
3ef93fc
StructField[]
joe-p Sep 25, 2024
e577cfb
specify utf-8
joe-p Sep 25, 2024
58a133d
AVMUint64 and AVMType
joe-p Sep 25, 2024
aa79e86
Apply suggestions from code review
joe-p Oct 2, 2024
35a19b3
ProgramSourceInfo
joe-p Oct 10, 2024
3476dad
added method as default arg source
joe-p Oct 10, 2024
1e31ba0
update template variables spec
joe-p Oct 10, 2024
19c6269
only inlcude pc and errorMessage in SourceInfo
joe-p Oct 11, 2024
017c6e3
Apply suggestions from code review
joe-p Oct 16, 2024
d58be41
fix typo
joe-p Oct 16, 2024
9ba13f1
remove ClearState
joe-p Oct 16, 2024
20369ed
defaultValue fields
joe-p Oct 16, 2024
027462f
make errorMessage optional, add optional teal and source to SourceInfo
joe-p Oct 23, 2024
12e3a53
teal string -> number
joe-p Oct 23, 2024
8e2db8d
expand cblocks reference
joe-p Oct 23, 2024
9a3f705
fix typo
joe-p Oct 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
343 changes: 343 additions & 0 deletions ARCs/arc-0056.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,343 @@
---
arc: 56
title: Extended App Description
description: Adds information to the ABI JSON description
author: Joe Polny (@joe-p), Rob Moore (@robdmoore)
discussions-to: https://github.com/algorandfoundation/ARCs/issues/258
status: Draft
type: Standards Track
category: ARC
created: 2023-11-14
requires: 4, 22, 28
---

## Abstract

This ARC takes the existing JSON description of a contract as described in [ARC-4](./arc-0004.md) and adds more fields for the purpose of client interaction

## Motivation

The data provided by ARC-4 is missing a lot of critical information that clients should know when interacting with an app. This means ARC-4 is insufficient to generate type-safe clients that provide a superior developer experience.

On the other hand, [ARC-32](./arc-0032.md) provides the vast majority of useful information that can be used to <a href="https://github.com/algorandfoundation/algokit-cli/blob/main/docs/features/generate.md#1-typed-clients">generate typed clients</a>, but requires a separate JSON file on top of the ARC-4 json file, which adds extra complexity and cognitive overhead.

## Specification

### Contract Interface

Every application is described via the following interface which is an extension of the `Contract` interface described in [ARC-4](./arc-0004.md).

```ts
/** Describes the entire contract. This interface is an extension of the interface described in ARC-4 */
interface Contract {
/** The ARCs used and/or supported by this contract. All contracts implicity support ARC4 and ARC56 */
joe-p marked this conversation as resolved.
Show resolved Hide resolved
joe-p marked this conversation as resolved.
Show resolved Hide resolved
arcs: number[];
/** A user-friendly name for the contract */
name: string;
/** Optional, user-friendly description for the interface */
desc?: string;
/**
* Optional object listing the contract instances across different networks
*/
networks?: {
/**
* The key is the base64 genesis hash of the network, and the value contains
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This description should probably be added to the networks key above otherwise you don't see it in intellisense

* information about the deployed contract in the network indicated by the
* key. A key containing the human-readable name of the network MAY be
* included, but the corresponding genesis hash key MUST also be defined
*/
[network: string]: {
/** The app ID of the deployed contract in this network */
appID: number;
joe-p marked this conversation as resolved.
Show resolved Hide resolved
};
};
/** Named structs use by the application */
structs: { [structName: StructName]: StructFields };
/** All of the methods that the contract implements */
methods: Method[];
state: {
/** Defines the values that should be used for GlobalNumUint, GlobalNumByteSlice, LocalNumUint, and LocalNumByteSlice when creating the application */
schema: {
global: {
ints: number;
bytes: number;
};
local: {
ints: number;
bytes: number;
};
};
/** Mapping of human-readable names to StorageKey objects */
keys: {
global: { [name: string]: StorageKey };
local: { [name: string]: StorageKey };
box: { [name: string]: StorageKey };
};
/** Mapping of human-readable names to StorageMap objects */
maps: {
global: { [name: string]: StorageMap };
local: { [name: string]: StorageMap };
box: { [name: string]: StorageMap };
};
};
/** Supported bare actions for the contract. An action is a combination of call/create and an OnComplete */
bareActions: {
/** OnCompletes this method allows when appID === 0 */
create: ("NoOp" | "OptIn" | "DeleteApplication")[];
/** OnCompletes this method allows when appID !== 0 */
call: (
| "NoOp"
| "OptIn"
| "CloseOut"
| "ClearState"
| "UpdateApplication"
| "DeleteApplication"
)[];
};
/** Information about the TEAL */
sourceInfo?: SourceInfo[];
joe-p marked this conversation as resolved.
Show resolved Hide resolved
/** The pre-compiled TEAL that may contain template variables. MUST be omitted if included as part of ARC23, but otherwise MUST be defined. */
source?: {
/** The approval program */
approval: string;
joe-p marked this conversation as resolved.
Show resolved Hide resolved
/** The clear program */
clear: string;
};
/** ARC-28 events that MAY be emitted by this contract */
events?: Array<Event>;
joe-p marked this conversation as resolved.
Show resolved Hide resolved
/** A mapping of template variable names as they appear in the teal (not including TMPL_ prefix) and their respecive types */

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There doesn't seem to be any harm if we include the TMPL_ prefix? The less assumptions (within the bounds of practicality) the better, no?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But if we use it for code generation then you’d have to strip the TMPL so we have to cater for it somewhere?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If certain tooling expects that as a prefix, that seems like the place to check for and strip that prefix, yeah?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shrug could go either way, I guess it's fair that this could be more general purpose and then the generators are clever enough to detect and strip TMPL_ when emitting code...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My rationale omitting the TMPL_ here and forcing tooling to use it is to make it easier to implement safely. For example, if ARC56 allowed anything as the variable name and the tooling didn't have to add TMPL_, it would need to check all the variable names against all opcodes. Otherwise an ARC56 file could have a template variable named byte which subsequently overwrites all byte opcodes which is not the intended effect of template variables.

templateVariables?: {
joe-p marked this conversation as resolved.
Show resolved Hide resolved
[name: string]: ABIType | AVMBytes | StructName;
};
}
```

### Method Interface

Every method in the contract is described via a `Method` interface. This interface is an extension of the one defined in [ARC-4](./arc-0004.md).

```ts
/** Describes a method in the contract. This interface is an extension of the interface described in ARC-4 */
interface Method {
/** The name of the method */
name: string;
/** Optional, user-friendly description for the method */
desc?: string;
/** The arguments of the method, in order */
args: Array<{
/** The type of the argument */
type: ABIType;
joe-p marked this conversation as resolved.
Show resolved Hide resolved
/** If the type is a struct, the name of the struct */
struct?: StructName;
joe-p marked this conversation as resolved.
Show resolved Hide resolved
/** Optional, user-friendly name for the argument */
name?: string;
/** Optional, user-friendly description for the argument */
desc?: string;
/** The default value that clients should use. MUST be base64 encoded bytes */
defaultValue?: string;
}>;
/** Information about the method's return value */
returns: {
/** The type of the return value, or "void" to indicate no return value. */
type: ABIType;
/** If the type is a struct, the name of the struct */
struct?: StructName;
/** Optional, user-friendly description for the return value */
desc?: string;
};
/** an action is a combination of call/create and an OnComplete */
actions: {
/** OnCompletes this method allows when appID === 0 */
create: ("NoOp" | "OptIn" | "DeleteApplication")[];
/** OnCompletes this method allows when appID !== 0 */
call: (
| "NoOp"
| "OptIn"
| "CloseOut"
| "ClearState"
joe-p marked this conversation as resolved.
Show resolved Hide resolved
| "UpdateApplication"
| "DeleteApplication"
)[];
};
/** If this method does not write anything to the ledger (ARC-22) */
readonly?: boolean;
/** ARC-28 events that MAY be emitted by this method */
events?: Array<Event>;
/** Information that clients can use when calling the method */
recommendations?: {
/** The number of inner transactions the caller should cover the fees for */
innerTransactionCount?: number;
/** Recommended box references to include */
boxes?: {
/** The app ID for the box */
app?: number;
/** The base64 encoded box key */
key: string;
/** The number of bytes being read from the box */
readBytes: number;
/** The number of bytes being written to the box */
writeBytes: number;
};
/** Recommended foreign accounts */
accounts?: string[];
/** Recommended foreign apps */
apps?: number[];
/** Recommended foreign assets */
assets?: number[];
};
}
```

### Event Interface

[ARC-28](./arc-0028.md) events are described using an extension of the original interface described in the ARC, with the addition of an optional struct field for arguments

```ts
interface Event {
/** The name of the event */
name: string;
/** Optional, user-friendly description for the event */
desc?: string;
/** The arguments of the event, in order */
args: Array<{
/** The type of the argument */
type: ABIType;
joe-p marked this conversation as resolved.
Show resolved Hide resolved
/** Optional, user-friendly name for the argument */
name?: string;
/** Optional, user-friendly description for the argument */
desc?: string;
/** If the type is a struct, the name of the struct */
struct?: StructName;
}>;
}
```

### Type Interfaces

The types defined in [ARC-4](./arc-0004.md) may not fully described the best way to use the ABI values as intended by the contract developers. These type interfaces are intended to supplement ABI types so clients can interact with the contract as intended.

```ts
/** An ABI-encoded type */
type ABIType = string;

/** The name of a defined struct */
type StructName = string;

/** Raw byteslice without the length prefixed that is specified in ARC-4 */
type AVMBytes = "bytes";

/** Mapping of named structs to the ABI type of their fields */
interface StructFields {
[fieldName: string]: ABIType | StructFields;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It occurs to me that this format requires property order to be maintained to be able to map them to the underlying tuple type. Are we comfortable that will always be the case across all languages?

https://www.rfc-editor.org/rfc/rfc7159 says (emphasis mine):

An object is an unordered collection of zero or more name/value pairs, where a name is a string and a value is a string, number, boolean, null, object, or array.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this a choice for the HLL. The struct have to have a specific order for ABI encoding, so the HLL can either implement an ordered struct or not expose ways of iterating keys/values (like in TEALScript)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But this spec determines the format of the json file? If the json file is an object and the programming language parsing it doesn’t implemented ordered iteration of object keys you are hosed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right ok I see what you are saying. So I think it should be StructFields = { fieldName: string, type: ABIType | StructFields }[] to preserve order as an array, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spot on I think

}
```

### Storage Interfaces

These interfaces properly describe how app storage is access within the contract

```ts
/** Describes a single key in app storage */
interface StorageKey {
/** Description of what this storage key holds */
desc?: string;
joe-p marked this conversation as resolved.
Show resolved Hide resolved
/** The type of the key */
keyType: ABIType | AVMBytes | StructName;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does StorageKey have keyType? Isn't this for singular, known values?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes but I figured it's still useful to know the type for things like explorers and frontends

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So it's intended to indicate the encoding of key? Maybe keyEncoding would be clearer? I don't think it benefits from consistency of naming with StorageMap.keyType since that seems of a different purpose - to indicate what sort of value to encode to combine with StorageMap.key in order to retrieve a value. Whereas StorageKey.key is already encoded and you may want to know how to interpret that value. So they're kind of opposites?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'm a bit confused about what keyType means?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wait, I think I see.

So what should this and the key value be for a string value like we had in arc-32?

Needing to have key as a base64 reduces the human readability of these files... I wonder if keyType should allow for a raw string to be used?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this would be the equivalent?

          key: Buffer.from('my key', 'utf-8').toString("base64"),
          keyType: 'bytes',

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But then that doesn't let explorers etc. know that the key is actually a (non-ARC4) string?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be confusing to have some keys be ASCII encoded and some keys be base64, thus everything base64 makes the most sense to me. I'm not sure if human readability is all that important. If you want to make it human readable just through it into Lora (once supported).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Human readable is a nice feature though - makes it easy to see things at a glance, I've certainly found that useful in the past.

Regardless if that's supported or not, it probably makes sense to support a keyType of utf-8, which would either mean non-base64 or base64 bytes, but representing a UTF-8 string so the likes of Lora can show it properly.

Also, similarly we should probably have a key type of avm_address or similar to denote when you use the public key bytes of an address directly (i.e. non-ARC4 encoding).

/** The type of the value */
valueType: ABIType | AVMBytes | StructName;
/** The bytes of the key encoded as base64 */
key: string;
}

/** Describes a mapping of key-value pairs in storage */
interface StorageMap {
/** Description of what the key-value pairs in this mapping hold */
desc?: string;
/** The type of the keys in the map */
keyType: ABIType | AVMBytes | StructName;
/** The type of the values in the map */
valueType: ABIType | AVMBytes | StructName;
/** The prefix of the map, encoded as a utf-8 string */
prefix?: string;
joe-p marked this conversation as resolved.
Show resolved Hide resolved
}
```

### SourceInfo Interface

This interface gives clients more information about the contract's source code.

```ts
interface SourceInfo {
joe-p marked this conversation as resolved.
Show resolved Hide resolved
/** The line of pre-compiled TEAL */
teal: number;
/** The program counter offset(s) that correspond to this line of TEAL */
pc?: Array<number>;
/** A human-readable string that describes the error when the program fails at this given line of TEAL */
errorMessage?: string;
}
```

### Template Variables

Template variables are variables in the TEAL that should be substitued prior to compilation. Template variables **MUST** be an argument to either `pushint` or `pusbyte` (not the pseudo opcodes `int` or `byte`). Using these opcodes is important to ensure that the program does not add them to the constant blocks.

#### Examples

##### Valid

```
pushint TMPL_FOO
pushbyte TMPL_BAR
```

#### Invalid

Template variables MUST use `pushint` or `pushbyte`

```
int TMPL_FOO
byte TMPL_BYTE
```

Template variables MUST be the entire argument

```
pushint 123TMPL_FOO
pusbyte 0xdeadTMPL_BAR
```

## Rationale

ARC-32 essentially addresses the same problem, but it requires the generation of two separate JSON files and the ARC-32 JSON file contains the ARC-4 JSON file within it (redundant information). The goal of this ARC is to create one JSON schema that is backwards compatible with ARC-4 clients, but contains the relevant information needed to automatically generate comprehensive client experiences.

### State

Describes all of the state that MAY exist in the app and how one should decode values. The schema provides the required schema when creating the app.

### Named Structs

It is common for high-level languages to support named structs, which gives names to the indexes of elements in an ABI tuple. The same structs should be useable on the client-side just as they are used in the contract.

### Action

This is one of the biggest deviation from ARC-32, but provides a much simpler interface to describe and understand what any given method can do.

## Backwards Compatibility

The JSON schema defined in this ARC should be compatible with all ARC-4 clients, provided they don't do any strict schema checking for extraneous fields.

## Test Cases

NA

## Reference Implementation

TODO

## Security Considerations

The type values used in methods **MUST** be correct, because if they were not then the method would not be callable. For state, however, it is possible to have an incorrect type encoding defined. Any significant security concern from this possibility is not immediately evident, but it is worth considering.

## Copyright

Copyright and related rights waived via <a href="https://creativecommons.org/publicdomain/zero/1.0/">CCO</a>.
Loading