Can a function that returns tuple data return an Object instead of an Array?

[luchenqun]

Ethers Version

6.11.1

Search Terms

tuple,object,array

Describe the Problem

struct Description {
    string moniker;
    string identity;
    string website;
    string securityContact;
    string details;
}

struct CommissionRates {
    uint256 rate;
    uint256 maxRate;
    uint256 maxChangeRate;
}

enum BondStatus {
    Unspecified,
    Unbonded,
    Unbonding,
    Bonded
}

struct Validator {
    address operatorAddress;
    string consensusPubkey;
    bool jailed;
    BondStatus status;
    uint256 tokens;
    uint256 delegatorShares;
    Description description;
    int64 unbondingHeight;
    int64 unbondingTime;
    Commission commission;
    uint256 minSelfDelegation;
}

  function validator(
      address validatorAddr
  ) external view returns (Validator calldata validator);

This is an interface I defined for querying validators based on addresses. The data returned from the query is as follows:

[
  "0x7a24464c2A92C3774f1C7b0FFCbeee759Fa9934E",
  "ckQp0mBGYUnbBf1v3PJC4nkdGDuSW2MYO5CazB83J+E=",
  false,
  "3",
  "100000000000000000000",
  "100000000000000000000000000000000000000",
  [
    "node0",
    "",
    "",
    "",
    ""
  ],
  "0",
  "0",
  [
    [
      "100000000000000000",
      "1000000000000000000",
      "1000000000000000000"
    ],
    "1712309039"
  ],
  "1"
]

However, returning data in the form of an Array is not very convenient for frontend usage. Is it possible to return the data in the form of an Object, like the following:

{
  "operatorAddress": "0x7a24464c2A92C3774f1C7b0FFCbeee759Fa9934E",
  "consensusPubkey": "ckQp0mBGYUnbBf1v3PJC4nkdGDuSW2MYO5CazB83J+E=",
  "jailed": false,
  "status": "Bonded",
  "tokens": "100000000000000000000",
  "delegatorShares": "100000000000000000000000000000000000000",
  "description": {
    "moniker": "node0",
    "identity": "",
    "website": "",
    "securityContact": "",
    "details": ""
  },
  "unbondingHeight": 0,
  "unbondingTime": 0,
  "commission": {
    "rate": "100000000000000000",
    "maxRate": "1000000000000000000",
    "maxChangeRate": "1000000000000000000"
  },
  "minSelfDelegation": "1"
}

Code Snippet

import { ethers } from 'ethers';
import fs from 'fs-extra';
import path from 'path';

BigInt.prototype.toJSON = function () {
  return this.toString();
};

export const main = async () => {
  const { rpc, contracts, stakingAddress } = await fs.readJSON('../cfg.json');
  const { abi } = await fs.readJSON(path.join(contracts, 'staking/IStaking.sol/IStaking.json'));
  const provider = new ethers.JsonRpcProvider(rpc);

  // input params
  const validatorAddress = '0x7a24464c2A92C3774f1C7b0FFCbeee759Fa9934E';

  const staking = new ethers.Contract(stakingAddress, abi, provider);
  const validator = await staking.validator(validatorAddress);
  console.log('validator', JSON.stringify(validator, undefined, 2));
};

main();

Contract ABI

{
    "inputs": [
        {
            "internalType": "address",
            "name": "validatorAddr",
            "type": "address"
        }
    ],
    "name": "validator",
    "outputs": [
        {
            "components": [
                {
                    "internalType": "address",
                    "name": "operatorAddress",
                    "type": "address"
                },
                {
                    "internalType": "string",
                    "name": "consensusPubkey",
                    "type": "string"
                },
                {
                    "internalType": "bool",
                    "name": "jailed",
                    "type": "bool"
                },
                {
                    "internalType": "enum BondStatus",
                    "name": "status",
                    "type": "uint8"
                },
                {
                    "internalType": "uint256",
                    "name": "tokens",
                    "type": "uint256"
                },
                {
                    "internalType": "uint256",
                    "name": "delegatorShares",
                    "type": "uint256"
                },
                {
                    "components": [
                        {
                            "internalType": "string",
                            "name": "moniker",
                            "type": "string"
                        },
                        {
                            "internalType": "string",
                            "name": "identity",
                            "type": "string"
                        },
                        {
                            "internalType": "string",
                            "name": "website",
                            "type": "string"
                        },
                        {
                            "internalType": "string",
                            "name": "securityContact",
                            "type": "string"
                        },
                        {
                            "internalType": "string",
                            "name": "details",
                            "type": "string"
                        }
                    ],
                    "internalType": "struct Description",
                    "name": "description",
                    "type": "tuple"
                },
                {
                    "internalType": "int64",
                    "name": "unbondingHeight",
                    "type": "int64"
                },
                {
                    "internalType": "int64",
                    "name": "unbondingTime",
                    "type": "int64"
                },
                {
                    "components": [
                        {
                            "components": [
                                {
                                    "internalType": "uint256",
                                    "name": "rate",
                                    "type": "uint256"
                                },
                                {
                                    "internalType": "uint256",
                                    "name": "maxRate",
                                    "type": "uint256"
                                },
                                {
                                    "internalType": "uint256",
                                    "name": "maxChangeRate",
                                    "type": "uint256"
                                }
                            ],
                            "internalType": "struct CommissionRates",
                            "name": "commissionRates",
                            "type": "tuple"
                        },
                        {
                            "internalType": "int64",
                            "name": "updateTime",
                            "type": "int64"
                        }
                    ],
                    "internalType": "struct Commission",
                    "name": "commission",
                    "type": "tuple"
                },
                {
                    "internalType": "uint256",
                    "name": "minSelfDelegation",
                    "type": "uint256"
                }
            ],
            "internalType": "struct Validator",
            "name": "validator",
            "type": "tuple"
        }
    ],
    "stateMutability": "view",
    "type": "function"
}

Errors

Can a function that returns tuple data return an Object instead of an Array?

Environment

node.js (v12 or newer)

Environment (Other)

no

[ricmoo]

The object returned from a call is a Result object, which sub-classes Array (so result[0] works), but the class is also implemented using an ES6 proxy, so if you have a property in your struct called foo then result.foo also works.

There is also a result.toObject() return a normal object with properties set on it and a result,toArray() which will return a non-proxy bare Array.

I think that should do what you need?

[luchenqun]

Thank you for your prompt reply. result.toObject() is very close to what I want. However, I found that the result returned by result.toObject() is as follows:

{
  "operatorAddress": "0x7a24464c2A92C3774f1C7b0FFCbeee759Fa9934E",
  "consensusPubkey": "ckQp0mBGYUnbBf1v3PJC4nkdGDuSW2MYO5CazB83J+E=",
  "jailed": false,
  "status": "3",
  "tokens": "100000000000000000000",
  "delegatorShares": "100000000000000000000000000000000000000",
  "description": [
    "node0",
    "",
    "",
    "",
    ""
  ],
  "unbondingHeight": "0",
  "unbondingTime": "0",
  "commission": [
    [
      "100000000000000000",
      "1000000000000000000",
      "1000000000000000000"
    ],
    "1712309039"
  ],
  "minSelfDelegation": "1"
}

Only the values corresponding to the keys in the first layer have been converted, while the second layer such as description and commission are still in array form. Do I have to recursively process the value corresponding to each key?

[ricmoo]

Ah yes. I do believe it isn’t recursive. There was a reason for that, which I cannot recall. I can likely add a recursive version, perhaps add an optional parameter to toObject(deep?: boolean)?

I think it had to do with certain ABI (which were popular) that do not include names for deeply nested structs. But an optional parameter seems safe.

Changing this issue to a feature request. :)

[luchenqun]

Thank you for your response. I think adding an optional parameter deep is a great design.

[ricmoo]

Added in v6.12.0. Try it out and let me know if you have any problems.

Thanks! :)

[luchenqun]

I just quickly tried it, and yes, the struct is working as expected for me now. However, there is still an issue with struct arrays. For example, with the following Solidity interface:

function validators(
    BondStatus status,
    PageRequest calldata pagination
) external view returns (Validator[] calldata validators, PageResponse calldata pageResponse);

When I call it using ethers like this:

const validators = await staking.validators(status, pageRequest);
console.log('validators', validators.toObject(true));

I get the following error.

node_modules/ethers/lib.esm/utils/errors.js:124
            error = new Error(message);
                    ^

Error: value at index ${ index } unnamed (operation="toObject()", code=UNSUPPORTED_OPERATION, version=6.12.0)
    at makeError (file:///Users/lcq/Code/ethos/precompile/node_modules/ethers/lib.esm/utils/errors.js:124:21)
    at assert (file:///Users/lcq/Code/ethos/precompile/node_modules/ethers/lib.esm/utils/errors.js:143:15)
    at file:///Users/lcq/Code/ethos/precompile/node_modules/ethers/lib.esm/abi/coders/abstract-coder.js:135:13
    at Array.reduce (<anonymous>)
    at Result.toObject (file:///Users/lcq/Code/ethos/precompile/node_modules/ethers/lib.esm/abi/coders/abstract-coder.js:134:28)
    at Proxy.<anonymous> (file:///Users/lcq/Code/ethos/precompile/node_modules/ethers/lib.esm/abi/coders/abstract-coder.js:93:42)
    at file:///Users/lcq/Code/ethos/precompile/node_modules/ethers/lib.esm/abi/coders/abstract-coder.js:142:35
    at Array.reduce (<anonymous>)
    at Result.toObject (file:///Users/lcq/Code/ethos/precompile/node_modules/ethers/lib.esm/abi/coders/abstract-coder.js:134:28)
    at Proxy.<anonymous> (file:///Users/lcq/Code/ethos/precompile/node_modules/ethers/lib.esm/abi/coders/abstract-coder.js:93:42) {
  code: 'UNSUPPORTED_OPERATION',
  operation: 'toObject()',
  shortMessage: 'value at index ${ index } unnamed'
}

[ricmoo]

Ah yes... Because Arrays use an anonymous coder for their child... Looking into it.

[ricmoo]

There isn't actually a (backwards compatible) way to detect if the type should be an array. In the future, I should have unpack return string | null | undefined, where null is unnamed, and undefined is "nameless" (such as an array). The change is quite simple, but breaks backwards compatibility.

For now, What do you think of this idea (inside toObject per child):

if (deep && child instanceof Result) {
  try {
    child = child.toObject(deep);
  } catch (error) {
    if (isError(error, "UNSUPPORTED_OPERATION") && error.operation === "toObject()") {
      child = child.toArray();
    } else {
      throw error;
    }
  }
}

This means that any Array will correctly get converted to an Array, but also any tuple with unnamed properties will get folded into an Array.

This may be what we actually want anyways, as it makes the method more robust against lossy ABI fragments.

However, to retain the top-level type of Record<string, any>, the top level must always be named. Perhaps in a future version the return type should be Record<string, ResultType> | Array<ResultType> (where ResultType is narrowed only to the types that Results can contain).

[luchenqun]

I just tried this way, and I can convert each element inside the array to an Object.

const validators = await staking.validators(status, pageRequest);
// console.log('validators', validators.toObject());
for (const validator of validators[0]) {
  console.log('validator', validator.toObject(true));
}

But what confuses me is why iterate over validators[0] instead of iterating over validators directly?

[ricmoo]

I think it's because your return type is (Validator[] calldata validators, PageResponse calldata pageResponse), which has 2 values.

So (I think?) what you would really want is const [ validators, pageResponses ] = await staking.validators(status, pageReqquest)?

Might be more obvious if you had const result = await stacking.validators(...); const validators = result[0]; for (const validator of validators) { ... }.

[luchenqun]

Sorry, it was my mistake. Your are completely correct. It should indeed be written like this:

const [validators, pageResponses] = await staking.validators(status, pageRequest);

[ricmoo]

Everything should be kosher with Array types now, as of v6.13.0.

Let me know if you have any more issues.

Thanks! :)

[luchenqun]

It seems like there is still an issue for array use `toObject(true)`

[ricmoo]

Aiya. I’ll look into it first thing in the morning. :s

[andrevmatos]

I also see this happening: specially, empty arrays in properties get converted to empty objects {} when toObject(true), which break other things. It'd be nice to have a standardized way to convert structs to objects, while deeply keeping arrays and unnamed tuples as such. Thank you for your great work on this!

[ricmoo]

@andrevmatos The objects returned are fully populated. The problem you are seeing is a limitation of console.log, but if a function returns a tuple, it is a Result instance. You can call .toObject() on it to reduce it to a normal JavaScript object though, but then you lose Array prototypes on it.

The Result class is a sub-class of Array with a bunch of extra features, including dynamic access to any value by name (like you are looking for).

Hope that helps. :)

[andrevmatos]

Yes, I understand that; What I'd like is some method which could convert a Result of struct MyStruct { address a; tuple(uint128 c; uint128 d)[] b } into {a: string; b: {c: bigint; d: bigint}[]}, i.e. named tuples to objects, and unnamed tuples and arrays to JSON arrays, for serialization;
Currently, Result.toObject(true) does almost that, but if b is empty, it returns {} (empty object), instead of [] (empty array), and can break serializers.