Design: Nymrefs for plethora of payments & purses

[zarutian]

Design issue: Vats that have lots of externally refered small objects in them get memory hoggy if those objects need to be all reified all the time.

Possible solution (or a start of one): Take the idea of KeyKos DataByte or seL4 badge and expand it to use an immutable string instead. A string specified at the time of the nymrefs creation. The size of such a string would be limited to 1 KibiBytes or even less.

So each vat has a device or some such we call nymref minter.
That minter gets exposed through nymrefManagerMaker inside the vat.
Each nymref is associated with one of these nymrefManagers.

  const prefix = "MyNymRefs";
  const nymman = nymrefManagerMaker(prefix);
  const nymrefA = nymman.mint("ABC");
  const nymrefstrA = nymman.deopaque(nymrefA);
  assert("ABC" == nymrefstrA);
  const nymrefAprime = nymman.mint(nymrefstrA);
  assert(nymrefA === nymrefAprime);
  const nymrefB = nymman.mint("DEF");
  assert(nymrefB !== nymrefA);
  const randoObj = {};
  assert(undefined == nymman.deopaque(randoObj));
  nymman.onMethodInvoke = (nymref, verb, args) => {
     return result;
  };
  nymman.onFunctionInvoke = (nymref, args) => {
    return result
  };
  nymman.onGet = (nymref, idx) => {
    return result;
  };
  nymman.onGC = (nymref) => {
    // returned value gets ignored.
  };

This can be used for ERTP Issuer Payments were the immutable string of each such nymref to a payment has two parts: an uuid or other such unique identifier (could be just a serial number), and the amount value. Then the vat where the issuer lives only has to keep track of the uuids of the live payments.

For purses, another nymrefManager would used and the immutable string of a nymref would only contain uuid or such.

This means the storage for the purses can be offloaded to some on-disk database.

Where would the immutable string of a nymref live? in the o+ or o- intervat identifiers. An example of such would be "o+42/MyNymRefs/17f5bd1a-b08c-4281-9c9a-e3ba17655fca 10000n".

Thoughts? Comments?

[zarutian]

An example of ERTP Issuer using nymref.

const makeIssuerKit_withNymrefPayments = (allegedName, amountMathKind) => {
  const liveSet = new Set();
  var paymentIdCounter = 0n;
  const nextPaymentId = () => (paymentIdCounter = paymentIdCounter + 1n);
  const nymmanPayments = nymrefManagerMaker("Payment_".concat(allegedName));
  nymmanPayments.onMethodInvoke = (nymref, verb, args) => {
    const nrstr = nymmanPayments.deopaque(nymref);
    if (undefined == nrstr) {
      throw new Error("this codepath should be unreachable");
    }
    if ("getAllegedBrand" == verb) { return brand; }
    throw new Error("no such method on a payment");
  };
  nymmanPayments.onGC = (nymref) => {
    const nymrefstr = nymmanPayments.deopaque(nymref);
    if (nymrefstr == undefined) { return; }
    const { id } = json.parse(nymrefstr);
    liveSet.delete(id);
  };
  const brand = harden({
    isMyIssuer: (other) => other == issuer,
    getAllegedName: () => allegedName,
  });
  const reiknir = makeAmountMath(brand, amountMathKind);
  const mint = harden({
    getIssuer: () => issuer,
    mintPayment: (newAmount) => {
      const value = reiknir.getValue(reiknir.coerce(newAmount));
      const id = nextPaymentId();
      liveSet.add(id);
      const j = json.stringify({ id , value});
      return nymmanPayments.mint(j);
    },
  });
  const issuer = harden({
    getAllegedName: () => allegedName,  
    getAmountMathKind: () => amountMathKind,
    getAmountOf: async (payment_) => {
      const payment = await payment_;
      const nymrefstr = nymmanPayments.deopaque(payment);
      if (undefined == nymrefstr) {
        throw new Error("thing given was not a payment of this issuance!");
      }
      const { id, value } = json.parse(nymrefstr);
      if (!liveSet.has(id)) { return reiknir.getEmpty(); }
      return reiknir.make(value);
    },
    getBrand: () => brand,
    makeEmptyPurse: () => new Error("this demonstration issuer does not support purses"),
    burn: async (payment_, optAmount) => {
      const payment = await payment_;
      const nymrefstr = nymmanPayments.deopaque(payment);
      if (undefined == nymrefstr) {
         throw new Error("thing given is not a payment of this issuance");
      }
      const { id, value } = json.parse(nymrefstr);
      if (!liveSet.has(id)) { throw new Error("it is dead, jim"); }
      const valueAmount = reiknir.make(value);
      if (optAmount == undefined) { optAmount = valueAmount; }
      if (!reiknir.isEqual(valueAmount, optAmount)) {
         throw new Error("the amount attempted to be burned was not equal to optAmount given");
      }
      liveSet.delete(id);
      return valueAmount;
    },
    claim: async (payment, optAmount) => {
      const valueAmount = await issuer.burn(payment, optAmount);
      return mint.mintPayment(valueAmount);
    },
    combine: async (paymentsArray_, optTotalAmount) => {
      const paymentsArray = await Promise.all(await paymentsArray_);
      if (!paymentsArray.reduce((a, i) => (a && issuer.isLive(i)), true)) {
        throw new Error("all must be alive and none dead");
      }
      paymentsArray.reduce((a, i) => {
        if (a.includes(i)) {
          throw new Error('aliased payment found');
        }
        a.push(i);
      }, []);
      const totalAmount = paymentsArray.reduce((a, i) => reiknir.add(a, issuer.getAmountOf(i)), reiknir.getEmpty());
      if (optTotalAmount == undefined) { optTotalAmount = totalAmount; }
      if (!reiknir.isEqual(totalAmount, optTotalAmount)) {
        throw new Error("optTotalAmount given not equal to totalAmount");
      }
      paymentsArray.forEach((payment) => issuer.burn(payment));
      return mint.mintPayment(totalAmount);
    },
    split: async (payment_, paymentAmountA) => {
      const payment = await payment_;
      const valueAmount = issuer.getAmountOf(payment);
      if (!reiknir.isGTE(valueAmount, paymentAmountA)) {
        throw new Error("insuffcient funds in payment given");
      }
      issuer.burn(payment);
      const paymentAmountB = reiknir.subtract(valueAmount, paymentAmountA);
      const paymentA = mint.mintPayment(paymentAmountA);
      const paymentB = mint.mintPayment(paymentAmountB);
      return harden([paymentA, paymentB]);
    },
    splitMany: async (payment_, amountArray) => {
      const payment = await payment_;
      const valueAmount = issuer.getAmountOf(payment);
      const totalAmount = amountArray.reduce((a, i) => reiknir.add(a, i), reiknir.getEmpty());
      if (!reiknir.isEqual(valueAmount, totalAmount)) {
        throw new Error("amounts in the array much cover the payment amount");
      }
      issuer.burn(payment);
      return harden(amountArray.map((amount) => mint.mintPayment(amount)));
    },
    isLive: async (payment_) => {
      const payment = await payment_;
      const nymrefstr = nymmanPayments.deopaque(payment);
      if (undefined == nymrefstr) { throw new Error("not a payment"); }
      const { id } = json.parse(nymrefstr);
      return liveSet.has(id);
    },
  });
  return harden({ issuer, mint, brand });
};

As you see, the only state kept in the vat where the issuer lives is the set of live payment ids.

[zarutian]

Not related but here is how you fake purses to support purse-less issuers like in the comment above:

import { E } from "@agoric/eventual-send";

const makePurse = (issuer) => {
  const innihald = [];
  const depositFacet = harden({
    receive: (payment) => purse.deposit(payment),
  });
  const reiknir = makeLocalAmountMath(issuer);
  const purse = harden({
    getCurrentAmount: () => innihald.map((it) => E(issuer).getAmountOf(it))
       .reduce(async (ac, it) => E(reiknir).add(await ac, await it), E(reiknir).getEmpty()),
    deposit: (payment, optAmount) => {
      const valueAmountP = E(issuer).getAmountOf(payment);
      return (async () => {
        const valueAmount = await valueAmountP;
        if (optAmount == undefined) { optAmount = valueAmount; }
        if (await E(reiknir).isEqual(valueAmount, optAmount)) {
          innihald.push(E(issuer).claim(payment));
          return valueAmount;
        } else {
          throw new Error("value amount of payment not equal to optAmount");
        }
      })();
    },
    getDepositFacet: () => depositFacet,
    getAllegedBrand: () => E(issuer).getBrand(),
    withdraw: async (amount) => {
      const funds = await purse.getCurrentAmount();
      if (! await E(reiknir).isGTE(funds, amount)) {
        throw new Error("insufficient funds for that withdrawal");
      }
      var tally = await E(reiknir).getEmpty();
      const greip = [];
      while (! await E(reiknir).isGTE(tally, amount)) {
        const p = innihald.shift();
        greip.push(p);
        tally = await E(reiknir).add(tally, await E(issuer).getAmountOf(p));
      }
      const p2 = E(issuer).combine(greip);
      if (await E(reiknir).isEqual(tally, amount)) {
        return p2;
      } else {
        const p3 = E(issuer).split(p2, amount);
        innihald.push(E.get(p3)[1]);
        return E.get(p3)[0];
      }
    },
    getCurrentAmountNotifier: () => throw new Error("currentAmountNotifier not supported in this purse implementation"),
    getCurrentAmountSubscription: () => throw new Error("not yet implemented"),
  });
  return purse;
};

But it looses the gurantee that .deposit() has regarding the passed in payment isnt a promise. But then again it is recommended to .then(), E.when(), or await on the promise returned by .deposit() specially when it was remote invoked.

[zarutian]

Low priorty paging of @michaelfig and @warner for when they are not swamped by the Beta launch.

[zarutian]

I just realized that this is basically a reinvention of virtual objects as described in #1960

Though the embedding of authorative data into the ref itself might be a new twist. I also note that this kind of technique might be usefull where macaroon-esque (or biscuit for that matter) or cert based bearer token is used. The added 'authorative data' can then live in the token itself.

[zarutian]

Easier link to virtual object documentation

[zarutian]

Unrelated but the liveSet in the example can be replaced by a Set implementation that only accepted positve bigints, sorted them, and stored the runs of (non-) existent entries like Apple Quickdraw masks stores scanlines.

( using wholeNums to run length encode preceeding non-existent entries followed by extistant entries. The repeat for how so ever many such runs pair you got)

[zarutian]

To clarify:

Where would the immutable string of a nymref live? in the o+ or o- intervat identifiers. An example of such would be "o+42/MyNymRefs/17f5bd1a-b08c-4281-9c9a-e3ba17655fca 10000n".

The immutable string of a nymref would only live in the c-list of a vat holding it and in any in transit messages where that nymref is being passed around.
That way only one entry per nymref manager need to exists in the kref table of the kernel and in the c-list of the vat where the nymref manager lives.
In essence, the c-list entries of nymrefs made by one nymref manager point to that nymref manager.

[zarutian]

jsdoc for idea above:

/*
 * @typedef {Object} NymRef
 */
/*
 * @typedef {Object} NymRefManager
 * @property {(string) => NymRef} mint
 * @property {(NymRef) => {string | undefined}} deopaque
 * @property {onMethodInvokeCallback} onMethodInvoke
 * @property {onFunctionInvokeCallback} onFunctionInvoke
 * @property {onGetCallback} onGet
 * @property {onGCCallback} onGC
 */
/*
 * @callback onMethodInvokeCallback
 * @param {NymRef} nymref
 * @param {string} verb
 * @param {Array<any>} args
 * @return {any}
 */
/*
 * @callback onFunctionInvokeCallback
 * @param {NymRef} nymref
 * @param {Array<any>} args
 * @return {any}
 */
/*
 * @callback onGetCallback
 * @param {NymRef} nymref
 * @param {any} idx
 * @return {any}
 */
/*
 * @callback onGCCallback
 * @param {NymRef} nymref
 * @return {void}
 */