Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: nft metadata service #769

Merged
merged 71 commits into from
Sep 15, 2023
Merged

feat: nft metadata service #769

merged 71 commits into from
Sep 15, 2023

Commits on Sep 14, 2023

  1. fix(cardano-services-client): use service version instead of root ver… 

    …sion
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    cf9b173 View commit details
    Browse the repository at this point in the history

  2. test(cardano-services): refactor HttpServer tests to work with non-ma… 

    …tching root and service version
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    c31603e View commit details
    Browse the repository at this point in the history

  3. feat!: update core types with deserialized PlutusData 

    BREAKING CHANGE: incompatible with previous revisions of cardano-services
- rename utxo and transactions PouchDB stores
- update type of Tx.witness.redeemers
- update type of Tx.witness.datums
- update type of TxOut.datum
- remove Cardano.Datum type

fix(cardano-services): correct chain history openApi endpoints path url to match version
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    d8cc93b View commit details
    Browse the repository at this point in the history

  4. fix(cardano-services): do not return datum hash for TxOut with inline… 

    … datum
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    8928869 View commit details
    Browse the repository at this point in the history

  5. feat(util-dev): add with-inline-datum.json chain sync data 

    in order to test ogmiosToCore inline datum mapper

to be used for testing cip68 metadata projection
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    ff0b923 View commit details
    Browse the repository at this point in the history

  6. refactor!: hoist metadatumToCip25 to NftMetadata.fromMetadatum 

    NftMetadata type shouldn't be coupled to cip25, because there are
now 2 sources for NFT metadata:
- cip25: auxiliary data (tx metadata)
- cip68/222: datum
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    c36d7ef View commit details
    Browse the repository at this point in the history

  7. refactor(core): loosen arg type of isPlutusData utils to accept 'unkn… 

    …own'
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    91e1e7a View commit details
    Browse the repository at this point in the history

  8. feat(core): add hexToBytes, utf8ToBytes, utf8ToHex utils

    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    d3da1a6 View commit details
    Browse the repository at this point in the history

  9. feat(core): add NftMetadata.fromPlutusData mapping from cip68 datum

    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    64b263b View commit details
    Browse the repository at this point in the history

  10. ci: enable unit and e2e tests for non-master targeting PRs

    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    45cfecb View commit details
    Browse the repository at this point in the history

  11. fix: correct ogmiosToCore auxiliaryData mapping 

    ogmiosToCore used jsonToMetadatum util, which mapping json
that is of different format from ogmios schema
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    eb0ddc0 View commit details
    Browse the repository at this point in the history

  12. refactor!: renamed field handle to handleResolutions

    @iadmytro @mkazlauskas
    iadmytro authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    8b3296e View commit details
    Browse the repository at this point in the history

  13. refactor(core): convert AssetNameLabelNum into object 

    enum is not convenient to use, because it's values aren't OpaqueNumber
    @greatertomi @mkazlauskas
    greatertomi authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    d2610d7 View commit details
    Browse the repository at this point in the history

  14. feat: add NFT metadata projection 

    - add Mappers.withCIP67
- add cip67 assetName to withMint Mapper
- add Mapper.withNftMetadata
- add storeNftMetadata operator
- refactor entities by adding OnDeleteCascadeRelationOptions relation option
    @greatertomi @mkazlauskas
    greatertomi authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    91fe7df View commit details
    Browse the repository at this point in the history

  15. refactor: update nullable TypeORM entity field types to include '| null' 

    querying a nullable column can return 'null' - entity types should reflect that
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    e654196 View commit details
    Browse the repository at this point in the history

  16. test(core): add a failing test for PlutusData

    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    bf6c9e2 View commit details
    Browse the repository at this point in the history

  17. fix(core): do not log a warning when nft metadata files are missing

    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    b79419b View commit details
    Browse the repository at this point in the history

  18. feat(core): added custom PlutusData serialization classes

    @AngelCastilloB @mkazlauskas
    AngelCastilloB authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    72e600c View commit details
    Browse the repository at this point in the history

  19. feat(projection): ignore burn transactions when projecting cip25 meta… 

    …data

cardano-foundation/CIPs#527
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    ab8bb29 View commit details
    Browse the repository at this point in the history

  20. fix!: convert tokens.quantity column to numeric 

    it does not fit to Postgres bigint

also, fix the parseBigInt transformer by serializing to str:
TypeORM seems to be working incosistently
and accepts js BigInt on INSERT, but not WHERE
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    31b0f0a View commit details
    Browse the repository at this point in the history

  21. test(projection-typeorm): hoist createRollForwardEventBasedOn util fr… 

    …om storeNftMetadata.test.ts
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    5e77b5d View commit details
    Browse the repository at this point in the history

  22. fix(util-dev): add missing 'inputSource' prop to some chainSync datasets

    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    0b236c9 View commit details
    Browse the repository at this point in the history

  23. feat: add address projection 

    - new mappers: withStakeKeyRegistrations, withAddresses
- new stores: storeStakeKeyRegistrations, storeAddresses
- new projection type/name in cardano-services: 'address'
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    416e5f5 View commit details
    Browse the repository at this point in the history

  24. test(cardano-services): re-generated fixtures

    @iccicci @mkazlauskas
    iccicci authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    0b6aa71 View commit details
    Browse the repository at this point in the history

  25. feat(cardano-services): create typeorm nft-metadata service

    @greatertomi @mkazlauskas
    greatertomi authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    72f6c1b View commit details
    Browse the repository at this point in the history

  26. refactor(cardano-services): extend typeormProvider with typeormService

    @greatertomi @mkazlauskas
    greatertomi authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    6016a65 View commit details
    Browse the repository at this point in the history

  27. refactor!: remove AssetInfo.history and AssetInfo.mintOrBurnCount

    @greatertomi @mkazlauskas
    greatertomi authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    4c0a7ee View commit details
    Browse the repository at this point in the history

  28. feat(cardano-services): create TypeormAssetProvider 

    - add use-typeorm-asset-provider option to cli
- update provider server to use both DbSync and Typeorm asset provider.
- fix typeorm stake pool provider test to use correct version
    @greatertomi @mkazlauskas
    greatertomi authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    aaf133b View commit details
    Browse the repository at this point in the history

  29. fix(core): bytes field on core plutus script type now contains the co… 

    …mpiled bytes instead of cbor
    @AngelCastilloB @mkazlauskas
    AngelCastilloB authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    15a6ba6 View commit details
    Browse the repository at this point in the history

  30. feat(core): added update field to the transaction body core type

    @AngelCastilloB @mkazlauskas
    AngelCastilloB authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    2e9c439 View commit details
    Browse the repository at this point in the history

  31. feat(core): added transaction body serialization classes

    @AngelCastilloB @mkazlauskas
    AngelCastilloB authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    9451a05 View commit details
    Browse the repository at this point in the history

  32. feat(core): replaced CML TransactionBody serialization class with out… 

    … own typescript native version
    @AngelCastilloB @mkazlauskas
    AngelCastilloB authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    0dfaeb7 View commit details
    Browse the repository at this point in the history

  33. test(wallet): fix transaction body initialization

    @AngelCastilloB @mkazlauskas
    AngelCastilloB authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    f9a1a58 View commit details
    Browse the repository at this point in the history

  34. docs(core): tagged all inline CDDL documentation on serialization cla… 

    …sses
    @AngelCastilloB @mkazlauskas
    AngelCastilloB authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    22318dd View commit details
    Browse the repository at this point in the history

  35. feat: added witness set serialization classes 

    Our new native serialization classes encode all arrays as definite length arrays (cannonical form),
the CML encoded arrays inside native scripts as indefinite length arrays, this results in a
difference of a few bytes on witness sets which contains nested native scripts.
    @AngelCastilloB @mkazlauskas
    AngelCastilloB authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    132599d View commit details
    Browse the repository at this point in the history

  36. feat(core): added serialization classes for tx auxiliary data

    @AngelCastilloB @mkazlauskas
    AngelCastilloB authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    4b49e57 View commit details
    Browse the repository at this point in the history

  37. feat(core): plutus data map now uses deep equality when being indexed… 

    … by key
    @AngelCastilloB @mkazlauskas
    AngelCastilloB authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    c34076c View commit details
    Browse the repository at this point in the history

  38. fix(core): fix circular dependency on cip67 module

    @AngelCastilloB @mkazlauskas
    AngelCastilloB authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    67f6892 View commit details
    Browse the repository at this point in the history

  39. feat(core): added native functions to convert between json and metadatum

    @AngelCastilloB @mkazlauskas
    AngelCastilloB authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    b0ba261 View commit details
    Browse the repository at this point in the history

  40. feat(key-management)!: remove deprecated trezor hardware wallet imple… 

    …mentation
    @AngelCastilloB @mkazlauskas
    AngelCastilloB authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    76bed53 View commit details
    Browse the repository at this point in the history

  41. feat!: remove the CML serialization code from core package 

    BREAKING CHANGES: CML serialization functions were replaced by the
new native serialization classes
    @AngelCastilloB @mkazlauskas
    AngelCastilloB authored and mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    62f4252 View commit details
    Browse the repository at this point in the history

  42. feat(projection): add cip67.byAssetId 

    that can be used to look up the label and decoded value by asset id
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    060ac99 View commit details
    Browse the repository at this point in the history

  43. refactor!: rename Mappers.Handle type to HandleOwnership 

    to not clash with Handle type from core package
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    4cb2f55 View commit details
    Browse the repository at this point in the history

  44. fix!: correct cip68 handle name (without label) 

    fix a bug where it would incorrectly map cip68 handle names
by including label as part of the Handle

also, add optional 'logger' parameter to 'withHandles'
operator (to be converted to required)

BREAKING CHANGE: withHandles now requires WithCIP67 props in
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    1711969 View commit details
    Browse the repository at this point in the history

  45. refactor!: make withHandles 'logger' argument required

    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    2267689 View commit details
    Browse the repository at this point in the history

  46. feat(projection): map 'extra' data with Mappers.withNftMetadata 

    also clean up the test by removing duplicate data
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    892c2eb View commit details
    Browse the repository at this point in the history

  47. fix(projection): keep latest nft metadata 

    there can be multiple 'copies' of nft metadata for an asset in a block

it was keeping the 1st one instead
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    3f8242a View commit details
    Browse the repository at this point in the history

  48. feat(core): export tryConvertPlutusMapToUtf8Record from Cardano.util 

    hoist from NftMetadata.fromPlutusData and add tests
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    645db52 View commit details
    Browse the repository at this point in the history

  49. feat(projection): add withHandleMetadata mapper 

    it maps 'og' from base nft metadata and images from 'extra'
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    9fc4722 View commit details
    Browse the repository at this point in the history

  50. feat(core): add AssetName.toUTF8 util

    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    5f13b4f View commit details
    Browse the repository at this point in the history

  51. refactor(projection): dedupe withHandles and withHandleMetadata imple… 

    …mentations

- hoist assetNameToUTF8Handle to util.ts
- use AssetName.toUTF8 core util

also, replace custom error with the existing InvalidStringError,
because it wasn't used anywhere
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    5b4704b View commit details
    Browse the repository at this point in the history

  52. test(projection): hoist common handle utils to shared module

    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    cadbedb View commit details
    Browse the repository at this point in the history

  53. refactor(projection-typeorm): rm redundant try-catch from storeHandles

    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    5aa9ebe View commit details
    Browse the repository at this point in the history

  54. feat(projection-typeorm): hydrate event with stored utxo row IDs

    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    8520e4e View commit details
    Browse the repository at this point in the history

  55. feat(projection-typeorm): add HandleMetadataEntity and storeHandleMet… 

    …adata
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    bca30bf View commit details
    Browse the repository at this point in the history

  56. test(projection-typeorm): split storeHandles.test.ts into 2 files 

    hoist shared utils to util.ts
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    716470a View commit details
    Browse the repository at this point in the history

  57. fix(projection): infer incoming event type for withNftMetadata

    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    53cd873 View commit details
    Browse the repository at this point in the history

  58. feat(projection-typeorm): add defaultForStakeCredential and defaultFo… 

    …rPaymentCredential columns
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    481996f View commit details
    Browse the repository at this point in the history

  59. feat(cardano-services): add HandleMetadata to handle projection 

    also add missing entity inter-dependency output->tokens
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    233ed70 View commit details
    Browse the repository at this point in the history

  60. feat(cardano-services): update handle projector to project 'default' … 

    …columns
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    903faf2 View commit details
    Browse the repository at this point in the history

  61. fix(cardano-services): return 0 for stake pool status 

    it would omit the prop if no stake pools of some status are in the database
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    6674b77 View commit details
    Browse the repository at this point in the history

  62. fix(wallet): do not track reference NFTs as handles 

    in reality those shouldn't be in user's wallet, but technically they can be
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    3b61c93 View commit details
    Browse the repository at this point in the history

  63. feat(e2e): hoist handle minting utils to src 

    to be used when generating cardano-services db snapshot
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    e49f3fd View commit details
    Browse the repository at this point in the history

  64. test(cardano-services): re-generate test DBs 

    add a cip68 handle to handle test db

disable some flaky tests: there are no 'retiring' pools in db

fix TypeormStakePoolProvider test setup to match partial strings
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    808dfde View commit details
    Browse the repository at this point in the history

  65. refactor(core)!: rename HandleResolution default props to credential 

    address could also be owned by script
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    877279f View commit details
    Browse the repository at this point in the history

  66. refactor(projection-typeorm): change NftMetadataEntity.image type to … 

    …Asset.Uri
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    361824b View commit details
    Browse the repository at this point in the history

  67. refactor(projection-typeorm): add NftMetadataEntity.userTokenAssetId … 

    …prop

it does not add any new columns, but simplifies queries
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    5ac43d1 View commit details
    Browse the repository at this point in the history

  68. feat(cardano-services): resolve default handle, image, pfp and bg

    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    9c446ac View commit details
    Browse the repository at this point in the history

  69. chore(wallet): set logger verbosity of duplicate utxo message to 'debug'

    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    dd1cc86 View commit details
    Browse the repository at this point in the history

  70. fix(core): return consistent bytes type in Serialization 

    in some cases it returned a Buffer, because Buffer.valueOf() => Buffer in nodejs
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    9331d01 View commit details
    Browse the repository at this point in the history

  71. fix(util): deserialize bytes as Uint8Array instead of Buffer 

    Buffer is Node.js-specific

ArrayBuffer and it's views is available in web environments as well
    @mkazlauskas
    mkazlauskas committed Sep 14, 2023
    Configuration menu
    Copy the full SHA
    78460d3 View commit details
    Browse the repository at this point in the history