Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement the calculate spower offchain feature #944

Merged
merged 102 commits into from
Jun 29, 2024

Commits on May 17, 2024

  1. 1. Add WorkReportsToProcess queue which will be processed by the offc…

    …hain crust-spower service.
    
    2. Add the WorkReportMetadata to this queue when the report_works call is successful.
    3. Refactor the update_files function to only calculate the changed_spower and changed_files_count based on the raw value directly. Do not invoke the market::upsert_replica and delete_replica anymore.
    wuhaixian1984 committed May 17, 2024
    Configuration menu
    Copy the full SHA
    f9e6a37 View commit details
    Browse the repository at this point in the history
  2. Fix the build warnings

    wuhaixian1984 committed May 17, 2024
    Configuration menu
    Copy the full SHA
    70c55dc View commit details
    Browse the repository at this point in the history

Commits on May 23, 2024

  1. Configuration menu
    Copy the full SHA
    8f60da5 View commit details
    Browse the repository at this point in the history

Commits on May 27, 2024

  1. 1. Refactor the UpdatedFilesToProcess data structure, to record the u…

    …pdated files replicas info at the specific report_block
    
    2. Add work_reports args to the update_replicas extrinsic call, which will then call the T::SworkerInterface::clear_process_work_reports to purge the successfully processed work reports from the WorkReportsToProcess db item
    wuhaixian1984 committed May 27, 2024
    Configuration menu
    Copy the full SHA
    9054268 View commit details
    Browse the repository at this point in the history

Commits on May 28, 2024

  1. 1. Add new events for pallet_market and pallet_swork for the update_r…

    …eplicas and update_spower extrinsic call
    
    2. Refactor some data structure names
    wuhaixian1984 committed May 28, 2024
    Configuration menu
    Copy the full SHA
    599a220 View commit details
    Browse the repository at this point in the history

Commits on May 30, 2024

  1. 1. Put the file info details inside the UpdatedFileToProcess structur…

    …e, so that we can replay this data in crust-spower service
    
    2. Refactor some detailed implementations for the upsert_replica and delete_replica to handle some detailed design considerations
    wuhaixian1984 committed May 30, 2024
    Configuration menu
    Copy the full SHA
    16cf42a View commit details
    Browse the repository at this point in the history
  2. 1. Implement the swork::update_spower extrinsic call

    2. Perform the illegal file replicas count substraction directly during the market::update_replicas call
    3. Remove some unused function parameter
    wuhaixian1984 committed May 30, 2024
    Configuration menu
    Copy the full SHA
    d30a270 View commit details
    Browse the repository at this point in the history

Commits on Jun 1, 2024

  1. 1. Refactor the market::update_replicas arguments, events, and db design

    2. Refactor the WorkReportsToProcess design (include db and events)
    wuhaixian1984 committed Jun 1, 2024
    Configuration menu
    Copy the full SHA
    250a08e View commit details
    Browse the repository at this point in the history

Commits on Jun 3, 2024

  1. Configuration menu
    Copy the full SHA
    617d342 View commit details
    Browse the repository at this point in the history

Commits on Jun 4, 2024

  1. Update the UPDATE_OFFSET from 200 to 100, which means the partial_upd…

    …ate_identities function will be called starting from 500th block in a report slot, while the original value is from 400th block in a slot.
    
    The 400th ~500 th block is reserved for the spower update.
    wuhaixian1984 committed Jun 4, 2024
    Configuration menu
    Copy the full SHA
    1c16e5f View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2024

  1. 1. Refactor the storage item and update_replicas process logic, don't…

    … store the UpdatedFilesToProcess right now.
    
    2. Perform the spower changed based on file_size during update_replicas
    3. Add FileClosed event
    4. Refactor the update_spower,  which doesn't update spower per blocks of files now
    wuhaixian1984 committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    8b41b01 View commit details
    Browse the repository at this point in the history

Commits on Jun 14, 2024

  1. Change the swork::update_spower call arguments type from BTreeMap to …

    …Vec.
    
    The reason is that the order of items in BTreeMap are not determined, when sending tx from polkadot.js, it may fail with error 'Invalid Transaction: Transaction has a bad signature'
    wuhaixian1984 committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    4e96583 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    30a5c53 View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. 1. Make the market::update_replicas charge no fee as this is a freque…

    …nt management extrinsic
    
    2. Fix bug for update_replicas with non-exist cids, need to add to the illegal_file_replicas_map as well under this scenario
    3. Use valid_at as the value of created_at
    4. Update the calculated_at field during maybe_reward_liquidator
    5. Update the calculate_spower to use the latest spower calculation logic
    wuhaixian1984 committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    794b0b5 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    d049b1b View commit details
    Browse the repository at this point in the history
  3. 1. Fix bug in update_illegal_file_replicas_count() method, need to us…

    …e get_current_reported_slot() instead of current_report_slot()
    
    2. Charge no fee for swork::update_spower extrinsic, since this is a frequent management call
    3. Remove unused commented out codes
    wuhaixian1984 committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    37f9e06 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    ef3aecc View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    a96d513 View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2024

  1. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    7cd54b6 View commit details
    Browse the repository at this point in the history
  2. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    e641b7b View commit details
    Browse the repository at this point in the history
  3. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    174ba68 View commit details
    Browse the repository at this point in the history
  4. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    a4a15b5 View commit details
    Browse the repository at this point in the history
  5. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    eb3cc51 View commit details
    Browse the repository at this point in the history
  6. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    3307965 View commit details
    Browse the repository at this point in the history
  7. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    5afa3dd View commit details
    Browse the repository at this point in the history
  8. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    1185b39 View commit details
    Browse the repository at this point in the history
  9. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    0752e8a View commit details
    Browse the repository at this point in the history
  10. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    6608116 View commit details
    Browse the repository at this point in the history
  11. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    b636f1d View commit details
    Browse the repository at this point in the history
  12. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    e0fe370 View commit details
    Browse the repository at this point in the history
  13. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    e040df2 View commit details
    Browse the repository at this point in the history
  14. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    06adc52 View commit details
    Browse the repository at this point in the history
  15. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    1137afc View commit details
    Browse the repository at this point in the history
  16. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    ca4aa01 View commit details
    Browse the repository at this point in the history
  17. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    5823b54 View commit details
    Browse the repository at this point in the history
  18. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    242f283 View commit details
    Browse the repository at this point in the history
  19. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    ebbbbe8 View commit details
    Browse the repository at this point in the history
  20. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    ce63ce8 View commit details
    Browse the repository at this point in the history
  21. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    1cbb684 View commit details
    Browse the repository at this point in the history
  22. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    db42e8d View commit details
    Browse the repository at this point in the history
  23. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    7c6139c View commit details
    Browse the repository at this point in the history
  24. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    98761ec View commit details
    Browse the repository at this point in the history
  25. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    e125def View commit details
    Browse the repository at this point in the history
  26. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    4aeae46 View commit details
    Browse the repository at this point in the history
  27. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    75734db View commit details
    Browse the repository at this point in the history
  28. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    3e95657 View commit details
    Browse the repository at this point in the history
  29. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    f797dfb View commit details
    Browse the repository at this point in the history
  30. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    ba5abd7 View commit details
    Browse the repository at this point in the history
  31. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    ab2eeaa View commit details
    Browse the repository at this point in the history
  32. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    1faffa2 View commit details
    Browse the repository at this point in the history
  33. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    55138b7 View commit details
    Browse the repository at this point in the history
  34. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    477229c View commit details
    Browse the repository at this point in the history
  35. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    e17b537 View commit details
    Browse the repository at this point in the history
  36. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    99ca6fd View commit details
    Browse the repository at this point in the history
  37. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    629f17c View commit details
    Browse the repository at this point in the history
  38. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    36eab47 View commit details
    Browse the repository at this point in the history
  39. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    42b221e View commit details
    Browse the repository at this point in the history
  40. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    3fdf39e View commit details
    Browse the repository at this point in the history
  41. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    1d55b53 View commit details
    Browse the repository at this point in the history
  42. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    73b1e16 View commit details
    Browse the repository at this point in the history
  43. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    18224f7 View commit details
    Browse the repository at this point in the history
  44. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    94fe0b1 View commit details
    Browse the repository at this point in the history
  45. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    bc930ce View commit details
    Browse the repository at this point in the history
  46. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    e9d8332 View commit details
    Browse the repository at this point in the history
  47. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    9dcb49b View commit details
    Browse the repository at this point in the history
  48. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    c8a4813 View commit details
    Browse the repository at this point in the history
  49. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    10b9f73 View commit details
    Browse the repository at this point in the history
  50. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    ef5f0d3 View commit details
    Browse the repository at this point in the history
  51. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    b9e0bfa View commit details
    Browse the repository at this point in the history
  52. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    50d93fd View commit details
    Browse the repository at this point in the history
  53. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    eed4039 View commit details
    Browse the repository at this point in the history
  54. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    960171d View commit details
    Browse the repository at this point in the history
  55. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    252f754 View commit details
    Browse the repository at this point in the history
  56. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    847197f View commit details
    Browse the repository at this point in the history
  57. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    32707aa View commit details
    Browse the repository at this point in the history
  58. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    d03d1da View commit details
    Browse the repository at this point in the history
  59. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    ad21a49 View commit details
    Browse the repository at this point in the history
  60. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    c1f5027 View commit details
    Browse the repository at this point in the history
  61. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    f099dca View commit details
    Browse the repository at this point in the history
  62. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    7fcd50c View commit details
    Browse the repository at this point in the history
  63. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    4d3a7d0 View commit details
    Browse the repository at this point in the history
  64. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    e207155 View commit details
    Browse the repository at this point in the history
  65. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    10eb0e3 View commit details
    Browse the repository at this point in the history
  66. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    ff33b05 View commit details
    Browse the repository at this point in the history
  67. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    4f7fcf4 View commit details
    Browse the repository at this point in the history
  68. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    9ea7703 View commit details
    Browse the repository at this point in the history
  69. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    d7f2d37 View commit details
    Browse the repository at this point in the history
  70. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    de7c327 View commit details
    Browse the repository at this point in the history
  71. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    0dd8e17 View commit details
    Browse the repository at this point in the history
  72. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    b5a8abd View commit details
    Browse the repository at this point in the history
  73. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    dda9161 View commit details
    Browse the repository at this point in the history
  74. Auto updated substrate references

    Crust Update Bot authored and wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    e9706da View commit details
    Browse the repository at this point in the history
  75. 1. Add WorkReportsToProcess queue which will be processed by the offc…

    …hain crust-spower service.
    
    2. Add the WorkReportMetadata to this queue when the report_works call is successful.
    3. Refactor the update_files function to only calculate the changed_spower and changed_files_count based on the raw value directly. Do not invoke the market::upsert_replica and delete_replica anymore.
    wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    b826935 View commit details
    Browse the repository at this point in the history
  76. Configuration menu
    Copy the full SHA
    579d19e View commit details
    Browse the repository at this point in the history
  77. 1. Add new events for pallet_market and pallet_swork for the update_r…

    …eplicas and update_spower extrinsic call
    
    2. Refactor some data structure names
    wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    beb668f View commit details
    Browse the repository at this point in the history
  78. 1. Implement the swork::update_spower extrinsic call

    2. Perform the illegal file replicas count substraction directly during the market::update_replicas call
    3. Remove some unused function parameter
    wuhaixian1984 committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    77818db View commit details
    Browse the repository at this point in the history
  79. Configuration menu
    Copy the full SHA
    cea8403 View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2024

  1. Configuration menu
    Copy the full SHA
    e65f77a View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    0325126 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    6e05661 View commit details
    Browse the repository at this point in the history
  4. Add back the comments

    wuhaixian1984 committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    e642db9 View commit details
    Browse the repository at this point in the history
  5. Remove unused code

    wuhaixian1984 committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    bff2821 View commit details
    Browse the repository at this point in the history