Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

LUCENE-10216: Use MergeScheduler and MergePolicy to run addIndexes(CodecReader[]) merges. #633

Merged
merged 60 commits into from
Jul 6, 2022
Merged

LUCENE-10216: Use MergeScheduler and MergePolicy to run addIndexes(CodecReader[]) merges. #633

merged 60 commits into from
Jul 6, 2022

Commits on May 30, 2022

  1. Use merge policy and merge scheduler to run addIndexes merges

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    5345ca7 View commit details
    Browse the repository at this point in the history

  2. wrapped reader does not see deletes - debug

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    77aadf3 View commit details
    Browse the repository at this point in the history

  3. Partially fixed tests in TestAddIndexes

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    d094aaf View commit details
    Browse the repository at this point in the history

  4. Use writer object to invoke addIndexes merge

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    6b754bb View commit details
    Browse the repository at this point in the history

  5. Use merge object info

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    0a76572 View commit details
    Browse the repository at this point in the history

  6. Add javadocs for new methods

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    631f956 View commit details
    Browse the repository at this point in the history

  7. TestAddIndexes passing

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    761c9a1 View commit details
    Browse the repository at this point in the history

  8. verify field info schemas upfront from incoming readers

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    26934d9 View commit details
    Browse the repository at this point in the history

  9. rename flag to track pooled readers

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    c21a05d View commit details
    Browse the repository at this point in the history

  10. Keep addIndexes API transactional

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    e538faa View commit details
    Browse the repository at this point in the history

  11. Maintain transactionality - register segments with iw after all merge… 

    …s complete
    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    c532e3f View commit details
    Browse the repository at this point in the history

  12. fix checkstyle

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    0a19796 View commit details
    Browse the repository at this point in the history

  13. PR comments

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    7848019 View commit details
    Browse the repository at this point in the history

  14. Fix pendingDocs - numDocs mismatch bug

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    3de64bc View commit details
    Browse the repository at this point in the history

  15. Tests with 1-1 merges and partial merge failures

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    02e15d2 View commit details
    Browse the repository at this point in the history

  16. variable renaming and better comments

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    5500f23 View commit details
    Browse the repository at this point in the history

  17. add test for partial merge failures. change tests to use 1-1 findmerges

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    e9c178e View commit details
    Browse the repository at this point in the history

  18. abort pending merges gracefully

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    a5327ac View commit details
    Browse the repository at this point in the history

  19. test null and empty merge specs

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    d857c0d View commit details
    Browse the repository at this point in the history

  20. test interim files are deleted

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    c8c9db3 View commit details
    Browse the repository at this point in the history

  21. test with empty readers

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    19248fc View commit details
    Browse the repository at this point in the history

  22. test cascading merges triggered

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    ea3f37d View commit details
    Browse the repository at this point in the history

  23. remove nocommits

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    6bf5715 View commit details
    Browse the repository at this point in the history

  24. gradle check errors

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    d978a11 View commit details
    Browse the repository at this point in the history

  25. remove unused line

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    9a7d569 View commit details
    Browse the repository at this point in the history

  26. remove printf

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    e5de05a View commit details
    Browse the repository at this point in the history

  27. spotless apply

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    9feff16 View commit details
    Browse the repository at this point in the history

  28. update TestIndexWriterOnDiskFull to accept mergeException from failin… 

    …g addIndexes calls
    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    67acc95 View commit details
    Browse the repository at this point in the history

  29. return singleton reader mergespec in NoMergePolicy

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    6abf56b View commit details
    Browse the repository at this point in the history

  30. rethrow exceptions seen in merge threads on failure

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    e450548 View commit details
    Browse the repository at this point in the history

  31. spotless apply

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    ae21b2a View commit details
    Browse the repository at this point in the history

  32. update test to new exception type thrown

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    7f194fb View commit details
    Browse the repository at this point in the history

  33. spotlessApply

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    4e81b2b View commit details
    Browse the repository at this point in the history

  34. test for maxDoc limit in IndexWriter

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    bc14626 View commit details
    Browse the repository at this point in the history

  35. spotlessApply

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    3a31d69 View commit details
    Browse the repository at this point in the history

  36. Use DocValuesIterator instead of DocValuesFieldExistsQuery for counti… 

    …ng soft deletes
    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    18dcc4a View commit details
    Browse the repository at this point in the history

  37. spotless apply

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    72bcb46 View commit details
    Browse the repository at this point in the history

  38. change exception message for closed IW

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    9fd2429 View commit details
    Browse the repository at this point in the history

  39. remove non-essential comments

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    bcd9219 View commit details
    Browse the repository at this point in the history

  40. update api doc string

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    a5107e6 View commit details
    Browse the repository at this point in the history

  41. doc string update

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    6ead169 View commit details
    Browse the repository at this point in the history

  42. spotless

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    6642a3a View commit details
    Browse the repository at this point in the history

  43. Changes file entry

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    e58b62c View commit details
    Browse the repository at this point in the history

  44. simplify findMerges API, add 1-1 merges to MockRandomMergePolicy

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    ecd42d8 View commit details
    Browse the repository at this point in the history

  45. update merge policies to new api

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    caf19f3 View commit details
    Browse the repository at this point in the history

  46. remove unused imports

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    79ad7e7 View commit details
    Browse the repository at this point in the history

  47. spotless apply

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    5faa8cf View commit details
    Browse the repository at this point in the history

  48. move changes entry to end of list

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    fa662f4 View commit details
    Browse the repository at this point in the history

  49. fix testAddIndicesWithSoftDeletes

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    bb83b4a View commit details
    Browse the repository at this point in the history

  50. test with 1-1 merge policy always enabled

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    be4899e View commit details
    Browse the repository at this point in the history

  51. please spotcheck

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    3b4da4b View commit details
    Browse the repository at this point in the history

  52. tidy

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    4601b84 View commit details
    Browse the repository at this point in the history

  53. test - never use 1-1 merge policy

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    ce3c08f View commit details
    Browse the repository at this point in the history

  54. use 1-1 merge policy randomly

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    b5ad19c View commit details
    Browse the repository at this point in the history

  55. Remove concurrent addIndexes findMerges from MockRandomMergePolicy

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    dc24ee5 View commit details
    Browse the repository at this point in the history

  56. Bug Fix: RuntimeException in addIndexes 

    Aborted pending merges were slipping through the merge exception check in
API, and getting caught later in the RuntimeException check.
    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    1794ab6 View commit details
    Browse the repository at this point in the history

  57. tidy

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    14fbefe View commit details
    Browse the repository at this point in the history

  58. Rebase on main. Move changes to 10.0

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    7ae05f6 View commit details
    Browse the repository at this point in the history

  59. Synchronize IW.AddIndexesMergeSource on outer class IW object

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    e923c16 View commit details
    Browse the repository at this point in the history

  60. tidy

    @vigyasharma
    vigyasharma committed May 30, 2022
    Configuration menu
    Copy the full SHA
    7199151 View commit details
    Browse the repository at this point in the history