Skip to content

Releases: 4dn-dcic/snovault

1.2.8

15 Oct 21:08
Compare
Choose a tag to compare

Fixed an issue with MPIndexer.update_objects that would cause some processes to be lost when counter did not increment but indexing was deferred. Also added a fix to Indexer.update_objects to actually return indexing errors, though this will only matter if single-process indexing is used.

1.2.7

06 Oct 21:01
79e864e
Compare
Choose a tag to compare

Add indexing_stats dictionary to document returned from /index-data. It times various parts of the indexing progress.

1.2.6

15 Sep 03:11
Compare
Choose a tag to compare

Use multiprocessing.cpu_count - 2 processes for the MPIndexer. Minimum number of processes is 1.

1.2.5

08 Sep 15:30
Compare
Choose a tag to compare

Fixed incorrect initialization of IndexerQueue.target_queues so now they are correctly ordered. Slight test update.

1.2.4

05 Sep 17:24
41be18c
Compare
Choose a tag to compare

Up SEARCH_MAX to support more pages.

1.2.3

26 Aug 18:56
Compare
Choose a tag to compare

Small release to fix ES SEARCH_MAX discrepancy between create_mapping and rev link searching function.

1.2.2

22 Aug 20:10
f2d173b
Compare
Choose a tag to compare

Release that improves indexing performance.

  • Sid checking now happens in indexer before /index-data view is called
  • Bugfix for the indexer that would catch KeyError/MissingIndexItemException before SidException
  • Bugfix for Indexer_queue.queue_targets not being ordered
  • es_index_listener now checks queue counts before calling/index
  • MPIndexer now always uses multiprocessing.cpu_count() - 1 for number if processes in pool
  • Some other random refactoring

1.2.1

12 Aug 20:03
48b2eea
Compare
Choose a tag to compare

Small release adding mimetype_map to ItemWithAttachment class to allow handling of mimetypes outside of the python-magic scope.

1.2.0

30 Jul 16:34
e52f7fe
Compare
Choose a tag to compare
Merge pull request #101 from 4dn-dcic/single_txn_indexing

1.2.0 -- Remove deferred queue

1.1.0

08 Jul 18:10
Compare
Choose a tag to compare

This release adds types from AbstractCollections to the /profiles/ view and adds a couple new keys to all types on that view. Additionally, some refactoring around TypeInfo/AbstractTypeInfo was done. See: #100