Skip to content

Latest commit

 

History

History
406 lines (350 loc) · 30.1 KB

NEWS.md

File metadata and controls

406 lines (350 loc) · 30.1 KB
Raw

2021-03-03 v3.3.0

  • MODSOURMAN-694 Improve sql query for retrieving job execution sourcechunks
  • MODSOURMAN-695 Upgrade RMB and Vertx versions that contain fixes for the connection pool
  • MODINVOICE-356 Fix progress bar stuck behaviour after the RecordTooLargeException
  • MODSOURMAN-624 Failed to handle DI_ERROR when 004 is invalid in MARC Holdings
  • MODSOURMAN-614 Authority: Add mapping rule for note types
  • MODSOURMAN-577 Optimistic locking: mod-source-record-manager modifications
  • MODSOURMAN-573 Create mapping rules for AUTHORITY records
  • MODDICORE-184 Update the MARC-Instance field mapping for InstanceType (336$a and $b)
  • MODSOURMAN-590 Save AUTHORITY rules to database
  • MODSOURMAN-595 "View all" and "Load more" buttons do not load all logs in Data Import
  • MODSOURMAN-570 Edit MARC Authorities via quickMARC | Handle events properly to Authorities
  • MODSOURMAN-594 Cannot build journal record when entity is empty
  • MODSOURMAN-228 Update the MARC-to-Instance mapping documentation
  • MODSOURMAN-605 Authority: update rules for name-title fields/properties
  • MODSOURMAN-619 Add an Authority info for Import Log for record page
  • MODSOURMAN-623 Generate IDs for Inventory authority
  • MODSOURMAN-598 Properly handle DB failures during events processing
  • MODDATAIMP-491 Improve logging to be able to trace the path of each record and file_chunks
  • MODDATAIMP-621 Fix saving of default mapping rules
  • MODDATAIMP-641 Fix NPE exception and fix the main reason of NPE (recordId = null).
  • MODSOURMAN-645 Update permissions related to Authority
  • MODSOURMAN-625 Invoice log detail sort by Record column in Data Import detail log not working properly
  • MODSOURMAN-432 Create job execution requires user with "personal" information
  • MODSOURMAN-656 Support Update MARC Authority Action
  • MODSOURMAN-638 Remove Kafka cache for StoreRecordsChunksKafkaHandler
  • MODSOURMAN-664 Remove Kafka cache for QuickMarcUpdateKafkaHandler
  • MODSOURMAN-639 Improve performance of saving journal records during import
  • MODSOURMAN-640 Remove Kafka cache for DataImportJournalKafkaHandler
  • MODSOURMAN-641 Remove Kafka cache by handling Constraint Violation Exceptions
  • MODDATAIMP-623 Remove Kafka cache initialization and Maven dependency
  • MODSOURMAN-668 Restructure job_execution_progress table for DataImportKafkaHandler
  • MODSOURMAN-675 Data Import handles repeated 020 $a:s in an unexpected manner when creating Instance Identifiers
  • MODSOURMAN-676 Provide Instance UUID for populating Inventory hotlinks for holdings/items
  • MODSOURMAN-682 Consume Authority log event
  • MODSOURMAN-699 Fix Can`t map 'RECORD' or/and 'MARC_BIBLIOGRAPHIC' statements from logs
  • MODSOURMAN-714 Legacy 999 (non-ff) fields cause data import failure

2022-02-24 v3.2.9

2022-02-09 v3.2.8

2021-12-15 v3.2.7

2021-12-03 v3.2.6

2021-11-12 v3.2.5

##2021-11-10 v3.2.4

  • MODSOURMAN-595 "View all" and "Load more" buttons do not load all logs in Data Import

2021-10-29 v3.2.3

  • MODSOURMAN-522 Fix the effect of DI_ERROR messages when trying to duplicate records on the import job progress bar
  • MODDICORE-184 Update the MARC-Instance field mapping for InstanceType (336$a and $b)
  • Updated data-import-processing-core to v3.2.2

2021-10-19 v3.2.2

2021-10-13 v3.2.1

2021-10-06 v3.2.0

  • MODSOURMAN-516 Send QM_COMPLETED event after processing finished
  • MODSOURMAN-517 Change quickMarc producers not to close after message sent
  • MODSOURMAN-524 Support MARC Holdings
  • MODSOURMAN-533 Upgrade to RAML Module Builder 33.x
  • Improved logging
  • Apply new version of clients generated by mod-data-import-converter-storage with updated Raml version
  • KAFKAWRAP-5 Add mechanism for detection and logging inability to create/connect Kafka consumers.
  • MODSOURMAN-550 Reduce BE response payload for DI Landing Page to increase performance
  • MODSOURMAN-540 Add default mapping profile for MARC holdings
  • MODSOURMAN-541 Update existing CLI endpoint GET /mapping-rules to support MARC Holdings default rules
  • MODSOURMAN-542 Update existing CLI endpoint PUT /mapping-rules to support MARC Holdings default rules
  • MODSOURMAN-543 Update existing CLI endpoint PUT /mapping-rules/restore to support MARC Holdings default rules
  • MODSOURMAN-547 Update cache for mapping-rules to support MARC Holdings default rules
  • MODSOURMAN-553 Update GET /change-manager/parsedRecords to have externalId param
  • MODSOURMAN-526 Verify persist value in DB during parsing 004 field
  • MODSOURMAN-544 Validate MARC Holdings 004 field from MARC Bib HRID
  • MODSOURMAN-546 Support edit Holdings via quickMarc
  • MODSOURMAN-464 Store snapshots of MappingRules and MappingParams to the database
  • MODSOURMAN-563 Add MARC-Instance field mapping for Cancelled system control number
  • MODSOURMAN-465 Remove MappingRules, MappingParams, and JobProfileSnapshot from the event payload
  • MODSOURMAN-466 Remove zipping mechanism for data import event payloads
  • MODDICORE-172 Add MARC-Instance field mapping for New identifier types

2021-08-04 v3.1.3

  • MODDICORE-166 Near the day boundary data import calculates today incorrectly.
  • MODSOURMAN-535 Data import can't retrieve location with code "olin".
  • MODPUBSUB-187 Add support for max.request.size configuration for Kafka messages
  • Update data-import-processing-core dependency to v3.1.4
  • Update folio-kafka-wrapper dependency to v3.1.4

2021-07-21 v3.1.2

  • MODSOURMAN-513 (Juniper) Data import stopped process before finishing: deadlock for "job_monitoring"
  • MODSOURMAN-508 Log details for Inventory single record imports for Overlays
  • MODSOURMAN-527 Cannot import EDIFACT invoices
  • Update data-import-processing-core dependency to v3.1.3

2021-06-25 v3.1.1

  • MODSOURMAN-497 Apply MarcRecordAnalyzer to determine MARC related specific type
  • MODSOURMAN-501 Change dataType to have have common type for MARC related subtypes
  • Update data-import-processing-core dependency to v3.1.2

2021-06-17 v3.1.0

2021-06-18 v3.0.8

2021-05-28 v3.0.7

  • MODSOURMAN-480 Create jobs with match profiles that include records with 999 fields cause errors in the srs-instance relationship

2021-05-22 v3.0.6

  • MODSOURMAN-457 Issue with Database migration for Iris release
  • MODSOURCE-278 Move logging of the records creation information to the handler receiving saved records
  • MODSOURCE-295 Set instanceHrid to externalIdsHolder when 999ff$i is present

2021-04-22 v3.0.5

##2021-04-14 v3.0.4

2021-04-12 v3.0.3

  • MODSOURMAN-419 SQL Exception WRT count function
  • MODSOURMAN-428 Ensure exactly one delivery approach for handler receiving stored records
  • MODSOURMAN-430 Ensure exactly one delivery approach for for data import log handler

2021-04-05 v3.0.2

  • MODSOURMAN-429 Add permission to /change-manager/jobExecutions/{id}/jobProfile

2021-03-28 v3.0.1

  • MODSOURMAN-421 Syntax problem for 561 field in default mapping rules
  • MODSOURMAN-422 Add record sequence number for records posted direct via API if it is not set
  • MODDATAIMP-388 Import job is not completed on file parsing error

2021-03-18 v3.0.0

2020-11-20 v2.4.3

2020-10-30 v2.4.2

  • MODSOURMAN-362 Mark job status with error when at least one record has not been parsed
  • MODSOURMAN-339 Disable CQL2PgJSON & CQLWrapper extra logging in mod-source-record-manager
  • MODSOURMAN-369 Upgrade to RMB v31.1.5

2020-10-26 v2.4.1

  • MODSOURMAN-363 Fix permissions issues
  • Update mod-pubsub-client to v1.3.1

2020-08-17 v2.4.0

  • MODSOURMAN-344 Fixed default MARC Bib-Instance mapping for 024 and 028 fields
  • MODSOURMAN-340 MARC field sort into numerical order when record is imported
  • MODSOURMAN-345 003 handling in SRS for MARC Bib records: Create
  • MODSOURMAN-346 Load MARC field protection settings to Mapping params
  • MODDATAIMP-324 Update all Data-Import modules to the new RMB version
  • MODINV-296 Added support for journalRecord saving on protected item status update
  • MODSOURMAN-361 Add capability to remove jobs that are stuck

2020-08-10 v2.3.2

  • MODSOURMAN-322 Add source-record states storing mechanism for QM edit workflow
  • MODSOURMAN-333 Replace incoming 999 ff fields if file is re-imported

2020-07-10 v2.3.1

  • MODSOURMAN-329 Set completed date on error status update for JobExecution

2020-06-25 v2.3.0

2020-06-11 v2.2.0

2020-04-23 v2.1.3

  • MODSOURMAN-303 Add actual state on creating record
  • MODSOURMAN-298 Added migration script to support RMB version update
  • MODDICORE-43 SRS MARC Bib: Fix formatting of 035 field constructed from incoming 001
  • MODSOURMAN-307 Progress for Cornell file duplicates and the job hangs

2020-04-07 v2.1.2

  • MODSOURMAN-296 Added filling connection parameters to the data import event payload
  • Updated dependency onn data-import-processing-core library

2020-03-27 v2.1.1

  • Updated mapping for instance type ID and instance format ID
  • Fixed duplicate languages in instance mapping
  • Updated dependency on data-import-processing-core library

2020-03-13 v2.1.0

  • Added get journalRecords endpoint
  • Incremented RMB version
  • MatchedId filled in with the same value as recordId
  • Exposed json schemas api
  • Added module registration as publisher/subscriber to mod-pubsub
  • Added mode of issuance mapping mechanism
  • Updated mapping for instance identifier types and unspecified instance type
  • Added JobExecutionProgress service
  • Added defaultMapping query param to choose between default mapping and application of JobProfiles
  • New endpoint for saving results of instance creation to journal was added
  • Implemented endpoint to handle DI_COMPLETED and DI_ERROR events.
METHOD URL DESCRIPTION
GET /metadata-provider/journalRecords/{jobExecutionId} Get list of the JournalRecords by jobExecution id
POST /change-manager/handlers/created-inventory-instance Handle event about created inventory instance
POST /change-manager/handlers/processing-result Handle DI_COMPLETED and DI_ERROR events

2020-02-10 v2.0.2

  • Added HrId handling on records parsing and after instance creation

2020-01-06 v2.0.1

  • Fixed MARC-Instance mapping for 260/264 $c
  • Using "unspecified" instance type(008) instead of stub value if no 336 field in MARC record

2019-12-04 v2.0.0

  • Added table schema for journal service
  • Implemented journal service
  • Added get JobExecutionLogDto endpoint
  • Applied new JVM features to manage container memory
  • Updated instance subject headings to include MARC 655 field
  • Updated RuleProcessor documentation

2019-11-04 v1.7.0

  • Wrote documentation for Marc-to-Instance mapping processor
  • Added order of the record in importing file
  • Create CLI way for individual tenant to update the default MARC-to-Instance map
  • Fixed sorting and filtering of logs
  • Deleted jobExecutionDto and logDto entities.
  • Response body for endpoint "/metadata-provider/jobExecutions" changed to JobExecutionCollection.
  • Deleted endpoint for logs retrieving: "/metadata-provider/logs";
  • Updated mapping for "Uniform title" instance alternative title type
  • Added loading sample data by "loadSample" tenant parameter.
  • Provided cql query support for sorting numeric data
  • Broken down source-record-manager interface into smaller ones: source-manager-job-executions, source-manager-records.
  • Changed 'hrId' field type to integer for jobExecution
METHOD URL DESCRIPTION
GET /metadata-provider/jobExecutions Get list of the JobExecutions by query
POST /change-manager/jobExecutions Initialize JobExecution entities
GET /change-manager/jobExecutions/{id} Get single JobExecution entity
PUT /change-manager/jobExecutions/{id}/status Update status of JobExecution by id
PUT /change-manager/jobExecutions/{id}/jobProfile Update jobProfile of single JobExecution entity
GET /change-manager/jobExecutions/{id}/children Get children JobExecutions by parent id
POST /change-manager/jobExecutions/{id}/records Receive chunk of raw records for JobExecution
DELETE /change-manager/jobExecutions/{id}/records Delete Job Execution and all associated records from SRS
GET /mapping-rules Get current mapping rules
PUT /mapping-rules Update current mapping rules
PUT /mapping-rules/restore Restore default mapping rules

2019-09-09 v1.6.1

  • Added instance-type settings loading for mapping
  • Added electronic access relationships loading for mapping
  • Added classification settings loading for mapping
  • Added instance-format settings loading for mapping
  • Added contributor name types setting loading for mapping
  • Added contributor types setting loading for mapping, simple mapping for primary sign and name
  • Added mapping for contributor type free text
  • Added instance-type identifiers settings loading for mapping
  • Applied caching for external mapping parameters
  • Added instance note types settings loading for mapping

2019-09-09 v1.6.0

  • Progress mechanism was updated
  • Changed RawRecordsDto schema with extended metadata information
  • Changed relations between UI and Backend statuses for job executions
  • Deleted stub data for job executions
  • Updated Instance schema
  • Removed partial success handling from SRS batch responses
  • Filtered out invalid Instances before saving to inventory
  • Added delete endpoint for job execution and all associated records from SRS
  • Added total records number to logDto
  • Changed logic of adding fields to MARC record resulting in update of leader value
  • Rule Processor integrated with Settings (mod-inventory-storage)
  • Filled in "fromModuleVersion" value for each "tables" and "scripts" section in schema.json
METHOD URL DESCRIPTION
DELETE /change-manager/jobExecutions/{id}/records Delete Job Execution and all associated records from SRS

2019-06-13 v1.5.0

  • Changed implementation for Job Execution human-readable id using DB sequence
  • JobExecution marked as Error if processing of at least one chunk failed
  • Updated Record-to-Instance mapping rules and Instance schema in accordance with breaking changes in mod-inventory
  • Progress mechanism was updated
  • Changed RawRecordsDto schema with extended metadata information
  • Optimized Record-to-Instance mapping (framed rules into 'entity')

2019-06-12 v1.4.1

  • Fixed mapping from Record to Instances
  • Fixed check whether processing is completed for all chunks

2019-06-12 v1.4.0

  • Added description for data-import flow
  • Fixed issue with saving ErrorRecords
  • Added batch update of ParsedRecords after assigning Instance id to MARC records
  • Added support for records processing in XML format
  • Applied parallel approach for mapping from records to instances
  • Use batch post to send Instances to the mod-inventory
  • MARC to Instance mapping was updated to the new one version

2019-05-17 v1.3.1

  • Filled complete date and stub HrID for Job Execution
  • Changed implementation for checking statuses for JobExecutionSourceChunk
  • Applied bug fixes to build 999 fields

2019-05-12 v1.3.0

  • Borrowed mapping of MARC to Instance logic from mod-data-loader. After parsing Records are mapped to Instances and saved in mod-inventory.
  • Added support for records coming for processing in json format.
  • Applied logic for expanding parsed MARC records with additional custom fields (999 field)

2019-03-25 v1.2.1

  • Set required recordType field for the Record entity
  • Removed deprecated IMPORT_IN_PROGRESS and IMPORT_FINISHED statuses for JobExecution
  • Set stub data for runBy, progress ans startedDate fields for JobExecution entity

2019-03-20 v1.2.0

  • Renamed endpoints

  • Configured log4j2 for logging

  • Raw records MARC parser was added

  • PUT endpoint for update status and jobProfile for single JobExecution was added

  • Created ChunkProcessing Service

  • Added Spring DI support

  • Changed project structure to contain server and client parts. Client builds as a lightweight java library.

    METHOD URL DESCRIPTION
    GET /metadata-provider/jobExecutions Get list of the JobExecutions DTO by query
    GET /metadata-provider/logs Get list of the Log entities by query
    POST /change-manager/jobExecutions Initialize JobExecution entities
    GET /change-manager/jobExecutions/{id} Get single JobExecution entity
    PUT /change-manager/jobExecutions/{id}/status Update status of JobExecution by id
    PUT /change-manager/jobExecutions/{id}/jobProfile Update jobProfile of single JobExecution entity
    GET /change-manager/jobExecutions/{id}/children Get children JobExecutions by parent id
    POST /change-manager/jobExecutions/{id}/records Receive chunk of raw records for JobExecution

2018-11-30 v0.1.0

  • Created ChangeManager component

  • Added API for managing JobExecution entities

    METHOD URL DESCRIPTION
    GET /metadata-provider/jobExecutions Get list of the JobExecutions DTO by query
    GET /metadata-provider/logs Get list of the Log entities by query
    POST /change-manager/jobExecutions Initialize JobExecution entities
    PUT /change-manager/jobExecution/{id} Update JobExecution entity