Signed-off-by: Vijayan Balasubramanian <[email protected]>

…-project#268)

Signed-off-by: Heemin Kim <[email protected]>

…t#267)

Signed-off-by: Heemin Kim <[email protected]>

Co-authored-by: mend-for-github-com[bot] <50673670+mend-for-github-com[bot]@users.noreply.github.com>

Signed-off-by: Vijayan Balasubramanian <[email protected]>

Signed-off-by: Naveen Tatikonda <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Martin Gaievski <[email protected]>

Signed-off-by: Junqiu Lei <[email protected]>

…project#353)

Signed-off-by: Heemin Kim <[email protected]>

* Update gradle version to 7.6 (opensearch-project#265)

Signed-off-by: Vijayan Balasubramanian <[email protected]>

* Implement creation of ip2geo feature

* Implementation of ip2geo datasource creation
* Implementation of ip2geo processor creation

Signed-off-by: Heemin Kim <[email protected]>
---------

Signed-off-by: Vijayan Balasubramanian <[email protected]>
Signed-off-by: Heemin Kim <[email protected]>
Co-authored-by: Vijayan Balasubramanian <[email protected]>

* Add Unit tests
* Set cache true for search query
* Remove in memory cache implementation (Two way door decision)
 * Relying on search cache without custom cache
* Renamed datasource state from FAILED to CREATE_FAILED
* Renamed class name from *Helper to *Facade
* Changed updateIntervalInDays to updateInterval
* Changed value type of default update_interval from TimeValue to Long
* Read setting value from cluster settings directly

Signed-off-by: Heemin Kim <[email protected]>

* Update gradle version to 7.6 (opensearch-project#265)

Signed-off-by: Vijayan Balasubramanian <[email protected]>

* Exclude lombok generated code from jacoco coverage report (opensearch-project#268)

Signed-off-by: Heemin Kim <[email protected]>

* Make jacoco report to be generated faster in local (opensearch-project#267)

Signed-off-by: Heemin Kim <[email protected]>

* Update dependency org.json:json to v20230227 (opensearch-project#273)

Co-authored-by: mend-for-github-com[bot] <50673670+mend-for-github-com[bot]@users.noreply.github.com>

* Baseline owners and maintainers (opensearch-project#275)

Signed-off-by: Vijayan Balasubramanian <[email protected]>

---------

Signed-off-by: Vijayan Balasubramanian <[email protected]>
Signed-off-by: Heemin Kim <[email protected]>
Co-authored-by: Vijayan Balasubramanian <[email protected]>
Co-authored-by: mend-for-github-com[bot] <50673670+mend-for-github-com[bot]@users.noreply.github.com>

Signed-off-by: Heemin Kim <[email protected]>

1. Change variable name from datasourceName to name
2. Change variable name from id to name
3. Added helper methods in test code

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

1. Make geodata index as hidden
2. Make geodata index as read only allow delete after creation is done
3. Refresh datasource index immediately after update

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

…h-project#297)

Signed-off-by: Heemin Kim <[email protected]>

…ject#299)

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

Make buildGeoJSONFeatureProcessorConfig method to be more general

Signed-off-by: Heemin Kim <[email protected]>

)

Signed-off-by: Heemin Kim <[email protected]>

1. Bugfix: Ingest metadata can be null if there is no processor created
2. Refactoring: Moved private method to another class for better testing support
3. Refactoring: Set some private static final variable as public so that unit test can use it
4. Refactoring: Changed string value to static variable

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

…ch-project#310)

* Release lock before response back to caller for update/delete API
* Release lock in background task for creation API
* Change index settings to improve indexing performance

Signed-off-by: Heemin Kim <[email protected]>

…-project#311)

read_only_allow_delete does not block write to an index.
The disk-based shard allocator may add and remove this block automatically.
Therefore, use index.blocks.write instead.

Signed-off-by: Heemin Kim <[email protected]>

…oject#313)

Signed-off-by: Heemin Kim <[email protected]>

…ch-project#317)

Signed-off-by: Heemin Kim <[email protected]>

…rceExtension (opensearch-project#319)

Signed-off-by: Heemin Kim <[email protected]>

…project#320)

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

Return null index name for expired data so that it can be deleted
by clean up process. Clean up process exclude current index from deleting.
Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

In the listener, we trigger a geoip data update

Signed-off-by: Heemin Kim <[email protected]>

Otherwise, opensearch does not clear old segment files

Signed-off-by: Heemin Kim <[email protected]>

* Removed first_only parameter
* Removed max_concurrency and batch_size setting

first_only parameter was added as current geoip processor has it.
However, the parameter have no benefit for ip2geo processor as we don't do a sequantial search for array data but use multi search.

max_concurrency and batch_size setting is removed as these are only reveal internal implementation and could be a future blocker to improve performance later.

Signed-off-by: Heemin Kim <[email protected]>

)

Signed-off-by: Heemin Kim <[email protected]>

…t#334)

We don't want to use restored GeoIP data indices. Therefore we
delete the indices once restoring process complete.

When GeoIP metadata index is restored, we create a new GeoIP data index instead.

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

This is not to block transport thread

Signed-off-by: Heemin Kim <[email protected]>

Cannot query index to get data to validate IP2Geo processor.
Will add validation when we decide to store some of data in cluster state metadata.

Signed-off-by: Heemin Kim <[email protected]>

By acquiring lock asychronously, the remaining part of the code
is being run by transport thread which does not allow blocking code.
We want only single update happen in a node using single thread. However,
it cannot be acheived if I acquire lock asynchronously and pass the listener.

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

1. Changed class name from Ip2GeoCache to Ip2GeoCachedDao
2. Moved the Ip2GeoCachedDao from cache to dao package

Signed-off-by: Heemin Kim <[email protected]>

Signed-off-by: Heemin Kim <[email protected]>

…oject#343)

Signed-off-by: Heemin Kim <[email protected]>

)

1. Use _primary preference to get datasource metadata so that it can read the latest data. RefreshPolicy.IMMEDIATE won't refresh replica shards immediately according to opensearch-project#346
2. Update datasource metadata index mapping
3. Move batch size from static value to setting

Signed-off-by: Heemin Kim <[email protected]>

…project#348)

Signed-off-by: Heemin Kim <[email protected]>

…project#354)

* Update packages according to a change in OpenSearch core

Signed-off-by: Heemin Kim <[email protected]>

* Update packages according to a change in OpenSearch core (opensearch-project#353)

Signed-off-by: Heemin Kim <[email protected]>

---------

Signed-off-by: Heemin Kim <[email protected]>