Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[text analytics] Updates to Healthcare design #16247

Merged
Merged
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
c00140c
Renamed healthcare methods
Jan 14, 2021
e69151c
Exposing job metadata on the healthcare polling operation
Jan 14, 2021
d005b70
Aligning with other similar work for Analyze
Feb 2, 2021
ac9faf0
Fixed merge conflicts
Feb 2, 2021
078e875
Regenerated with custom pollers; updated tests and a few other files
Feb 2, 2021
c2a09ff
Renamed HealthcareEntityLink -> HealthcareEntityDataSource and also r…
Feb 3, 2021
fb37651
Fixed missed renames in __init__.py
Feb 3, 2021
f827c6b
Updated links and other things in the readme
Feb 3, 2021
864b1b2
Updated samples readme
Feb 3, 2021
2649001
Merge branch 'master' of https://github.com/Azure/azure-sdk-for-pytho…
Feb 3, 2021
eed2d38
Reverted an unintentional change in analyze_sentiment
Feb 3, 2021
429c4ea
Moved cancellation to the poller; fixed some linting issues
Feb 4, 2021
ef6c210
Removed an unused model
Feb 4, 2021
91ba6a1
PR comments and updated changelog
Feb 4, 2021
a78decd
Added related_entities; Fixed merge conflicts
Feb 5, 2021
7da7797
Fixed a few issues caused by merge conflicts; made HealthcareEntity h…
Feb 5, 2021
0a14704
Fixed a problem with cancellation; updated samples
Feb 5, 2021
d952ded
Merge conflicts
Feb 5, 2021
82bc4e4
PR comments
Feb 5, 2021
03d68a7
PR comments
Feb 5, 2021
00c00a5
Fixed bidirectional check; updated/added tests
Feb 5, 2021
f2dedef
added a test for async
Feb 5, 2021
bae8200
PR comments and fixing test issues
Feb 5, 2021
4563059
Fixed a linting issue
Feb 5, 2021
f9e5776
PR comments and fixed a test issue
Feb 5, 2021
3745666
Fixed cancellation issue
Feb 5, 2021
0971671
Another cancellation test fix
Feb 5, 2021
7771a70
PR comments
Feb 5, 2021
acc58ed
Async syntax issue
Feb 5, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions sdk/textanalytics/azure-ai-textanalytics/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,15 @@ of action results, in the same order they've been inputted. The actual document
each action result.

**New Features**
- No longer need to specify `api_version=TextAnalyticsApiVersion.V3_1_PREVIEW_3` when calling `begin_analyze` and `begin_analyze_healthcare`. `begin_analyze_healthcare` is still in gated preview though.
- Added a new parameter `string_index_type` to the service client methods `begin_analyze_healthcare`, `analyze_sentiment`, `recognize_entities`, `recognize_pii_entities`, and `recognize_linked_entities`.
- Renamed `begin_analyze_healthcare` to `begin_analyze_healthcare_entities`.
- Renamed `AnalyzeHealthcareResult` to `AnalyzeHealthcareEntitiesResult` and `AnalyzeHealthcareResultItem` to `AnalyzeHealthcareEntitiesResultItem`.
- Renamed `HealthcareEntityLink` to `HealthcareEntityDataSource` and renamed its properties `id` to `entity_id` and `data_source` to `name`.
- Removed `relations` from `AnalyzeHealthcareEntitiesResultItem` and added `related_entities` to `HealthcareEntity`.
- Moved the cancellation logic for the Analyze Healthcare Entities service from
the service client to the poller object returned from `begin_analyze_healthcare_entities`.
- Exposed Analyze Healthcare Entities operation metadata on the poller object returned from `begin_analyze_healthcare_entities`.
- No longer need to specify `api_version=TextAnalyticsApiVersion.V3_1_PREVIEW_3` when calling `begin_analyze` and `begin_analyze_healthcare_entities`. `begin_analyze_healthcare_entities` is still in gated preview though.
- Added a new parameter `string_index_type` to the service client methods `begin_analyze_healthcare_entities`, `analyze_sentiment`, `recognize_entities`, `recognize_pii_entities`, and `recognize_linked_entities` which tells the service how to interpret string offsets.
- Added property `length` to `CategorizedEntity`, `SentenceSentiment`, `LinkedEntityMatch`, `AspectSentiment`, `OpinionSentiment`, `PiiEntity` and
`HealthcareEntity`.

Expand Down
30 changes: 15 additions & 15 deletions sdk/textanalytics/azure-ai-textanalytics/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Text Analytics is a cloud-based service that provides advanced natural language
- Language Detection
- Key Phrase Extraction
- Batch Analysis
- Healthcare Analysis (Gated Preview)
- Healthcare Entities Analysis (Gated Preview)

[Source code][source_code] | [Package (PyPI)][ta_pypi] | [API reference documentation][ta_ref_docs]| [Product documentation][ta_product_documentation] | [Samples][ta_samples]

Expand Down Expand Up @@ -224,7 +224,7 @@ The following section provides several code snippets covering some of the most c
- [Recognize PII Entities](#recognize-pii-entities "Recognize pii entities")
- [Extract Key Phrases](#extract-key-phrases "Extract key phrases")
- [Detect Language](#detect-language "Detect language")
- [Healthcare Analysis](#healthcare-analysis "Healthcare analysis")
- [Healthcare Entities Analysis](#healthcare-entities-analysis "Healthcare Entities Analysis")
- [Batch Analysis](#batch-analysis "Batch analysis")

### Analyze sentiment
Expand Down Expand Up @@ -443,9 +443,9 @@ The returned response is a heterogeneous list of result and error objects: list[
Please refer to the service documentation for a conceptual discussion of [language detection][language_detection]
and [language and regional support][language_and_regional_support].

### Healthcare Analysis
### Healthcare Entities Analysis

The example below extracts entities recognized within the healthcare domain, and identifies relationships between entities within the input document and links to known sources of information in various well known databases, such as UMLS, CHV, MSH, etc. This sample demonstrates the usage for [long-running operations](#long-running-operations).
The example below extracts entities recognized within the healthcare domain, and identifies relationships between entities within the input document and links to known sources of information in various well known databases, such as UMLS, CHV, MSH, etc. This sample demonstrates the usage for [long-running operations](#long-running-operations).

```python
from azure.core.credentials import AzureKeyCredential
Expand All @@ -458,24 +458,24 @@ text_analytics_client = TextAnalyticsClient(endpoint, credential)

documents = ["Subject is taking 100mg of ibuprofen twice daily"]

poller = text_analytics_client.begin_analyze_healthcare(documents, show_stats=True)
poller = text_analytics_client.begin_analyze_healthcare_entities(documents, show_stats=True)
result = poller.result()

docs = [doc for doc in result if not doc.is_error]

print("Results of Healthcare Analysis:")
print("Results of Healthcare Entities Analysis:")
for idx, doc in enumerate(docs):
for entity in doc.entities:
print("Entity: {}".format(entity.text))
print("...Category: {}".format(entity.category))
print("...Subcategory: {}".format(entity.subcategory))
print("...Offset: {}".format(entity.offset))
print("...Confidence score: {}".format(entity.confidence_score))
if entity.links is not None:
print("...Links:")
for link in entity.links:
print("......ID: {}".format(link.id))
print("......Data source: {}".format(link.data_source))
if entity.data_sources is not None:
print("...Data Sources:")
for data_source in entity.data_sources:
print("......Entity ID: {}".format(data_source.entity_id))
print("......Name: {}".format(data_source.name))
for relation in doc.relations:
print("Relation:")
print("...Source: {}".format(relation.source.text))
Expand All @@ -485,7 +485,7 @@ for idx, doc in enumerate(docs):
print("------------------------------------------")
```

Note: The Healthcare Analysis service is currently available only in API version v3.1-preview.3 in gated preview. Since this is a gated preview, AAD is not supported. More information [here](https://docs.microsoft.com/azure/cognitive-services/text-analytics/how-tos/text-analytics-for-health?tabs=ner#request-access-to-the-public-preview).
Note: The Healthcare Entities Analysis service is currently available only in API version v3.1-preview.3 in gated preview. Since this is a gated preview, AAD is not supported. More information [here](https://docs.microsoft.com/azure/cognitive-services/text-analytics/how-tos/text-analytics-for-health?tabs=ner#request-access-to-the-public-preview).

### Batch Analysis

Expand Down Expand Up @@ -633,7 +633,7 @@ Common scenarios
- Recognize linked entities: [sample_recognize_linked_entities.py][recognize_linked_entities_sample] ([async version][recognize_linked_entities_sample_async])
- Extract key phrases: [sample_extract_key_phrases.py][extract_key_phrases_sample] ([async version][extract_key_phrases_sample_async])
- Detect language: [sample_detect_language.py][detect_language_sample] ([async version][detect_language_sample_async])
- Healthcare Analysis: [sample_analyze_healthcare.py][analyze_healthcare_sample] ([async version][analyze_healthcare_sample_async])
- Healthcare Entities Analysis: [sample_analyze_healthcare_entities.py][analyze_healthcare_entities_sample] ([async version][analyze_healthcare_entities_sample_async])
- Batch Analysis: [sample_analyze_batch_actions.py][analyze_sample] ([async version][analyze_sample_async])

Advanced scenarios
Expand Down Expand Up @@ -722,8 +722,8 @@ This project has adopted the [Microsoft Open Source Code of Conduct][code_of_con
[recognize_linked_entities_sample_async]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_recognize_linked_entities_async.py
[recognize_pii_entities_sample]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_recognize_pii_entities.py
[recognize_pii_entities_sample_async]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_recognize_pii_entities_async.py
[analyze_healthcare_sample]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_analyze_healthcare.py
[analyze_healthcare_sample_async]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_analyze_healthcare_async.py
[analyze_healthcare_entities_sample]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_analyze_healthcare_entities.py
[analyze_healthcare_entities_sample_async]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_analyze_healthcare_entities_async.py
[analyze_sample]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_analyze_batch_actions.py
[analyze_sample_async]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_analyze_batch_actions_async.py
[opinion_mining_sample]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_analyze_sentiment_with_opinion_mining.py
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,9 @@
RecognizePiiEntitiesResult,
PiiEntity,
PiiEntityDomainType,
AnalyzeHealthcareResultItem,
AnalyzeHealthcareEntitiesResultItem,
HealthcareEntity,
HealthcareRelation,
HealthcareEntityLink,
HealthcareEntityDataSource,
RecognizeEntitiesAction,
RecognizePiiEntitiesAction,
ExtractKeyPhrasesAction,
Expand All @@ -44,7 +43,7 @@
AnalyzeBatchActionsType,
AnalyzeBatchActionsError,
)
from._paging import AnalyzeHealthcareResult
from ._paging import AnalyzeHealthcareEntitiesResult

__all__ = [
'TextAnalyticsApiVersion',
Expand Down Expand Up @@ -73,11 +72,10 @@
'RecognizePiiEntitiesResult',
'PiiEntity',
'PiiEntityDomainType',
'AnalyzeHealthcareResultItem',
'AnalyzeHealthcareResult',
'AnalyzeHealthcareEntitiesResultItem',
'AnalyzeHealthcareEntitiesResult',
'HealthcareEntity',
'HealthcareRelation',
'HealthcareEntityLink',
'HealthcareEntityDataSource',
'RecognizeEntitiesAction',
'RecognizePiiEntitiesAction',
'ExtractKeyPhrasesAction',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,10 @@
# Licensed under the MIT License.
# ------------------------------------

from azure.core.exceptions import HttpResponseError
from azure.core.polling import AsyncLROPoller
from azure.core.polling.base_polling import OperationFailed, BadStatus
from azure.core.polling.async_base_polling import AsyncLROBasePolling
from azure.core.polling import AsyncLROPoller
from azure.core.polling._async_poller import PollingReturnType


Expand Down Expand Up @@ -76,6 +77,95 @@ async def _poll(self): # pylint:disable=invalid-overridden-method
self._pipeline_response.http_response
)

class AnalyzeHealthcareEntitiesAsyncLROPollingMethod(TextAnalyticsAsyncLROPollingMethod):

def __init__(self, *args, **kwargs):
self._text_analytics_client = kwargs.pop("text_analytics_client")
super(AnalyzeHealthcareEntitiesAsyncLROPollingMethod, self).__init__(*args, **kwargs)

@property
def _current_body(self):
from ._generated.v3_1_preview_3.models import JobMetadata
return JobMetadata.deserialize(self._pipeline_response)

@property
def created_on(self):
if not self._current_body:
return None
return self._current_body.created_date_time

@property
def expires_on(self):
if not self._current_body:
return None
return self._current_body.expiration_date_time

@property
def last_modified_on(self):
if not self._current_body:
return None
return self._current_body.last_update_date_time

@property
def id(self):
if not self._current_body:
return None
return self._current_body.job_id


class AnalyzeHealthcareEntitiesAsyncLROPoller(AsyncLROPoller[PollingReturnType]):

@property
def created_on(self):
return self._polling_method.created_on

@property
def expires_on(self):
return self._polling_method.expires_on

@property
def last_modified_on(self):
return self._polling_method.last_modified_on

@property
def id(self):
return self._polling_method.id

async def cancel( # type: ignore
self,
**kwargs
):
"""Cancel the operation currently being polled.

:keyword int polling_interval: The polling interval to use to poll the cancellation status.
The default value is 5 seconds.
:return: Returns an instance of an LROPoller that returns None.
:rtype: ~azure.core.polling.LROPoller[None]
:raises: Warning when the operation has already reached a terminal state.

.. admonition:: Example:

.. literalinclude:: ../samples/async_samples/sample_analyze_healthcare_entities_with_cancellation_async.py
:start-after: [START analyze_healthcare_entities_with_cancellation_async]
:end-before: [END analyze_healthcare_entities_with_cancellation_async]
:language: python
:dedent: 8
:caption: Cancel an existing health operation.
"""
polling_interval = kwargs.pop("polling_interval", 5)
await self._polling_method.update_status()

try:
return await getattr(self._polling_method, "_text_analytics_client").begin_cancel_health_job(
self.id,
polling=TextAnalyticsAsyncLROPollingMethod(timeout=polling_interval)
)

except HttpResponseError as error:
from ._response_handlers import process_http_response_error
process_http_response_error(error)


class AsyncAnalyzeBatchActionsLROPollingMethod(TextAnalyticsAsyncLROPollingMethod):

@property
Expand Down Expand Up @@ -137,6 +227,7 @@ def id(self):
return None
return self._current_body.job_id


class AsyncAnalyzeBatchActionsLROPoller(AsyncLROPoller[PollingReturnType]):

@property
Expand Down Expand Up @@ -166,10 +257,12 @@ def actions_succeeded_count(self):
@property
def last_modified_on(self):
return self._polling_method.last_modified_on

@property
def total_actions_count(self):
return self._polling_method.total_actions_count

@property
def id(self):
return self._polling_method.id

Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@
from azure.core.async_paging import AsyncItemPaged


class AnalyzeHealthcareResultAsync(AsyncItemPaged):
class AnalyzeHealthcareEntitiesResultAsync(AsyncItemPaged):
def __init__(self, *args, **kwargs):
self.model_version = kwargs.pop('model_version')
self.statistics = kwargs.pop('statistics')
super(AnalyzeHealthcareResultAsync, self).__init__(*args, **kwargs)
super(AnalyzeHealthcareEntitiesResultAsync, self).__init__(*args, **kwargs)


class AnalyzeResultAsync(AsyncItemPaged):
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
import warnings

# FIXME: have to manually reconfigure import path for multiapi operation mixin
from .._lro import AnalyzeHealthcareEntitiesLROPoller, AnalyzeHealthcareEntitiesLROPollingMethod
from .._lro import AnalyzeBatchActionsLROPoller, AnalyzeBatchActionsLROPollingMethod
from azure.core.exceptions import ClientAuthenticationError, HttpResponseError, ResourceExistsError, ResourceNotFoundError, map_error
from azure.core.pipeline import PipelineResponse
Expand Down Expand Up @@ -162,12 +163,12 @@ def begin_health(
:type string_index_type: str or ~azure.ai.textanalytics.v3_1_preview_3.models.StringIndexType
:keyword callable cls: A custom type or function that will be passed the direct response
:keyword str continuation_token: A continuation token to restart a poller from a saved state.
:keyword polling: Pass in True if you'd like the AnalyzeBatchActionsLROPollingMethod polling method,
:keyword polling: Pass in True if you'd like the AnalyzeHealthcareEntitiesLROPollingMethod polling method,
False for no polling, or your own initialized polling object for a personal polling strategy.
:paramtype polling: bool or ~azure.core.polling.PollingMethod
:keyword int polling_interval: Default waiting time between two polls for LRO operations if no Retry-After header is present.
:return: An instance of AnalyzeBatchActionsLROPoller that returns either HealthcareJobState or the result of cls(response)
:rtype: ~...._lro.AnalyzeBatchActionsLROPoller[~azure.ai.textanalytics.v3_1_preview_3.models.HealthcareJobState]
:return: An instance of AnalyzeHealthcareEntitiesLROPoller that returns either HealthcareJobState or the result of cls(response)
maririos marked this conversation as resolved.
Show resolved Hide resolved
:rtype: ~...._lro.AnalyzeHealthcareEntitiesLROPoller[~azure.ai.textanalytics.v3_1_preview_3.models.HealthcareJobState]
:raises ~azure.core.exceptions.HttpResponseError:
"""
api_version = self._get_api_version('begin_health')
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
import warnings

# FIXME: have to manually reconfigure import path for multiapi operation mixin
from ..._async_lro import AnalyzeHealthcareEntitiesAsyncLROPoller, AnalyzeHealthcareEntitiesAsyncLROPollingMethod
from ..._async_lro import AsyncAnalyzeBatchActionsLROPoller, AsyncAnalyzeBatchActionsLROPollingMethod
from azure.core.exceptions import ClientAuthenticationError, HttpResponseError, ResourceExistsError, ResourceNotFoundError, map_error
from azure.core.pipeline import PipelineResponse
Expand Down Expand Up @@ -141,7 +142,7 @@ async def begin_health(
model_version: Optional[str] = None,
string_index_type: Optional[Union[str, "_models.StringIndexType"]] = "TextElements_v8",
**kwargs
) -> AsyncAnalyzeBatchActionsLROPoller["_models.HealthcareJobState"]:
) -> AnalyzeHealthcareEntitiesAsyncLROPoller["_models.HealthcareJobState"]:
"""Submit healthcare analysis job.

Start a healthcare analysis job to recognize healthcare related entities (drugs, conditions,
Expand All @@ -158,12 +159,12 @@ async def begin_health(
:type string_index_type: str or ~azure.ai.textanalytics.v3_1_preview_3.models.StringIndexType
:keyword callable cls: A custom type or function that will be passed the direct response
:keyword str continuation_token: A continuation token to restart a poller from a saved state.
:keyword polling: Pass in True if you'd like the AsyncAnalyzeBatchActionsLROPollingMethod polling method,
:keyword polling: Pass in True if you'd like the AnalyzeHealthcareEntitiesAsyncLROPollingMethod polling method,
False for no polling, or your own initialized polling object for a personal polling strategy.
:paramtype polling: bool or ~azure.core.polling.AsyncPollingMethod
:keyword int polling_interval: Default waiting time between two polls for LRO operations if no Retry-After header is present.
:return: An instance of AsyncAnalyzeBatchActionsLROPoller that returns either HealthcareJobState or the result of cls(response)
:rtype: ~...._async_lro.AsyncAnalyzeBatchActionsLROPoller[~azure.ai.textanalytics.v3_1_preview_3.models.HealthcareJobState]
:return: An instance of AnalyzeHealthcareEntitiesAsyncLROPoller that returns either HealthcareJobState or the result of cls(response)
:rtype: ~....._async_lro.AnalyzeHealthcareEntitiesAsyncLROPoller[~azure.ai.textanalytics.v3_1_preview_3.models.HealthcareJobState]
:raises ~azure.core.exceptions.HttpResponseError:
"""
api_version = self._get_api_version('begin_health')
Expand Down
Loading