-
Notifications
You must be signed in to change notification settings - Fork 2.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[text analytics] Updates to Healthcare design #16247
Changes from 24 commits
c00140c
e69151c
d005b70
ac9faf0
078e875
c2a09ff
fb37651
f827c6b
864b1b2
2649001
eed2d38
429c4ea
ef6c210
91ba6a1
a78decd
7da7797
0a14704
d952ded
82bc4e4
03d68a7
00c00a5
f2dedef
bae8200
4563059
f9e5776
3745666
0971671
7771a70
acc58ed
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -9,7 +9,7 @@ Text Analytics is a cloud-based service that provides advanced natural language | |
- Language Detection | ||
- Key Phrase Extraction | ||
- Batch Analysis | ||
- Healthcare Analysis (Gated Preview) | ||
- Healthcare Entities Analysis (Gated Preview) | ||
|
||
[Source code][source_code] | [Package (PyPI)][ta_pypi] | [API reference documentation][ta_ref_docs]| [Product documentation][ta_product_documentation] | [Samples][ta_samples] | ||
|
||
|
@@ -224,7 +224,7 @@ The following section provides several code snippets covering some of the most c | |
- [Recognize PII Entities](#recognize-pii-entities "Recognize pii entities") | ||
- [Extract Key Phrases](#extract-key-phrases "Extract key phrases") | ||
- [Detect Language](#detect-language "Detect language") | ||
- [Healthcare Analysis](#healthcare-analysis "Healthcare analysis") | ||
- [Healthcare Entities Analysis](#healthcare-analysis "Healthcare analysis") | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this needs to be There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed! |
||
- [Batch Analysis](#batch-analysis "Batch analysis") | ||
|
||
### Analyze sentiment | ||
|
@@ -443,9 +443,8 @@ The returned response is a heterogeneous list of result and error objects: list[ | |
Please refer to the service documentation for a conceptual discussion of [language detection][language_detection] | ||
and [language and regional support][language_and_regional_support]. | ||
|
||
### Healthcare Analysis | ||
|
||
The example below extracts entities recognized within the healthcare domain, and identifies relationships between entities within the input document and links to known sources of information in various well known databases, such as UMLS, CHV, MSH, etc. This sample demonstrates the usage for [long-running operations](#long-running-operations). | ||
### Healthcare Entities Analysis | ||
The example below extracts entities recognized within the healthcare domain, and identifies relationships between entities within the input document and links to known sources of information in various well known databases, such as UMLS, CHV, MSH, etc. This sample demonstrates the usage for [long-running operations](#long-running-operations). | ||
|
||
```python | ||
from azure.core.credentials import AzureKeyCredential | ||
|
@@ -458,24 +457,24 @@ text_analytics_client = TextAnalyticsClient(endpoint, credential) | |
|
||
documents = ["Subject is taking 100mg of ibuprofen twice daily"] | ||
|
||
poller = text_analytics_client.begin_analyze_healthcare(documents, show_stats=True) | ||
poller = text_analytics_client.begin_analyze_healthcare_entities(documents, show_stats=True) | ||
result = poller.result() | ||
|
||
docs = [doc for doc in result if not doc.is_error] | ||
|
||
print("Results of Healthcare Analysis:") | ||
print("Results of Healthcare Entities Analysis:") | ||
for idx, doc in enumerate(docs): | ||
for entity in doc.entities: | ||
print("Entity: {}".format(entity.text)) | ||
print("...Category: {}".format(entity.category)) | ||
print("...Subcategory: {}".format(entity.subcategory)) | ||
print("...Offset: {}".format(entity.offset)) | ||
print("...Confidence score: {}".format(entity.confidence_score)) | ||
if entity.links is not None: | ||
print("...Links:") | ||
for link in entity.links: | ||
print("......ID: {}".format(link.id)) | ||
print("......Data source: {}".format(link.data_source)) | ||
if entity.data_sources is not None: | ||
print("...Data Sources:") | ||
for data_source in entity.data_sources: | ||
print("......Entity ID: {}".format(data_source.entity_id)) | ||
print("......Name: {}".format(data_source.name)) | ||
for relation in doc.relations: | ||
print("Relation:") | ||
print("...Source: {}".format(relation.source.text)) | ||
|
@@ -485,7 +484,7 @@ for idx, doc in enumerate(docs): | |
print("------------------------------------------") | ||
``` | ||
|
||
Note: The Healthcare Analysis service is currently available only in API version v3.1-preview.3 in gated preview. Since this is a gated preview, AAD is not supported. More information [here](https://docs.microsoft.com/azure/cognitive-services/text-analytics/how-tos/text-analytics-for-health?tabs=ner#request-access-to-the-public-preview). | ||
Note: The Healthcare Entities Analysis service is currently available only in API version v3.1-preview.3 in gated preview. Since this is a gated preview, AAD is not supported. More information [here](https://docs.microsoft.com/azure/cognitive-services/text-analytics/how-tos/text-analytics-for-health?tabs=ner#request-access-to-the-public-preview). | ||
|
||
### Batch Analysis | ||
|
||
|
@@ -633,7 +632,7 @@ Common scenarios | |
- Recognize linked entities: [sample_recognize_linked_entities.py][recognize_linked_entities_sample] ([async version][recognize_linked_entities_sample_async]) | ||
- Extract key phrases: [sample_extract_key_phrases.py][extract_key_phrases_sample] ([async version][extract_key_phrases_sample_async]) | ||
- Detect language: [sample_detect_language.py][detect_language_sample] ([async version][detect_language_sample_async]) | ||
- Healthcare Analysis: [sample_analyze_healthcare.py][analyze_healthcare_sample] ([async version][analyze_healthcare_sample_async]) | ||
- Healthcare Entities Analysis: [sample_analyze_healthcare_entities.py][analyze_healthcare_entities_sample] ([async version][analyze_healthcare_entities_sample_async]) | ||
- Batch Analysis: [sample_analyze_batch_actions.py][analyze_sample] ([async version][analyze_sample_async]) | ||
|
||
Advanced scenarios | ||
|
@@ -722,8 +721,8 @@ This project has adopted the [Microsoft Open Source Code of Conduct][code_of_con | |
[recognize_linked_entities_sample_async]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_recognize_linked_entities_async.py | ||
[recognize_pii_entities_sample]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_recognize_pii_entities.py | ||
[recognize_pii_entities_sample_async]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_recognize_pii_entities_async.py | ||
[analyze_healthcare_sample]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_analyze_healthcare.py | ||
[analyze_healthcare_sample_async]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_analyze_healthcare_async.py | ||
[analyze_healthcare_entities_sample]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_analyze_healthcare_entities.py | ||
[analyze_healthcare_entities_sample_async]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_analyze_healthcare_entities_async.py | ||
[analyze_sample]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_analyze_batch_actions.py | ||
[analyze_sample_async]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/textanalytics/azure-ai-textanalytics/samples/async_samples/sample_analyze_batch_actions_async.py | ||
[opinion_mining_sample]: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/textanalytics/azure-ai-textanalytics/samples/sample_analyze_sentiment_with_opinion_mining.py | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -4,9 +4,10 @@ | |
# Licensed under the MIT License. | ||
# ------------------------------------ | ||
|
||
from azure.core.exceptions import HttpResponseError | ||
from azure.core.polling import AsyncLROPoller | ||
from azure.core.polling.base_polling import OperationFailed, BadStatus | ||
from azure.core.polling.async_base_polling import AsyncLROBasePolling | ||
from azure.core.polling import AsyncLROPoller | ||
from azure.core.polling._async_poller import PollingReturnType | ||
|
||
|
||
|
@@ -76,6 +77,101 @@ async def _poll(self): # pylint:disable=invalid-overridden-method | |
self._pipeline_response.http_response | ||
) | ||
|
||
class AnalyzeHealthcareEntitiesAsyncLROPollingMethod(TextAnalyticsAsyncLROPollingMethod): | ||
|
||
def __init__(self, *args, **kwargs): | ||
self._text_analytics_client = kwargs.pop("text_analytics_client") | ||
super(AnalyzeHealthcareEntitiesAsyncLROPollingMethod, self).__init__(*args, **kwargs) | ||
|
||
@property | ||
def _current_body(self): | ||
from ._generated.v3_1_preview_3.models import JobMetadata | ||
return JobMetadata.deserialize(self._pipeline_response) | ||
|
||
@property | ||
def created_on(self): | ||
if not self._current_body: | ||
return None | ||
return self._current_body.created_date_time | ||
|
||
@property | ||
def expires_on(self): | ||
if not self._current_body: | ||
return None | ||
return self._current_body.expiration_date_time | ||
|
||
@property | ||
def last_modified_on(self): | ||
if not self._current_body: | ||
return None | ||
return self._current_body.last_update_date_time | ||
|
||
@property | ||
def id(self): | ||
if not self._current_body: | ||
return None | ||
return self._current_body.job_id | ||
|
||
|
||
class AnalyzeHealthcareEntitiesAsyncLROPoller(AsyncLROPoller[PollingReturnType]): | ||
|
||
@property | ||
def created_on(self): | ||
return self._polling_method.created_on | ||
|
||
@property | ||
def expires_on(self): | ||
return self._polling_method.expires_on | ||
|
||
@property | ||
def last_modified_on(self): | ||
return self._polling_method.last_modified_on | ||
|
||
@property | ||
def id(self): | ||
return self._polling_method.id | ||
|
||
async def cancel( # type: ignore | ||
self, | ||
**kwargs | ||
): | ||
"""Cancel the operation currently being polled. | ||
|
||
:keyword int polling_interval: The polling interval to use to poll the cancellation status. | ||
The default value is 5 seconds. | ||
:return: Returns an instance of an LROPoller that returns None. | ||
:rtype: ~azure.core.polling.LROPoller[None] | ||
:raises: Warning when the operation has already reached a terminal state. | ||
|
||
.. admonition:: Example: | ||
|
||
.. literalinclude:: ../samples/async_samples/sample_analyze_healthcare_entities_with_cancellation_async.py | ||
:start-after: [START analyze_healthcare_entities_with_cancellation_async] | ||
:end-before: [END analyze_healthcare_entities_with_cancellation_async] | ||
:language: python | ||
:dedent: 8 | ||
:caption: Cancel an existing health operation. | ||
""" | ||
polling_interval = kwargs.pop("polling_interval", 5) | ||
|
||
terminal_states = ["cancelled", "cancelling", "failed", "succeeded", "partiallyCompleted", "rejected"] | ||
await self._polling_method.update_status() | ||
|
||
if self._polling_method.status() in terminal_states: | ||
raise Warning("Operation with ID '%s' is already in a terminal state and cannot be cancelled." \ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @abhahn the more I think about it, the more I think we should not check the status, and just make the network call. We try to do as little client side validation as possible |
||
% self.id) | ||
|
||
try: | ||
return await getattr(self._polling_method, "_text_analytics_client").begin_cancel_health_job( | ||
self.id, | ||
polling=TextAnalyticsAsyncLROPollingMethod(timeout=polling_interval) | ||
) | ||
|
||
except HttpResponseError as error: | ||
from ._response_handlers import process_http_response_error | ||
process_http_response_error(error) | ||
|
||
|
||
class AsyncAnalyzeBatchActionsLROPollingMethod(TextAnalyticsAsyncLROPollingMethod): | ||
|
||
@property | ||
|
@@ -137,6 +233,7 @@ def id(self): | |
return None | ||
return self._current_body.job_id | ||
|
||
|
||
class AsyncAnalyzeBatchActionsLROPoller(AsyncLROPoller[PollingReturnType]): | ||
|
||
@property | ||
|
@@ -166,10 +263,12 @@ def actions_succeeded_count(self): | |
@property | ||
def last_modified_on(self): | ||
return self._polling_method.last_modified_on | ||
|
||
@property | ||
def total_actions_count(self): | ||
return self._polling_method.total_actions_count | ||
|
||
@property | ||
def id(self): | ||
return self._polling_method.id | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please explain the role of that new parameter.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also please talk briefly about the new graph design.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you share a link to your changelog? I got up really early this morning to work on this PR and am close to the time where I will be logging off for the day. If you can share an example I will add it here, otherwise it will have to wait.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, JS Changelog: https://github.com/Azure/azure-sdk-for-js/blob/master/sdk/textanalytics/ai-text-analytics/CHANGELOG.md#510-beta4-unreleased
here is the snippet:
[Breaking] The healthcare entities returned by beginAnalyzeHealthcare are now organized as a directed graph where the edges represent a certain type of healthcare relationship between the source and target entities. Edges are stored in the relatedEntities property.