Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[text analytics] analyze changes #16418

Merged
merged 30 commits into from
Feb 4, 2021
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
9252158
begin_analyze -> begin_analyze_batch_actions
iscai-msft Jan 29, 2021
2d49745
rename task classes (i.e. EntitiesRecognitionTask) to action
iscai-msft Jan 29, 2021
a834254
return results directly on the AnalyzeBatchActionsResult
iscai-msft Jan 29, 2021
092d47b
rename parameters
iscai-msft Jan 29, 2021
b087a57
switch mentions to task to action
iscai-msft Jan 29, 2021
3064965
change input of actions to list, return actions same order as input
iscai-msft Jan 30, 2021
3274fd8
add initial code and test for poller metadata
iscai-msft Jan 30, 2021
510c97b
fix multi page case
iscai-msft Feb 1, 2021
50f1a28
regen with custom pollers and polling methods
iscai-msft Feb 2, 2021
09d9135
fix pollers and callbacks
iscai-msft Feb 2, 2021
a8a68f8
clean up sync tests
iscai-msft Feb 2, 2021
806a8e5
fix async tests
iscai-msft Feb 2, 2021
ff538ab
fix async tests and generated code
iscai-msft Feb 2, 2021
7e4cc1e
Merge branch 'analyze_redesign' of https://github.com/iscai-msft/azur…
iscai-msft Feb 2, 2021
4fd6637
fix healthcare tests
iscai-msft Feb 2, 2021
3885590
pylint and mypy
iscai-msft Feb 2, 2021
527dc58
Merge branch 'master' of https://github.com/Azure/azure-sdk-for-pytho…
iscai-msft Feb 2, 2021
5707d8e
update samples
iscai-msft Feb 2, 2021
a21646b
revert eng tools change
iscai-msft Feb 2, 2021
18602d3
remove erroneous async test in sync tests
iscai-msft Feb 2, 2021
8ed20e1
fix domain filter naming and remove await in test
iscai-msft Feb 2, 2021
f4a4253
rename sample_analyze -> sample_analyze_batch_actions
iscai-msft Feb 3, 2021
8866ba4
task docstring -> action
iscai-msft Feb 3, 2021
ed075f4
address krista's comments
Feb 3, 2021
c1f7bdc
remove playback test only headers
iscai-msft Feb 3, 2021
20eee41
add support for heterogeneous list of good and bad responses
iscai-msft Feb 4, 2021
6ed4e1e
Merge branch 'master' of https://github.com/Azure/azure-sdk-for-pytho…
iscai-msft Feb 4, 2021
04fff3a
fix phi
iscai-msft Feb 4, 2021
e97eb9e
fix sync sample, fix typo in name
iscai-msft Feb 4, 2021
54cc95b
fix repr
iscai-msft Feb 4, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions sdk/textanalytics/azure-ai-textanalytics/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,15 @@

## 5.1.0b5 (Unreleased)

**Breaking Changes**

- Rename `begin_analyze` to `begin_analyze_batch_actions`.
- Now instead of separate parameters for all of the different types of actions you can pass to `begin_analyze_batch_actions`, we accept one parameter `actions`,
which is a list of actions you would like performed. The results of the actions are returned in the same order as when inputted.
- The response object from `begin_analyze_batch_actions` has also changed. Now, after the completion of your long running operation, we return a paged iterable
of action results, in the same order they've been inputted. The actual document results for each action are included under property `document_results` of
each action result.

**New Features**
- No longer need to specify `api_version=TextAnalyticsApiVersion.V3_1_PREVIEW_3` when calling `begin_analyze` and `begin_analyze_healthcare`. `begin_analyze_healthcare` is still in gated preview though.

Expand Down
231 changes: 131 additions & 100 deletions sdk/textanalytics/azure-ai-textanalytics/README.md

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,12 @@
HealthcareEntity,
HealthcareRelation,
HealthcareEntityLink,
EntitiesRecognitionTask,
PiiEntitiesRecognitionTask,
KeyPhraseExtractionTask,
TextAnalysisResult,
RequestStatistics
RecognizeEntitiesAction,
RecognizePiiEntitiesAction,
ExtractKeyPhrasesAction,
AnalyzeBatchActionsResult,
RequestStatistics,
AnalyzeBatchActionsType,
)
from._paging import AnalyzeHealthcareResult

Expand Down Expand Up @@ -76,11 +77,12 @@
'HealthcareEntity',
'HealthcareRelation',
'HealthcareEntityLink',
'EntitiesRecognitionTask',
'PiiEntitiesRecognitionTask',
'KeyPhraseExtractionTask',
'TextAnalysisResult',
'RequestStatistics'
'RecognizeEntitiesAction',
'RecognizePiiEntitiesAction',
'ExtractKeyPhrasesAction',
'AnalyzeBatchActionsResult',
'RequestStatistics',
'AnalyzeBatchActionsType',
]

__version__ = VERSION
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@

from azure.core.polling.base_polling import OperationFailed, BadStatus
from azure.core.polling.async_base_polling import AsyncLROBasePolling
from azure.core.polling import AsyncLROPoller
from azure.core.polling._async_poller import PollingReturnType


_FINISHED = frozenset(["succeeded", "cancelled", "failed", "partiallysucceeded"])
Expand Down Expand Up @@ -73,3 +75,101 @@ async def _poll(self): # pylint:disable=invalid-overridden-method
TextAnalyticsAsyncLROPollingMethod._raise_if_bad_http_status_and_method(
self._pipeline_response.http_response
)

class AsyncAnalyzeBatchActionsLROPollingMethod(TextAnalyticsAsyncLROPollingMethod):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adding metadata to the poller


@property
def _current_body(self):
kristapratico marked this conversation as resolved.
Show resolved Hide resolved
from ._generated.v3_1_preview_3.models import JobMetadata
return JobMetadata.deserialize(self._pipeline_response)

@property
def created_on(self):
if not self._current_body:
return None
return self._current_body.created_date_time

@property
def display_name(self):
if not self._current_body:
return None
return self._current_body.display_name

@property
def expires_on(self):
if not self._current_body:
return None
return self._current_body.expiration_date_time

@property
def actions_failed_count(self):
if not self._current_body:
return None
return self._current_body.additional_properties['tasks']['failed']

@property
def actions_in_progress_count(self):
if not self._current_body:
return None
return self._current_body.additional_properties['tasks']['inProgress']

@property
def actions_succeeded_count(self):
if not self._current_body:
return None
return self._current_body.additional_properties['tasks']["completed"]

@property
def last_modified_on(self):
if not self._current_body:
return None
return self._current_body.last_update_date_time

@property
def total_actions_count(self):
if not self._current_body:
return None
return self._current_body.additional_properties['tasks']["total"]

@property
def id(self):
if not self._current_body:
return None
return self._current_body.job_id

class AsyncAnalyzeBatchActionsLROPoller(AsyncLROPoller[PollingReturnType]):

@property
def created_on(self):
return self._polling_method.created_on

@property
def display_name(self):
return self._polling_method.display_name

@property
def expires_on(self):
return self._polling_method.expires_on

@property
def actions_failed_count(self):
return self._polling_method.actions_failed_count

@property
def actions_in_progress_count(self):
return self._polling_method.actions_in_progress_count

@property
def actions_succeeded_count(self):
return self._polling_method.actions_succeeded_count

@property
def last_modified_on(self):
return self._polling_method.last_modified_on
@property
def total_actions_count(self):
return self._polling_method.total_actions_count

@property
def id(self):
return self._polling_method.id
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@
from typing import TYPE_CHECKING
import warnings

# FIXME: have to manually reconfigure import path for multiapi operation mixin
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can ignore all of the generated files

from .._lro import AnalyzeBatchActionsLROPoller, AnalyzeBatchActionsLROPollingMethod
from azure.core.exceptions import ClientAuthenticationError, HttpResponseError, ResourceExistsError, ResourceNotFoundError, map_error
from azure.core.pipeline import PipelineResponse
from azure.core.pipeline.transport import HttpRequest, HttpResponse
Expand Down Expand Up @@ -82,12 +84,12 @@ def begin_analyze(
:type body: ~azure.ai.textanalytics.v3_1_preview_3.models.AnalyzeBatchInput
:keyword callable cls: A custom type or function that will be passed the direct response
:keyword str continuation_token: A continuation token to restart a poller from a saved state.
:keyword polling: Pass in True if you'd like the LROBasePolling polling method,
:keyword polling: Pass in True if you'd like the AnalyzeBatchActionsLROPollingMethod polling method,
False for no polling, or your own initialized polling object for a personal polling strategy.
:paramtype polling: bool or ~azure.core.polling.PollingMethod
:keyword int polling_interval: Default waiting time between two polls for LRO operations if no Retry-After header is present.
:return: An instance of LROPoller that returns either None or the result of cls(response)
:rtype: ~azure.core.polling.LROPoller[None]
:return: An instance of AnalyzeBatchActionsLROPoller that returns either AnalyzeJobState or the result of cls(response)
:rtype: ~...._lro.AnalyzeBatchActionsLROPoller[~azure.ai.textanalytics.v3_1_preview_3.models.AnalyzeJobState]
:raises ~azure.core.exceptions.HttpResponseError:
"""
api_version = self._get_api_version('begin_analyze')
Expand Down Expand Up @@ -160,12 +162,12 @@ def begin_health(
:type string_index_type: str or ~azure.ai.textanalytics.v3_1_preview_3.models.StringIndexType
:keyword callable cls: A custom type or function that will be passed the direct response
:keyword str continuation_token: A continuation token to restart a poller from a saved state.
:keyword polling: Pass in True if you'd like the LROBasePolling polling method,
:keyword polling: Pass in True if you'd like the AnalyzeBatchActionsLROPollingMethod polling method,
False for no polling, or your own initialized polling object for a personal polling strategy.
:paramtype polling: bool or ~azure.core.polling.PollingMethod
:keyword int polling_interval: Default waiting time between two polls for LRO operations if no Retry-After header is present.
:return: An instance of LROPoller that returns either None or the result of cls(response)
:rtype: ~azure.core.polling.LROPoller[None]
:return: An instance of AnalyzeBatchActionsLROPoller that returns either HealthcareJobState or the result of cls(response)
:rtype: ~...._lro.AnalyzeBatchActionsLROPoller[~azure.ai.textanalytics.v3_1_preview_3.models.HealthcareJobState]
:raises ~azure.core.exceptions.HttpResponseError:
"""
api_version = self._get_api_version('begin_health')
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@
from typing import Any, Callable, Dict, Generic, List, Optional, TypeVar, Union
import warnings

# FIXME: have to manually reconfigure import path for multiapi operation mixin
from ..._async_lro import AsyncAnalyzeBatchActionsLROPoller, AsyncAnalyzeBatchActionsLROPollingMethod
from azure.core.exceptions import ClientAuthenticationError, HttpResponseError, ResourceExistsError, ResourceNotFoundError, map_error
from azure.core.pipeline import PipelineResponse
from azure.core.pipeline.transport import AsyncHttpResponse, HttpRequest
Expand Down Expand Up @@ -68,7 +70,7 @@ async def begin_analyze(
self,
body: Optional["_models.AnalyzeBatchInput"] = None,
**kwargs
) -> AsyncLROPoller[None]:
) -> AsyncAnalyzeBatchActionsLROPoller["_models.AnalyzeJobState"]:
"""Submit analysis job.

Submit a collection of text documents for analysis. Specify one or more unique tasks to be
Expand All @@ -78,12 +80,12 @@ async def begin_analyze(
:type body: ~azure.ai.textanalytics.v3_1_preview_3.models.AnalyzeBatchInput
:keyword callable cls: A custom type or function that will be passed the direct response
:keyword str continuation_token: A continuation token to restart a poller from a saved state.
:keyword polling: Pass in True if you'd like the AsyncLROBasePolling polling method,
:keyword polling: Pass in True if you'd like the AsyncAnalyzeBatchActionsLROPollingMethod polling method,
False for no polling, or your own initialized polling object for a personal polling strategy.
:paramtype polling: bool or ~azure.core.polling.AsyncPollingMethod
:keyword int polling_interval: Default waiting time between two polls for LRO operations if no Retry-After header is present.
:return: An instance of AsyncLROPoller that returns either None or the result of cls(response)
:rtype: ~azure.core.polling.AsyncLROPoller[None]
:return: An instance of AsyncAnalyzeBatchActionsLROPoller that returns either AnalyzeJobState or the result of cls(response)
:rtype: ~...._async_lro.AsyncAnalyzeBatchActionsLROPoller[~azure.ai.textanalytics.v3_1_preview_3.models.AnalyzeJobState]
:raises ~azure.core.exceptions.HttpResponseError:
"""
api_version = self._get_api_version('begin_analyze')
Expand Down Expand Up @@ -139,7 +141,7 @@ async def begin_health(
model_version: Optional[str] = None,
string_index_type: Optional[Union[str, "_models.StringIndexType"]] = "TextElements_v8",
**kwargs
) -> AsyncLROPoller[None]:
) -> AsyncAnalyzeBatchActionsLROPoller["_models.HealthcareJobState"]:
"""Submit healthcare analysis job.

Start a healthcare analysis job to recognize healthcare related entities (drugs, conditions,
Expand All @@ -156,12 +158,12 @@ async def begin_health(
:type string_index_type: str or ~azure.ai.textanalytics.v3_1_preview_3.models.StringIndexType
:keyword callable cls: A custom type or function that will be passed the direct response
:keyword str continuation_token: A continuation token to restart a poller from a saved state.
:keyword polling: Pass in True if you'd like the AsyncLROBasePolling polling method,
:keyword polling: Pass in True if you'd like the AsyncAnalyzeBatchActionsLROPollingMethod polling method,
False for no polling, or your own initialized polling object for a personal polling strategy.
:paramtype polling: bool or ~azure.core.polling.AsyncPollingMethod
:keyword int polling_interval: Default waiting time between two polls for LRO operations if no Retry-After header is present.
:return: An instance of AsyncLROPoller that returns either None or the result of cls(response)
:rtype: ~azure.core.polling.AsyncLROPoller[None]
:return: An instance of AsyncAnalyzeBatchActionsLROPoller that returns either HealthcareJobState or the result of cls(response)
:rtype: ~...._async_lro.AsyncAnalyzeBatchActionsLROPoller[~azure.ai.textanalytics.v3_1_preview_3.models.HealthcareJobState]
:raises ~azure.core.exceptions.HttpResponseError:
"""
api_version = self._get_api_version('begin_health')
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,37 @@
},
"constant": {
},
"call": "credential, endpoint"
"call": "credential, endpoint",
"service_client_specific": {
"sync": {
"api_version": {
"signature": "api_version=None, # type: Optional[str]",
"description": "API version to use if no profile is provided, or if missing in profile.",
"docstring_type": "str",
"required": false
},
"profile": {
"signature": "profile=KnownProfiles.default, # type: KnownProfiles",
"description": "A profile definition, from KnownProfiles to dict.",
"docstring_type": "azure.profiles.KnownProfiles",
"required": false
}
},
"async": {
"api_version": {
"signature": "api_version: Optional[str] = None,",
"description": "API version to use if no profile is provided, or if missing in profile.",
"docstring_type": "str",
"required": false
},
"profile": {
"signature": "profile: KnownProfiles = KnownProfiles.default,",
"description": "A profile definition, from KnownProfiles to dict.",
"docstring_type": "azure.profiles.KnownProfiles",
"required": false
}
}
}
},
"config": {
"credential": true,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ async def entities_recognition_general(

if response.status_code not in [200]:
map_error(status_code=response.status_code, response=response, error_map=error_map)
error = self._deserialize(_models.ErrorResponse, response)
error = self._deserialize.failsafe_deserialize(_models.ErrorResponse, response)
raise HttpResponseError(response=response, model=error)

deserialized = self._deserialize('EntitiesResult', pipeline_response)
Expand Down Expand Up @@ -159,7 +159,7 @@ async def entities_linking(

if response.status_code not in [200]:
map_error(status_code=response.status_code, response=response, error_map=error_map)
error = self._deserialize(_models.ErrorResponse, response)
error = self._deserialize.failsafe_deserialize(_models.ErrorResponse, response)
raise HttpResponseError(response=response, model=error)

deserialized = self._deserialize('EntityLinkingResult', pipeline_response)
Expand Down Expand Up @@ -234,7 +234,7 @@ async def key_phrases(

if response.status_code not in [200]:
map_error(status_code=response.status_code, response=response, error_map=error_map)
error = self._deserialize(_models.ErrorResponse, response)
error = self._deserialize.failsafe_deserialize(_models.ErrorResponse, response)
raise HttpResponseError(response=response, model=error)

deserialized = self._deserialize('KeyPhraseResult', pipeline_response)
Expand Down Expand Up @@ -310,7 +310,7 @@ async def languages(

if response.status_code not in [200]:
map_error(status_code=response.status_code, response=response, error_map=error_map)
error = self._deserialize(_models.ErrorResponse, response)
error = self._deserialize.failsafe_deserialize(_models.ErrorResponse, response)
raise HttpResponseError(response=response, model=error)

deserialized = self._deserialize('LanguageResult', pipeline_response)
Expand Down Expand Up @@ -386,7 +386,7 @@ async def sentiment(

if response.status_code not in [200]:
map_error(status_code=response.status_code, response=response, error_map=error_map)
error = self._deserialize(_models.ErrorResponse, response)
error = self._deserialize.failsafe_deserialize(_models.ErrorResponse, response)
raise HttpResponseError(response=response, model=error)

deserialized = self._deserialize('SentimentResponse', pipeline_response)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ def entities_recognition_general(

if response.status_code not in [200]:
map_error(status_code=response.status_code, response=response, error_map=error_map)
error = self._deserialize(_models.ErrorResponse, response)
error = self._deserialize.failsafe_deserialize(_models.ErrorResponse, response)
raise HttpResponseError(response=response, model=error)

deserialized = self._deserialize('EntitiesResult', pipeline_response)
Expand Down Expand Up @@ -165,7 +165,7 @@ def entities_linking(

if response.status_code not in [200]:
map_error(status_code=response.status_code, response=response, error_map=error_map)
error = self._deserialize(_models.ErrorResponse, response)
error = self._deserialize.failsafe_deserialize(_models.ErrorResponse, response)
raise HttpResponseError(response=response, model=error)

deserialized = self._deserialize('EntityLinkingResult', pipeline_response)
Expand Down Expand Up @@ -241,7 +241,7 @@ def key_phrases(

if response.status_code not in [200]:
map_error(status_code=response.status_code, response=response, error_map=error_map)
error = self._deserialize(_models.ErrorResponse, response)
error = self._deserialize.failsafe_deserialize(_models.ErrorResponse, response)
raise HttpResponseError(response=response, model=error)

deserialized = self._deserialize('KeyPhraseResult', pipeline_response)
Expand Down Expand Up @@ -318,7 +318,7 @@ def languages(

if response.status_code not in [200]:
map_error(status_code=response.status_code, response=response, error_map=error_map)
error = self._deserialize(_models.ErrorResponse, response)
error = self._deserialize.failsafe_deserialize(_models.ErrorResponse, response)
raise HttpResponseError(response=response, model=error)

deserialized = self._deserialize('LanguageResult', pipeline_response)
Expand Down Expand Up @@ -395,7 +395,7 @@ def sentiment(

if response.status_code not in [200]:
map_error(status_code=response.status_code, response=response, error_map=error_map)
error = self._deserialize(_models.ErrorResponse, response)
error = self._deserialize.failsafe_deserialize(_models.ErrorResponse, response)
raise HttpResponseError(response=response, model=error)

deserialized = self._deserialize('SentimentResponse', pipeline_response)
Expand Down
Loading