Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Credentials in azure-identity should be pickle-able #37451

Closed
needuv opened this issue Sep 18, 2024 · 4 comments
Closed

Credentials in azure-identity should be pickle-able #37451

needuv opened this issue Sep 18, 2024 · 4 comments
Assignees
Labels
Azure.Identity needs-author-feedback Workflow: More information is needed from author to address the issue. no-recent-activity There has been no recent activity on this issue.

Comments

@needuv
Copy link
Member

needuv commented Sep 18, 2024

Is your feature request related to a problem? Please describe.

The azure-ai-evaluation SDK exposes a credential field in its evaluators (example: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/evaluators/_content_safety/_violence.py#L14) to be used when calling the Azure AI RAI service. Right now, this credential is optional, and at execution time we choose DefaultAzureCredential if the user did not pass anything in. However, we need to make this credential parameter required in the evaluators since choosing a default credential is not safe SDK behavior.

If the evaluator is being run on a single input OR when it is used in the batch evaluate API with no credential being passed in, the customer will not see any errors. However, if the user passes a credential to the evaluator and uses it in the evaluate function, it will not work since evaluate uses multiprocessing to do batch runs of multiple evaluators. Multiprocessing requires the evaluator to be pickle-able, but the credentials in azure-identity are not pickle-able, which breaks this scenario.

Describe the solution you'd like
The credentials in azure-identity should be pickle-able so that they can be used in evaluators when doing batch evaluation using evaluate.

@xiangyan99
Copy link
Member

I believe the identity object is pickle-able.

Could you share a repro if it is not the case?

@xiangyan99 xiangyan99 added the needs-author-feedback Workflow: More information is needed from author to address the issue. label Sep 24, 2024
Copy link

Hi @needuv. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

Copy link

github-actions bot commented Oct 2, 2024

Hi @needuv, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

@github-actions github-actions bot added the no-recent-activity There has been no recent activity on this issue. label Oct 2, 2024
@needuv
Copy link
Member Author

needuv commented Oct 2, 2024

Hi @xiangyan99, we just tested with the latest azure-identity and confirmed we are not seeing the errors with pickling that we saw before with multiprocess. Will close this issue

@needuv needuv closed this as completed Oct 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Azure.Identity needs-author-feedback Workflow: More information is needed from author to address the issue. no-recent-activity There has been no recent activity on this issue.
Projects
Development

No branches or pull requests

4 participants