-
Notifications
You must be signed in to change notification settings - Fork 377
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FEATURE] Create a new endpoint supporting bulk annotation #4367
Comments
🤖 Hello @frascuchon! 👋 I'm Dosu, a friendly bot here to assist you with bugs, answer your queries, and guide you on becoming a contributor. While we wait for a human maintainer, feel free to ask me anything about the Argilla repository, I'm here to help! Thank you for your feature request. Based on the current structure of the Argilla codebase, it seems feasible to create a new endpoint for supporting bulk annotation. Currently, the handling of annotations is done through the However, without information on the current API structure and the endpoint used for creating a single annotation, it's challenging to provide a detailed plan on how to implement this feature. The implementation would largely depend on these details. For instance, if the current API uses REST, the new endpoint could be something like In both cases, the server-side logic would need to be updated to process multiple annotations at once. This could involve iterating over the array of annotations and creating/updating each one in the database. Error handling would also need to be considered, such as what to do if some annotations are successfully processed but others are not. Here's a rough example of what the server-side code could look like: @app.route('/annotations/bulk', methods=['POST'])
def create_bulk_annotations():
annotations = request.json
results = []
for annotation in annotations:
try:
# This is a placeholder for the actual logic to create an annotation
result = create_annotation(annotation)
results.append(result)
except Exception as e:
# Handle the error appropriately
pass
return jsonify(results), 200 Please note that this is a very simplified example and the actual implementation would likely be more complex. If you could provide more details about the current API structure and the endpoint used for creating a single annotation, I could give a more accurate and detailed plan for this feature. Dosu Sources
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot. |
# Description This PR adds a new endpoint to create responses in bulk to support the new bulk annotation feature on Argilla. The new endpoint is `POST /api/v1/me/responses/bulk` with the following implementations details: * It can accept a maximum of `50` responses to be created or updated. * Responses are created or updated for the current user doing the request. * The endpoint is not transactional. * One response failing to be created or updated is not avoiding to process the rest of the responses in the request. * If the request is successful the number and order of the items in the request body and response will match. Example of a body request with a response to update and a second response to update (but using an invalid question): ```json { "items": [ { "values": {"prompt-quality": {"value": 10}}, "status": "submitted", "record_id": "8df12fc4-aaa4-4c5c-a883-1b50ab877c97" }, { "values": {"non-existent-question": {"value": 1}}, "status": "draft", "record_id": "f62786d4-eba7-4ecb-8225-730509150668" } ] } ``` > [!NOTE] > For updating responses we don't need the `id` of the response, from the context of the request we have the current `user_id` and the `record_id` is mandatory using this endpoint. These two attributes are the ones used to unambiguously know when to create or update a response. A response for the previous request with the first response successfully updated and an error trying to create the second response: ```json { "items": [ { "item": { "id": "e27cdcde-1393-4bfb-a42a-d557a1975b35", "values": {"prompt-quality": {"value": 10}}, "status": "submitted", "user_id": "3063e7c6-5b2a-4952-83b1-8e05277e8b51", "record_id": "8df12fc4-aaa4-4c5c-a883-1b50ab877c97", "inserted_at": "2023-12-05T12:39:19.165603", "updated_at": "2023-12-05T12:50:26.593117", }, "error": null }, { "item": null, "error": { "detail": "found responses for non configured questions: ['non-existent-question']" } } ] } ``` /cc @frascuchon @gabrielmbmb <del>there are some `TODO` comments on the code to discuss possible improvements.</del> /cc @damianpumar Closes #4367 **Type of change** (Please delete options that are not relevant. Remember to title the PR according to the type of change) - [x] New feature (non-breaking change which adds functionality) **How Has This Been Tested** - [x] Running tests locally using SQLite as database. - [x] Running tests locally using PostgreSQL as database. **Checklist** - [ ] I added relevant documentation - [ ] follows the style guidelines of this project - [x] I did a self-review of my code - [ ] I made corresponding changes to the documentation - [ ] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] I filled out [the contributor form](https://tally.so/r/n9XrxK) (see text above) - [x] I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/)
# Generic Pull Request Template This PRs include a small change increasing the maximum number of items that can be part of a bulk responses request from `50` to `100`. Records pagination is expected to be `100` as maximum so it makes sense that the endpoint for bulk annotation/responses also allow to get a maximum of `100` responses to create/update. Refs #4367 **Type of change** (Please delete options that are not relevant. Remember to title the PR according to the type of change) - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Refactor (change restructuring the codebase without changing functionality) - [ ] Improvement (change adding some improvement to an existing functionality) - [ ] Documentation update **How Has This Been Tested** - [x] Running tests locally. **Checklist** - [ ] I added relevant documentation - [x] follows the style guidelines of this project - [x] I did a self-review of my code - [ ] I made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] I filled out [the contributor form](https://tally.so/r/n9XrxK) (see text above) - [ ] <del>I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/)</del>
No description provided.
The text was updated successfully, but these errors were encountered: