Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Validated status of annotated data gets set prematurely in MultiLabelText Classification #2160

Closed
dhruvsakalley opened this issue Jan 9, 2023 · 3 comments
Closed

Validated status of annotated data gets set prematurely in MultiLabelText Classification #2160

dhruvsakalley opened this issue Jan 9, 2023 · 3 comments
Assignees
@leiyre @keithCuniah
Labels
type: enhancement Indicates new feature requests
Milestone
v1.4.0

Comments

@dhruvsakalley
Copy link

Describe the bug
In case of labeling for a multilabeled classification exercise, the logic of considering a datapoint validated if any one class is annotated is only partially correct. The action of marking a datapoint completely validated must be configurable to be explicit rather than just defaulting implicit.

To Reproduce
Steps to reproduce the behavior:

  1. Create multilabel text classification exercise with significantly high number of labels
  2. Make only one class selection validated gets toggled, other class annotations do not affect the status

Expected behavior
Validate behavior should be configurable to be explicit toggle in case of multilabel text classification
@dhruvsakalley dhruvsakalley added the type: bug Indicates an unexpected problem or unintended behavior label Jan 9, 2023
@dvsrepo
Copy link
Member

dvsrepo commented Jan 9, 2023

Thanks @dhruvsakalley, this is important feedback and I fully agree with your views on this.

There's some redesign work going on, led by @Amelie-V, we'll link this feedback with that issue and get back to you.

@Amelie-V
Copy link
Member

Hi, thanks for your feedback @dhruvsakalley!
We are improving annotation status logic on individual and bulk annotation actions.

This was referenced Jan 31, 2023
Closed
Closed
@keithCuniah keithCuniah mentioned this issue Feb 1, 2023
Merged
7 tasks
leiyre added a commit that referenced this issue Feb 1, 2023
@leiyre
feat: Two steps validation for multilabel (#2273)
3701c4e 
# Description

Move records to PENDING status (instead of VALIDATED) when a labels
change in multilabel Text Classifier

Closes #2265
See #2264

Note :  this will also correct this issue #2160 

**Type of change**

(Please delete options that are not relevant. Remember to title the PR
according to the type of change)

- [x] New feature (non-breaking change which adds functionality)

**How Has This Been Tested**

(Please describe the tests that you ran to verify your changes. And
ideally, reference `tests`)

- [x] update ClassifierAnnotationArea

**Checklist**

- [x] I have merged the original branch into my forked branch
- [x] follows the style guidelines of this project
- [x] I did a self-review of my code
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
@frascuchon frascuchon added this to the v1.4.0 milestone Feb 7, 2023
@frascuchon frascuchon added type: enhancement Indicates new feature requests and removed type: bug Indicates an unexpected problem or unintended behavior labels Feb 14, 2023
@frascuchon
Copy link
Member

Fixed in #2273

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@leiyre leiyre

@keithCuniah keithCuniah

Labels
type: enhancement Indicates new feature requests
Projects
None yet
Milestone
v1.4.0
Development

Successfully merging a pull request may close this issue.

feat(UI): Improve the bulk annotation process argilla-io/argilla
6 participants
@dvsrepo @frascuchon @dhruvsakalley @leiyre @Amelie-V @keithCuniah