Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: improve protected fields #3342

Merged
merged 9 commits into from
Jul 6, 2023
Merged

Conversation

frascuchon
Copy link
Member

@frascuchon frascuchon commented Jul 5, 2023

Description

This PR reviews and improves the protected metadata field's functionality:

  • Values for protected fields won't be truncated
  • Support for other than text content. Raw json dict are now supported.

Closes #3331
Closes #3332

Type of change

(Please delete options that are not relevant. Remember to title the PR according to the type of change)

  • Bug fix (non-breaking change which fixes an issue)
  • Improvement (change adding some improvement to an existing functionality)

How Has This Been Tested

(Please describe the tests that you ran to verify your changes. And ideally, reference tests)

tested with a local instance

Checklist

  • I added relevant documentation
  • follows the style guidelines of this project
  • I did a self-review of my code
  • I made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I filled out the contributor form (see text above)
  • I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/)

@codecov
Copy link

codecov bot commented Jul 5, 2023

Codecov Report

Patch coverage: 94.37% and project coverage change: +0.32 🎉

Comparison is base (6630d7b) 90.13% compared to head (8c913d1) 90.46%.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #3342      +/-   ##
===========================================
+ Coverage    90.13%   90.46%   +0.32%     
===========================================
  Files          233      239       +6     
  Lines        12493    12837     +344     
===========================================
+ Hits         11261    11613     +352     
+ Misses        1232     1224       -8     
Flag Coverage Δ
pytest ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ack/integrations/huggingface/card/_dataset_card.py 100.00% <ø> (ø)
.../feedback/integrations/huggingface/card/_parser.py 100.00% <ø> (ø)
src/argilla/tasks/database/migrate.py 39.13% <ø> (-4.87%) ⬇️
src/argilla/server/settings.py 77.65% <50.00%> (-3.52%) ⬇️
src/argilla/server/database.py 76.47% <60.00%> (-23.53%) ⬇️
src/argilla/client/models.py 94.41% <75.00%> (-0.42%) ⬇️
...lient/feedback/integrations/huggingface/dataset.py 85.36% <85.36%> (ø)
src/argilla/server/contexts/datasets.py 93.15% <90.68%> (-2.87%) ⬇️
src/argilla/server/apis/v0/handlers/workspaces.py 89.39% <90.90%> (+1.06%) ⬆️
src/argilla/client/feedback/config.py 92.59% <91.66%> (-7.41%) ⬇️
... and 52 more

... and 9 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@frascuchon frascuchon self-assigned this Jul 5, 2023
@frascuchon frascuchon marked this pull request as ready for review July 5, 2023 11:16
CHANGELOG.md Outdated Show resolved Hide resolved
Copy link
Member

@davidberenstein1957 davidberenstein1957 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but I have some small remarks.

@@ -22,9 +22,12 @@
DEFAULT_MAX_KEYWORD_LENGTH = 128
DEFAULT_TELEMETRY_KEY = "C6FkcaoCbt78rACAgvyBxGBcMB3dM3nn"

# The metadata field name prefix defined for protected (non-searchable) values
PROTECTED_METADATA_FIELD_PREFIX = "_"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add this to the documentation somewhere?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than here?

src/argilla/server/daos/models/records.py Show resolved Hide resolved
src/argilla/server/daos/models/records.py Outdated Show resolved Hide resolved
@frascuchon frascuchon merged commit d49ae88 into develop Jul 6, 2023
@frascuchon frascuchon deleted the feat/improve-protected-fields branch July 6, 2023 10:45
leiyre pushed a commit that referenced this pull request Jul 6, 2023
* develop:
  feat: disable concurrency tests GitHub (#3355)
  feat: improve protected fields (#3342)
  chore: added telemetry to ArgillaTrainer (#3336)
  Feat/update start page code (#3327)
  [DOCS] Added information about tests in developer documentation (#3147)

# Conflicts:
#	docs/_source/_common/snippets/start_page.md
#	frontend/components/commons/datasets-list/DatasetsEmpty.vue
leiyre pushed a commit that referenced this pull request Jul 6, 2023
* develop:
  feat: disable concurrency tests GitHub (#3355)
  feat: improve protected fields (#3342)
  chore: added telemetry to ArgillaTrainer (#3336)
  Feat/update start page code (#3327)
  [DOCS] Added information about tests in developer documentation (#3147)

# Conflicts:
#	docs/_source/_common/snippets/start_page.md
#	frontend/components/commons/datasets-list/DatasetsEmpty.vue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants