Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Data integration API implementation #53

Merged
merged 24 commits into from
Jun 27, 2024
Merged

Data integration API implementation #53

merged 24 commits into from
Jun 27, 2024

Commits on Jun 6, 2024

  1. feat(api): Add new function bdi.match_columns()

    @aecio
    aecio committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    d3ff7f0 View commit details
    Browse the repository at this point in the history

  2. feat(api): Add new function bdi.top_matches()

    @aecio
    aecio committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    ca8f0b8 View commit details
    Browse the repository at this point in the history

Commits on Jun 7, 2024

  1. feat(api): Added bdi.materialize_mapping() and basic value mappers

    @aecio
    aecio committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    ee9c09f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'devel' into new_api

    @aecio
    aecio committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    49ab313 View commit details
    Browse the repository at this point in the history

  3. Change column matching API to be stateless

    @aecio
    aecio committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    adae6d7 View commit details
    Browse the repository at this point in the history

Commits on Jun 10, 2024

  1. Adding TwoPhase ColumnMatch algorithm based on CTLearning

    @EduardoPena
    EduardoPena committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    bc79764 View commit details
    Browse the repository at this point in the history

  2. Formating with black, small fix for columns with no matches

    @EduardoPena
    EduardoPena committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    ed4d42c View commit details
    Browse the repository at this point in the history

  3. Formating using black variation

    @EduardoPena
    EduardoPena committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    5e1f147 View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2024

  1. Moving model_name,top_k to construtor parameters for better class reuse

    @EduardoPena
    EduardoPena committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    c85de2b View commit details
    Browse the repository at this point in the history

  2. Merge pull request #55 from VIDA-NYU/new_api_column_matching_algs 

    Adding TwoPhase ColumnMatch algorithm based on CTLearning, Fix case where MatchAlgs do not find matches
    @EduardoPena
    EduardoPena authored Jun 11, 2024
    Configuration menu
    Copy the full SHA
    176fd14 View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2024

  1. feat(api): add match_values() and preview_value_mappings()

    @aecio
    aecio committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    838f7e6 View commit details
    Browse the repository at this point in the history

  2. test(api): Add end-to-end API integration test

    @aecio
    aecio committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    1e71758 View commit details
    Browse the repository at this point in the history

  3. feat(api): Add bdi.preview_domains()

    @aecio
    aecio committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    3fc6b94 View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2024

  1. refactor(api): Make API inputs more compatible

    @aecio
    aecio committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    ebe528b View commit details
    Browse the repository at this point in the history

  2. feat(api): Support an object as a method in match_columns()

    @aecio
    aecio committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    c1d3812 View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. Cast dictionary key to string

    @roquelopez
    roquelopez committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    c84023a View commit details
    Browse the repository at this point in the history

  2. Fix #57

    @roquelopez
    roquelopez committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    718b3d6 View commit details
    Browse the repository at this point in the history

  3. Change default algorithm for column mapping

    @roquelopez
    roquelopez committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    e55f070 View commit details
    Browse the repository at this point in the history

  4. Make EditAlgorithm to return values between 0 and 1

    @roquelopez
    roquelopez committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    b4f8fb1 View commit details
    Browse the repository at this point in the history

  5. Update threshold to fix test

    @roquelopez
    roquelopez committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    39fdfd9 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. refactor: Extract TopkColumnMatcher from ContrastiveLearningAPI 

    This change will allows spliting the column embedding phase
from the selection of top-k most similar embeddings. Besides
allowing the introduction of alternative embedding models in
the future, this will be allow the implementation of alternative
TopkColumnMatcher's that instead of selecting the top-k based on
embedding similarities, will return the results based on
interactions of the user through with automatic recomendations
through a Jupyter notebook and visualizations.
    @aecio
    aecio committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    be7c1bd View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. Additional documentation and implementation improvements

    @aecio
    aecio committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    154f5e9 View commit details
    Browse the repository at this point in the history

  2. Rename notebook to doc_gdc_harmonization.ipynb

    @aecio
    aecio committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    5f65b62 View commit details
    Browse the repository at this point in the history

  3. Fix API documentation

    @roquelopez
    roquelopez committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    732c83d View commit details
    Browse the repository at this point in the history