Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(integrations-service): Add new integrations & refactor integrations service #540

Merged
merged 32 commits into from
Oct 1, 2024

Commits on Sep 22, 2024

  1. fix: Remove unnecessary docker compose dependencies

    Julep Developers committed Sep 22, 2024
    Configuration menu
    Copy the full SHA
    4ace5fa View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2024

  1. fix: Fix litellm config for voyage-3

    Signed-off-by: Diwank Singh Tomer <[email protected]>
    creatorrr committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    d633c77 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    50f96d3 View commit details
    Browse the repository at this point in the history
  3. Implement ToolCallStep & Fix transition after PromptStep (#513)

    <!-- ELLIPSIS_HIDDEN -->
    
    
    
    > [!IMPORTANT]
    > Implement `ToolCallStep` and fix transition logic after `PromptStep`
    in workflow execution.
    > 
    >   - **ToolCallStep Implementation**:
    > - Implements `tool_call_step()` in `tool_call_step.py` to handle tool
    calls, including generating a call ID and validating tool names.
    > - Updates `STEP_TO_ACTIVITY` in `task_execution/__init__.py` to map
    `ToolCallStep` to `tool_call_step()`.
    >   - **PromptStep Transition Fix**:
    > - Updates transition logic in `task_execution/__init__.py` to handle
    tool calls after a `PromptStep`.
    > - Removes unused code related to tool calls in `PromptStep` handling.
    >   - **State Machine Update**:
    > - Updates `valid_transitions` in `tasks.py` to allow 'wait' to 'step'
    transition.
    > 
    > <sup>This description was created by </sup>[<img alt="Ellipsis"
    src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=julep-ai%2Fjulep&utm_source=github&utm_medium=referral)<sup>
    for 5ab9e3a. It will automatically
    update as commits are pushed.</sup>
    
    <!-- ELLIPSIS_HIDDEN -->
    HamadaSalhab authored Sep 23, 2024
    Configuration menu
    Copy the full SHA
    ecc95ea View commit details
    Browse the repository at this point in the history
  4. doc: Update README.md with announcement update (#517)

    Co-authored-by: Julep Developers <[email protected]>
    creatorrr and Julep Developers authored Sep 23, 2024
    Configuration menu
    Copy the full SHA
    6896c23 View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2024

  1. feat: Add basic support for integration tools to ToolStep (#519)

    Signed-off-by: Diwank Singh Tomer <[email protected]>
    
    <!-- ELLIPSIS_HIDDEN -->
    
    
    ----
    
    > [!IMPORTANT]
    > This PR updates the handling of integrations and systems by adding new
    models, updating workflows, and modifying session options, along with
    dependency updates and a migration script.
    > 
    >   - **Behavior**:
    > - Adds `execute_integration` function in `execute_integration.py` to
    handle integration tool calls.
    > - Updates `prompt_step.py` to handle unwrapping of prompt responses
    and tool call results.
    > - Modifies `tool_call_step.py` to handle tool calls using `Tool`
    model.
    >   - **Models**:
    >     - Adds `IntegrationDef` and `SystemDef` models in `Tools.py`.
    > - Updates `CreateToolRequest`, `PatchToolRequest`,
    `UpdateToolRequest`, and `Tool` to use `IntegrationDef` and `SystemDef`.
    > - Adds `forward_tool_results` option to session models in
    `Sessions.py`.
    >   - **Workflow**:
    > - Updates `TaskExecutionWorkflow` in `task_execution/__init__.py` to
    handle integration tool calls.
    >   - **Dependencies**:
    > - Updates `@typespec/*` dependencies in `package.json` to version
    `0.60.x`.
    >   - **Migration**:
    > - Adds migration script
    `migrate_1727235852_add_forward_tool_calls_option.py` to add
    `forward_tool_calls` option to sessions.
    > 
    > <sup>This description was created by </sup>[<img alt="Ellipsis"
    src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=julep-ai%2Fjulep&utm_source=github&utm_medium=referral)<sup>
    for a49aa12. It will automatically
    update as commits are pushed.</sup>
    
    
    <!-- ELLIPSIS_HIDDEN -->
    
    ---------
    
    Signed-off-by: Diwank Singh Tomer <[email protected]>
    creatorrr authored Sep 25, 2024
    Configuration menu
    Copy the full SHA
    a4151d0 View commit details
    Browse the repository at this point in the history
  2. feat(integration-service): Add integrations service (#520)

    Introduces a new integration service with FastAPI, Docker setup, and support for multiple integrations like DALL-E and Wikipedia.
    
    - Integration Service:
      + Adds integrations-service with Docker setup in docker-compose.yml and Dockerfile.
      + FastAPI application in web.py with routers for execution and integration management.
    - Models:
      + Defines models for DalleImageGenerator, DuckDuckGoSearch, HackerNews, Weather, and Wikipedia in models directory.
      + IntegrationExecutionRequest and IntegrationExecutionResponse for handling execution requests.
    - Utilities:
      + Implements execute_integration in execute_integration.py to handle integration execution logic.
      + Integration utilities for DALL-E, DuckDuckGo, Hacker News, Weather, and Wikipedia in utils/integrations.
    - Configuration:
      + Adds pyproject.toml for dependency management with Poetry.
      + Adds pytype.toml for type checking configuration.
    HamadaSalhab authored Sep 25, 2024
    Configuration menu
    Copy the full SHA
    d8072a0 View commit details
    Browse the repository at this point in the history
  3. feat(agents-api,integrations): Working integrations for tool-call step (

    #521)
    
    - **fix(typespec,agents-api): Rename integration providers**
    - **feat(agents-api,integrations): Working integrations for tool-call
    step**
    
    <!-- ELLIPSIS_HIDDEN -->
    
    ----
    
    > [!IMPORTANT]
    > Add integration service handling for tool-call steps in agents API,
    including provider renames and workflow updates.
    > 
    >   - **Integrations**:
    > - Implement `run_integration_service` in `clients/integrations.py` to
    handle integration service calls.
    > - Update `execute_integration` in `activities/execute_integration.py`
    to use `run_integration_service` for non-dummy providers.
    > - Add `INTEGRATION_SERVICE_URL` to `.env.example`, `env.py`, and
    `docker-compose.yml`.
    >   - **Renames**:
    > - Rename integration providers in `Tools.py` and `scalars.tsp` (e.g.,
    `dall-e` to `dalle_image_generator`).
    >   - **Workflows**:
    > - Update `task_execution/__init__.py` to handle integration tool calls
    using `execute_integration`.
    >   - **Tests**:
    >     - Add `integration_example.yaml` for sample tasks.
    > - Add tests for integration tool calls in
    `test_execution_workflow.py`.
    > - Add `patch_integration_service` in `utils.py` for mocking
    integration service calls.
    > 
    > <sup>This description was created by </sup>[<img alt="Ellipsis"
    src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=julep-ai%2Fjulep&utm_source=github&utm_medium=referral)<sup>
    for f13f8dd. It will automatically
    update as commits are pushed.</sup>
    
    <!-- ELLIPSIS_HIDDEN -->
    
    ---------
    
    Signed-off-by: Diwank Singh Tomer <[email protected]>
    creatorrr authored Sep 25, 2024
    Configuration menu
    Copy the full SHA
    ed33cf9 View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2024

  1. fix(agents-api): Fix wait for input step (#522)

    <!-- ELLIPSIS_HIDDEN -->
    
    
    
    > [!IMPORTANT]
    > Removes unnecessary transition call in `WaitForInputStep` case in
    `TaskExecutionWorkflow.run()` and updates tests accordingly.
    > 
    >   - **Behavior**:
    > - Removes `await transition(context, type="wait", output=output)` from
    `WaitForInputStep` case in `run()` in `__init__.py`.
    >   - **Tests**:
    > - Updates `test_execution_workflow.py` to ensure workflow behaves
    correctly without the removed transition call.
    > 
    > <sup>This description was created by </sup>[<img alt="Ellipsis"
    src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=julep-ai%2Fjulep&utm_source=github&utm_medium=referral)<sup>
    for 6600cee. It will automatically
    update as commits are pushed.</sup>
    
    <!-- ELLIPSIS_HIDDEN -->
    HamadaSalhab authored Sep 26, 2024
    Configuration menu
    Copy the full SHA
    833433a View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    087495b View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2024

  1. Configuration menu
    Copy the full SHA
    683522c View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    6c0a919 View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2024

  1. Configuration menu
    Copy the full SHA
    9db28a6 View commit details
    Browse the repository at this point in the history
  2. feat(agents-api): Add support for reading setup args from metadata an…

    …d Upgrade to python 3.12 (#525)
    
    Signed-off-by: Diwank Singh Tomer <[email protected]>
    
    <!-- ELLIPSIS_HIDDEN -->
    
    
    ----
    
    > [!IMPORTANT]
    > Upgrade to Python 3.12 and add support for reading setup arguments
    from metadata in `agents-api`.
    > 
    >   - **Python Version Upgrade**:
    > - Upgrade Python to 3.12 in `.tool-versions`, `Dockerfile`,
    `Dockerfile.worker`, `pyproject.toml`, and `pytype.toml`.
    >   - **Setup Arguments from Metadata**:
    > - `execute_integration()` in `execute_integration.py` now reads setup
    arguments from metadata.
    > - `get_tool_args_from_metadata()` in `get_tool_args_from_metadata.py`
    updated to handle `arg_type` for "args" and "setup".
    >   - **CI and Documentation**:
    >     - Delete `generate-docs.yml` workflow.
    > - Update `lint-and-format.yml` to focus on `agents-api` and use Python
    3.12.
    >     - Update `pytype` version in `pyproject.toml`.
    > - Change `x-tool-parameters` to `x-integration-args` in
    `julep-concepts.md`.
    >     - Adjust backup schedule in `docker-compose.yml` from 1h to 3h.
    > 
    > <sup>This description was created by </sup>[<img alt="Ellipsis"
    src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=julep-ai%2Fjulep&utm_source=github&utm_medium=referral)<sup>
    for b1c0615. It will automatically
    update as commits are pushed.</sup>
    
    
    <!-- ELLIPSIS_HIDDEN -->
    
    ---------
    
    Signed-off-by: Diwank Singh Tomer <[email protected]>
    creatorrr authored Sep 28, 2024
    Configuration menu
    Copy the full SHA
    e345dfe View commit details
    Browse the repository at this point in the history
  3. feat: Add docker bake builder (#528)

    Introduce Docker Bake for building and pushing images with updated GitHub Actions workflows and Dockerfiles.
    
      - **GitHub Actions Workflows**:
        - Add `.github/workflows/dev-push-to-hub.yml` and `.github/workflows/main-push-to-hub.yml` for building and pushing images on `dev` and `main` branch merges.
        - Remove `push-to-hub.yml` workflow.
      - **Docker Build System**:
        - Introduce `docker-bake.hcl` for defining Docker build targets and configurations.
        - Update Dockerfiles in `agents-api`, `agents-api.worker`, `agents-api.migration` to use Docker Bake.
      - **Documentation**:
        - Update `CONTRIBUTING.md` with instructions for building Docker images using Docker Bake.
      - **Miscellaneous**:
        - Add submodules for `sdks/node-sdk` and `sdks/python-sdk`.
    creatorrr authored Sep 28, 2024
    Configuration menu
    Copy the full SHA
    5060ee3 View commit details
    Browse the repository at this point in the history
  4. fix: Minor fix to docker bake github actions (#529)

      - **GitHub Actions**:
        - Add `docker-bake-on-pr.yml` for baking images on pull requests.
        - Rename `dev-push-to-hub.yml` to `bake-push-to-hub.yml` and `main-push-to-hub.yml` to `latest-push-to-hub.yml`.
        - Update `bake-push-to-hub.yml` and `latest-push-to-hub.yml` to use `docker/bake-action@v5` and remove branch name and git sha setting steps.
    creatorrr authored Sep 28, 2024
    Configuration menu
    Copy the full SHA
    de2be8b View commit details
    Browse the repository at this point in the history
  5. feat: Add changelog from release notes (#530)

    Update GitHub Actions workflow to generate changelog from release notes on release event.
    
      - **Workflow Changes**:
        - Renames workflow from `Changelog CI` to `Changelog on release` in `.github/workflows/changelog-ci.yml`.
        - Changes trigger from `pull_request` to `release` with type `published`.
      - **Actions**:
        - Updates `actions/checkout` from `v2` to `v4` with `fetch-depth: 0` and `ref: dev`.
        - Replaces `saadmk11/[email protected]` with `rhysd/changelog-from-release/action@v3` to generate changelog from release notes.
    creatorrr authored Sep 28, 2024
    Configuration menu
    Copy the full SHA
    cc8adca View commit details
    Browse the repository at this point in the history
  6. Update changelog for v0.4.0 (#531)

    Co-authored-by: creatorrr <[email protected]>
    github-actions[bot] and creatorrr authored Sep 28, 2024
    Configuration menu
    Copy the full SHA
    be4d40f View commit details
    Browse the repository at this point in the history
  7. fix: Bake on release as well (#532)

    Enhance workflows to bake images on release and generate changelogs from release notes.
    
      - **Workflows**:
        - Update `.github/workflows/bake-push-to-hub.yml` to trigger on `release` events in addition to `push` to `dev` and tags `v*`.
        - Update `.github/workflows/changelog-ci.yml` to trigger on `release` events and generate changelog from release notes using `rhysd/changelog-from-release/action@v3`.
    creatorrr authored Sep 28, 2024
    Configuration menu
    Copy the full SHA
    98d322b View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2024

  1. Configuration menu
    Copy the full SHA
    79a2114 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    9f46e02 View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2024

  1. Configuration menu
    Copy the full SHA
    e9b7075 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    b9c3c9e View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    23f1964 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    d67c375 View commit details
    Browse the repository at this point in the history
  5. feat(integrations-service): Add brave & browserbase integrations | Sw…

    …itch to synchronous methods
    HamadaSalhab committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    a568f4b View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2024

  1. Reformat

    HamadaSalhab committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    d977e98 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    ff15631 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    53bb994 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    aeb20a8 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    c1828e6 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    a6f7e6a View commit details
    Browse the repository at this point in the history