Layer-dynamic-routes.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UxqB7_Ieur0s"
      },
      "source": [
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb) [![Open nbviewer](https://raw.githubusercontent.com/pinecone-io/examples/master/assets/nbviewer-shield.svg)](https://nbviewer.org/github/aurelio-labs/semantic-router/blob/main/docs/02-dynamic-routes.ipynb)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EduhQaNAur0u"
      },
      "source": [
        "# Dynamic Routes"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_4JgNeX4ur0v"
      },
      "source": [
        "In semantic-router there are two types of routes that can be chosen. Both routes belong to the `Route` object, the only difference between them is that _static_ routes return a `Route.name` when chosen, whereas _dynamic_ routes use an LLM call to produce parameter input values.\n",
        "\n",
        "For example, a _static_ route will tell us if a query is talking about mathematics by returning the route name (which could be `\"math\"` for example). A _dynamic_ route can generate additional values, so it may decide a query is talking about maths, but it can also generate Python code that we can later execute to answer the user's query, this output may look like `\"math\", \"import math; output = math.sqrt(64)`.\n",
        "\n",
        "***⚠️ Note: We have a fully local version of dynamic routes available at [docs/05-local-execution.ipynb](https://github.com/aurelio-labs/semantic-router/blob/main/docs/05-local-execution.ipynb). The local 05 version tends to outperform the OpenAI version we demo in this notebook, so we'd recommend trying [05](https://github.com/aurelio-labs/semantic-router/blob/main/docs/05-local-execution.ipynb)!***"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bbmw8CO4ur0v"
      },
      "source": [
        "## Installing the Library"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "dLElfRhgur0v"
      },
      "outputs": [],
      "source": [
        "!pip install -qU semantic-router==0.0.32\n",
        "!pip install unifyai"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BixZd6Eour0w"
      },
      "source": [
        "## Initializing Routes and RouteLayer"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PxnW9qBvur0x"
      },
      "source": [
        "Dynamic routes are treated in the same way as static routes, let's begin by initializing a `RouteLayer` consisting of static routes."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "kc9Ty6Lgur0x"
      },
      "outputs": [],
      "source": [
        "from semantic_router import Route\n",
        "\n",
        "# Define routes for Math and Coding\n",
        "math_route = Route(\n",
        "    name=\"math\",\n",
        "    utterances=[\n",
        "        \"solve for x in the equation\",\n",
        "        \"what is the integral of\",\n",
        "        \"how to calculate the derivative\",\n",
        "        \"mathematical proofs\",\n",
        "        \"how do you find the percentage of this number\"\n",
        "    ],\n",
        ")\n",
        "\n",
        "coding_route = Route(\n",
        "    name=\"coding\",\n",
        "    utterances=[\n",
        "        \"how to write a for loop in Python\",\n",
        "        \"explain the use of classes in Java\",\n",
        "        \"what is recursion in programming\",\n",
        "        \"how do i optimise this problem using hash tables\",\n",
        "        \"suggest a more efficient data structure for this problem\"\n",
        "    ],\n",
        ")\n",
        "\n",
        "# List of all routes\n",
        "routes = [math_route, coding_route]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "voWyqmffur0x"
      },
      "source": [
        "We initialize our `RouteLayer` with our `encoder` and `routes`. We can use popular encoder APIs like `CohereEncoder` and `OpenAIEncoder`, or local alternatives like `FastEmbedEncoder`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BI9AiDspur0y",
        "outputId": "27329a54-3f16-44a5-ac20-13a6b26afb97"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m2024-01-08 11:12:24 INFO semantic_router.utils.logger Initializing RouteLayer\u001b[0m\n"
          ]
        }
      ],
      "source": [
        "import os\n",
        "from getpass import getpass\n",
        "from semantic_router import RouteLayer\n",
        "from semantic_router.encoders import CohereEncoder, OpenAIEncoder\n",
        "\n",
        "# dashboard.cohere.ai\n",
        "# os.environ[\"COHERE_API_KEY\"] = os.getenv(\"COHERE_API_KEY\") or getpass(\n",
        "#     \"Enter Cohere API Key: \"\n",
        "# )\n",
        "# platform.openai.com\n",
        "os.environ[\"OPENAI_API_KEY\"] = os.getenv(\"OPENAI_API_KEY\") or getpass(\n",
        "    \"Enter OpenAI API Key: \"\n",
        ")\n",
        "\n",
        "# encoder = CohereEncoder()\n",
        "encoder = OpenAIEncoder()\n",
        "\n",
        "rl = RouteLayer(encoder=encoder, routes=routes)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GuLCeIS5ur0y"
      },
      "source": [
        "We run the solely static routes layer:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_rNREh7gur0y",
        "outputId": "f3a1dc0b-d760-4efb-b634-d3547011dcb7"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "RouteChoice(name='chitchat', function_call=None)"
            ]
          },
          "execution_count": 4,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "rl(\"Solve the equation 5-x=12 for x?\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from unify import Unify  \n",
        "# Environment setup for API keys\n",
        "os.environ[\"UNIFY_KEY\"] = os.getenv(\"UNIFY_KEY\") or getpass(\"Enter Unify API Key: \")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "#Unify process query\n",
        "def process_query(query):\n",
        "    route_choice = rl(query)\n",
        "    print(f\"Route chosen: {route_choice.name}\")\n",
        "\n",
        "    if route_choice.name == \"math\":\n",
        "        # Initialize Unify with the endpoint for math queries\n",
        "        unify = Unify(\n",
        "            api_key=os.environ[\"UNIFY_KEY\"],\n",
        "            endpoint=\"gpt-4@anyscale\"  # Use the correct endpoint for math queries\n",
        "        )\n",
        "        # Generate the response using Unify\n",
        "        response = unify.generate(user_prompt=query)\n",
        "        return response\n",
        "\n",
        "    elif route_choice.name == \"coding\":\n",
        "        # Initialize Unify with the endpoint for coding queries\n",
        "        unify = Unify(\n",
        "            api_key=os.environ[\"UNIFY_KEY\"],\n",
        "            endpoint=\"deepseek-coder-33b-instruct@anyscale\"  # Use the correct endpoint for coding queries\n",
        "        )\n",
        "        # Generate the response using Unify\n",
        "        response = unify.generate(user_prompt=query)\n",
        "        return response\n",
        "\n",
        "    else:\n",
        "        return \"This query does not fall under a supported category.\"\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Process query test\n",
        "process_query(\"Solve the equation 5-x=12 for x?\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "McbLKO26ur0y"
      },
      "source": [
        "## Creating a Dynamic Route"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ANAoEjxYur0y"
      },
      "source": [
        "As with static routes, we must create a dynamic route before adding it to our route layer. To make a route dynamic, we need to provide a `function_schema`. The function schema provides instructions on what a function is, so that an LLM can decide how to use it correctly."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "5jaF1Xa5ur0y"
      },
      "outputs": [],
      "source": [
        "from datetime import datetime\n",
        "from zoneinfo import ZoneInfo\n",
        "\n",
        "\n",
        "def get_time(timezone: str) -> str:\n",
        "    \"\"\"Finds the current time in a specific timezone.\n",
        "\n",
        "    :param timezone: The timezone to find the current time in, should\n",
        "        be a valid timezone from the IANA Time Zone Database like\n",
        "        \"America/New_York\" or \"Europe/London\". Do NOT put the place\n",
        "        name itself like \"rome\", or \"new york\", you must provide\n",
        "        the IANA format.\n",
        "    :type timezone: str\n",
        "    :return: The current time in the specified timezone.\"\"\"\n",
        "    now = datetime.now(ZoneInfo(timezone))\n",
        "    return now.strftime(\"%H:%M\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "YyFKV8jMur0z",
        "outputId": "29cf80f4-552c-47bb-fbf9-019f5dfdf00a"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'06:13'"
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "get_time(\"America/New_York\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4qyaRuNXur0z"
      },
      "source": [
        "To get the function schema we can use the `get_schema` function from the `function_call` module."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tOjuhp5Xur0z",
        "outputId": "ca88a3ea-d70a-4950-be9a-63fab699de3b"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'name': 'get_time',\n",
              " 'description': 'Finds the current time in a specific timezone.\\n\\n:param timezone: The timezone to find the current time in, should\\n    be a valid timezone from the IANA Time Zone Database like\\n    \"America/New_York\" or \"Europe/London\". Do NOT put the place\\n    name itself like \"rome\", or \"new york\", you must provide\\n    the IANA format.\\n:type timezone: str\\n:return: The current time in the specified timezone.',\n",
              " 'signature': '(timezone: str) -> str',\n",
              " 'output': \"<class 'str'>\"}"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from semantic_router.utils.function_call import get_schema\n",
        "\n",
        "schema = get_schema(get_time)\n",
        "schema"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HcF7jGjAur0z"
      },
      "source": [
        "We use this to define our dynamic route:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "iesBG9P3ur0z"
      },
      "outputs": [],
      "source": [
        "time_route = Route(\n",
        "    name=\"get_time\",\n",
        "    utterances=[\n",
        "        \"what is the time in new york city?\",\n",
        "        \"what is the time in london?\",\n",
        "        \"I live in Rome, what time is it?\",\n",
        "    ],\n",
        "    function_schema=schema,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZiUs3ovpur0z"
      },
      "source": [
        "Add the new route to our `layer`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-0vY8PRXur0z",
        "outputId": "db01e14c-eab3-4f93-f4c2-e30f508c8b5d"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m2024-01-08 11:15:26 INFO semantic_router.utils.logger Adding `get_time` route\u001b[0m\n"
          ]
        }
      ],
      "source": [
        "rl.add(time_route)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7yoE0IrNur0z"
      },
      "source": [
        "Now we can ask our layer a time related question to trigger our new dynamic route."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "id": "Wfb68M0-ur0z",
        "outputId": "79923883-2a4d-4744-f8ce-e818cb5f14c3"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\u001b[32m2024-01-08 11:16:24 INFO semantic_router.utils.logger Extracting function input...\u001b[0m\n"
          ]
        },
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'06:16'"
            ]
          },
          "execution_count": 11,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "out = rl(\"what is the time in new york city?\")\n",
        "get_time(**out.function_call)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Qt0vkq2Xur00"
      },
      "source": [
        "Our dynamic route provides both the route itself _and_ the input parameters required to use the route."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J0oD1dxIur00"
      },
      "source": [
        "---"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "decision-layer",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.5"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}