Skip to content

Commit

Permalink
Structured output improvements (#286)
Browse files Browse the repository at this point in the history 
* Adding an "install dependencies" section and updating the intro wording.
* Rephrasing text to make it easier to understand.
* Adding a "Next Steps" section
* Using Flash instead of Pro
  • Loading branch information
@Giom-V
Giom-V committed Sep 16, 2024
1 parent b12f60b commit 9fa7f11
Show file tree
Hide file tree
Showing 3 changed files with 163 additions and 35 deletions.
40 changes: 35 additions & 5 deletions quickstarts/Enum.ipynb
View file

Large diffs are not rendered by default.

136 changes: 111 additions & 25 deletions quickstarts/JSON_mode.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@
"\n",
"<table align=\"left\">\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://colab.research.google.com/github/google-gemini/cookbook/blob/main/quickstarts/JSON_mode.ipynb\"><img src=\"../images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
" <a target=\"_blank\" href=\"https://colab.research.google.com/github/google-gemini/cookbook/blob/main/quickstarts/JSON_mode.ipynb\"><img src=\"https://github.com/google-gemini/cookbook/blob/0f9a09fd0db1b54cc1af0f87f11f88b493e0cb3b/images/colab_logo_32px.png?raw=1\" />Run in Google Colab</a>\n",
" </td>\n",
"</table>"
]
Expand All @@ -54,23 +54,42 @@
"source": [
"The Gemini API can be used to generate a JSON output if you set the schema that you would like to use.\n",
"\n",
"**Note**: JSON schemas are only supported by Gemini 1.5 Pro right now."
"Two methods are available. You can either set the desired output in the prompt or supply a schema to the model separately."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "857d8bf104ed"
},
"source": [
"### Install dependencies"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 2,
"metadata": {
"id": "qLuL9m7KhvxR"
},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[?25l \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/164.0 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[91m╸\u001b[0m \u001b[32m163.8/164.0 kB\u001b[0m \u001b[31m8.8 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m164.0/164.0 kB\u001b[0m \u001b[31m3.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h\u001b[?25l \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/725.4 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[91m╸\u001b[0m \u001b[32m716.8/725.4 kB\u001b[0m \u001b[31m38.0 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m725.4/725.4 kB\u001b[0m \u001b[31m16.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h"
]
}
],
"source": [
"!pip install -U -q \"google-generativeai>=0.7.2\""
]
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 3,
"metadata": {
"id": "ATIbQM0NHhkj"
},
Expand All @@ -89,14 +108,14 @@
"id": "B-axqBTM8Lbd"
},
"source": [
"## Configure your API key\n",
"### Configure your API key\n",
"\n",
"To run the following cell, your API key must be stored in a Colab Secret named `GOOGLE_API_KEY`. If you don't already have an API key, or you're not sure how to create a Colab Secret, see [Authentication](https://github.com/google-gemini/cookbook/blob/main/quickstarts/Authentication.ipynb) for an example."
"To run the following cell, your API key must be stored in a Colab Secret named `GOOGLE_API_KEY`. If you don't already have an API key, or you're not sure how to create a Colab Secret, see [Authentication](../quickstarts/Authentication.ipynb) for an example."
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 4,
"metadata": {
"id": "d6lYXRcjthKV"
},
Expand All @@ -114,14 +133,14 @@
"id": "hD3qXcOTRD3z"
},
"source": [
"## Activate JSON Mode\n",
"## Set your constrained output in the prompt\n",
"\n",
" Activate JSON mode by specifying `respose_mime_type` in the `generation_config` parameter:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": 5,
"metadata": {
"id": "i5Rod-lXRIhf"
},
Expand All @@ -137,12 +156,12 @@
"id": "4071a6143d31"
},
"source": [
"For this first example just describe the schema you want back:"
"For this first example just describe the schema you want back in the prompt:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 6,
"metadata": {
"id": "K8ezjNb0RJ6Y"
},
Expand All @@ -156,7 +175,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 7,
"metadata": {
"id": "ggudoxK8RMlb"
},
Expand All @@ -176,7 +195,7 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 8,
"metadata": {
"id": "b99ee66972f5"
},
Expand Down Expand Up @@ -205,7 +224,7 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 9,
"metadata": {
"id": "WLDPREpmSMu5"
},
Expand Down Expand Up @@ -244,23 +263,25 @@
"id": "K9nIks0R-tIa"
},
"source": [
"## Generate JSON from schema\n",
"## Supply the schema to the model directly\n",
"\n",
"While `gemini-1.5-flash` models only accept a text description of the JSON you want back, `gemini-1.5-pro` models support \"controlled generation\" (aka \"constrained decoding\"). This allows you to pass a schema object (or a python type equivalent) and the output will strictly follow that schema.\n",
"The newest models (1.5 and beyond) allow you to pass a schema object (or a python type equivalent) directly and the output will strictly follow that schema.\n",
"\n",
"Following the same example as the previous section, here's that recipe type:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 10,
"metadata": {
"id": "JiIxKaLl4R0f"
},
"outputs": [],
"source": [
"class Recipe(typing.TypedDict):\n",
" recipe_name: str"
" recipe_name: str\n",
" recipe_description: str\n",
" recipe_ingredients: list[str]"
]
},
{
Expand All @@ -274,16 +295,16 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": 11,
"metadata": {
"id": "8oe-tL8MDGtx"
},
"outputs": [],
"source": [
"model = genai.GenerativeModel(model_name=\"models/gemini-1.5-pro-latest\")\n",
"model = genai.GenerativeModel(model_name=\"models/gemini-1.5-flash-latest\")\n",
"\n",
"result = model.generate_content(\n",
" \"List a few popular cookie recipes\",\n",
" \"List a few imaginative cookie recipes along with a one-sentence description as if you were a gourmet restaurant and their main ingredients\",\n",
" generation_config=genai.GenerationConfig(\n",
" response_mime_type=\"application/json\",\n",
" response_schema = list[Recipe]),\n",
Expand All @@ -293,7 +314,7 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": 12,
"metadata": {
"id": "slYcVAcqaDQY"
},
Expand All @@ -302,12 +323,77 @@
"name": "stdout",
"output_type": "stream",
"text": [
"[{\"recipe_name\": \"Chocolate Chip Cookies\"}, {\"recipe_name\": \"Peanut Butter Cookies\"}, {\"recipe_name\": \"Oatmeal Raisin Cookies\"}, {\"recipe_name\": \"Sugar Cookies\"}, {\"recipe_name\": \"Snickerdoodles\"}] \n"
"[\n",
" {\n",
" \"recipe_description\": \"A delicate and fragrant cookie infused with the bright citrus notes of bergamot and the warming spice of cardamom.\",\n",
" \"recipe_ingredients\": [\n",
" \"Bergamot zest\",\n",
" \"Cardamom pods\",\n",
" \"Almond flour\",\n",
" \"Butter\",\n",
" \"Sugar\"\n",
" ],\n",
" \"recipe_name\": \"Bergamot Cardamom Sabl\\u00e9s\"\n",
" },\n",
" {\n",
" \"recipe_description\": \"A luxurious and decadent cookie with a rich chocolate flavor and a hint of salty caramel.\",\n",
" \"recipe_ingredients\": [\n",
" \"Dark chocolate\",\n",
" \"Sea salt\",\n",
" \"Caramel sauce\",\n",
" \"Butter\",\n",
" \"Flour\"\n",
" ],\n",
" \"recipe_name\": \"Salted Caramel Chocolate Chunk Cookies\"\n",
" },\n",
" {\n",
" \"recipe_description\": \"A whimsical and colorful cookie inspired by a summer garden, featuring a blend of fresh herbs and bright citrus.\",\n",
" \"recipe_ingredients\": [\n",
" \"Lavender flowers\",\n",
" \"Lemon zest\",\n",
" \"Rosemary\",\n",
" \"Butter\",\n",
" \"Sugar\",\n",
" \"Flour\"\n",
" ],\n",
" \"recipe_name\": \"Lavender Lemon Rosemary Shortbread\"\n",
" }\n",
"]\n"
]
}
],
"source": [
"print(result.text)"
"print(json.dumps(json.loads(result.text), indent=4))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "239af844c650"
},
"source": [
"It is the recommended method if you're using a compatible model."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5ef9efdb62d3"
},
"source": [
"## Next Steps\n",
"### Useful API references:\n",
"\n",
"Check the [structured ouput](https://ai.google.dev/gemini-api/docs/structured-output) documentation or the [`GenerationConfig`](https://ai.google.dev/api/generate-content#generationconfig) API reference for more details\n",
"\n",
"### Related examples\n",
"\n",
"* The constrained output is used in the [Text summarization](../examples/json_capabilities/Text_Summarization.ipynb) example to provide the model a format to summarize a story (genre, characters, etc...)\n",
"* The [Object detection](../examples/Object_detection.ipynb) examples are using the JSON constrained output to uniiformize the output of the detection.\n",
"\n",
"### Continue your discovery of the Gemini API\n",
"\n",
"JSON is not the only way to constrain the output of the model, you can also use an [Enum](../quickstarts/Enum.ipynb). [Function calling](../quickstarts/Function_calling.ipynb) and [Code execution](../quickstarts/Code_Execution.ipynb) are other ways to enhance your model by either using your own functions or by letting the model write and run them."
]
}
],
Expand Down
22 changes: 17 additions & 5 deletions quickstarts/rest/JSON_mode_REST.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,13 @@
"id": "JMNKdTpTGZET"
},
"source": [
"This notebook provides a code example that shows you how to get started with JSON mode using `curl`.\n",
"The Gemini API can be used to generate a JSON output if you set the schema that you would like to use. This notebook provides a code example that shows you how to get started with JSON mode using `curl`.\n",
"\n",
"You can run this in Google Colab, or you can copy/paste the `curl` commands into your terminal.\n",
"\n",
"To run this notebook, your API key must be stored it in a Colab Secret named GOOGLE_API_KEY. If you are running in a different environment, you can store your key in an environment variable. See [Authentication](https://github.com/google-gemini/gemini-api-cookbook/blob/main/quickstarts/Authentication.ipynb) to learn more."
"### Authentication\n",
"\n",
"To run this notebook, your API key must be stored it in a Colab Secret named GOOGLE_API_KEY. If you are running in a different environment, you can store your key in an environment variable. See [Authentication](../../quickstarts/Authentication.ipynb) to learn more."
]
},
{
Expand Down Expand Up @@ -120,7 +122,7 @@
"source": [
"%%bash\n",
"\n",
"curl \"https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-pro-latest:generateContent?key=$GOOGLE_API_KEY\" \\\n",
"curl \"https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash-latest:generateContent?key=$GOOGLE_API_KEY\" \\\n",
"-H 'Content-Type: application/json' \\\n",
"-d '{\n",
" \"contents\": [{\n",
Expand Down Expand Up @@ -151,9 +153,19 @@
"id": "jXkRYBhbB_b2"
},
"source": [
"## Learning more\n",
"## Next Steps\n",
"### Useful API references:\n",
"\n",
"Check the [structured ouput](https://ai.google.dev/gemini-api/docs/structured-output) documentation or the [`GenerationConfig`](https://ai.google.dev/api/generate-content#generationconfig) API reference for more details \n",
"\n",
"### Related examples\n",
"\n",
"* The constrained output is used in the [Text summarization](../../examples/json_capabilities/Text_Summarization.ipynb) example to provide the model a format to summarize a story (genre, characters, etc...)\n",
"* The [Object detection](../../examples/Object_detection.ipynb) examples are using the JSON constrained output to normalize the output of the detection.\n",
"\n",
"### Continue your discovery of the Gemini API\n",
"\n",
"See the [JSON mode documentation](https://ai.google.dev/docs/gemini_api_overview#json) and the [REST API reference](https://ai.google.dev/api/rest/v1beta/GenerationConfig) for `generationConfig` to learn more.\n"
"JSON is not the only way to constrain the output of the model, you can also use an [Enum](../../quickstarts/Enum.ipynb). [Function calling](../../quickstarts/Function_calling.ipynb) and [Code execution](../../quickstarts/Code_Execution.ipynb) are other ways to enhance your model by using your own functions or by letting the model write and run them."
]
}
],
Expand Down

0 comments on commit 9fa7f11

Please sign in to comment.