Skip to content

Commit

Permalink
New prompting examples
Browse files Browse the repository at this point in the history
  • Loading branch information
ddebowczyk committed Sep 13, 2024
1 parent 67e85c5 commit 1944149
Show file tree
Hide file tree
Showing 2 changed files with 70 additions and 0 deletions.
69 changes: 69 additions & 0 deletions docs/cookbook/examples/prompting/follow_up_questions.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
---
title: 'Generate Follow-Up Questions'
docname: 'follow_up_questions'
---

## Overview

Models can sometimes correctly answer sub-problems but incorrectly answer the overall query. This is known as the compositionality gap1.

How can we encourage a model to use the answers to sub-problems to correctly generate the overall solution?

Self-Ask is a technique which use a single prompt to:
- decide if follow-up questions are required
- generate the follow-up questions
- answer the follow-up questions
- answer the main query

## Example

```php
<?php

$loader = require 'vendor/autoload.php';
$loader->add('Cognesy\\Instructor\\', __DIR__ . '../../src/');

use Cognesy\Instructor\Instructor;
use Cognesy\Instructor\Schema\Attributes\Description;

class FollowUp {
#[Description("The follow-up question")]
public string $question;
#[Description("The answer to the follow-up question")]
public string $answer;
}

class Response {
public bool $followUpsRequired;
/** @var FollowUp[] */
public array $followUps;
public string $finalAnswer;
}

class RespondWithFollowUp {
private $prompt = <<<QUERY
Query: {query}
Are follow-up questions needed?
If so, generate follow-up questions, their answers, and then the final answer to the query.
QUERY;

public function __invoke(string $query) : Response {
return (new Instructor)->respond(
messages: str_replace('{query}', $query, $this->prompt),
responseModel: Response::class,
);
}
}

$response = (new RespondWithFollowUp)(
query: "Who was president of the U.S. when superconductivity was discovered?",
);

echo "Answer:\n";
dump($response);
?>
```

## References

1. [Measuring and Narrowing the Compositionality Gap in Language Models](https://arxiv.org/abs/2210.03350)
1 change: 1 addition & 0 deletions docs/mint.json
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,7 @@
"cookbook/examples/prompting/clarify_ambiguity",
"cookbook/examples/prompting/define_style",
"cookbook/examples/prompting/emotional_stimuli",
"cookbook/examples/prompting/follow_up_questions",
"cookbook/examples/prompting/repeat_query",
"cookbook/examples/prompting/simulate_perspective"
]
Expand Down

0 comments on commit 1944149

Please sign in to comment.