Skip to content

Commit

Permalink
Update parallelism.qmd
Browse files Browse the repository at this point in the history
  • Loading branch information
jjallaire authored Oct 5, 2024
1 parent b871e65 commit 09d4786
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions docs/parallelism.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,7 @@ graders = [model.generate(prompt) for model in models]
grader_outputs = await asyncio.gather(*graders)
```

Note that we don't await the call to `model.generate()` when building our list of graders. Rather the call to `asyncio.gather()` will await each of these requests and return when they have all completed. Inspect's internal handling of `max_connections` for model APIs will throttle these requests, so don't need to worry about how many you put in flight.
Note that we don't await the call to `model.generate()` when building our list of graders. Rather the call to `asyncio.gather()` will await each of these requests and return when they have all completed. Inspect's internal handling of `max_connections` for model APIs will throttle these requests, so there is no need to worry about how many you put in flight.

#### Web Requests

Expand Down Expand Up @@ -276,4 +276,4 @@ except TimeoutError:
...
```

If a timeout occurs, then a `TimeoutError` will be thrown (which your code should generally handle in whatever manner is appropriate).
If a timeout occurs, then a `TimeoutError` will be thrown (which your code should generally handle in whatever manner is appropriate).

0 comments on commit 09d4786

Please sign in to comment.