Enable ToolFunctionDef instantiation for custom function handlers #130
Assignees
Labels
enhancement
New feature or request
Comments
|
@widoriezebos Thanks for using simple-openai. I've prepared an example for you, just be sure to use the release 3.2.0 +: Java Codepackage io.github.sashirestela.openai.demo;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonPropertyDescription;
import io.github.sashirestela.openai.SimpleOpenAI;
import io.github.sashirestela.openai.common.function.FunctionDef;
import io.github.sashirestela.openai.common.function.FunctionExecutor;
import io.github.sashirestela.openai.common.function.Functional;
import io.github.sashirestela.openai.common.tool.ToolCall;
import io.github.sashirestela.openai.domain.chat.Chat.Choice;
import io.github.sashirestela.openai.domain.chat.ChatMessage.AssistantMessage;
import io.github.sashirestela.openai.domain.chat.ChatMessage.ResponseMessage;
import io.github.sashirestela.openai.domain.chat.ChatMessage.ToolMessage;
import io.github.sashirestela.openai.domain.chat.ChatMessage.UserMessage;
import io.github.sashirestela.openai.domain.chat.Chat;
import io.github.sashirestela.openai.domain.chat.ChatMessage;
import io.github.sashirestela.openai.domain.chat.ChatRequest;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Stream;
public class ConversationDemo {
private SimpleOpenAI openAI;
private FunctionExecutor functionExecutor;
private int indexTool;
private StringBuilder content;
private StringBuilder functionArgs;
public ConversationDemo() {
openAI = SimpleOpenAI.builder().apiKey(System.getenv("OPENAI_API_KEY")).build();
}
public void prepareConversation() {
List<FunctionDef> functionList = new ArrayList<>();
functionList.add(FunctionDef.builder()
.name("getCurrentTemperature")
.description("Get the current temperature for a specific location")
.functionalClass(CurrentTemperature.class)
.build());
functionList.add(FunctionDef.builder()
.name("getRainProbability")
.description("Get the probability of rain for a specific location")
.functionalClass(RainProbability.class)
.build());
functionExecutor = new FunctionExecutor(functionList);
}
public void runConversation() {
List<ChatMessage> messages = new ArrayList<>();
var myMessage = System.console().readLine("\nWelcome! Write any message: ");
messages.add(UserMessage.of(myMessage));
while (!myMessage.toLowerCase().equals("exit")) {
var chatStream = openAI.chatCompletions()
.createStream(ChatRequest.builder()
.model("gpt-4o")
.messages(messages)
.tools(functionExecutor.getToolFunctions())
.temperature(0.2)
.stream(true)
.build())
.join();
indexTool = -1;
content = new StringBuilder();
functionArgs = new StringBuilder();
var response = getResponse(chatStream);
if (response.getMessage().getContent() != null) {
messages.add(AssistantMessage.of(response.getMessage().getContent()));
}
if (response.getFinishReason().equals("tool_calls")) {
messages.add(response.getMessage());
for (var chatToollCall : response.getMessage().getToolCalls()) {
var result = functionExecutor.execute(chatToollCall.getFunction());
messages.add(ToolMessage.of(result.toString(), chatToollCall.getId()));
}
} else {
myMessage = System.console().readLine("\n\nWrite any message (or write 'exit' to finish): ");
messages.add(UserMessage.of(myMessage));
}
}
}
private Choice getResponse(Stream<Chat> chatStream) {
var choice = new Choice();
choice.setIndex(0);
var chatMsgResponse = new ResponseMessage();
List<ToolCall> toolCalls = new ArrayList<>();
chatStream.forEach(responseChunk -> {
var choices = responseChunk.getChoices();
if (choices.size() > 0) {
var innerChoice = choices.get(0);
var delta = innerChoice.getMessage();
if (delta.getRole() != null) {
chatMsgResponse.setRole(delta.getRole());
} else if (delta.getContent() != null && !delta.getContent().isEmpty()) {
content.append(delta.getContent());
System.out.print(delta.getContent());
} else if (delta.getToolCalls() != null) {
var toolCall = delta.getToolCalls().get(0);
if (toolCall.getIndex() != indexTool) {
if (toolCalls.size() > 0) {
toolCalls.get(toolCalls.size() - 1).getFunction().setArguments(functionArgs.toString());
functionArgs = new StringBuilder();
}
toolCalls.add(toolCall);
indexTool++;
} else {
functionArgs.append(toolCall.getFunction().getArguments());
}
} else {
if (content.length() > 0) {
chatMsgResponse.setContent(content.toString());
}
if (toolCalls.size() > 0) {
toolCalls.get(toolCalls.size() - 1).getFunction().setArguments(functionArgs.toString());
chatMsgResponse.setToolCalls(toolCalls);
}
choice.setMessage(chatMsgResponse);
choice.setFinishReason(innerChoice.getFinishReason());
}
}
});
return choice;
}
public static void main(String[] args) {
var demo = new ConversationDemo();
demo.prepareConversation();
demo.runConversation();
}
public static class CurrentTemperature implements Functional {
@JsonPropertyDescription("The city and state, e.g., San Francisco, CA")
@JsonProperty(required = true)
public String location;
@JsonPropertyDescription("The temperature unit to use. Infer this from the user's location.")
@JsonProperty(required = true)
public String unit;
@Override
public Object execute() {
double centigrades = Math.random() * (40.0 - 10.0) + 10.0;
double fahrenheit = centigrades * 9.0 / 5.0 + 32.0;
String shortUnit = unit.substring(0, 1).toUpperCase();
return shortUnit.equals("C") ? centigrades : (shortUnit.equals("F") ? fahrenheit : 0.0);
}
}
public static class RainProbability implements Functional {
@JsonPropertyDescription("The city and state, e.g., San Francisco, CA")
@JsonProperty(required = true)
public String location;
@Override
public Object execute() {
return Math.random() * 100;
}
}
}Console OutputLog DetailPlease, let me know if you have any questions/thoughts/concerns/doubts. Show us your loveIf you find this project valuable there are a few ways you can show us your love, preferably all of them 🙂:
|
|
Thank you very much for your swift response! This is the Accumulator I created: and it is used like |
|
I have a followup request. My goal is to have my tool (Morpheus) use SimpleAI as a client plugin. This has some restrictions for me though; I have my own version of Function metadata (+ some state) that I need to wrap into a Tool (and therefore ToolFunctionDef). The point is that I have a different way of specifying request and response types. They are not attributes of the function itself; which means I do not have a Functional for the Request (a separate class). I would like to instantiate ToolFunctionDef myself, basically I would like do myself elsewhere like (note different argument) However, since @AllArgsConstructor(access = AccessLevel.PROTECTED) is specified for ToolFunctionDef there is no way to do that. If this can be made public (or expose via a builder); I can plugin my own Request class and then go on and write my own FunctionExecutor to call my functions. (Which is the next exercise to tackle) Is there a pressing reason to have @AllArgsConstructor(access = AccessLevel.PROTECTED)? |
|
@widoriezebos I think I could remove that restriction for |
sashirestela
changed the title
sashirestela
changed the title
|
Awesome. Thank you @sashirestela |
|
Did a local patch of ToolFunctionDef and was able to implement a custom FunctionExecuter. Everything works just fine. Streaming, functions, custom FunctionExecuter all ok now. Good stuff; looking forward to the version with this in so I can remove the patch. Thanks for your support! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When combining the Chat streaming API with functions, the incoming deltas need to be merged into a single message in order to call one or more functions. Currently the classes involved make it a bit hard to write an Accumulator that appends the deltas into a single message with one or more Tools.
Maybe I missed it; is there a way to have function calling support with the streaming API?
If there is not; it would be good to at least have setters (next to getters) for all the messages involved so the accumulator can append.
Even better would be a listener mechanism that notifies some listener of incoming deltas until the entire message was received
I’m happy to write an accumulator but would appreciate setters (that currently do not exist). Alternatively a factory method for construction of messages would also be appreciated.
Looking forward to your view on this. Thanks in advance!
The text was updated successfully, but these errors were encountered: