autogen with external web app

[mlops-v2-gs]

I'm trying to get autogen to work with an external web app. I want user to provide input fairly frequently and my human_input_mode is set to ALWAYS.
I'm sure there is a way and I'm missing a trick but I'm really struggling to find out a solution to handle user interactions during the chat.

For example, if the user says I want you to comeup with a plan, then autogen replies with a plan and user can go back and forth reviewing this by providing feedback. Obvisouly totally works fine on the console based conversation :)
But how do I deploy handle this within a web app? Is there a guideline somewhere on how to do this?

[sonichi]

I've seen a number of examples. @victordibia shall we add to FAQ?

[mlops-v2-gs]

thanks @sonichi That's really glad to hear. Any with human_input_mode with ALWAYS? I've seen a one with NEVER and that's not really helping in my context. Could you be kind enough to share a link please?

[ip-aka-ilamparithi]

@mlops-v2-gs - did you find any solution for this? if so can you share the github repo?

[hughlv]

@mlops-v2-gs - did you find any solution for this? if so can you share the github repo?

I can share one solution which is available in my side-project for autogen: https://github.com/tiwater/flowgen

Currently the get_human_input is implmeneted in ConversableAgent, which is the base class of AssistantAgent, you don't have an elegant way to override the behavior so I 'hijacked' this method in my code as follows:

# Replace the default get_human_input function for status control
def custom_get_human_input(self, prompt: str) -> str:
    # Set wait_for_human_input to True
    print('__STATUS_WAIT_FOR_HUMAN_INPUT__', prompt, flush=True)
    reply = input(prompt)
    # Restore the status to running
    print('__STATUS_RECEIVED_HUMAN_INPUT__', prompt, flush=True)
    return reply

autogen.ConversableAgent.get_human_input = custom_get_human_input

then you can monitor the STDOUT for __STATUS_WAIT_FOR_HUMAN_INPUT__ and feed your inputs to STDIN. I used this mechanism on my ui layer for interaction, and I suppose this can fit your requirement.

The related code:

Subprocess monitoring:
https://github.com/tiwater/flowgen/blob/18f71869d8fd1bb442648142552a2c12e98a7eee/api/app/core/runner.py#L68C1-L84C25

UI:
https://github.com/tiwater/flowgen/blob/18f71869d8fd1bb442648142552a2c12e98a7eee/ui/src/app/%5Blocale%5D/components/Chat.tsx#L63C1-L105C16

I used PocketBase as the data platform. The autogen caller on server side will add message to pb and my nextjs project will wait for specific message starts with __STATUS_WAIT_FOR_HUMAN_INPUT__ and then set the status to waiting-for-input.

This solution is a bit dirty however works fine. Because the autogen code is generated so here I attached one for your reference:

# This file is auto-generated by [FlowGen](https://github.com/tiwater/flowgen)
# Last generated: 2024-01-04 17:29:29
#
# Autoflow Name: Simple Chat
# Description: 
"""
ChatGPT-alike Simple Chat involves human.
"""
#

"""
Notes:

This flow has set the Human Input Mode to ALWAYS. That means whenever receive a message from Assistant, you as user need to respond by typing message in the chatbox. 'exit' will quit the conversation.

1. Send a simple message such as `What day is today?`.
2. If need to quit, send 'exit'
3. Sometimes Assistant will send back some code ask you to execute, you can simply press Enter and the code will be executed.
"""


from dotenv import load_dotenv
load_dotenv()  # This will load all environment variables from .env

import argparse
import os
import time
from termcolor import colored

# Parse command line arguments
parser = argparse.ArgumentParser(description='Start a chat with agents.')
parser.add_argument('message', type=str, help='The message to send to agent.')
args = parser.parse_args()

import autogen

# openai, whisper and moviepy are optional dependencies, currently only used in video transcript example
# However, we beleive they are useful for other future examples, so we include them here as part of standard imports
from openai import OpenAI
import whisper
from moviepy.editor import VideoFileClip

from IPython import get_ipython

try:
  from autogen import AssistantAgent
  from autogen import UserProxyAgent
except ImportError as e:
  print(colored(f'Error: {e}', 'red'))
  raise

# Replace the default get_human_input function for status control
def custom_get_human_input(self, prompt: str) -> str:
    # Set wait_for_human_input to True
    print('__STATUS_WAIT_FOR_HUMAN_INPUT__', prompt, flush=True)
    reply = input(prompt)
    # Restore the status to running
    print('__STATUS_RECEIVED_HUMAN_INPUT__', prompt, flush=True)
    return reply

autogen.ConversableAgent.get_human_input = custom_get_human_input


config_list = autogen.config_list_from_json(
    "OAI_CONFIG_LIST",
)

llm_config = {
    "config_list": config_list,
    "temperature": 0.5,
    "max_tokens": 1024,
}

node_vbhhpjj8xo = AssistantAgent(
    name="Assistant",
    llm_config=llm_config,
)


user_proxy = UserProxyAgent(
    name="UserProxy",
    system_message="Hello AI",
    human_input_mode="ALWAYS",
    is_termination_msg=lambda x: x.get("content", "") and x.get("content", "").rstrip().endswith("TERMINATE"),
    max_consecutive_auto_reply=1,
    code_execution_config={
      "work_dir": "work_dir",
    },
)
# Function template content generator
# register the functions

# Start the conversation
user_proxy.initiate_chat(
    node_vbhhpjj8xo,
    message=args.message,
)