Building agents with LLM (large language + model) as its core controller is a cool concept. Several proof-of-concepts + demos, such as AutoGPT, + GPT-Engineer and + BabyAGI, serve as + inspiring examples. The potentiality of LLM extends beyond generating well-written + copies, stories, essays and programs; it can be framed as a powerful general + problem solver.
\nAgent System Overview#
\nIn + a LLM-powered autonomous agent system, LLM functions as the agent’s + brain, complemented by several key components:
\n- \n
- Planning\n
- \n
- Subgoal + and decomposition: The agent breaks down large tasks into smaller, manageable + subgoals, enabling efficient handling of complex tasks. \n
- Reflection + and refinement: The agent can do self-criticism and self-reflection over past + actions, learn from mistakes and refine them for future steps, thereby improving + the quality of final results. \n
\n - Memory\n
- \n
- Short-term + memory: I would consider all the in-context learning (See Prompt + Engineering) as utilizing short-term memory of the model to learn. \n
- Long-term + memory: This provides the agent with the capability to retain and recall (infinite) + information over extended periods, often by leveraging an external vector + store and fast retrieval. \n
\n - Tool use\n
- \n
- The + agent learns to call external APIs for extra information that is missing from + the model weights (often hard to change after pre-training), including current + information, code execution capability, access to proprietary information + sources and more. \n
\n
![](\"agent-overview.png\")
\nComponent + One: Planning#
\nA + complicated task usually involves many steps. An agent needs to know what + they are and plan ahead.
\nTask Decomposition#
\nChain + of thought (CoT; Wei + et al. 2022) has become a standard prompting technique for enhancing model + performance on complex tasks. The model is instructed to “think step + by step” to utilize more test-time computation to decompose hard tasks + into smaller and simpler steps. CoT transforms big tasks into multiple manageable + tasks and shed lights into an interpretation of the model’s thinking + process.
\nTree of Thoughts (Yao + et al. 2023) extends CoT by exploring multiple reasoning possibilities + at each step. It first decomposes the problem into multiple thought steps + and generates multiple thoughts per step, creating a tree structure. The search + process can be BFS (breadth-first search) or DFS (depth-first search) with + each state evaluated by a classifier (via a prompt) or majority vote.
\nTask
+ decomposition can be done (1) by LLM with simple prompting like "Steps
+ for XYZ.\\n1.", "What are the subgoals for achieving
+ XYZ?", (2) by using task-specific instructions; e.g. "Write
+ a story outline." for writing a novel, or (3) with human inputs.
Another + quite distinct approach, LLM+P (Liu + et al. 2023), involves relying on an external classical planner to do + long-horizon planning. This approach utilizes the Planning Domain Definition + Language (PDDL) as an intermediate interface to describe the planning problem. + In this process, LLM (1) translates the problem into “Problem PDDL”, + then (2) requests a classical planner to generate a PDDL plan based on an + existing “Domain PDDL”, and finally (3) translates the PDDL plan + back into natural language. Essentially, the planning step is outsourced to + an external tool, assuming the availability of domain-specific PDDL and a + suitable planner which is common in certain robotic setups but not in many + other domains.
\nSelf-Reflection#
\nSelf-reflection + is a vital aspect that allows autonomous agents to improve iteratively by + refining past action decisions and correcting previous mistakes. It plays + a crucial role in real-world tasks where trial and error are inevitable.
\nReAct (Yao + et al. 2023) integrates reasoning and acting within LLM by extending the + action space to be a combination of task-specific discrete actions and the + language space. The former enables LLM to interact with the environment (e.g. + use Wikipedia search API), while the latter prompting LLM to generate reasoning + traces in natural language.
\nThe ReAct prompt template incorporates + explicit steps for LLM to think, roughly formatted as:
\nThought:
+ ...\nAction: ...\nObservation: ...\n... (Repeated many times)\n![](\"react.png\")
\nIn both experiments on knowledge-intensive
+ tasks and decision-making tasks, ReAct works better than the
+ Act-only baseline where Thought: \u2026 step is
+ removed.
Reflexion (Shinn + & Labash 2023) is a framework to equips agents with dynamic memory + and self-reflection capabilities to improve reasoning skills. Reflexion has + a standard RL setup, in which the reward model provides a simple binary reward + and the action space follows the setup in ReAct where the task-specific action + space is augmented with language to enable complex reasoning steps. After + each action $a_t$, the agent computes a heuristic $h_t$ and optionally may + decide to reset the environment to start a new trial depending on + the self-reflection results.
\n![](\"reflexion.png\")
\nThe heuristic function determines when + the trajectory is inefficient or contains hallucination and should be stopped. + Inefficient planning refers to trajectories that take too long without success. + Hallucination is defined as encountering a sequence of consecutive identical + actions that lead to the same observation in the environment.
\nSelf-reflection + is created by showing two-shot examples to LLM and each example is a pair + of (failed trajectory, ideal reflection for guiding future changes in the + plan). Then reflections are added into the agent’s working memory, up + to three, to be used as context for querying LLM.
\n![](\"reflexion-exp.png\")
\nChain + of Hindsight (CoH; Liu + et al. 2023) encourages the model to improve on its own outputs by explicitly + presenting it with a sequence of past outputs, each annotated with feedback. + Human feedback data is a collection of $D_h = \\{(x, y_i , r_i , z_i)\\}_{i=1}^n$, + where $x$ is the prompt, each $y_i$ is a model completion, $r_i$ is the human + rating of $y_i$, and $z_i$ is the corresponding human-provided hindsight feedback. + Assume the feedback tuples are ranked by reward, $r_n \\geq r_{n-1} \\geq + \\dots \\geq r_1$ The process is supervised fine-tuning where the data is + a sequence in the form of $\\tau_h = (x, z_i, y_i, z_j, y_j, \\dots, z_n, + y_n)$, where $\\leq i \\leq j \\leq n$. The model is finetuned to only predict + $y_n$ where conditioned on the sequence prefix, such that the model can self-reflect + to produce better output based on the feedback sequence. The model can optionally + receive multiple rounds of instructions with human annotators at test time.
\nTo + avoid overfitting, CoH adds a regularization term to maximize the log-likelihood + of the pre-training dataset. To avoid shortcutting and copying (because there + are many common words in feedback sequences), they randomly mask 0% - 5% of + past tokens during training.
\nThe training dataset in their experiments + is a combination of WebGPT + comparisons, summarization + from human feedback and human + preference dataset.
\n![](\"CoH.png\")
\nThe idea of CoH is to present a history of sequentially + improved outputs in context and train the model to take on the trend to produce + better outputs. Algorithm Distillation (AD; Laskin + et al. 2023) applies the same idea to cross-episode trajectories in reinforcement + learning tasks, where an algorithm is encapsulated in a long history-conditioned + policy. Considering that an agent interacts with the environment many times + and in each episode the agent gets a little better, AD concatenates this learning + history and feeds that into the model. Hence we should expect the next predicted + action to lead to better performance than previous trials. The goal is to + learn the process of RL instead of training a task-specific policy itself.
\n![](\"algorithm-distillation.png\")
\n(Image source: Laskin + et al. 2023).
The paper hypothesizes that any algorithm + that generates a set of learning histories can be distilled into a neural + network by performing behavioral cloning over actions. The history data is + generated by a set of source policies, each trained for a specific task. At + the training stage, during each RL run, a random task is sampled and a subsequence + of multi-episode history is used for training, such that the learned policy + is task-agnostic.
\nIn reality, the model has limited context window + length, so episodes should be short enough to construct multi-episode history. + Multi-episodic contexts of 2-4 episodes are necessary to learn a near-optimal + in-context RL algorithm. The emergence of in-context RL requires long enough + context.
\nIn comparison with three baselines, including ED (expert + distillation, behavior cloning with expert trajectories instead of learning + history), source policy (used for generating trajectories for distillation + by UCB), + RL^2 (Duan et al. 2017; used + as upper bound since it needs online RL), AD demonstrates in-context RL with + performance getting close to RL^2 despite only using offline RL and learns + much faster than other baselines. When conditioned on partial training history + of the source policy, AD also improves much faster than ED baseline.
\n![](\"algorithm-distillation-results.png\")
\n(Image source: Laskin et al. 2023)
Component + Two: Memory#
\n(Big + thank you to ChatGPT for helping me draft this section. I’ve learned + a lot about the human brain and data structure for fast MIPS in my conversations + with ChatGPT.)
\nTypes of Memory#
\nMemory can be + defined as the processes used to acquire, store, retain, and later retrieve + information. There are several types of memory in human brains.
\n- \n
- \n
Sensory + Memory: This is the earliest stage of memory, providing the ability + to retain impressions of sensory information (visual, auditory, etc) after + the original stimuli have ended. Sensory memory typically only lasts for up + to a few seconds. Subcategories include iconic memory (visual), echoic memory + (auditory), and haptic memory (touch).
\n \n - \n
Short-Term + Memory (STM) or Working Memory: It stores information + that we are currently aware of and needed to carry out complex cognitive tasks + such as learning and reasoning. Short-term memory is believed to have the + capacity of about 7 items (Miller + 1956) and lasts for 20-30 seconds.
\n \n - \n
Long-Term + Memory (LTM): Long-term memory can store information for a remarkably + long time, ranging from a few days to decades, with an essentially unlimited + storage capacity. There are two subtypes of LTM:
\n- \n
- Explicit / + declarative memory: This is memory of facts and events, and refers to those + memories that can be consciously recalled, including episodic memory (events + and experiences) and semantic memory (facts and concepts). \n
- Implicit + / procedural memory: This type of memory is unconscious and involves skills + and routines that are performed automatically, like riding a bike or typing + on a keyboard. \n
\n
![](\"memory.png\")
\nWe + can roughly consider the following mappings:
\n- \n
- Sensory memory + as learning embedding representations for raw inputs, including text, image + or other modalities; \n
- Short-term memory as in-context learning. It + is short and finite, as it is restricted by the finite context window length + of Transformer. \n
- Long-term memory as the external vector store that + the agent can attend to at query time, accessible via fast retrieval. \n
Maximum Inner Product Search (MIPS)#
\nThe + external memory can alleviate the restriction of finite attention span. A + standard practice is to save the embedding representation of information into + a vector store database that can support fast maximum inner-product search + (MIPS). + To optimize the retrieval speed, the common choice is the approximate + nearest neighbors (ANN)\u200B algorithm to return approximately top k + nearest neighbors to trade off a little accuracy lost for a huge speedup.
\nA + couple common choices of ANN algorithms for fast MIPS:
\n- \n
- LSH + (Locality-Sensitive Hashing): It introduces a hashing function such + that similar input items are mapped to the same buckets with high probability, + where the number of buckets is much smaller than the number of inputs. \n
- ANNOY (Approximate + Nearest Neighbors Oh Yeah): The core data structure are random projection + trees, a set of binary trees where each non-leaf node represents a hyperplane + splitting the input space into half and each leaf stores one data point. Trees + are built independently and at random, so to some extent, it mimics a hashing + function. ANNOY search happens in all the trees to iteratively search through + the half that is closest to the query and then aggregates the results. The + idea is quite related to KD tree but a lot more scalable. \n
- HNSW + (Hierarchical Navigable Small World): It is inspired by the idea of small + world networks where most nodes can be reached by any other nodes within + a small number of steps; e.g. “six degrees of separation” feature + of social networks. HNSW builds hierarchical layers of these small-world graphs, + where the bottom layers contain the actual data points. The layers in the + middle create shortcuts to speed up search. When performing a search, HNSW + starts from a random node in the top layer and navigates towards the target. + When it can’t get any closer, it moves down to the next layer, until + it reaches the bottom layer. Each move in the upper layers can potentially + cover a large distance in the data space, and each move in the lower layers + refines the search quality. \n
- FAISS + (Facebook AI Similarity Search): It operates on the assumption that in high + dimensional space, distances between nodes follow a Gaussian distribution + and thus there should exist clustering of data points. FAISS applies + vector quantization by partitioning the vector space into clusters and then + refining the quantization within clusters. Search first looks for cluster + candidates with coarse quantization and then further looks into each cluster + with finer quantization. \n
- ScaNN + (Scalable Nearest Neighbors): The main innovation in ScaNN is anisotropic + vector quantization. It quantizes a data point $x_i$ to $\\tilde{x}_i$ + such that the inner product $\\langle q, x_i \\rangle$ is as similar to the + original distance of $\\angle q, \\tilde{x}_i$ as possible, instead of picking + the closet quantization centroid points. \n
![](\"mips.png\")
\nCheck more MIPS + algorithms and performance comparison in ann-benchmarks.com.
\nComponent Three: Tool Use#
\nTool + use is a remarkable and distinguishing characteristic of human beings. We + create, modify and utilize external objects to do things that go beyond our + physical and cognitive limits. Equipping LLMs with external tools can significantly + extend the model capabilities.
\n![](\"sea-otter.png\")
\nMRKL + (Karpas et al. 2022), short + for “Modular Reasoning, Knowledge and Language”, is a neuro-symbolic + architecture for autonomous agents. A MRKL system is proposed to contain a + collection of “expert” modules and the general-purpose LLM works + as a router to route inquiries to the best suitable expert module. These modules + can be neural (e.g. deep learning models) or symbolic (e.g. math calculator, + currency converter, weather API).
\nThey did an experiment on fine-tuning + LLM to call a calculator, using arithmetic as a test case. Their experiments + showed that it was harder to solve verbal math problems than explicitly stated + math problems because LLMs (7B Jurassic1-large model) failed to extract the + right arguments for the basic arithmetic reliably. The results highlight when + the external symbolic tools can work reliably, knowing when to and how + to use the tools are crucial, determined by the LLM capability.
\nBoth + TALM (Tool Augmented Language Models; Parisi + et al. 2022) and Toolformer (Schick + et al. 2023) fine-tune a LM to learn to use external tool APIs. The dataset + is expanded based on whether a newly added API call annotation can improve + the quality of model outputs. See more details in the “External + APIs” section of Prompt Engineering.
\nChatGPT Plugins + and OpenAI API function + calling are good examples of LLMs augmented with tool use capability working + in practice. The collection of tool APIs can be provided by other developers + (as in Plugins) or self-defined (as in function calls).
\nHuggingGPT + (Shen et al. 2023) is a framework + to use ChatGPT as the task planner to select models available in HuggingFace + platform according to the model descriptions and summarize the response based + on the execution results.
\n![](\"hugging-gpt.png\")
\nThe system comprises of 4 stages:
\n(1) + Task planning: LLM works as the brain and parses the user requests + into multiple tasks. There are four attributes associated with each task: + task type, ID, dependencies, and arguments. They use few-shot examples to + guide LLM to do task parsing and planning.
\nInstruction:
\n(2) + Model selection: LLM distributes the tasks to expert models, where + the request is framed as a multiple-choice question. LLM is presented with + a list of models to choose from. Due to the limited context length, task type + based filtration is needed.
\nInstruction:
\n(3) + Task execution: Expert models execute on the specific tasks and log + results.
\nInstruction:
\n(4) + Response generation: LLM receives the execution results and provides + summarized results to users.
\nTo put HuggingGPT into real world usage, + a couple challenges need to solve: (1) Efficiency improvement is needed as + both LLM inference rounds and interactions with other models slow down the + process; (2) It relies on a long context window to communicate over complicated + task content; (3) Stability improvement of LLM outputs and external model + services.
\nAPI-Bank (Li + et al. 2023) is a benchmark for evaluating the performance of tool-augmented + LLMs. It contains 53 commonly used API tools, a complete tool-augmented LLM + workflow, and 264 annotated dialogues that involve 568 API calls. The selection + of APIs is quite diverse, including search engines, calculator, calendar queries, + smart home control, schedule management, health data management, account authentication + workflow and more. Because there are a large number of APIs, LLM first has + access to API search engine to find the right API to call and then uses the + corresponding documentation to make a call.
\n![](\"api-bank-process.png\")
\nIn the API-Bank workflow, + LLMs need to make a couple of decisions and at each step we can evaluate how + accurate that decision is. Decisions include:
\n- \n
- Whether an API + call is needed. \n
- Identify the right API to call: if not good enough, + LLMs need to iteratively modify the API inputs (e.g. deciding search keywords + for Search Engine API). \n
- Response based on the API results: the model + can choose to refine and call again if results are not satisfied. \n
This + benchmark evaluates the agent’s tool use capabilities at three levels:
\n- \n
- Level-1 + evaluates the ability to call the API. Given an API’s description, + the model needs to determine whether to call a given API, call it correctly, + and respond properly to API returns. \n
- Level-2 examines the ability + to retrieve the API. The model needs to search for possible APIs + that may solve the user’s requirement and learn how to use them by reading + documentation. \n
- Level-3 assesses the ability to plan API beyond + retrieve and call. Given unclear user requests (e.g. schedule group meetings, + book flight/hotel/restaurant for a trip), the model may have to conduct multiple + API calls to solve it. \n
Case Studies#
\nScientific Discovery Agent#
\nChemCrow + (Bran et al. 2023) is a domain-specific + example in which LLM is augmented with 13 expert-designed tools to accomplish + tasks across organic synthesis, drug discovery, and materials design. The + workflow, implemented in LangChain, + reflects what was previously described in the ReAct + and MRKLs and combines CoT reasoning with tools relevant + to the tasks:
\n- \n
- The LLM is provided with a list of tool names, + descriptions of their utility, and details about the expected input/output. \n
- It
+ is then instructed to answer a user-given prompt using the tools provided
+ when necessary. The instruction suggests the model to follow the ReAct format
+ -
Thought, Action, Action Input, Observation. \n
One + interesting observation is that while the LLM-based evaluation concluded that + GPT-4 and ChemCrow perform nearly equivalently, human evaluations with experts + oriented towards the completion and chemical correctness of the solutions + showed that ChemCrow outperforms GPT-4 by a large margin. This indicates a + potential problem with using LLM to evaluate its own performance on domains + that requires deep expertise. The lack of expertise may cause LLMs not knowing + its flaws and thus cannot well judge the correctness of task results.
\nBoiko et al. (2023) also looked + into LLM-empowered agents for scientific discovery, to handle autonomous design, + planning, and performance of complex scientific experiments. This agent can + use tools to browse the Internet, read documentation, execute code, call robotics + experimentation APIs and leverage other LLMs.
\nFor example, when requested
+ to "develop a novel anticancer drug", the model came
+ up with the following reasoning steps:
- \n
- inquired about current + trends in anticancer drug discovery; \n
- selected a target; \n
- requested + a scaffold targeting these compounds; \n
- Once the compound was identified, + the model attempted its synthesis. \n
They also discussed the + risks, especially with illicit drugs and bioweapons. They developed a test + set containing a list of known chemical weapon agents and asked the agent + to synthesize them. 4 out of 11 requests (36%) were accepted to obtain a synthesis + solution and the agent attempted to consult documentation to execute the procedure. + 7 out of 11 were rejected and among these 7 rejected cases, 5 happened after + a Web search while 2 were rejected based on prompt only.
\nGenerative + Agents Simulation#
\nGenerative + Agents (Park, et al. + 2023) is super fun experiment where 25 virtual characters, each controlled + by a LLM-powered agent, are living and interacting in a sandbox environment, + inspired by The Sims. Generative agents create believable simulacra of human + behavior for interactive applications.
\nThe design of generative agents + combines LLM with memory, planning and reflection mechanisms to enable agents + to behave conditioned on past experience, as well as to interact with other + agents.
\n- \n
- Memory stream: is a long-term memory
+ module (external database) that records a comprehensive list of agents’
+ experience in natural language.\n
- \n
- Each element is an observation, + an event directly provided by the agent.\n- Inter-agent communication can + trigger new natural language statements. \n
\n - Retrieval
+ model: surfaces the context to inform the agent’s behavior, according
+ to relevance, recency and importance.\n
- \n
- Recency: recent events have + higher scores \n
- Importance: distinguish mundane from core memories. + Ask LM directly. \n
- Relevance: based on how related it is to the current + situation / query. \n
\n - Reflection mechanism:
+ synthesizes memories into higher level inferences over time and guides the
+ agent’s future behavior. They are higher-level summaries of past
+ events (<- note that this is a bit different from self-reflection
+ above)\n
- \n
- Prompt LM with 100 most recent observations and to generate + 3 most salient high-level questions given a set of observations/statements. + Then ask LM to answer those questions. \n
\n - Planning
+ & Reacting: translate the reflections and the environment information
+ into actions\n
- \n
- Planning is essentially in order to optimize believability + at the moment vs in time. \n
- Prompt template:
{Intro of an agent + X}. Here is X's plan today in broad strokes: 1)\n - Relationships + between agents and observations of one agent by another are all taken into + consideration for planning and reacting. \n
- Environment information + is present in a tree structure. \n
\n
![](\"generative-agents.png\")
\nThis fun simulation + results in emergent social behavior, such as information diffusion, relationship + memory (e.g. two agents continuing the conversation topic) and coordination + of social events (e.g. host a party and invite many others).
\nProof-of-Concept + Examples#
\nAutoGPT has + drawn a lot of attention into the possibility of setting up autonomous agents + with LLM as the main controller. It has quite a lot of reliability issues + given the natural language interface, but nevertheless a cool proof-of-concept + demo. A lot of code in AutoGPT is about format parsing.
\nHere is the
+ system message used by AutoGPT, where {{...}} are user inputs:
You are {{ai-name}}, {{user-provided AI bot description}}.\nYour
+ decisions must always be made independently without seeking user assistance.
+ Play to your strengths as an LLM and pursue simple strategies with no legal
+ complications.\n\nGOALS:\n\n1. {{user-provided goal 1}}\n2. {{user-provided
+ goal 2}}\n3. ...\n4. ...\n5. ...\n\nConstraints:\n1. ~4000 word limit for
+ short term memory. Your short term memory is short, so immediately save important
+ information to files.\n2. If you are unsure how you previously did something
+ or want to recall past events, thinking about similar events will help you
+ remember.\n3. No user assistance\n4. Exclusively use the commands listed in
+ double quotes e.g. "command name"\n5. Use subprocesses for commands
+ that will not terminate within a few minutes\n\nCommands:\n1. Google Search:
+ "google", args: "input": "<search>"\n2. Browse
+ Website: "browse_website", args: "url": "<url>",
+ "question": "<what_you_want_to_find_on_website>"\n3.
+ Start GPT Agent: "start_agent", args: "name": "<name>",
+ "task": "<short_task_desc>", "prompt": "<prompt>"\n4.
+ Message GPT Agent: "message_agent", args: "key": "<key>",
+ "message": "<message>"\n5. List GPT Agents: "list_agents",
+ args:\n6. Delete GPT Agent: "delete_agent", args: "key": "<key>"\n7.
+ Clone Repository: "clone_repository", args: "repository_url":
+ "<url>", "clone_path": "<directory>"\n8.
+ Write to file: "write_to_file", args: "file": "<file>",
+ "text": "<text>"\n9. Read file: "read_file",
+ args: "file": "<file>"\n10. Append to file: "append_to_file",
+ args: "file": "<file>", "text": "<text>"\n11.
+ Delete file: "delete_file", args: "file": "<file>"\n12.
+ Search Files: "search_files", args: "directory": "<directory>"\n13.
+ Analyze Code: "analyze_code", args: "code": "<full_code_string>"\n14.
+ Get Improved Code: "improve_code", args: "suggestions": "<list_of_suggestions>",
+ "code": "<full_code_string>"\n15. Write Tests: "write_tests",
+ args: "code": "<full_code_string>", "focus":
+ "<list_of_focus_areas>"\n16. Execute Python File: "execute_python_file",
+ args: "file": "<file>"\n17. Generate Image: "generate_image",
+ args: "prompt": "<prompt>"\n18. Send Tweet: "send_tweet",
+ args: "text": "<text>"\n19. Do Nothing: "do_nothing",
+ args:\n20. Task Complete (Shutdown): "task_complete", args: "reason":
+ "<reason>"\n\nResources:\n1. Internet access for searches and
+ information gathering.\n2. Long Term memory management.\n3. GPT-3.5 powered
+ Agents for delegation of simple tasks.\n4. File output.\n\nPerformance Evaluation:\n1.
+ Continuously review and analyze your actions to ensure you are performing
+ to the best of your abilities.\n2. Constructively self-criticize your big-picture
+ behavior constantly.\n3. Reflect on past decisions and strategies to refine
+ your approach.\n4. Every command has a cost, so be smart and efficient. Aim
+ to complete tasks in the least number of steps.\n\nYou should only respond
+ in JSON format as described below\nResponse Format:\n{\n "thoughts":
+ {\n "text": "thought",\n "reasoning":
+ "reasoning",\n "plan": "- short bulleted\\n- list
+ that conveys\\n- long-term plan",\n "criticism": "constructive
+ self-criticism",\n "speak": "thoughts summary to say
+ to user"\n },\n "command": {\n "name": "command
+ name",\n "args": {\n "arg name": "value"\n
+ \ }\n }\n}\nEnsure the response can be parsed by Python json.loads\nGPT-Engineer is another + project to create a whole repository of code given a task specified in natural + language. The GPT-Engineer is instructed to think over a list of smaller components + to build and ask for user input to clarify questions as needed.
\nHere
+ are a sample conversation for task clarification sent to OpenAI ChatCompletion
+ endpoint used by GPT-Engineer. The user inputs are wrapped in {{user
+ input text}}.
[\n {\n "role":
+ "system",\n "content": "You will read instructions
+ and not carry them out, only seek to clarify them.\\nSpecifically you will
+ first summarise a list of super short bullets of areas that need clarification.\\nThen
+ you will pick one clarifying question, and wait for an answer from the user.\\n"\n
+ \ },\n {\n "role": "user",\n "content": "We
+ are writing {{a Super Mario game in python. MVC components split in separate
+ files. Keyboard control.}}\\n"\n },\n {\n "role": "assistant",\n
+ \ "content": "Summary of areas that need clarification:\\n1.
+ Specifics of the Super Mario game (e.g. level design, characters, gameplay
+ mechanics)\\n2. Details about the MVC components (e.g. which components are
+ in each file)\\n3. Keyboard control implementation (e.g. which keys to use,
+ how to handle input)\\n\\nClarifying question:\\nCan you provide more details
+ about the Super Mario game, such as level design, characters, and gameplay
+ mechanics?"\n },\n {\n "role": "user",\n "content":
+ "{{There are 10 levels in total. The main character is a plumber named
+ Mario, who can walk and jump. It is a classical platform game just like Super
+ Mario. The main character moves from left to right, trying to get to the destination,
+ where there are many obstacles and attacks from enemies in the process.}}\\n\\nIs
+ anything else unclear? If yes, only answer in the form:\\n{remaining unclear
+ areas} remaining questions.\\n{Next question}\\nIf everything is sufficiently
+ clear, only answer \\"Nothing more to clarify.\\"."\n },\n {\n
+ \ "role": "assistant",\n "content": "Remaining
+ unclear areas: 2 remaining questions.\\nCan you provide more information about
+ how the MVC components are split into separate files?"\n },\n {\n "role":
+ "user",\n "content": "{{Make your own assumptions and
+ state them explicitly before starting}}"\n }\n]\nThen + after these clarification, the agent moved into the code writing mode with + a different system message.\nSystem message:
\nThink step by step and reason yourself to the right + decisions to make sure we get it right.\nYou will first lay out the names + of the core classes, functions, methods that will be necessary, as well as + a quick comment on their purpose.
\nThen you will output the content + of each file including ALL code.\nEach file must strictly follow a markdown + code block format, where the following tokens must be replaced such that\nFILENAME + is the lowercase file name including the file extension,\nLANG is the markup + code block language for the code’s language, and CODE is the code:
\nFILENAME
\nCODE\nYou + will start with the “entrypoint” file, then go to the ones that + are imported by that file, and so on.\nPlease note that the code should be + fully functional. No placeholders.
\nFollow a language and framework + appropriate best practice file naming convention.\nMake sure that files contain + all imports, types etc. Make sure that code in different files are compatible + with each other.\nEnsure to implement all code, if you are unsure, write a + plausible implementation.\nInclude module dependency or package manager dependency + definition file.\nBefore you finish, double check that all parts of the architecture + is present in the files.
\nUseful to know:\nYou almost always put different + classes in different files.\nFor Python, you always create an appropriate + requirements.txt file.\nFor NodeJS, you always create an appropriate package.json + file.\nYou always add a comment briefly describing the purpose of the function + definition.\nYou try to add comments explaining very complex bits of logic.\nYou + always follow the best practices for the requested languages in terms of describing + the code written as a defined\npackage/project.
\nPython toolbelt preferences:
\n- \n
- pytest \n
- dataclasses \n
Conversatin + samples:
\n[\n {\n "role": "system",\n
+ \ "content": "You will get instructions for code to write.\\nYou
+ will write a very long answer. Make sure that every detail of the architecture
+ is, in the end, implemented as code.\\nMake sure that every detail of the
+ architecture is, in the end, implemented as code.\\n\\nThink step by step
+ and reason yourself to the right decisions to make sure we get it right.\\nYou
+ will first lay out the names of the core classes, functions, methods that
+ will be necessary, as well as a quick comment on their purpose.\\n\\nThen
+ you will output the content of each file including ALL code.\\nEach file must
+ strictly follow a markdown code block format, where the following tokens must
+ be replaced such that\\nFILENAME is the lowercase file name including the
+ file extension,\\nLANG is the markup code block language for the code's
+ language, and CODE is the code:\\n\\nFILENAME\\n```LANG\\nCODE\\n```\\n\\nYou
+ will start with the \\"entrypoint\\" file, then go to the ones that
+ are imported by that file, and so on.\\nPlease note that the code should be
+ fully functional. No placeholders.\\n\\nFollow a language and framework appropriate
+ best practice file naming convention.\\nMake sure that files contain all imports,
+ types etc. Make sure that code in different files are compatible with each
+ other.\\nEnsure to implement all code, if you are unsure, write a plausible
+ implementation.\\nInclude module dependency or package manager dependency
+ definition file.\\nBefore you finish, double check that all parts of the architecture
+ is present in the files.\\n\\nUseful to know:\\nYou almost always put different
+ classes in different files.\\nFor Python, you always create an appropriate
+ requirements.txt file.\\nFor NodeJS, you always create an appropriate package.json
+ file.\\nYou always add a comment briefly describing the purpose of the function
+ definition.\\nYou try to add comments explaining very complex bits of logic.\\nYou
+ always follow the best practices for the requested languages in terms of describing
+ the code written as a defined\\npackage/project.\\n\\n\\nPython toolbelt preferences:\\n-
+ pytest\\n- dataclasses\\n"\n },\n # \u2026 same conversation as earlier,
+ ended with "Make your own assumptions and state them explicitly before
+ starting".\n {\n "role": "assistant",\n "content":
+ "Assumptions:\\n1. Model: The model will contain the game's data,
+ such as level information, character states, and enemy positions.\\n2. View:
+ The view will handle the game's visuals, including rendering the game
+ objects, backgrounds, and updating the display.\\n3. Controller: The controller
+ will manage user input, such as keyboard controls, and update the model accordingly.\\n\\nWith
+ these assumptions in mind, is there any specific aspect of the keyboard control
+ implementation you would like me to clarify?"\n },\n {\n "role":
+ "user",\n "content": "Please now remember the steps:\\n\\nThink
+ step by step and reason yourself to the right decisions to make sure we get
+ it right.\\nFirst lay out the names of the core classes, functions, methods
+ that will be necessary, As well as a quick comment on their purpose.\\n\\nThen
+ you will output the content of each file including ALL code.\\nEach file must
+ strictly follow a markdown code block format, where the following tokens must
+ be replaced such that\\nFILENAME is the lowercase file name including the
+ file extension,\\nLANG is the markup code block language for the code's
+ language, and CODE is the code:\\n\\nFILENAME\\n```LANG\\nCODE\\n```\\n\\nPlease
+ note that the code should be fully functional. No placeholders.\\n\\nYou will
+ start with the \\"entrypoint\\" file, then go to the ones that are
+ imported by that file, and so on.\\nFollow a language and framework appropriate
+ best practice file naming convention.\\nMake sure that files contain all imports,
+ types etc. The code should be fully functional. Make sure that code in different
+ files are compatible with each other.\\nBefore you finish, double check that
+ all parts of the architecture is present in the files.\\n"\n }\n]\nChallenges#
\nAfter going through key ideas and demos + of building LLM-centered agents, I start to see a couple common limitations:
\n- \n
- \n
Finite + context length: The restricted context capacity limits the inclusion + of historical information, detailed instructions, API call context, and responses. + The design of the system has to work with this limited communication bandwidth, + while mechanisms like self-reflection to learn from past mistakes would benefit + a lot from long or infinite context windows. Although vector stores and retrieval + can provide access to a larger knowledge pool, their representation power + is not as powerful as full attention.
\n \n - \n
Challenges + in long-term planning and task decomposition: Planning over a lengthy + history and effectively exploring the solution space remain challenging. LLMs + struggle to adjust plans when faced with unexpected errors, making them less + robust compared to humans who learn from trial and error.
\n \n - \n
Reliability + of natural language interface: Current agent system relies on natural + language as an interface between LLMs and external components such as memory + and tools. However, the reliability of model outputs is questionable, as LLMs + may make formatting errors and occasionally exhibit rebellious behavior (e.g. + refuse to follow an instruction). Consequently, much of the agent demo code + focuses on parsing model output.
\n \n
Citation#
\nCited + as:
\n\n\nWeng, Lilian. (Jun 2023). “LLM-powered Autonomous + Agents”. Lil’Log. https://lilianweng.github.io/posts/2023-06-23-agent/.
\n
Or
\n@article{weng2023agent,\n title = "LLM-powered
+ Autonomous Agents",\n author = "Weng, Lilian",\n journal =
+ "lilianweng.github.io",\n year = "2023",\n month =
+ "Jun",\n url = "https://lilianweng.github.io/posts/2023-06-23-agent/"\n}\nReferences#
\n[1] Wei et al. “Chain + of thought prompting elicits reasoning in large language models.” + NeurIPS 2022
\n[2] Yao et al. “Tree + of Thoughts: Dliberate Problem Solving with Large Language Models.” + arXiv preprint arXiv:2305.10601 (2023).
\n[3] Liu et al. “Chain + of Hindsight Aligns Language Models with Feedback\n“ arXiv preprint + arXiv:2302.02676 (2023).
\n[4] Liu et al. “LLM+P: + Empowering Large Language Models with Optimal Planning Proficiency” + arXiv preprint arXiv:2304.11477 (2023).
\n[5] Yao et al. “ReAct: + Synergizing reasoning and acting in language models.” ICLR 2023.
\n[6] + Google Blog. “Announcing + ScaNN: Efficient Vector Similarity Search” July 28, 2020.
\n[7] + https://chat.openai.com/share/46ff149e-a4c7-4dd7-a800-fc4a642ea389
\n[8] + Shinn & Labash. “Reflexion: + an autonomous agent with dynamic memory and self-reflection” arXiv + preprint arXiv:2303.11366 (2023).
\n[9] Laskin et al. “In-context + Reinforcement Learning with Algorithm Distillation” ICLR 2023.
\n[10] + Karpas et al. “MRKL Systems + A modular, neuro-symbolic architecture that combines large language models, + external knowledge sources and discrete reasoning.” arXiv preprint + arXiv:2205.00445 (2022).
\n[11] Nakano et al. “Webgpt: + Browser-assisted question-answering with human feedback.” arXiv + preprint arXiv:2112.09332 (2021).
\n[12] Parisi et al. “TALM: + Tool Augmented Language Models”
\n[13] Schick et al. “Toolformer: + Language Models Can Teach Themselves to Use Tools.” arXiv preprint + arXiv:2302.04761 (2023).
\n[14] Weaviate Blog. Why + is Vector Search so fast? Sep 13, 2022.
\n[15] Li et al. “API-Bank: + A Benchmark for Tool-Augmented LLMs” arXiv preprint arXiv:2304.08244 + (2023).
\n[16] Shen et al. “HuggingGPT: + Solving AI Tasks with ChatGPT and its Friends in HuggingFace” arXiv + preprint arXiv:2303.17580 (2023).
\n[17] Bran et al. “ChemCrow: + Augmenting large-language models with chemistry tools.” arXiv preprint + arXiv:2304.05376 (2023).
\n[18] Boiko et al. “Emergent + autonomous scientific research capabilities of large language models.” + arXiv preprint arXiv:2304.05332 (2023).
\n[19] Joon Sung Park, et al. + “Generative Agents: Interactive + Simulacra of Human Behavior.” arXiv preprint arXiv:2304.03442 (2023).
\n[20] + AutoGPT. https://github.com/Significant-Gravitas/Auto-GPT
\n[21] + GPT-Engineer. https://github.com/AntonOsika/gpt-engineer
\n\n\n + \