14 Front. Comput. Sci.,2024,0(0): 1–42
ActionGoal: Theagentcanperformactionswith maintainsamemorystream,andbeforetakingeach
variousobjectives. Here,wepresentseveralrepre- action, it retrieves recent, relevant and important
sentative examples: (1) Task Completion. In this information from the memory steam to guide the
scenario, the agent’s actions are aimed at accom- agentactions. InGITM[16], inordertoachievea
plishingspecifictasks,suchascraftinganironpick- low-level sub-goal, the agent queries its memory to
axe in Minecraft [38] or completing a function in determine if there are any successful experiences
software development [18]. These actions usually relatedtothe task. If similartaskshavebeencom-
havewell-definedobjectives,andeachactioncon- pletedpreviously,theagentinvokesthepreviously
tributes to the completion of the final task. Ac- successfulactionstohandlethecurrenttaskdirectly.
tions aimed at this type of goal are very common In collaborative agents such as ChatDev [18] and
in existing literature. (2) Communication. In this MetaGPT [23], different agents may communicate
case,theactionsaretakentocommunicatewiththe with each other. In this process, the conversation
other agents or real humans for sharing informa- historyinadialogisrememberedintheagentmem-
tion or collaboration. For example, the agents in ories. Each utterance generated by the agent is
ChatDev [18] may communicate with each other influencedbyitsmemory. (2)ActionviaPlanFol-
to collectively accomplish software development lowing. In this strategy, the agent takes actions
tasks. In InnerMonologue[61], theagent actively followingitspre-generatedplans. Forinstance,in
engages in communication with humans and ad- DEPS [33], for a given task, the agent first makes
justs its actionstrategies based on humanfeedback. actionplans. Iftherearenosignalsindicatingplan
(3)EnvironmentExploration. Inthisexample,the failure,theagentwillstrictlyadheretotheseplans.
agent aims to explore unfamiliar environments to InGITM[16],theagentmakeshigh-levelplansby
expanditsperceptionandstrikeabalancebetween decomposingthetaskintomanysub-goals. Based
exploringandexploiting. Forinstance,theagentin ontheseplans,theagenttakesactionstosolveeach
Voyager [38] may explore unknown skills in their sub-goalsequentiallytocompletethefinaltask.
task completionprocess, and continuallyrefine the
Action Space: Action space refers to the set of
skillexecutioncodebasedonenvironmentfeedback
possibleactionsthatcanbeperformedbytheagent.
throughtrialanderror.
In general, we can roughly divide these actions
ActionProduction: DifferentfromordinaryLLMs, into two classes: (1) external tools and (2) inter-
nal knowledge of the LLMs. In the following, we
where the model input and output are directly as-
sociated, the agent may take actions via different introducetheseactionsmoreindetail.
strategies and sources. In the following, we intro- • External Tools. While LLMs have been
ducetwotypesofcommonlyusedactionproduction demonstrated to be effective in accomplishing a
strategies. (1) Action viaMemory Recollection. In large amount of tasks, they may not work well
this strategy, the action is generated by extracting forthedomainswhichneedcomprehensiveexpert
information from the agent memory according to knowledge. Inaddition,LLMsmayalsoencounter
thecurrent task. The taskandthe extractedmemo- hallucination problems, which are hard to be re-
riesare usedasprompts totrigger theagentactions. solvedbythemselves. Toalleviatetheaboveprob-
For example, inGenerativeAgents[20], the agent lems,theagentsareempoweredwiththecapability