1) FixBug

2) Add Documents abount TASK/TODO Refactor
This commit is contained in:
Liu Zhicong
2024-01-06 13:08:41 -08:00
parent 3663f140fc
commit c975ba2053
22 changed files with 398 additions and 109 deletions
+265
View File
@@ -0,0 +1,265 @@
# AI Function list (Local)
根据新的Function/Action 定义,我们需要记录系统目前提供的所有注册到GlobaToolsLibrary里的function,方便配置组合
功能扩展只需要扩展function就好了,Action可以通过Function直接得到。
本文档提到的AI Function一般都是Local Function, 基于这个架构,我们可以用Web3的思路整合AI Service。
## Context Prepare
很多AIFUnction的正确调用都需要在参数里提供一个状态上下文。这个上下文的准备一般由LLProcess完成.而可以通过提示词实时改变的参数是AIFunction的真参数。目前需要做Context Prepare的主要和Agent相关的基础函数。
## Agent Core
Agent Core中提供的,与Agent/Workflow 状态关联的基础函数,是Agent的核心设计。对Agent来说,这些基础能力通常都是打开的,我们需要非常谨慎的思考哪些功能应该放到Agent Core中。
根据我们的架构设计,Agent Core的函数包含 Agent的Memory + TODO能力(Plan)。通过这些基础函数,支持了Agent的
- Process Message
- Planing: 让Agent可以基于Zone的权限Process Task/Todo
- Self-Think: Process的经验进行总结
- Self-Improve: 对Agent的提示词进行持续的自我改进。
### logs
logs通常代表Agent的短期记忆,目前有两种Log:chatlog和worklog
目前我们并不鼓励Agent提供短期记忆的调整能力,都是通过标准流程得到一定实践范围的的,完整的短期记忆。
set_message //增加tag
get_chatlogs
get_worklogs
set_worklogs //增加tag
### Memory
Memory 代表Agent的长期记忆
通过LLM驱动的Self-Think流程来更新。基本上是在根据短期记忆提炼对人、对事的看法和终结。也包含一些基于提示词工程的自我能力提升(比如复用已知的,好用的工具,或则复用自己为了解决某个特定问题已经制作并使用成功的工具)
get_contact_summary
update_contact_summary
get_sth_summary
update_sth_summary
### Workspace (TODOList)
Workspace为Agent提供了可以完成TODO的工具和保存工作结果的状态空间(FileSystem)。每个Agent默认有自己的private workspaceWorkflow为Agent提供了可以共享的workspaceWorkspace尽量基于文件系统构造,也方便Agent与人类协同工作。
#### Task/Todo 管理
context中需要一个隐藏的_workspace
```
_self = parameters.get("_workspace")
```
##### agent.workspace.list_task
##### agent.workspace.create_task
```json
{
"title": {"type": "string", "description": "The title of the task."},
"detail": {"type": "string", "description": "The detail of the task."},
"tags": {"type": "string array", "description": "The tags of the task."},
"due_date": {"type": "string", "description": "The due date of the task."},
"parent_id": {"type": "string", "description": "The parent id of the task."},
}
```
##### agent.workspace.cancel_task
```json
{
"task_id": {"type": "string", "description": "The id of the task to cancel."},
}
```
##### agent.workspace.update_task
##### agent.workspace.get_sub_tasks
##### agent.workspace.create_todos
##### agent.workspace.list_todos
##### agent.workspace.get_todo
##### agent.workspace.update_todo
#### 核心的完成TODO的能力
##### Code Interprete
##### Send Msg (系统原生能力)
## Knowledge Base
Knowledge Base对大部分Agent来说,是一个获得私有信息,并让LLM处理结果更好的基础设施(RAG支持)。少部分Agent会使用相关API,结合Knowledge Base所服务的目标来整理Knowledge Base.
### 搜索
#### 矢量搜索
#### 传统的文本搜索搜索
#### 根据已经存在的数据库描述,构造SQL搜索
### 当成文件系统浏览
## AIGC
一系列AIGC函数,是LLM打通AIGC能力,形成新生产力的基础。
### aigc.text_2_image
文生图.返回的是生成图片的路径。
```json
{
"prompt": "Description of the content of the painting",
}
```
### aigc.image_2_text
图生文,返回的是图片的描述
(TODO:是否需要有一个提示词来要求针对特定问题对图片进行描述)
```json
{
"image_path": {"type": "string", "description": "image file path"}
}
```
### aigc.voice_to_text
```json
{
"audio_file": {"type": "string", "description": "Audio file path"},
"model": {"type": "string", "description": "Recognition model", "enum": ["openai-whisper"]},
"prompt": {"type": "string", "description": "Prompt statement, can be None"},
"response_format": {"type": "string", "description": "Return format", "enum": ["text", "json", "srt", "verbose_json", "vtt"]},
}
```
## system
访问当前系统的基础设施
### system.now
返回当前时间
### system.calender
保存在当前Zone上的日历,默认是与Zone Owner相关的。也可以以自动形式同步别人的日历
这个组件对AI成为个人助理非常重要,当与旧世界互通时,其细节的复杂度也是非常高的。
这是一个典型的看起来容易做起来难度很大的基础组件,是思考和验证LLM对传统软件复杂度进行降维的一个关键实践。
#### system.calender.get_events
#### system.calender.add_event
#### system.calender.delete_event
#### system.calender.update_event
### system.contacts
访问用户的联系人列表。在OOD System 中,联系人列表是非常重要的系统基础设施,为一系列的权限控制提供了基础信息。
#### system.contacts.get
通过contanct的名字得到contact的完整信息(json格式)
```json
{"name":"name"}
```
#### system.contacts.set
设置 contact 信息,注意这里使用了一个可扩展结构,我们可能需要定义一些标准的必填信息。
```json
{"name":"name","contact_info":"A json to descrpit contact"}
```
### System.shell
#### system.shell.exec
执行Shell命令(目前只支持Linux Bash
## web
访问web的函数。使用下列函数要确保Agent有访问互联网的权限。
### web.search.duckduckgo
使用搜索引擎搜索互联网
```json
{
"query": {"type": "string", "description": "The query to search for."}
}
```
## 常见的llm_context(能力分组)
为典型的LLM处理过程,进行了分组。主要是为了节约Token。
使用Action比使用inner function更节约。
### Process消息组 (定制度高)
得到潜在的Task并创建,在这个过程中可能需要查询已有的任务(防止重复创建)
action:craete_task, function:list_task
通常Agent在Process Message时,会表现出和其处理TODO接近的能力,核心的区别在于Process Message是立刻处理并给出结果,而变成Task更多的是当成一个异步的任务
为了能处理回复,查询历史沟通记录(寻找记忆的过程)
为了能处理回复,查询KB的过程
REMARK:目前LLM的主要问题是,如果开放的function, LLM会倾向于优先使用,是否可以做成“如果用户对答案不满意”,再使用?
### Review Task
对Task的首次执行,Review的目的
- 拆分创建TODO(使用create_todos
- 对简单的任务立刻执行并记录更新结果 (通常是)
- 认为超出自己的能力范围,标记为无法处理或转交给合适的人(Agent (使用update_task)
- 对已有任务进行查询(list_task,query_task
### Do TODO(定制度高)
DO行为是复杂的,我们会精细的区分TODO的首次执行和失败后再次执行。失败后再次执行会得到之前的记录摘要,并有查询之前工作日志的能力。
Do TODO是Agent的另一个核心行为,这里会根据Agent的设定,集成更多的能力
系统为Do提供的默认支持:(按难度逐步增加)
- 写文档 : 不需要任何外部支持
- 运行AIGC AIGC的函数组
- 收集,整理信息(通过互联网或查询知识库): web.search.duckduckgo
- 发送消息,系统自带,但可能需要依赖一些通讯录浏览/查找函数
- 执行自己的代码/编写代码并执行 system.shell.execcode_interprete (这是一个重点模块!)
- 运行网络服务 :智能合约的有通用套路,非智能合约的需要有一个完整的SOP来支持
根据任务要求保存工作成功是手动的,这里有一组workspace级别的文件系统API。
保存工作记录的行为是自动的,默认所有的Action都执行成就算是DoComplte,会自动的更新状态
有的Do可能需要自我迭代一下,这和大多数Behavor只有一次LLM调用有所不同。
### Check TODO/Task
为了解决LLM不可避免的幻视加入的Check流程。该流程会根据TODO的目标,对TODO的结果进行判定
使用 update_todo 更新TODO的状态
当所有的sub_todos都完成后,会check task的目标是否达到
当所有的sub_task都完成后,会check task的目标是否达到
因此,提供的函数主要是得到 todo/task的更深入细节的函数(访问相关log),已经读取相关 工程成果文件 的函数
### Self-Think
获得logs 和summary
进行update
### Learning
得到logs和summary
浏览和整理KB
### Self-Improve
@@ -0,0 +1,26 @@
# 从日程管理软件到个人助理Agent
## 背景
在计算的时代,我们想要有开发一个好用的日程管理服务,很快就会发现这并不简单。比如我们会有各种各样的重复任务提醒要求:比如每个工作周周五的下午去游泳,每个工作周的第一天要开例会,和某人的纪念日要做什么等等。允许用户创建,并识别这些复杂的条件是非常繁琐的。而在计算机发明以前,你的秘书只需要一张纸就可以搞定各种复杂的日程管理:秘书把你的要求原样记录下,然后秘书只要每天Review一遍,确定好今天的行程就好。
一些有更复杂的条件的日程,秘书可以通过定期Review代办来搞定。
理解上述区别,就理解了 PC 和 PI (CPU vs LLM)的核心区别。
## 基于Agent 的日程管理基础设施
为了支持秘书Agent,aios所构建的机制是:
1. Task创建: 模拟秘书在记事本上记录下日程相关的任务(注意不要让Agent创建重复的任务)
2. ListTask,可查询未标记为结束的Task
3. 大Review(定期LLM调用),Agent浏览未完成的Task,并给这些Task一个比较精确的`下次处理时间`
4. 当处理事件的时间点到达时,Agent会尝试处理Task,此时决定是否满足日程发生的条件,并发送正确的提醒信息。
上面的所有的Agent处理,都是让 LLM处理一段文本,并做出反应。因此没有存储格式的需求,用自然语言记录下各种条件就可以了。
## 过渡:混合使用日程管理软件和 秘书Agent
1. 秘书Agent主要使用日程管理软件来查询已有日程
2. Agent创建通过日程管理软件的目的,通常是为了公开日程,或则创建的是具有`一定协作需求`,或有`公开需求``确定`日程
3. Agent依旧依靠自己的LLM Based逻辑来Review并处理日程, 通过日程管理软件得到的需要处理的日程会有特殊标记,以能正确的操作来保持处理结果。
+4 -4
View File
@@ -115,22 +115,22 @@ class ChatSessionDB:
action_result = None
mentions = None
if msg.mentions:
mentions = json.dumps(msg.mentions)
mentions = json.dumps(msg.mentions,ensure_ascii=False)
match msg.msg_type:
case AgentMsgType.TYPE_MSG:
pass
case AgentMsgType.TYPE_ACTION:# THIS Action is not AIAction
action_name = msg.func_name
action_params = json.dumps(msg.args)
action_params = json.dumps(msg.args,ensure_ascii=False)
action_result = msg.result_str
case AgentMsgType.TYPE_INTERNAL_CALL:
action_name = msg.func_name
action_params = json.dumps(msg.args)
action_params = json.dumps(msg.args,ensure_ascii=False)
action_result = msg.result_str
case AgentMsgType.TYPE_EVENT:
action_name = msg.event_name
action_params = json.dumps(msg.event_args)
action_params = json.dumps(msg.event_args,ensure_ascii=False)
if tags is None:
tags = []
+41 -47
View File
@@ -4,7 +4,7 @@ import json
import logging
from typing import Optional,Set,List,Dict,Callable
from ..proto.ai_function import AIFunction,AIAction,SimpleAIAction
from ..proto.ai_function import AIFunction,AIAction, AIFunction2Action,SimpleAIAction
logger = logging.getLogger(__name__)
@@ -15,10 +15,7 @@ class LLMProcessContext:
@staticmethod
def function2action(ai_func:AIFunction) -> AIAction:
async def exec_func(params:Dict) -> str:
return await ai_func.execute(params)
return SimpleAIAction(ai_func.get_id(),ai_func.get_detail_description(),exec_func)
return AIFunction2Action(ai_func)
@staticmethod
def aifunctions_to_inner_functions(all_inner_function:List[AIFunction]) -> List[Dict]:
@@ -30,7 +27,7 @@ class LLMProcessContext:
this_func["name"] = func_name
this_func["description"] = inner_func.get_description()
this_func["parameters"] = inner_func.get_openai_parameters()
result_len += len(json.dumps(this_func)) / 4
result_len += len(json.dumps(this_func,ensure_ascii=False)) / 4
result_func.append(this_func)
return result_func
@@ -117,7 +114,7 @@ class SimpleLLMContext(LLMProcessContext):
self.actions: Dict[str,AIAction] = {}
self.action_sets : Dict[str,Dict[str,AIAction]] = {}
def load_action_set_from_config(self,preset,config:Dict[str,str]) -> bool:
def load_action_set_from_config(self,preset,config:Dict[str,str]) -> Dict:
if preset is None:
result = {}
else:
@@ -217,44 +214,43 @@ class SimpleLLMContext(LLMProcessContext):
self.func_sets = self.parent.func_sets
action_def:Dict= config.get("actions")
if action_def is None:
logger.error(f"load_from_config failed! actions not found!")
return False
self.actions = self.load_action_set_from_config(self.actions,action_def)
if self.actions is None:
logger.error(f"load_from_config failed! load_action_set_from_config failed!")
return False
for set_name in action_def.keys():
if set_name == "enable":
continue
if set_name == "disable":
continue
sub_set = config.get(set_name)
self.action_sets[set_name] = self.load_action_set_from_config(None,sub_set)
if self.action_sets[set_name] is None:
if action_def:
self.actions = self.load_action_set_from_config(self.actions,action_def)
if self.actions is None:
logger.error(f"load_from_config failed! load_action_set_from_config failed!")
return False
for set_name in action_def.keys():
if set_name == "enable":
continue
if set_name == "disable":
continue
sub_set = config.get(set_name)
self.action_sets[set_name] = self.load_action_set_from_config(None,sub_set)
if self.action_sets[set_name] is None:
logger.error(f"load_from_config failed! load_action_set_from_config failed!")
return False
function_def:Dict = config.get("functions")
self.functions = self.load_function_set_from_config(self.functions,function_def)
if self.functions is None:
logger.error(f"load_from_config failed! load_function_set_from_config failed!")
return False
for set_name in function_def.keys():
if set_name == "enable":
continue
if set_name == "disable":
continue
sub_set = config.get(set_name)
self.func_sets[set_name] = self.load_function_set_from_config(None,sub_set)
if self.func_sets[set_name] is None:
if function_def:
self.functions = self.load_function_set_from_config(self.functions,function_def)
if self.functions is None:
logger.error(f"load_from_config failed! load_function_set_from_config failed!")
return False
for set_name in function_def.keys():
if set_name == "enable":
continue
if set_name == "disable":
continue
sub_set = config.get(set_name)
self.func_sets[set_name] = self.load_function_set_from_config(None,sub_set)
if self.func_sets[set_name] is None:
logger.error(f"load_from_config failed! load_function_set_from_config failed!")
return False
#values_def = config.get("values")
#if values_def:
# for key,value in values_def.items():
@@ -280,6 +276,7 @@ class SimpleLLMContext(LLMProcessContext):
# func = self.func_sets[set_name].get(func_name)
# if func is not None:
# return func
return None
def get_function_set(self,set_name:str = None) -> List[AIFunction]:
if set_name is None:
@@ -291,15 +288,12 @@ class SimpleLLMContext(LLMProcessContext):
return None
# def get_ai_action(self,op_name:str) -> AIOperation:
# op = self.actions.get(op_name)
# if op is not None:
# return op
# for set_name in self.action_sets.keys():
# op = self.action_sets[set_name].get(op_name)
# if op is not None:
# return op
# return None
def get_ai_action(self,op_name:str) -> AIAction:
for action in self.actions.values():
if action.get_name() == op_name:
return action
return None
def get_action_set(self,set_name:str = None) -> List[AIFunction]:
if set_name is None:
+6 -6
View File
@@ -46,9 +46,9 @@ class BaseLLMProcess(ABC):
async def prepare_prompt(self,input:Dict) -> LLMPrompt:
pass
@abstractmethod
async def get_inner_function_for_exec(self,func_name:str) -> AIFunction:
return GlobaToolsLibrary.get_instance().get_tool_function(func_name)
pass
@abstractmethod
async def post_llm_process(self,actions:List[ActionNode],input:Dict,llm_result:LLMResult) -> bool:
@@ -89,7 +89,7 @@ class BaseLLMProcess(ABC):
try:
func_name = inner_func_call_node.get("name")
arguments = json.loads(inner_func_call_node.get("arguments"))
logger.info(f"LLMProcess execute inner func:{func_name} :\n\t {json.dumps(arguments)}")
logger.info(f"LLMProcess execute inner func:{func_name} :\n\t {json.dumps(arguments,ensure_ascii=False)}")
func_node : AIFunction = await self.get_inner_function_for_exec(func_name)
if func_node is None:
@@ -455,7 +455,7 @@ class LLMAgentMessageProcess(BaseLLMProcess):
logger.info(f"enable kb")
prompt.append_system_message(json.dumps(system_prompt_dict))
prompt.append_system_message(json.dumps(system_prompt_dict,ensure_ascii=False))
## 扩展已知信息 (这可能是一个LLM过程)
prompt.append_system_message(await self.get_extend_known_info(msg,prompt))
@@ -555,8 +555,8 @@ class ReviewTaskProcess(BaseLLMProcess):
system_prompt_dict["role_description"] = self.role_description
system_prompt_dict["process_rule"] = self.process_description
system_prompt_dict["reply_format"] = self.reply_format
prompt.append_system_message(json.dumps(system_prompt_dict))
prompt.append_user_message(json.dumps(agent_task.to_dict()))
prompt.append_system_message(json.dumps(system_prompt_dict,ensure_ascii=False))
prompt.append_user_message(json.dumps(agent_task.to_dict(),ensure_ascii=False))
return prompt
+12 -10
View File
@@ -76,7 +76,7 @@ class LocalAgentTaskManger(AgentTaskManager):
self._save_obj_path(task.task_id,task_path)
logger.info("create_task at %s",detail_path)
async with aiofiles.open(detail_path, mode='w', encoding="utf-8") as f:
await f.write(json.dumps(task.to_dict()))
await f.write(json.dumps(task.to_dict(),ensure_ascii=False))
return "create task ok"
except Exception as e:
logger.error("create_task failed:%s",e)
@@ -97,7 +97,7 @@ class LocalAgentTaskManger(AgentTaskManager):
todo_path = f"{self.root_path}/{owner_task_path}/#{step_order} {todo.title}.todo"
self._save_obj_path(todo.todo_id,todo_path)
async with aiofiles.open(todo_path, mode='w', encoding="utf-8") as f:
await f.write(json.dumps(todo.to_dict()))
await f.write(json.dumps(todo.to_dict(),ensure_ascii=False))
logger.info("create_todos at %s OK!",todo_path)
step_order += 1
except Exception as e:
@@ -121,7 +121,7 @@ class LocalAgentTaskManger(AgentTaskManager):
logs = []
logs.append(log.to_dict())
json_obj["logs"] = logs
await f.write(json.dumps(json_obj))
await f.write(json.dumps(json_obj,ensure_ascii=False))
async def get_worklog(self,obj_id:str)->List[AgentWorkLog]:
@@ -263,7 +263,7 @@ class LocalAgentTaskManger(AgentTaskManager):
detail_path = f"{self.root_path}/{task.task_path}/detail"
try:
async with aiofiles.open(detail_path, mode='w', encoding="utf-8") as f:
await f.write(json.dumps(task.to_dict()))
await f.write(json.dumps(task.to_dict(),ensure_ascii=False))
except Exception as e:
logger.error("update_task failed:%s",e)
return str(e)
@@ -277,7 +277,7 @@ class LocalAgentTaskManger(AgentTaskManager):
try:
async with aiofiles.open(todo_path, mode='w', encoding="utf-8") as f:
await f.write(json.dumps(todo.to_dict()))
await f.write(json.dumps(todo.to_dict(),ensure_ascii=False))
except Exception as e:
logger.error("update_todo failed:%s",e)
return str(e)
@@ -311,18 +311,20 @@ class LocalAgentTaskManger(AgentTaskManager):
class AgentWorkspace:
def __init__(self,owner_agent_id:str) -> None:
self.agent_id : str = owner_agent_id
self.task_mgr : AgentTaskManager = LocalAgentTaskManger(owner_agent_id)
def __init__(self,owner_id:str) -> None:
self.owner_id : str = owner_id
self.task_mgr : AgentTaskManager = LocalAgentTaskManger(owner_id)
@staticmethod
def register_ai_functions():
async def create_task(params):
_workspace = params.get("_workspace")
_agent_id = params.get("_agentid")
if _workspace is None:
return "_workspace not found"
if params.get("creator") is None:
params["creator"] = _agent_id
taskObj = AgentTask.create_by_dict(params)
parent_id = params.get("parent")
return await _workspace.task_mgr.create_task(taskObj,parent_id)
@@ -371,7 +373,7 @@ class AgentWorkspace:
return "_workspace not found"
all_task = await _workspace.task_mgr.list_task(None)
if all_task:
return json.dumps([task.to_dict() for task in all_task])
return json.dumps([task.to_dict() for task in all_task],ensure_ascii=False)
else :
return "no task"
list_task_ai_function = SimpleAIFunction("agent.workspace.list_task",
@@ -44,7 +44,7 @@ class DuckDuckGoTextSearchFunction(AIFunction):
max_results=self.max_results
)]
return json.dumps(results)
return json.dumps(results,,ensure_ascii=False)
def is_local(self) -> bool:
return True
+2 -2
View File
@@ -87,7 +87,7 @@ class SimpleEnvironment(BaseEnvironment):
return func_list
def add_ai_operation(self,op:AIAction) -> None:
self.operations[op.get_name()] = op
self.operations[op.get_id()] = op
def get_ai_operation(self,op_name:str) -> AIAction:
op = self.operations.get(op_name)
@@ -114,7 +114,7 @@ class CompositeEnvironment(SimpleEnvironment):
self.functions[func.get_id()] = func
operations = env.get_all_ai_operations()
for op in operations:
self.operations[op.get_name()] = op
self.operations[op.get_id()] = op
def get_value(self,key:str) -> Optional[str]:
for env in self.envs:
+1 -1
View File
@@ -214,7 +214,7 @@ class SimpleKnowledgeDB:
def query_docs_by_tag(self, tag: str) -> List[str]:
conn = self._get_conn()
cursor = conn.cursor()
tag_json = json.dumps(tag) # 将标签转换为 JSON 字符串
tag_json = json.dumps(tag,ensure_ascii=False) # 将标签转换为 JSON 字符串
cursor.execute('''
SELECT documents.doc_path
FROM documents
+3 -3
View File
@@ -126,7 +126,7 @@ class CalenderEnvironment(SimpleEnvironment):
_event["location"] = row[5]
_event["details"] = row[6]
result[row[0]] = _event
return json.dumps(result, indent=4, sort_keys=True)
return json.dumps(result, indent=4, sort_keys=True,ensure_ascii=False)
async def _get_events_by_time_range(self,start_time, end_time):
async with aiosqlite.connect(self.db_file) as db:
@@ -152,7 +152,7 @@ class CalenderEnvironment(SimpleEnvironment):
if not have_result:
return "No event."
return json.dumps(result, indent=4, sort_keys=True)
return json.dumps(result, indent=4, sort_keys=True,ensure_ascii=False)
async def _update_event(self,event_id, new_title=None, new_participants=None, new_location=None, new_details=None ,start_time=None, end_time=None):
fields_to_update = []
@@ -214,7 +214,7 @@ class CalenderEnvironment(SimpleEnvironment):
cm = ContactManager.get_instance()
contact : Contact = cm.find_contact_by_name(name)
if contact:
s = json.dumps(contact.to_dict())
s = json.dumps(contact.to_dict(),ensure_ascii=False)
return f"Execute get_contact OK , contact {name} is {s}"
else:
return f"Execute get_contact OK , contact {name} not found!"
+3 -3
View File
@@ -205,7 +205,7 @@ class TodoListEnvironment(SimpleEnvironment):
self._save_todo_path(todo.todo_id,todo_path)
logger.info("create_todo %s",detail_path)
async with aiofiles.open(detail_path, mode='w', encoding="utf-8") as f:
await f.write(json.dumps(todo.to_dict()))
await f.write(json.dumps(todo.to_dict(),ensure_ascii=False))
except Exception as e:
logger.error("create_todo failed:%s",e)
return str(e)
@@ -221,7 +221,7 @@ class TodoListEnvironment(SimpleEnvironment):
todo.state = new_stat
detail_path = f"{full_path}/detail"
async with aiofiles.open(detail_path, mode='w', encoding="utf-8") as f:
await f.write(json.dumps(todo.to_dict()))
await f.write(json.dumps(todo.to_dict()),ensure_ascii=False)
return None
else:
return "todo not found."
@@ -270,7 +270,7 @@ class TodoListEnvironment(SimpleEnvironment):
logs = []
logs.append(result.to_dict())
json_obj["logs"] = logs
await f.write(json.dumps(json_obj))
await f.write(json.dumps(json_obj),ensure_ascii=False)
class WorkspaceEnvironment(CompositeEnvironment):
+1
View File
@@ -63,6 +63,7 @@ class KnowledgeObject(ABC):
# Convert the object_type and desc to string and compute the SHA256 hash
data = json.dumps(
{"object_type": self.object_type, "desc": self.desc},
ensure_ascii=False,
cls=ObjectEnhancedJSONEncoder,
)
sha256 = hashlib.sha256()
+3 -3
View File
@@ -154,7 +154,7 @@ class AgentMsg:
@staticmethod
def create_image_body(images: [str], prompt: str = None):
return json.dumps({"images": images, "prompt": prompt})
return json.dumps({"images": images, "prompt": prompt}, ensure_ascii=False)
@staticmethod
def parse_image_body(image_body: str) -> Tuple[str, List[str]]:
@@ -163,7 +163,7 @@ class AgentMsg:
@staticmethod
def create_video_body(video: str, prompt: str = None):
return json.dumps({"video": video, "prompt": prompt})
return json.dumps({"video": video, "prompt": prompt}, ensure_ascii=False)
@staticmethod
def parse_video_body(video_body: str) -> Tuple[str, str]:
@@ -172,7 +172,7 @@ class AgentMsg:
@staticmethod
def create_audio_body(audio: str, prompt: str = None):
return json.dumps({"audio": audio, "prompt": prompt})
return json.dumps({"audio": audio, "prompt": prompt}, ensure_ascii=False)
@staticmethod
def parse_audio_body(audio_body: str) -> Tuple[str, str]:
+12 -11
View File
@@ -60,7 +60,7 @@ class AIFunction:
else:
parameters_str += f"{k} (Optional): {v.description},"
if len(parameters_str) > 0:
return f"{self.get_description} Parameters: {parameters_str}"
return f"{self.get_description()} Parameters: {parameters_str}"
return f"f{self.get_description()}, no parameters"
@abstractmethod
@@ -246,12 +246,16 @@ class SimpleAIFunction(AIFunction):
class AIAction:
@abstractmethod
def get_name(self) -> str:
def get_id(self) -> str:
"""
return the name of the operation (should be snake case)
"""
pass
def get_name(self)->str:
return self.get_id().split('.')[-1].strip()
@abstractmethod
def get_description(self) -> str:
"""
@@ -274,7 +278,7 @@ class SimpleAIAction(AIAction):
self.description = description
self.func_handler = func_handler
def get_name(self) -> str:
def get_id(self) -> str:
return self.op
def get_description(self) -> str:
@@ -289,17 +293,14 @@ class SimpleAIAction(AIAction):
class AIFunction2Action(AIAction):
def __init__(self, func: AIFunction) -> None:
self.func = func
super().__init__()
self.ai_func = func
@abstractmethod
def get_name(self) -> str:
return self.func.get_id()
def get_id(self) -> str:
return self.ai_func.get_id()
@abstractmethod
def get_description(self) -> str:
return self.func.get_detail_description()
return self.ai_func.get_detail_description()
@abstractmethod
async def execute(self, params: dict) -> str:
self.func.execute(**params)
return await self.ai_func.execute(params)
+3 -3
View File
@@ -95,11 +95,11 @@ class LLMPrompt:
def as_str(self)->str:
result_str = ""
if self.system_message:
result_str = json.dumps(self.system_message)
result_str = json.dumps(self.system_message,ensure_ascii=False)
if self.messages:
result_str += json.dumps(self.messages)
result_str += json.dumps(self.messages,ensure_ascii=False)
if self.inner_functions:
result_str += json.dumps(self.inner_functions)
result_str += json.dumps(self.inner_functions,ensure_ascii=False)
return result_str
+1 -1
View File
@@ -92,7 +92,7 @@ class UserConfig:
os.makedirs(directory)
async with aiofiles.open(self.user_config_path,"w") as f:
toml_str = toml.dumps(will_save_config)
toml_str = toml.dumps(will_save_config,ensure_ascii=False)
await f.write(toml_str)
except Exception as e:
logger.error(f"save user config to {self.user_config_path} failed!")
@@ -132,7 +132,7 @@ class MetaDatabase:
create_time = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
summary = metadata.get("summary", "")
catalogs = json.dumps(metadata.get("catalogs", {}))
catalogs = json.dumps(metadata.get("catalogs", {}),ensure_ascii=False)
title = metadata.get("title","")
tags = ','.join(metadata.get("tags", []))
@@ -151,7 +151,7 @@ class MetaDatabase:
title = meta.get("title", "")
summary = meta.get("summary", "")
catalogs = json.dumps(meta.get("catalogs", {}))
catalogs = json.dumps(meta.get("catalogs", {}),ensure_ascii=False)
tags = ','.join(meta.get("tags", []))
cursor.execute('''
@@ -224,7 +224,7 @@ class MetaDatabase:
def query_docs_by_tag(self, tag: str) -> List[str]:
conn = self._get_conn()
cursor = conn.cursor()
tag_json = json.dumps(tag) # 将标签转换为 JSON 字符串
tag_json = json.dumps(tag,ensure_ascii=False) # 将标签转换为 JSON 字符串
cursor.execute('''
SELECT documents.doc_path
FROM documents
@@ -450,7 +450,7 @@ class ParseLocalDocument:
org_path = meta.get("original_path")
known_obj["original_path"] = org_path
return f"# Known information:\n## Current directory structure:\n{kb_tree}\n## Knowlege Metadata:\n{json.dumps(known_obj)}\n"
return f"# Known information:\n## Current directory structure:\n{kb_tree}\n## Knowlege Metadata:\n{json.dumps(known_obj,ensure_ascii=False)}\n"
def _token_len(self, text: str) -> int:
return CustomAIAgent("", "gpt-4-1106-preview", self.token_limit).token_len(text=text)
@@ -563,7 +563,7 @@ class ParseLocalDocument:
if bookmarks:
catalogs = []
self._parse_pdf_bookmarks(bookmarks,catalogs)
metadata["catalogs"] = json.dumps(catalogs)
metadata["catalogs"] = json.dumps(catalogs,ensure_ascii=False)
except Exception as e:
logger.warn("parse pdf bookmarks failed:%s",e)
@@ -55,7 +55,7 @@ class FilesystemEnvironment(SimpleEnvironment):
item_type = "directory" if is_dir else "file"
items.append({"name": entry.name, "type": item_type})
return json.dumps(items)
return json.dumps(items,ensure_ascii=False)
# inner_function
async def read(self,path:str) -> str:
@@ -120,7 +120,7 @@ class FilesystemEnvironment(SimpleEnvironment):
try:
file_path = self.root_path + path
stat = os.stat(file_path)
return json.dumps(stat)
return json.dumps(stat,ensure_ascii=False)
except Exception as e:
return str(e)
+1 -1
View File
@@ -106,7 +106,7 @@ class Issue:
child = desc_list.pop()
root["child"] = child
root = child
return json.dumps(root)
return json.dumps(root,ensure_ascii=False)
@classmethod
+1 -1
View File
@@ -32,7 +32,7 @@ class Mail:
"date": self.date,
"content": self.content
}
return json.dumps(prompt)
return json.dumps(prompt,ensure_ascii=False)
@classmethod
def prompt_desc(cls) -> dict:
+1 -1
View File
@@ -223,7 +223,7 @@ class OpenAI_ComputeNode(ComputeNode):
max_tokens=result_token,
)
else:
logger.info(f"call openai {mode_name} prompts: \n\t {prompts} \nfunctions: \n\t{json.dumps(llm_inner_functions)}")
logger.info(f"call openai {mode_name} prompts: \n\t {prompts} \nfunctions: \n\t{json.dumps(llm_inner_functions,ensure_ascii=False)}")
resp = await client.chat.completions.create(model=mode_name,
messages=prompts,
response_format = response_format,
+4 -4
View File
@@ -156,11 +156,11 @@ class WhisperComputeNode(ComputeNode):
result.result_str = text
result.result = text
elif response_format == "json":
result.result_str = json.dumps({"text": text})
result.result_str = json.dumps({"text": text},ensure_ascii=False)
resp.text = text
result.result = resp
elif response_format == "verbose_json":
result.result_str = json.dumps({"text": text, "segments": results})
result.result_str = json.dumps({"text": text, "segments": results},ensure_ascii=False)
latest_resp.text = text
latest_resp.segments = results
result.result = latest_resp
@@ -190,9 +190,9 @@ class WhisperComputeNode(ComputeNode):
result.set_from_task(task)
result.worker_id = self.node_id
if response_format == "json":
result.result_str = json.dumps({"text": resp.text})
result.result_str = json.dumps({"text": resp.text},ensure_ascii=False)
elif response_format == "verbose_json":
result.result_str = json.dumps({"text": resp.text, "segments": resp.segments})
result.result_str = json.dumps({"text": resp.text, "segments": resp.segments},ensure_ascii=False)
elif response_format == "srt" or response_format == "vtt" or response_format == "text":
result.result_str = resp
else: