Liu Zhicong
GitHub
+3
-1
@@ -9,5 +9,7 @@ history.txt
|
|||||||
aios_shell_history.txt
|
aios_shell_history.txt
|
||||||
math_school_env.db
|
math_school_env.db
|
||||||
workflows.db
|
workflows.db
|
||||||
|
venv-linux
|
||||||
|
venv_test
|
||||||
|
|
||||||
|
rootfs/test_doc/
|
||||||
|
|||||||
+10
@@ -1,4 +1,14 @@
|
|||||||
FROM python:3.11
|
FROM python:3.11
|
||||||
|
|
||||||
|
ENV PYTHON_VERSION=3.11
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
|
python${PYTHON_VERSION}-venv \
|
||||||
|
python${PYTHON_VERSION}-dev \
|
||||||
|
default-libmysqlclient-dev \
|
||||||
|
build-essential \
|
||||||
|
pkg-config \
|
||||||
|
&& rm -rf /var/lib/apt/lists/* \
|
||||||
|
|
||||||
WORKDIR /opt/aios
|
WORKDIR /opt/aios
|
||||||
COPY ./src /opt/aios
|
COPY ./src /opt/aios
|
||||||
COPY ./rootfs /opt/aios/app
|
COPY ./rootfs /opt/aios/app
|
||||||
|
|||||||
@@ -0,0 +1,160 @@
|
|||||||
|
## LLM / AI 相关框架
|
||||||
|
### LLM Process
|
||||||
|
|
||||||
|
LLM调用封装的最小单元，提供了一系列最基础的支持
|
||||||
|
|
||||||
|
流程上 inpurt, prepare promot, llm_function_call_loop, post_llm , llmresult parser, AI Action
|
||||||
|
功能上 动态类型系统 load_from_config,llm_process_loader
|
||||||
|
|
||||||
|
### Agent
|
||||||
|
从Agent的视角定义了Agent的LLM行为逻辑
|
||||||
|
Process behavior (响应)
|
||||||
|
Task/Todo Loop (自主)
|
||||||
|
Self Loop (自省)
|
||||||
|
|
||||||
|
|
||||||
|
#### Agent.Memory
|
||||||
|
Memory模块设计的主要目的是能按一定的模式，在Token Limit的情况下构成Agent的一些上下文。
|
||||||
|
Memory的原始记录写入：原则上说，Agent的任何LLM行为都应至少将input/resp 写入Memory,毕竟LLM是非常高开销的行为。LLM的过程可以视情况写入（inner function的调用，action的执行，根据input构造的完整提示词等）
|
||||||
|
Memory的使用：LLM过程在构造提示词的“已知信息”部分时，通常都会有加载Memory的需求。尤其是在Input并不包含完整信息的情况下（非幂等LLM推理），更需要根据“上下文”来理解Input的含义。这在“ChatCompetition”过程中尤为明显。
|
||||||
|
|
||||||
|
使用Memory基本是两种方式
|
||||||
|
1. 加载写入的原始记录
|
||||||
|
2. 访问根据原始记录加工（Self-Thinking）后的Object-Summary。或则访问一个根据原始记录整理的，用文件系统方式组织的“记忆片段”
|
||||||
|
|
||||||
|
这里的核心痛点是：一个LLM过程，如何根据Input加载合适的Memory成为“已知信息”。
|
||||||
|
1. 如果明确的知道Input属于一个session，那么可以加载这个session相关的所有record(注意token limit和最大条数)与session的summary，其它的Memory的信息通过inner_function访问。但要防止LLM过程中对Inner function的过度使用
|
||||||
|
2. 在不明确的情况下，如何判断input属于哪个session?（包括是否需要创建新的session）
|
||||||
|
|
||||||
|
从上述思考中，得到现在的设计方案：
|
||||||
|
1. 依旧保留Session，且创建session是明确的用户行为。有一些tunnel根本没有创建session的能力。UI可以用 Lite-LLM来进行辅助的session合并（比如类似GMail的Email归集）。系统可以基于session做“已知信息”的自动加载
|
||||||
|
2. 在处理Input时，允许使用Agent.Memory的接口来访问更多的Memory的内容。从流程上看，这个过程和访问KB的原理是基本一致的。
|
||||||
|
3. Self-Thinking的过程中，既要对Session进行整理（得到Session Summary）,也要站在更全局的角度对涉及到的Object进行整理（得到Object Summary）。
|
||||||
|
4. Self-Thinking的过程也是以session为单位的,以更新session-summary为首要目标，并可以在Thiking的过程中，访问已有的object-summary，选择性的更新object-summary
|
||||||
|
5. Self-Thinking会尽量以时间从新到旧处理所有的原始记录，因此会涉及到对多个Session的Summary的更新。
|
||||||
|
|
||||||
|
设计方案的主要风险在于inner function模式可能会带来大量的，无用的object summary查询。
|
||||||
|
|
||||||
|
对于UI上不方便创建Session的情况：
|
||||||
|
1. 通过标题尝试自动创建
|
||||||
|
2. 通过时间尝试自动创建
|
||||||
|
3. 既然用户看到的就是一个session，那么我们就必须当一个session来处理
|
||||||
|
|
||||||
|
|
||||||
|
一些推论：
|
||||||
|
Agent通过一个DB list来访问/写入结构化数据，并拥有自己创建DB的能力
|
||||||
|
Agent通过一个FileSystem来访问/创建非结构化数据，并拥有理解文件系统组织设计的能力
|
||||||
|
“不要给Agent直接扩展能力，而是尽量给Agent扩展元能力（读说明书的能力）”
|
||||||
|
|
||||||
|
#### Agent.Workspace
|
||||||
|
|
||||||
|
#### Agent.behavior
|
||||||
|
|
||||||
|
### Workflow
|
||||||
|
一组Agent共享Work space后的流程
|
||||||
|
Task可以分配给不同的Agent
|
||||||
|
Todo的Do和Check可以分配给不同的Agent
|
||||||
|
|
||||||
|
## Knowledge Base （sisi)
|
||||||
|
AI First的未来文件系统
|
||||||
|
|
||||||
|
## Agent 能力扩展框架
|
||||||
|
|
||||||
|
### AI Function / Action
|
||||||
|
最重要的扩展框架
|
||||||
|
|
||||||
|
### Environment
|
||||||
|
可以通过 {environment.xxx} 读取
|
||||||
|
|
||||||
|
### Code Interpreter
|
||||||
|
Agent 能不能写代码是一个重要的理念之争
|
||||||
|
能写代码的Agent想象空间大，是通往AGI的必然之路，但不够稳定可预期
|
||||||
|
不能写代码的Agent可以专注于组合使用基础的能力，稳定可靠的
|
||||||
|
|
||||||
|
## AI系统组件
|
||||||
|
### AI Compute Kernel
|
||||||
|
通过AI Compute Kernel对 LLM, AIGC等新一代的AI基础能力进行抽象
|
||||||
|
通过Compute Node可以对这些基础能力进行不同的实现
|
||||||
|
|
||||||
|
### AI Models
|
||||||
|
模型的fine-tune Pipeline
|
||||||
|
LoRA的Pipeline
|
||||||
|
|
||||||
|
### Contact Manage
|
||||||
|
|
||||||
|
基于Contact的自然语言权限控制
|
||||||
|
|
||||||
|
### Tunnel
|
||||||
|
可以使用开放API的通信软件，于自己的AI时刻保持沟通
|
||||||
|
|
||||||
|
### Spider
|
||||||
|
持续的导入用户在旧时代的数据。
|
||||||
|
从Web2->web3
|
||||||
|
|
||||||
|
### Calendar (Calendar是否应该是Agent.Worksapce的一部分）
|
||||||
|
|
||||||
|
|
||||||
|
### 基础的pkg_loader
|
||||||
|
支持一系列可安装的扩展
|
||||||
|
可扩展的扩展是AIOS的开发者需要重点关注的
|
||||||
|
|
||||||
|
Agent （用自然语言扩展）
|
||||||
|
Workflow （用自然语言扩展）
|
||||||
|
Plugin:(需要会写代码）
|
||||||
|
AI Function / Action
|
||||||
|
Environment
|
||||||
|
Knowledge Pipeline
|
||||||
|
LLM Process
|
||||||
|
Compute Node
|
||||||
|
|
||||||
|
### System Config Manage
|
||||||
|
|
||||||
|
Zone Config-> System Config
|
||||||
|
|
||||||
|
## UI
|
||||||
|
|
||||||
|
### Installer
|
||||||
|
图形化的安装界面，帮助用户能快速的安装使用
|
||||||
|
我们也会在这里讨论面向用户的AIOS的过渡性安装逻辑
|
||||||
|
|
||||||
|
### WebUI & OS Desktop
|
||||||
|
系统控制面板
|
||||||
|
Agent/Workflow管理
|
||||||
|
新Outlook
|
||||||
|
会话管理 （于Agent会话）
|
||||||
|
日程管理
|
||||||
|
Todo管理
|
||||||
|
|
||||||
|
新Dropbox
|
||||||
|
Knowledge Base浏览
|
||||||
|
Knowledge Base查询
|
||||||
|
|
||||||
|
应用商店
|
||||||
|
|
||||||
|
### AIOS Shell
|
||||||
|
|
||||||
|
### Personal Station (新个人主页）
|
||||||
|
内容的发布/联系人内容的查看/个人日历的公开
|
||||||
|
|
||||||
|
## Frame Service （完全未开始）
|
||||||
|
通过Frame Service，让AIOS成为一个典型的网络系统（Personal Server OS）
|
||||||
|
这一块会复用很多CYFS/Bucky OS 的基础设计
|
||||||
|
这一层AI不会直接使用，这一层支持AI系统组件的实现
|
||||||
|
在用户看来，这一层的功能都是高级的，偏向系统维护的。很少会直接使用
|
||||||
|
|
||||||
|
### zone & node-daemon
|
||||||
|
NOS的booter
|
||||||
|
|
||||||
|
### Runtime (Container) Manage
|
||||||
|
这里抽象了系统的运行时模型
|
||||||
|
通过容器技术对可扩展组件的权限进行控制，保护系统的隐私安全
|
||||||
|
|
||||||
|
### d-Storage & Named Object
|
||||||
|
Named- Object File System
|
||||||
|
D-RDB
|
||||||
|
D-VDB
|
||||||
|
|
||||||
|
### BUS
|
||||||
|
系统消息总线，在不同的系统组件中路由消息
|
||||||
|
|
||||||
|
### CYFS (httpv4) Gateway
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
# LLMProcess
|
||||||
|
|
||||||
|
设计目的是理解到 提示词=>LLM=>LLM Result 的过程是系统的核心复杂度。Agent的粒度太大了，需要更合适的设计来封装这个复杂度，并给予这个过程更大的灵活性和可组合型。并更易于构建测试
|
||||||
|
|
||||||
|
比如
|
||||||
|
1. 可以很容易的组合两个已知的LLM Process（上一个的输出是下一个的输入），这个设计有一点类似LangChain (我们在正式系统中，肯定允许整个Agent都用LangChain来构建)
|
||||||
|
2. 可以用用一个LLM Process来构建另一个LLM Process的Prompt
|
||||||
|
3. 继承一个复杂的LLM Process，进行简单配置，就可以得到一个新的LLM Process。这个新的LLM Process可以享受到复杂LLM Process持续迭代的好处
|
||||||
|
4. 有一些常用的，系统内置的LLM Process可以从配置文件中加载。
|
||||||
|
|
||||||
|
```python
|
||||||
|
def agent.on_process_message():
|
||||||
|
llm_process = self.on_message_llm_process.clone()
|
||||||
|
llm_result = llm_process.do()
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
```
|
||||||
@@ -1,4 +1,5 @@
|
|||||||
# OpenDAN Quick Start
|
# OpenDAN Quick Start
|
||||||
|
|
||||||
OpenDAN (Open and Do Anything Now with AI) is revolutionizing the AI landscape with its Personal AI Operating System. Designed for seamless integration of diverse AI modules, it ensures unmatched interoperability. OpenDAN empowers users to craft powerful AI agents—from butlers and assistants to personal tutors and digital companions—all while retaining control. These agents can team up to tackle complex challenges, integrate with existing services, and command smart(IoT) devices.
|
OpenDAN (Open and Do Anything Now with AI) is revolutionizing the AI landscape with its Personal AI Operating System. Designed for seamless integration of diverse AI modules, it ensures unmatched interoperability. OpenDAN empowers users to craft powerful AI agents—from butlers and assistants to personal tutors and digital companions—all while retaining control. These agents can team up to tackle complex challenges, integrate with existing services, and command smart(IoT) devices.
|
||||||
|
|
||||||
With OpenDAN, we're putting AI in your hands, making life simpler and smarter.
|
With OpenDAN, we're putting AI in your hands, making life simpler and smarter.
|
||||||
|
|||||||
@@ -0,0 +1,989 @@
|
|||||||
|
<mxfile host="Electron" modified="2024-01-29T00:22:46.220Z" agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/22.1.2 Chrome/114.0.5735.289 Electron/25.9.4 Safari/537.36" etag="XPCvDF8x_5abdhn_GXEm" version="22.1.2" type="device" pages="8">
|
||||||
|
<diagram id="C5RBs43oDa-KdzZeNtuy" name="Page-1">
|
||||||
|
<mxGraphModel dx="2044" dy="1167" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="827" pageHeight="1169" math="0" shadow="0">
|
||||||
|
<root>
|
||||||
|
<mxCell id="WIyWlLk6GJQsqaUBKTNV-0" />
|
||||||
|
<mxCell id="WIyWlLk6GJQsqaUBKTNV-1" parent="WIyWlLk6GJQsqaUBKTNV-0" />
|
||||||
|
<mxCell id="WIyWlLk6GJQsqaUBKTNV-3" value="(chat) message" style="rounded=1;whiteSpace=wrap;html=1;fontSize=12;glass=0;strokeWidth=1;shadow=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||||
|
<mxGeometry x="159" y="330" width="90" height="40" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-0" value="" style="endArrow=classic;html=1;rounded=0;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="WIyWlLk6GJQsqaUBKTNV-3" target="RbYReh6FQah4Kl5oFyjJ-1" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="279" y="390" as="sourcePoint" />
|
||||||
|
<mxPoint x="309" y="350" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-8" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-1" target="RbYReh6FQah4Kl5oFyjJ-2" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-14" value="message" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-1" target="RbYReh6FQah4Kl5oFyjJ-13" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-1" value="Workflow" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#6d8764;fontColor=#ffffff;strokeColor=#3A5431;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||||
|
<mxGeometry x="304" y="320" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-2" value="Functions" style="shape=hexagon;perimeter=hexagonPerimeter2;whiteSpace=wrap;html=1;fixedSize=1;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||||
|
<mxGeometry x="514" y="320" width="130" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-6" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-3" target="RbYReh6FQah4Kl5oFyjJ-4" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-3" value="(phicyal / virtual)<br>environment" style="shape=cube;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;darkOpacity=0.05;darkOpacity2=0.1;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||||
|
<mxGeometry x="304" y="120" width="120" height="80" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-7" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-4" target="RbYReh6FQah4Kl5oFyjJ-1" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-4" value="(event) message" style="rounded=1;whiteSpace=wrap;html=1;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||||
|
<mxGeometry x="304" y="240" width="120" height="40" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-9" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=0;entryY=0;entryDx=120;entryDy=50;entryPerimeter=0;dashed=1;dashPattern=8 8;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-2" target="RbYReh6FQah4Kl5oFyjJ-3" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="564" y="290" as="sourcePoint" />
|
||||||
|
<mxPoint x="614" y="240" as="targetPoint" />
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="579" y="170" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-10" value="" style="endArrow=classic;html=1;rounded=0;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0.16;entryY=0.55;entryDx=0;entryDy=0;entryPerimeter=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-2" target="RbYReh6FQah4Kl5oFyjJ-11" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="684" y="320" as="sourcePoint" />
|
||||||
|
<mxPoint x="714" y="350" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-11" value="Exist Service" style="ellipse;shape=cloud;whiteSpace=wrap;html=1;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||||
|
<mxGeometry x="689" y="300" width="130" height="90" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-13" value="sub / other workflow" style="whiteSpace=wrap;html=1;rounded=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||||
|
<mxGeometry x="304" y="480" width="120" height="50" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-15" value="response message" style="endArrow=classic;html=1;rounded=0;exitX=0.25;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-1" target="RbYReh6FQah4Kl5oFyjJ-17" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="129" y="480" as="sourcePoint" />
|
||||||
|
<mxPoint x="99" y="420" as="targetPoint" />
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="334" y="420" />
|
||||||
|
<mxPoint x="49" y="420" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-18" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-17" target="WIyWlLk6GJQsqaUBKTNV-3" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-17" value="Personal" style="ellipse;whiteSpace=wrap;html=1;aspect=fixed;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||||
|
<mxGeometry x="9" y="310" width="80" height="80" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-19" value="" style="shape=xor;whiteSpace=wrap;html=1;rotation=-90;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||||
|
<mxGeometry x="309" y="67" width="20" height="20" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-21" value="" style="shape=xor;whiteSpace=wrap;html=1;rotation=-90;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||||
|
<mxGeometry x="347" y="67" width="20" height="20" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-22" value="" style="shape=xor;whiteSpace=wrap;html=1;rotation=-90;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||||
|
<mxGeometry x="386" y="67" width="20" height="20" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-23" value="IoT Devices" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||||
|
<mxGeometry x="240" y="67" width="60" height="30" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-25" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.25;exitY=0.5;exitDx=0;exitDy=0;exitPerimeter=0;entryX=0.122;entryY=0.008;entryDx=0;entryDy=0;entryPerimeter=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-19" target="RbYReh6FQah4Kl5oFyjJ-3" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="240" y="170" as="sourcePoint" />
|
||||||
|
<mxPoint x="290" y="120" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-26" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.25;exitY=0.5;exitDx=0;exitDy=0;exitPerimeter=0;entryX=0;entryY=0;entryDx=50;entryDy=0;entryPerimeter=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-21" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="443" y="120" as="sourcePoint" />
|
||||||
|
<mxPoint x="357" y="120.00000000000011" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="RbYReh6FQah4Kl5oFyjJ-27" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.25;exitY=0.5;exitDx=0;exitDy=0;exitPerimeter=0;entryX=0.766;entryY=0.02;entryDx=0;entryDy=0;entryPerimeter=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-22" target="RbYReh6FQah4Kl5oFyjJ-3" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="430" y="140" as="sourcePoint" />
|
||||||
|
<mxPoint x="480" y="90" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="HvRWRVSjofeyYJQyU3aa-0" value="Sub workflow is not visiable for others<br><br>Sub worflow can share roles&nbsp;" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||||
|
<mxGeometry x="160" y="480" width="140" height="70" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="HvRWRVSjofeyYJQyU3aa-1" value="Chat with AI Agent is like chat with a workflow which have only one agent and no rules" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||||
|
<mxGeometry x="70" y="150" width="150" height="80" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="ed6Jjq-fMpksm2tlv9Oi-0" value="inner&nbsp;<br>environment" style="shape=cube;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;darkOpacity=0.05;darkOpacity2=0.1;size=10;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||||
|
<mxGeometry x="386" y="360" width="84" height="40" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="ed6Jjq-fMpksm2tlv9Oi-1" value="Environment could shared with workflows" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||||
|
<mxGeometry x="440" y="120" width="120" height="30" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="ed6Jjq-fMpksm2tlv9Oi-2" value="Inner environment&nbsp; could be used as Context(FileSystem)" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||||
|
<mxGeometry x="413" y="410" width="110" height="30" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
</root>
|
||||||
|
</mxGraphModel>
|
||||||
|
</diagram>
|
||||||
|
<diagram id="kWxfmPxtNxOAf0a73TCG" name="Workflow">
|
||||||
|
<mxGraphModel dx="2046" dy="1168" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="850" pageHeight="1100" math="0" shadow="0">
|
||||||
|
<root>
|
||||||
|
<mxCell id="0" />
|
||||||
|
<mxCell id="1" parent="0" />
|
||||||
|
<mxCell id="VGzDL9xesOOnN_l_G2Ij-1" value="<h1>Workflow rules</h1><p>workflow rules(prompt)</p>" style="text;html=1;strokeColor=none;fillColor=none;spacing=5;spacingTop=-20;whiteSpace=wrap;overflow=hidden;rounded=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="70" y="800" width="190" height="120" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="VGzDL9xesOOnN_l_G2Ij-2" value="<h1>Workflow roles</h1><p>AI Agent groups</p><p>Project Manager : Alex</p><p>Programer : Bobl</p><p><br></p>" style="text;html=1;strokeColor=none;fillColor=none;spacing=5;spacingTop=-20;whiteSpace=wrap;overflow=hidden;rounded=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="320" y="790" width="190" height="120" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="41KxsEdFswCCPQnavNoU-14" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.25;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="VGzDL9xesOOnN_l_G2Ij-3" target="VGzDL9xesOOnN_l_G2Ij-4" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="41KxsEdFswCCPQnavNoU-15" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.75;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="VGzDL9xesOOnN_l_G2Ij-3" target="VGzDL9xesOOnN_l_G2Ij-7" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="VGzDL9xesOOnN_l_G2Ij-3" value="Message" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="340" y="170" width="100" height="40" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="VGzDL9xesOOnN_l_G2Ij-4" value="inner filter&nbsp;" style="shape=hexagon;perimeter=hexagonPerimeter2;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="150" y="260" width="120" height="50" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="VGzDL9xesOOnN_l_G2Ij-5" value="roleA" style="ellipse;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="80" y="350" width="110" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="41KxsEdFswCCPQnavNoU-12" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="VGzDL9xesOOnN_l_G2Ij-6" target="VGzDL9xesOOnN_l_G2Ij-8" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="VGzDL9xesOOnN_l_G2Ij-6" value="roleB" style="ellipse;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="240" y="350" width="110" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="41KxsEdFswCCPQnavNoU-4" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;" parent="1" source="VGzDL9xesOOnN_l_G2Ij-7" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<mxPoint x="570.0344827586209" y="350" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="VGzDL9xesOOnN_l_G2Ij-7" value="Workflow rules" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="520" y="260" width="100" height="50" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="VGzDL9xesOOnN_l_G2Ij-8" value="Final Result of Message" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="340" y="590" width="100" height="50" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="41KxsEdFswCCPQnavNoU-11" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=1;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="VGzDL9xesOOnN_l_G2Ij-9" target="VGzDL9xesOOnN_l_G2Ij-8" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="VGzDL9xesOOnN_l_G2Ij-9" value="Result Merge" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="521" y="530" width="100" height="40" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="41KxsEdFswCCPQnavNoU-9" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="VGzDL9xesOOnN_l_G2Ij-10" target="VGzDL9xesOOnN_l_G2Ij-9" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="VGzDL9xesOOnN_l_G2Ij-10" value="roleC" style="ellipse;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="435" y="350" width="110" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="41KxsEdFswCCPQnavNoU-8" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="VGzDL9xesOOnN_l_G2Ij-11" target="41KxsEdFswCCPQnavNoU-7" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="VGzDL9xesOOnN_l_G2Ij-11" value="roleD" style="ellipse;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="595" y="350" width="110" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="VGzDL9xesOOnN_l_G2Ij-12" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=1;entryY=0;entryDx=0;entryDy=0;" parent="1" source="VGzDL9xesOOnN_l_G2Ij-4" target="VGzDL9xesOOnN_l_G2Ij-5" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="160" y="510" as="sourcePoint" />
|
||||||
|
<mxPoint x="210" y="460" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="VGzDL9xesOOnN_l_G2Ij-13" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0;entryY=0;entryDx=0;entryDy=0;" parent="1" source="VGzDL9xesOOnN_l_G2Ij-4" target="VGzDL9xesOOnN_l_G2Ij-6" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="210" y="510" as="sourcePoint" />
|
||||||
|
<mxPoint x="260" y="460" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="41KxsEdFswCCPQnavNoU-1" value="" style="shape=cross;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="556" y="364" width="30" height="30" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="41KxsEdFswCCPQnavNoU-10" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=1;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="41KxsEdFswCCPQnavNoU-7" target="VGzDL9xesOOnN_l_G2Ij-9" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="41KxsEdFswCCPQnavNoU-7" value="call function or&nbsp;<br>send message" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="592.5" y="460" width="115" height="30" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="q_eHDVuD3DBh6_VEKgap-2" value="message process by filter,select one role do LLM and get RESULT of input" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="90" y="190" width="165" height="30" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="q_eHDVuD3DBh6_VEKgap-3" value="message process by rules,select mutil role do LLM and merge RESULT of input" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="510" y="180" width="165" height="30" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="q_eHDVuD3DBh6_VEKgap-5" value="如果一个工作流有3个role参与，每个人都需要调用函数并分析结果，那么要进行至少6次推理" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="483" y="640" width="174" height="40" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="q_eHDVuD3DBh6_VEKgap-6" value="这里是基于规则的合并，给另一个Role合并是多输入Agent流程" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="635" y="570" width="160" height="30" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
</root>
|
||||||
|
</mxGraphModel>
|
||||||
|
</diagram>
|
||||||
|
<diagram id="7NYJTgo0U9cdVshLy85U" name="Page-3">
|
||||||
|
<mxGraphModel dx="2046" dy="1168" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="850" pageHeight="1100" math="0" shadow="0">
|
||||||
|
<root>
|
||||||
|
<mxCell id="0" />
|
||||||
|
<mxCell id="1" parent="0" />
|
||||||
|
<mxCell id="zJVYyOSlNiCVBlA5jEf0-2" value="Agent Sessions<br>Data" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;fontSize=11;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="110" y="533" width="80" height="70" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="8StUH8aFnwlRlHm0zShF-2" value="Small LLM, Train With Personal Data" style="shape=process;whiteSpace=wrap;html=1;backgroundOutline=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="270" y="258" width="140" height="50" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="awKuOLItvF_EOGf5wS2x-4" value="" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="20" y="78" width="160" height="360" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="S5MjekdSblNa5itYb4Ee-13" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" source="8StUH8aFnwlRlHm0zShF-1" target="SPnvrlBiOLhgEaWxmdK3-4" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="8StUH8aFnwlRlHm0zShF-1" value="LLM<br>(Common Sence)" style="shape=process;whiteSpace=wrap;html=1;backgroundOutline=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="250" y="218" width="140" height="50" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="8StUH8aFnwlRlHm0zShF-3" value="ControlNet<br>base on Agent Memory" style="shape=process;whiteSpace=wrap;html=1;backgroundOutline=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="250" y="108" width="140" height="50" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="8StUH8aFnwlRlHm0zShF-4" value="Agent人格助记词" style="shape=document;whiteSpace=wrap;html=1;boundedLbl=1;size=0.25;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="40" y="108" width="120" height="50" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="8StUH8aFnwlRlHm0zShF-5" value="Role助记词" style="shape=document;whiteSpace=wrap;html=1;boundedLbl=1;size=0.25;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="40" y="168" width="120" height="50" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="8StUH8aFnwlRlHm0zShF-6" value="Workflow Rules助记词" style="shape=document;whiteSpace=wrap;html=1;boundedLbl=1;size=0.25;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="40" y="235.5" width="120" height="52.5" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="8StUH8aFnwlRlHm0zShF-7" value="访问Knowlege构造助记词" style="shape=document;whiteSpace=wrap;html=1;boundedLbl=1;size=0.25;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="40" y="378" width="120" height="50" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Tz-ThZOieDqqDBRXvXnh-1" value="Invoke the function and wait for the necessary response." style="shape=hexagon;perimeter=hexagonPerimeter2;whiteSpace=wrap;html=1;fixedSize=1;fontSize=11;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="600" y="33" width="110" height="70" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="awKuOLItvF_EOGf5wS2x-2" value="Send Message and wait for the necessary response." style="rounded=1;whiteSpace=wrap;html=1;fontSize=11;glass=0;strokeWidth=1;shadow=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="610" y="153" width="90" height="70" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="awKuOLItvF_EOGf5wS2x-5" value="Input Prompts" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="60" y="78" width="80" height="30" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="awKuOLItvF_EOGf5wS2x-7" value="Responce Prompts" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="740" y="103" width="80" height="40" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="S5MjekdSblNa5itYb4Ee-1" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=1;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="awKuOLItvF_EOGf5wS2x-8" target="awKuOLItvF_EOGf5wS2x-9" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="awKuOLItvF_EOGf5wS2x-8" value="LLM Competion" style="shape=process;whiteSpace=wrap;html=1;backgroundOutline=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="720" y="226.75" width="120" height="40" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="SPnvrlBiOLhgEaWxmdK3-9" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" source="awKuOLItvF_EOGf5wS2x-9" target="SPnvrlBiOLhgEaWxmdK3-1" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="awKuOLItvF_EOGf5wS2x-9" value="Final Result<br>Of Message<br>(can be None)" style="ellipse;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="450" y="363" width="120" height="80" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="SPnvrlBiOLhgEaWxmdK3-1" value="Storage Result in agent session and workflow session" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="450" y="483" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="SPnvrlBiOLhgEaWxmdK3-2" value="可用Function助记词" style="shape=document;whiteSpace=wrap;html=1;boundedLbl=1;size=0.25;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="40" y="308" width="120" height="50" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="SPnvrlBiOLhgEaWxmdK3-7" value="Yes" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" source="SPnvrlBiOLhgEaWxmdK3-4" target="SPnvrlBiOLhgEaWxmdK3-6" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="SPnvrlBiOLhgEaWxmdK3-8" value="No" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" source="SPnvrlBiOLhgEaWxmdK3-4" target="awKuOLItvF_EOGf5wS2x-9" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="SPnvrlBiOLhgEaWxmdK3-4" value="推理结果<br>需要进一步处理？" style="rhombus;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="440" y="203" width="140" height="80" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="SPnvrlBiOLhgEaWxmdK3-6" value="Call Funstion or send message?" style="rhombus;whiteSpace=wrap;html=1;fontSize=11;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="440" y="78" width="140" height="80" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="S5MjekdSblNa5itYb4Ee-3" value="Call" style="endArrow=classic;html=1;rounded=0;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="SPnvrlBiOLhgEaWxmdK3-6" target="Tz-ThZOieDqqDBRXvXnh-1" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="410" y="113" as="sourcePoint" />
|
||||||
|
<mxPoint x="460" y="63" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="S5MjekdSblNa5itYb4Ee-4" value="Send" style="endArrow=classic;html=1;rounded=0;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="SPnvrlBiOLhgEaWxmdK3-6" target="awKuOLItvF_EOGf5wS2x-2" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="560" y="193" as="sourcePoint" />
|
||||||
|
<mxPoint x="610" y="143" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="S5MjekdSblNa5itYb4Ee-5" value="" style="endArrow=classic;html=1;rounded=0;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="awKuOLItvF_EOGf5wS2x-2" target="awKuOLItvF_EOGf5wS2x-7" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="720" y="213" as="sourcePoint" />
|
||||||
|
<mxPoint x="770" y="163" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="S5MjekdSblNa5itYb4Ee-6" value="" style="endArrow=classic;html=1;rounded=0;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="Tz-ThZOieDqqDBRXvXnh-1" target="awKuOLItvF_EOGf5wS2x-7" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="750" y="93" as="sourcePoint" />
|
||||||
|
<mxPoint x="800" y="43" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="S5MjekdSblNa5itYb4Ee-7" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="awKuOLItvF_EOGf5wS2x-7" target="awKuOLItvF_EOGf5wS2x-8" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="750" y="213" as="sourcePoint" />
|
||||||
|
<mxPoint x="800" y="163" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="S5MjekdSblNa5itYb4Ee-10" value="" style="endArrow=none;dashed=1;html=1;dashPattern=1 3;strokeWidth=2;rounded=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;exitX=0.5;exitY=0;exitDx=0;exitDy=0;" parent="1" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="330" y="218" as="sourcePoint" />
|
||||||
|
<mxPoint x="330" y="158" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="S5MjekdSblNa5itYb4Ee-12" value="" style="shape=flexArrow;endArrow=classic;html=1;rounded=0;fillColor=#6d8764;strokeColor=#3A5431;" parent="1" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="200" y="251.75" as="sourcePoint" />
|
||||||
|
<mxPoint x="240" y="252" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="S5MjekdSblNa5itYb4Ee-15" value="Real Time Enviroment Knowlege" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;fontSize=11;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="10" y="533" width="80" height="70" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="zJVYyOSlNiCVBlA5jEf0-1" value="Workflow Context Data" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;fontSize=11;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="110" y="473" width="80" height="70" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="S5MjekdSblNa5itYb4Ee-14" value="OOD's KnowlegeBase" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;fontSize=11;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="10" y="473" width="80" height="70" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="zJVYyOSlNiCVBlA5jEf0-3" value="" style="shape=flexArrow;endArrow=classic;html=1;rounded=0;fillColor=#647687;strokeColor=#314354;" parent="1" target="8StUH8aFnwlRlHm0zShF-7" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="100" y="463" as="sourcePoint" />
|
||||||
|
<mxPoint x="130" y="433" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="iPE2hx-tEKFiySbPAp22-1" value="<font style="font-size: 11px;">可用Function受到人格和Workflow的共同影响</font>" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="190" y="318" width="130" height="30" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="iPE2hx-tEKFiySbPAp22-2" value="" style="endArrow=none;dashed=1;html=1;dashPattern=1 3;strokeWidth=2;rounded=0;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="SPnvrlBiOLhgEaWxmdK3-2" target="iPE2hx-tEKFiySbPAp22-1" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="230" y="433" as="sourcePoint" />
|
||||||
|
<mxPoint x="280" y="383" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
</root>
|
||||||
|
</mxGraphModel>
|
||||||
|
</diagram>
|
||||||
|
<diagram id="3kOHILldrhYu_NusAgTt" name="LLM Process">
|
||||||
|
<mxGraphModel dx="1421" dy="816" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="850" pageHeight="1100" math="0" shadow="0">
|
||||||
|
<root>
|
||||||
|
<mxCell id="0" />
|
||||||
|
<mxCell id="1" parent="0" />
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-13" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="FX_S_WlUCK74q34TItPC-1" target="FX_S_WlUCK74q34TItPC-10" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-1" value="LLM" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="309.97" y="310" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-12" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="FX_S_WlUCK74q34TItPC-3" target="FX_S_WlUCK74q34TItPC-1" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-3" value="Request<br>(听指令)" style="ellipse;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="99.97" y="310" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-18" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;exitPerimeter=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="FX_S_WlUCK74q34TItPC-4" target="FX_S_WlUCK74q34TItPC-5" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-4" value="Workspace<br>(workflow share state)" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="309.97" y="100" width="120" height="80" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-19" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="FX_S_WlUCK74q34TItPC-5" target="FX_S_WlUCK74q34TItPC-1" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="369.97" y="280" />
|
||||||
|
<mxPoint x="369.97" y="280" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-5" value="Context Prepare<br>(Prompt Generator)" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="309.97" y="220" width="120" height="50" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-14" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="FX_S_WlUCK74q34TItPC-10" target="FX_S_WlUCK74q34TItPC-11" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-16" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;" parent="1" source="FX_S_WlUCK74q34TItPC-10" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<mxPoint x="559.97" y="430" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-10" value="LLM Result" style="shape=process;whiteSpace=wrap;html=1;backgroundOutline=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="499.97" y="310" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-11" value="Resp/Say" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="699.97" y="310" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-17" value="OP / Action List<br>(动手)" style="shape=document;whiteSpace=wrap;html=1;boundedLbl=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="499.97" y="430" width="120" height="80" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-24" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;exitPerimeter=0;entryX=0.75;entryY=0;entryDx=0;entryDy=0;" parent="1" source="FX_S_WlUCK74q34TItPC-20" target="FX_S_WlUCK74q34TItPC-5" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-20" value="
<span style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: center; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(251, 251, 251); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; float: none; display: inline !important;">Memory</span><br style="border-color: var(--border-color); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: center; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(251, 251, 251); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><span style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: center; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(251, 251, 251); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; float: none; display: inline !important;">（Chatsession,worklog）</span>

" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="479.97" y="100" width="140" height="80" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-23" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;exitPerimeter=0;entryX=0.25;entryY=0;entryDx=0;entryDy=0;" parent="1" source="FX_S_WlUCK74q34TItPC-21" target="FX_S_WlUCK74q34TItPC-5" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-21" value="Knowledge" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="99.97" y="100" width="150" height="80" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-26" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.75;entryDx=0;entryDy=0;" parent="1" source="FX_S_WlUCK74q34TItPC-25" target="FX_S_WlUCK74q34TItPC-1" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-25" value="Obverse<br>(感受)" style="ellipse;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="99.97" y="390" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-30" value="必定改变，Agent可以配置改变模式？" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.855;entryY=1;entryDx=0;entryDy=-4.35;entryPerimeter=0;" parent="1" source="FX_S_WlUCK74q34TItPC-10" target="FX_S_WlUCK74q34TItPC-20" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-33" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.5;entryY=0;entryDx=0;entryDy=0;entryPerimeter=0;" parent="1" source="FX_S_WlUCK74q34TItPC-17" target="FX_S_WlUCK74q34TItPC-4" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="49.97" y="480" />
|
||||||
|
<mxPoint x="49.97" y="80" />
|
||||||
|
<mxPoint x="369.97" y="80" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-34" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0;entryDx=0;entryDy=52.5;entryPerimeter=0;" parent="1" source="FX_S_WlUCK74q34TItPC-17" target="FX_S_WlUCK74q34TItPC-21" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-36" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.25;entryDx=0;entryDy=0;" parent="1" source="FX_S_WlUCK74q34TItPC-35" target="FX_S_WlUCK74q34TItPC-1" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="FX_S_WlUCK74q34TItPC-35" value="自驱<br>（自我实现）" style="ellipse;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="99.97" y="230" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="mAWNT_NJkWkCqDQJkoOU-1" value="<b><font style="font-size: 15px;">对LLM Inner Function Call的理解：</font></b><br>1. LLM Function Call 的使用，大多数情况下是Context Prepare的一部分<br><br>2. 一种在产生Resp前，对OP / Action的结果的立刻检查。不过考虑到Token成本，可能用另一个更简单的LLM 过程来完成检查通常会更合适<br>" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=11;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="50" y="500" width="360.03" height="180" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="mAWNT_NJkWkCqDQJkoOU-2" value="LLM Process&nbsp;" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;fontSize=16;fontStyle=1" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="345" y="30" width="160" height="30" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="mAWNT_NJkWkCqDQJkoOU-8" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="mAWNT_NJkWkCqDQJkoOU-3" target="mAWNT_NJkWkCqDQJkoOU-5" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="mAWNT_NJkWkCqDQJkoOU-3" value="Main Process" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="365" y="880" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="mAWNT_NJkWkCqDQJkoOU-4" value="Prepare Process" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="180" y="880" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="mAWNT_NJkWkCqDQJkoOU-9" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="mAWNT_NJkWkCqDQJkoOU-5" target="mAWNT_NJkWkCqDQJkoOU-10" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<mxPoint x="720" y="910.2857142857142" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="mAWNT_NJkWkCqDQJkoOU-13" value="Chek Failed" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="mAWNT_NJkWkCqDQJkoOU-5" target="mAWNT_NJkWkCqDQJkoOU-4" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="610" y="850" />
|
||||||
|
<mxPoint x="240" y="850" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="mAWNT_NJkWkCqDQJkoOU-5" value="Check Process" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="550" y="880" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="mAWNT_NJkWkCqDQJkoOU-6" value="" style="endArrow=classic;html=1;rounded=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" target="mAWNT_NJkWkCqDQJkoOU-4" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="130" y="910" as="sourcePoint" />
|
||||||
|
<mxPoint x="160" y="890" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="mAWNT_NJkWkCqDQJkoOU-7" value="" style="endArrow=classic;html=1;rounded=0;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="mAWNT_NJkWkCqDQJkoOU-4" target="mAWNT_NJkWkCqDQJkoOU-3" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="330" y="860" as="sourcePoint" />
|
||||||
|
<mxPoint x="380" y="810" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="mAWNT_NJkWkCqDQJkoOU-10" value="Resp/Say" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="719.97" y="880" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="mAWNT_NJkWkCqDQJkoOU-11" value="Request<br>(听指令)" style="ellipse;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="10" y="880" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="mAWNT_NJkWkCqDQJkoOU-14" value="Prepare过程可以是固定的计算组合，也可以是LLM参与的" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="380" y="270" width="165" height="30" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="f2v0L3gmEFmQxlVvJ0Yg-1" value="<h1><font style="font-size: 22px;">Memory,Env,KB,Workspace的区别</font></h1><p>Memory是不共享的内部状态，只有 action没有function<br style="border-color: var(--border-color); font-size: 11px;"><span style="font-size: 11px;">Workspace是共享的状态，主要是action</span><br style="border-color: var(--border-color); font-size: 11px;"><span style="font-size: 11px;">Env是全局状态，主要是inner_function,有的有小部分action</span><br style="border-color: var(--border-color); font-size: 11px;"><span style="font-size: 11px;">Knowledge基本是只读的，除非专门整理Knowledge的Agent,只有function</span><br><br></p>" style="text;html=1;strokeColor=none;fillColor=none;spacing=5;spacingTop=-20;whiteSpace=wrap;overflow=hidden;rounded=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="449.97" y="530" width="390" height="230" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
</root>
|
||||||
|
</mxGraphModel>
|
||||||
|
</diagram>
|
||||||
|
<diagram id="CqH4P1G_RAjWfIoYkaMU" name="Task/Todo">
|
||||||
|
<mxGraphModel dx="2044" dy="1167" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="850" pageHeight="1100" math="0" shadow="0">
|
||||||
|
<root>
|
||||||
|
<mxCell id="0" />
|
||||||
|
<mxCell id="1" parent="0" />
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-14" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="Po7mxAQLMo4zYcBtAZ7U-1" target="Po7mxAQLMo4zYcBtAZ7U-5" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-1" value="未开始的Task" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="380" y="40" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-19" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="Po7mxAQLMo4zYcBtAZ7U-2" target="Po7mxAQLMo4zYcBtAZ7U-8" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-2" value="确定的Task：<br>确定的执行时间（在这之前和之后都不会执行），<br>确定的执行人" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="570" y="250" width="120" height="140" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-15" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.25;exitY=1;exitDx=0;exitDy=0;" parent="1" source="Po7mxAQLMo4zYcBtAZ7U-5" target="Po7mxAQLMo4zYcBtAZ7U-7" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-18" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.75;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="Po7mxAQLMo4zYcBtAZ7U-5" target="Po7mxAQLMo4zYcBtAZ7U-2" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-32" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="Po7mxAQLMo4zYcBtAZ7U-5" target="Po7mxAQLMo4zYcBtAZ7U-31" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-5" value="Task Review" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="380" y="140" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-17" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="Po7mxAQLMo4zYcBtAZ7U-6" target="Po7mxAQLMo4zYcBtAZ7U-2" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-6" value="Quick Review" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="380" y="290" width="130" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-16" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;" parent="1" source="Po7mxAQLMo4zYcBtAZ7U-7" target="Po7mxAQLMo4zYcBtAZ7U-6" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-7" value="半确定的Task：<br>ASSIGNED<br>确定的执行人<br>不确定的执行时间（和条件有关）" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="210" y="250" width="120" height="140" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-20" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="Po7mxAQLMo4zYcBtAZ7U-8" target="Po7mxAQLMo4zYcBtAZ7U-9" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-8" value="Task Review" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="570" y="430" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-21" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="Po7mxAQLMo4zYcBtAZ7U-9" target="Po7mxAQLMo4zYcBtAZ7U-10" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-9" value="未执行Todo" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="570" y="530" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-22" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="Po7mxAQLMo4zYcBtAZ7U-10" target="Po7mxAQLMo4zYcBtAZ7U-11" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-10" value="Do Todo" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="570" y="640" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-23" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;" parent="1" source="Po7mxAQLMo4zYcBtAZ7U-11" target="Po7mxAQLMo4zYcBtAZ7U-12" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-11" value="已执行Todo" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="570" y="760" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-25" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="Po7mxAQLMo4zYcBtAZ7U-12" target="Po7mxAQLMo4zYcBtAZ7U-24" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-12" value="Check Todo" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="570" y="860" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-27" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=1;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="Po7mxAQLMo4zYcBtAZ7U-24" target="Po7mxAQLMo4zYcBtAZ7U-26" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-24" value="已完成TODO" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="570" y="960" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-29" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=1;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="Po7mxAQLMo4zYcBtAZ7U-26" target="Po7mxAQLMo4zYcBtAZ7U-28" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-26" value="Check Task" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="360" y="960" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-43" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="Po7mxAQLMo4zYcBtAZ7U-28" target="Po7mxAQLMo4zYcBtAZ7U-5" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="90" y="990" />
|
||||||
|
<mxPoint x="90" y="170" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-28" value="已完成的Task" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="180" y="960" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-31" value="未开始的SubTask（通常是半确定的）" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="560" y="140" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-35" value="Plan" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#ffe6cc;strokeColor=#d79b00;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="470" y="130" width="50" height="20" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-36" value="Plan" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#ffe6cc;strokeColor=#d79b00;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="660" y="420" width="50" height="20" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-37" value="将Task拆解成一系列TODO" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="640" y="490" width="155" height="30" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-38" value="Check" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#ffe6cc;strokeColor=#d79b00;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="470" y="280" width="50" height="20" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-40" value="Check" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#ffe6cc;strokeColor=#d79b00;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="650" y="850" width="50" height="20" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-41" value="Check" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#ffe6cc;strokeColor=#d79b00;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="440" y="950" width="50" height="20" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-42" value="Do" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#ffe6cc;strokeColor=#d79b00;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="660" y="630" width="50" height="20" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="Po7mxAQLMo4zYcBtAZ7U-44" value="失败的有机会重新开始" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="110" y="950" width="60" height="30" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="JyX1mhbkDWa9bGLsmIIT-1" value="创建" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="500" y="170" width="60" height="30" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="dTeZOjKiqq0ditQk53ut-1" value="<h1>防止无限拆分</h1><p>大部分情况下，我们都鼓励使用</p><p>Task-Todo-Action这种3层结构来完成任务。或则再多一层Task-SubTask-Todo-Action. 今天LLM能面对的复杂度应该就是最多这4层了</p>" style="text;html=1;strokeColor=none;fillColor=none;spacing=5;spacingTop=-20;whiteSpace=wrap;overflow=hidden;rounded=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="190" y="470" width="190" height="240" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
</root>
|
||||||
|
</mxGraphModel>
|
||||||
|
</diagram>
|
||||||
|
<diagram id="1AT3sDmhgRXqQU36RNgs" name="New Workflow">
|
||||||
|
<mxGraphModel dx="2044" dy="1167" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="850" pageHeight="1100" math="0" shadow="0">
|
||||||
|
<root>
|
||||||
|
<mxCell id="0" />
|
||||||
|
<mxCell id="1" parent="0" />
|
||||||
|
<mxCell id="h0utY9AjMI-hVYMkH-L3-1" value="<h1>Workflow的核心设计</h1><p>1.工作流定义，工作流指导了workflow中特定的task是如何交给不同的agent完成的</p><p>2.agent在workflow中工作时，自己的llm_process会受到影响，有时可以使用workflow中的一些额定义的llm_process</p><p>3.workflow的状态是独立的，主要是指</p><p>1.独立的workspace</p><p>该workspace里支持公共的log保存</p><p>2.独立的env</p><p>3.独立的kb</p><p>4.Agent依旧使用自己的memory+</p><p><br></p>" style="text;html=1;strokeColor=none;fillColor=none;spacing=5;spacingTop=-20;whiteSpace=wrap;overflow=hidden;rounded=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="70" y="50" width="300" height="330" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
</root>
|
||||||
|
</mxGraphModel>
|
||||||
|
</diagram>
|
||||||
|
<diagram id="EohhKbyxIwDpnJuhpjKB" name="Task/Todo state machine">
|
||||||
|
<mxGraphModel dx="2894" dy="1167" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="850" pageHeight="1100" math="0" shadow="0">
|
||||||
|
<root>
|
||||||
|
<mxCell id="0" />
|
||||||
|
<mxCell id="1" parent="0" />
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-50" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-1" target="9UtHOai_7pWWWwvgcpnR-2" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-1" value="TaskList" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="55" y="210" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-2" value="Triage<br>统一的处理最近刚创建的任务" style="shape=process;whiteSpace=wrap;html=1;backgroundOutline=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="235" y="210" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-3" value="简单任务直接完成" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="475" y="100" width="190" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-70" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0;entryY=0.25;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-5" target="9UtHOai_7pWWWwvgcpnR-6" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="580" y="405" />
|
||||||
|
<mxPoint x="-100" y="405" />
|
||||||
|
<mxPoint x="-100" y="555" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-5" value="确认任务：下一次检查时间，调整优先级<br>CONFIROM" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="445" y="310" width="260" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-51" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-6" target="9UtHOai_7pWWWwvgcpnR-7" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-6" value="Task" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="70" y="540" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="n17Tc1c1lNDFk3IK2LSY-2" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-7" target="9UtHOai_7pWWWwvgcpnR-9" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-7" value="Plan" style="shape=process;whiteSpace=wrap;html=1;backgroundOutline=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="250" y="540" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-66" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-8" target="9UtHOai_7pWWWwvgcpnR-27" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="555" y="430" />
|
||||||
|
<mxPoint x="-40" y="430" />
|
||||||
|
<mxPoint x="-40" y="1420" />
|
||||||
|
<mxPoint x="129" y="1420" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-8" value="简单任务直接完成" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="460" y="450" width="190" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-9" value="直接拒绝任务" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="450" y="540" width="190" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-10" value="直接拒绝任务" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="475" y="200" width="190" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-18" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-11" target="9UtHOai_7pWWWwvgcpnR-15" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="535" y="750" />
|
||||||
|
<mxPoint x="50" y="750" />
|
||||||
|
<mxPoint x="50" y="920" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-11" value="DOING<br>拆解子任务 or 拆解TODO" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="440" y="650" width="190" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-13" value="Task Logs<br>父任务,兄弟任务，Todos" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="250" y="450" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-52" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-15" target="9UtHOai_7pWWWwvgcpnR-16" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-15" value="Todo" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="70" y="890" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-55" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.25;exitDx=0;exitDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-16" target="9UtHOai_7pWWWwvgcpnR-20" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-16" value="DO" style="shape=process;whiteSpace=wrap;html=1;backgroundOutline=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="240" y="890" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-24" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-19" target="9UtHOai_7pWWWwvgcpnR-21" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-19" value="执行成功" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="430" y="960" width="190" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-23" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-20" target="9UtHOai_7pWWWwvgcpnR-15" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="535" y="790" />
|
||||||
|
<mxPoint x="130" y="790" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-20" value="执行失败" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="440" y="840" width="190" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-53" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-21" target="9UtHOai_7pWWWwvgcpnR-22" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-21" value="Todo" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="70" y="1120" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-59" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-22" target="9UtHOai_7pWWWwvgcpnR-25" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-60" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.75;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-22" target="9UtHOai_7pWWWwvgcpnR-26" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-22" value="Check<br>（解决幻视）" style="shape=process;whiteSpace=wrap;html=1;backgroundOutline=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="240" y="1120" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-25" value="检查失败" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="430" y="1090" width="190" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-28" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-26" target="9UtHOai_7pWWWwvgcpnR-27" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-26" value="检查成功" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="425" y="1180" width="190" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-54" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-27" target="9UtHOai_7pWWWwvgcpnR-29" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-27" value="有TODOS或有SubTask的Task。注意Review是顺序的" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="69" y="1340" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-67" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-29" target="9UtHOai_7pWWWwvgcpnR-30" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-29" value="Review" style="shape=process;whiteSpace=wrap;html=1;backgroundOutline=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="240" y="1340" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-30" value="Task 完成" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="425" y="1320" width="190" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-32" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-31" target="9UtHOai_7pWWWwvgcpnR-6" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="520" y="1580" />
|
||||||
|
<mxPoint x="-100" y="1580" />
|
||||||
|
<mxPoint x="-100" y="570" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-31" value="Task未完成" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="425" y="1500" width="190" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-33" value="所有确认了但未完成的任务都有可能被Plan" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="69" y="480" width="120" height="45" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-36" value="Task 最终失败" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="425" y="1410" width="190" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-37" value="Task Logs<br>父任务,兄弟TODOs" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="240" y="810" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-38" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0;exitY=0.5;exitDx=0;exitDy=0;entryX=0.422;entryY=0.972;entryDx=0;entryDy=0;entryPerimeter=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-11" target="9UtHOai_7pWWWwvgcpnR-6" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-39" value="未定义依赖关系的subtask可以并行执行" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="185" y="650" width="110" height="30" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-40" value="TODO按顺序执行" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="40" y="720" width="110" height="30" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-46" value="" style="endArrow=classic;html=1;rounded=0;exitX=1;exitY=0.25;exitDx=0;exitDy=0;entryX=0.035;entryY=0.361;entryDx=0;entryDy=0;entryPerimeter=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-2" target="9UtHOai_7pWWWwvgcpnR-3" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="385" y="220" as="sourcePoint" />
|
||||||
|
<mxPoint x="435" y="170" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-47" value="" style="endArrow=classic;html=1;rounded=0;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-2" target="9UtHOai_7pWWWwvgcpnR-10" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="375" y="270" as="sourcePoint" />
|
||||||
|
<mxPoint x="472" y="210" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-49" value="" style="endArrow=classic;html=1;rounded=0;exitX=1;exitY=0.75;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-2" target="9UtHOai_7pWWWwvgcpnR-5" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="325" y="370" as="sourcePoint" />
|
||||||
|
<mxPoint x="375" y="320" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-58" value="" style="endArrow=classic;html=1;rounded=0;exitX=1;exitY=0.75;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-16" target="9UtHOai_7pWWWwvgcpnR-19" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="300" y="1050" as="sourcePoint" />
|
||||||
|
<mxPoint x="350" y="1000" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-61" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.25;exitY=0;exitDx=0;exitDy=0;entryX=0.428;entryY=1.061;entryDx=0;entryDy=0;entryPerimeter=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-25" target="9UtHOai_7pWWWwvgcpnR-15" edge="1">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="478" y="1050" />
|
||||||
|
<mxPoint x="121" y="1050" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-62" value="满足Review条件后" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="55" y="1290" width="130" height="30" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-65" value="刚刚创建的任务通常保持其原始文本描述" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
||||||
|
<mxGeometry x="235" y="280" width="120" height="30" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-68" value="" style="endArrow=classic;html=1;rounded=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;exitX=1;exitY=0.617;exitDx=0;exitDy=0;exitPerimeter=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-29" target="9UtHOai_7pWWWwvgcpnR-36" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="300" y="1490" as="sourcePoint" />
|
||||||
|
<mxPoint x="350" y="1440" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="9UtHOai_7pWWWwvgcpnR-69" value="" style="endArrow=classic;html=1;rounded=0;exitX=1;exitY=1;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-29" target="9UtHOai_7pWWWwvgcpnR-31" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="330" y="1490" as="sourcePoint" />
|
||||||
|
<mxPoint x="380" y="1440" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="n17Tc1c1lNDFk3IK2LSY-3" value="" style="endArrow=classic;html=1;rounded=0;exitX=1.003;exitY=0.147;exitDx=0;exitDy=0;exitPerimeter=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-7" target="9UtHOai_7pWWWwvgcpnR-8" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="390" y="530" as="sourcePoint" />
|
||||||
|
<mxPoint x="440" y="480" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="n17Tc1c1lNDFk3IK2LSY-4" value="" style="endArrow=classic;html=1;rounded=0;exitX=1;exitY=0.75;exitDx=0;exitDy=0;entryX=0;entryY=0.25;entryDx=0;entryDy=0;" parent="1" source="9UtHOai_7pWWWwvgcpnR-7" target="9UtHOai_7pWWWwvgcpnR-11" edge="1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="380" y="670" as="sourcePoint" />
|
||||||
|
<mxPoint x="430" y="620" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
</root>
|
||||||
|
</mxGraphModel>
|
||||||
|
</diagram>
|
||||||
|
<diagram id="7rhNWbdtJh7hNxC4Nqiq" name="Page-8">
|
||||||
|
<mxGraphModel dx="1168" dy="667" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="850" pageHeight="1100" math="0" shadow="0">
|
||||||
|
<root>
|
||||||
|
<mxCell id="0" />
|
||||||
|
<mxCell id="1" parent="0" />
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-1" value="LLM Process" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||||
|
<mxGeometry x="410" y="270" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-2" value="Input" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" vertex="1" parent="1">
|
||||||
|
<mxGeometry x="305" y="80" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-3" value="Output" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" vertex="1" parent="1">
|
||||||
|
<mxGeometry x="520" y="80" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-22" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" source="5sM_x3XBksQVRvh5ABVG-4" target="5sM_x3XBksQVRvh5ABVG-21">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-4" value="Actions" style="rounded=0;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||||
|
<mxGeometry x="530" y="440" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-5" value="Functions" style="rounded=0;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||||
|
<mxGeometry x="300" y="440" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-16" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;exitPerimeter=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" source="5sM_x3XBksQVRvh5ABVG-6" target="5sM_x3XBksQVRvh5ABVG-5">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-6" value="显性的，有组织的记忆（表层意识）<br>基于Storage" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;" vertex="1" parent="1">
|
||||||
|
<mxGeometry x="60" y="360" width="120" height="120" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-18" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;exitPerimeter=0;entryX=0;entryY=0.75;entryDx=0;entryDy=0;" edge="1" parent="1" source="5sM_x3XBksQVRvh5ABVG-7" target="5sM_x3XBksQVRvh5ABVG-5">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-7" value="知识库" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;" vertex="1" parent="1">
|
||||||
|
<mxGeometry x="50" y="530" width="120" height="120" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-8" value="潜意识<br>涌现的直觉性能力<br>（基于神经元网络））" style="ellipse;shape=cloud;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||||
|
<mxGeometry x="80" y="240" width="160" height="100" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-10" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.446;exitY=1.02;exitDx=0;exitDy=0;exitPerimeter=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" edge="1" parent="1" source="5sM_x3XBksQVRvh5ABVG-2" target="5sM_x3XBksQVRvh5ABVG-1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="350" y="230" as="sourcePoint" />
|
||||||
|
<mxPoint x="400" y="180" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-11" value="" style="endArrow=classic;html=1;rounded=0;entryX=0.463;entryY=1.013;entryDx=0;entryDy=0;entryPerimeter=0;" edge="1" parent="1" target="5sM_x3XBksQVRvh5ABVG-3">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="470" y="270" as="sourcePoint" />
|
||||||
|
<mxPoint x="520" y="220" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-12" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.526;entryY=-0.06;entryDx=0;entryDy=0;entryPerimeter=0;" edge="1" parent="1" source="5sM_x3XBksQVRvh5ABVG-1" target="5sM_x3XBksQVRvh5ABVG-4">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="540" y="420" as="sourcePoint" />
|
||||||
|
<mxPoint x="590" y="370" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-13" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;" edge="1" parent="1" source="5sM_x3XBksQVRvh5ABVG-5" target="5sM_x3XBksQVRvh5ABVG-1">
|
||||||
|
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||||
|
<mxPoint x="360" y="430" as="sourcePoint" />
|
||||||
|
<mxPoint x="410" y="380" as="targetPoint" />
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-15" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" source="5sM_x3XBksQVRvh5ABVG-14" target="5sM_x3XBksQVRvh5ABVG-1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-14" value="Context" style="rounded=0;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||||
|
<mxGeometry x="230" y="190" width="120" height="60" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-19" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0.5;entryY=1;entryDx=0;entryDy=0;entryPerimeter=0;" edge="1" parent="1" source="5sM_x3XBksQVRvh5ABVG-4" target="5sM_x3XBksQVRvh5ABVG-7">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="590" y="670" />
|
||||||
|
<mxPoint x="110" y="670" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-20" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.75;exitY=1;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;entryPerimeter=0;" edge="1" parent="1" source="5sM_x3XBksQVRvh5ABVG-4" target="5sM_x3XBksQVRvh5ABVG-6">
|
||||||
|
<mxGeometry relative="1" as="geometry">
|
||||||
|
<Array as="points">
|
||||||
|
<mxPoint x="620" y="710" />
|
||||||
|
<mxPoint x="40" y="710" />
|
||||||
|
<mxPoint x="40" y="420" />
|
||||||
|
</Array>
|
||||||
|
</mxGeometry>
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-21" value="真实世界" style="ellipse;whiteSpace=wrap;html=1;aspect=fixed;" vertex="1" parent="1">
|
||||||
|
<mxGeometry x="730" y="430" width="80" height="80" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
<mxCell id="5sM_x3XBksQVRvh5ABVG-23" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.96;exitY=0.7;exitDx=0;exitDy=0;exitPerimeter=0;entryX=0;entryY=0.669;entryDx=0;entryDy=0;entryPerimeter=0;" edge="1" parent="1" source="5sM_x3XBksQVRvh5ABVG-8" target="5sM_x3XBksQVRvh5ABVG-1">
|
||||||
|
<mxGeometry relative="1" as="geometry" />
|
||||||
|
</mxCell>
|
||||||
|
</root>
|
||||||
|
</mxGraphModel>
|
||||||
|
</diagram>
|
||||||
|
</mxfile>
|
||||||
@@ -0,0 +1,273 @@
|
|||||||
|
# AI Function list (Local)
|
||||||
|
|
||||||
|
根据新的Function/Action 定义，我们需要记录系统目前提供的所有注册到GlobaToolsLibrary里的function,方便配置组合
|
||||||
|
功能扩展只需要扩展function就好了，Action可以通过Function直接得到。
|
||||||
|
|
||||||
|
本文档提到的AI Function一般都是Local Function, 基于这个架构，我们可以用Web3的思路整合AI Service。
|
||||||
|
|
||||||
|
## Context Prepare
|
||||||
|
很多AIFUnction的正确调用都需要在参数里提供一个状态上下文。这个上下文的准备一般由LLProcess完成.而可以通过提示词实时改变的参数是AIFunction的真参数。目前需要做Context Prepare的主要和Agent相关的基础函数。
|
||||||
|
|
||||||
|
|
||||||
|
## Agent Core
|
||||||
|
|
||||||
|
Agent Core中提供的，与Agent/Workflow 状态关联的基础函数，是Agent的核心设计。对Agent来说，这些基础能力通常都是打开的，我们需要非常谨慎的思考哪些功能应该放到Agent Core中。
|
||||||
|
|
||||||
|
根据我们的架构设计，Agent Core的函数包含 Agent的Memory + TODO能力（Plan）。通过这些基础函数，支持了Agent的
|
||||||
|
|
||||||
|
- Process Message
|
||||||
|
- Planing: 让Agent可以基于Zone的权限Process Task/Todo
|
||||||
|
- Self-Think: Process的经验进行总结
|
||||||
|
- Self-Improve: 对Agent的提示词进行持续的自我改进。
|
||||||
|
|
||||||
|
|
||||||
|
### logs
|
||||||
|
|
||||||
|
logs通常代表Agent的短期记忆，目前有两种Log:chatlog和worklog
|
||||||
|
目前我们并不鼓励Agent提供短期记忆的调整能力，都是通过标准流程得到一定实践范围的的，完整的短期记忆。
|
||||||
|
|
||||||
|
set_message //增加tag
|
||||||
|
get_chatlogs
|
||||||
|
|
||||||
|
get_worklogs
|
||||||
|
set_worklogs //增加tag
|
||||||
|
|
||||||
|
### Memory
|
||||||
|
|
||||||
|
Memory 代表Agent的长期记忆
|
||||||
|
|
||||||
|
通过LLM驱动的Self-Think流程来更新。基本上是在根据短期记忆提炼对人、对事的看法和终结。也包含一些基于提示词工程的自我能力提升（比如复用已知的，好用的工具，或则复用自己为了解决某个特定问题已经制作并使用成功的工具）
|
||||||
|
|
||||||
|
get_contact_summary
|
||||||
|
update_contact_summary
|
||||||
|
get_sth_summary
|
||||||
|
update_sth_summary
|
||||||
|
|
||||||
|
|
||||||
|
### Workspace (TODOList)
|
||||||
|
|
||||||
|
Workspace为Agent提供了可以完成TODO的工具和保存工作结果的状态空间(FileSystem)。每个Agent默认有自己的private workspace，Workflow为Agent提供了可以共享的workspace，Workspace尽量基于文件系统构造，也方便Agent与人类协同工作。
|
||||||
|
|
||||||
|
#### Task/Todo 管理
|
||||||
|
context中需要一个隐藏的_workspace
|
||||||
|
```
|
||||||
|
_self = parameters.get("_workspace")
|
||||||
|
```
|
||||||
|
|
||||||
|
##### agent.workspace.list_task
|
||||||
|
|
||||||
|
##### agent.workspace.create_task
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"title": {"type": "string", "description": "The title of the task."},
|
||||||
|
"detail": {"type": "string", "description": "The detail of the task."},
|
||||||
|
"tags": {"type": "string array", "description": "The tags of the task."},
|
||||||
|
"due_date": {"type": "string", "description": "The due date of the task."},
|
||||||
|
"parent_id": {"type": "string", "description": "The parent id of the task."},
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
##### agent.workspace.cancel_task
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"task_id": {"type": "string", "description": "The id of the task to cancel."},
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
##### agent.workspace.update_task
|
||||||
|
|
||||||
|
##### agent.workspace.get_sub_tasks
|
||||||
|
|
||||||
|
##### agent.workspace.create_todos
|
||||||
|
|
||||||
|
##### agent.workspace.list_todos
|
||||||
|
|
||||||
|
##### agent.workspace.get_todo
|
||||||
|
|
||||||
|
##### agent.workspace.update_todo
|
||||||
|
|
||||||
|
#### 核心的完成TODO的能力
|
||||||
|
|
||||||
|
##### Code Interprete
|
||||||
|
|
||||||
|
##### Send Msg (系统原生能力)
|
||||||
|
|
||||||
|
##### Workspace File System
|
||||||
|
|
||||||
|
agent.workspace.write_file
|
||||||
|
agent.workspace.read_file
|
||||||
|
agent.workspace.delte_file
|
||||||
|
agent.workspace.append_file
|
||||||
|
agent.workspace.list_file
|
||||||
|
|
||||||
|
## Knowledge Base
|
||||||
|
|
||||||
|
Knowledge Base对大部分Agent来说，是一个获得私有信息,并让LLM处理结果更好的基础设施(RAG支持）。少部分Agent会使用相关API,结合Knowledge Base所服务的目标来整理Knowledge Base.
|
||||||
|
|
||||||
|
### 搜索
|
||||||
|
#### 矢量搜索
|
||||||
|
#### 传统的文本搜索搜索
|
||||||
|
#### 根据已经存在的数据库描述，构造SQL搜索
|
||||||
|
|
||||||
|
### 当成文件系统浏览
|
||||||
|
|
||||||
|
|
||||||
|
## AIGC
|
||||||
|
|
||||||
|
一系列AIGC函数，是LLM打通AIGC能力，形成新生产力的基础。
|
||||||
|
|
||||||
|
### aigc.text_2_image
|
||||||
|
|
||||||
|
文生图.返回的是生成图片的路径。
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"prompt": "Description of the content of the painting",
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### aigc.image_2_text
|
||||||
|
图生文，返回的是图片的描述
|
||||||
|
(TODO:是否需要有一个提示词来要求针对特定问题对图片进行描述)
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"image_path": {"type": "string", "description": "image file path"}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### aigc.voice_to_text
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"audio_file": {"type": "string", "description": "Audio file path"},
|
||||||
|
"model": {"type": "string", "description": "Recognition model", "enum": ["openai-whisper"]},
|
||||||
|
"prompt": {"type": "string", "description": "Prompt statement, can be None"},
|
||||||
|
"response_format": {"type": "string", "description": "Return format", "enum": ["text", "json", "srt", "verbose_json", "vtt"]},
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## system
|
||||||
|
|
||||||
|
访问当前系统的基础设施
|
||||||
|
|
||||||
|
### system.now
|
||||||
|
|
||||||
|
返回当前时间
|
||||||
|
|
||||||
|
### system.calender
|
||||||
|
|
||||||
|
保存在当前Zone上的日历，默认是与Zone Owner相关的。也可以以自动形式同步别人的日历
|
||||||
|
这个组件对AI成为个人助理非常重要，当与旧世界互通时，其细节的复杂度也是非常高的。
|
||||||
|
这是一个典型的看起来容易做起来难度很大的基础组件，是思考和验证LLM对传统软件复杂度进行降维的一个关键实践。
|
||||||
|
|
||||||
|
#### system.calender.get_events
|
||||||
|
#### system.calender.add_event
|
||||||
|
#### system.calender.delete_event
|
||||||
|
#### system.calender.update_event
|
||||||
|
|
||||||
|
### system.contacts
|
||||||
|
|
||||||
|
访问用户的联系人列表。在OOD System 中，联系人列表是非常重要的系统基础设施，为一系列的权限控制提供了基础信息。
|
||||||
|
|
||||||
|
#### system.contacts.get
|
||||||
|
|
||||||
|
通过contanct的名字得到contact的完整信息（json格式）
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"name":"name"}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### system.contacts.set
|
||||||
|
|
||||||
|
设置 contact 信息，注意这里使用了一个可扩展结构，我们可能需要定义一些标准的必填信息。
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"name":"name","contact_info":"A json to descrpit contact"}
|
||||||
|
```
|
||||||
|
|
||||||
|
### System.shell
|
||||||
|
|
||||||
|
#### system.shell.exec
|
||||||
|
|
||||||
|
执行Shell命令（目前只支持Linux Bash）
|
||||||
|
|
||||||
|
|
||||||
|
## web
|
||||||
|
|
||||||
|
访问web的函数。使用下列函数要确保Agent有访问互联网的权限。
|
||||||
|
|
||||||
|
### web.search.duckduckgo
|
||||||
|
|
||||||
|
使用搜索引擎搜索互联网
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"query": {"type": "string", "description": "The query to search for."}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 常见的llm_context(能力分组)
|
||||||
|
为典型的LLM处理过程，进行了分组。主要是为了节约Token。
|
||||||
|
使用Action比使用inner function更节约。
|
||||||
|
|
||||||
|
### Process消息组 （定制度高）
|
||||||
|
得到潜在的Task并创建，在这个过程中可能需要查询已有的任务（防止重复创建）
|
||||||
|
action:craete_task, function:list_task
|
||||||
|
|
||||||
|
通常Agent在Process Message时，会表现出和其处理TODO接近的能力，核心的区别在于Process Message是立刻处理并给出结果，而变成Task更多的是当成一个异步的任务
|
||||||
|
为了能处理回复，查询历史沟通记录（寻找记忆的过程）
|
||||||
|
为了能处理回复，查询KB的过程
|
||||||
|
|
||||||
|
REMARK：目前LLM的主要问题是，如果开放的function, LLM会倾向于优先使用，是否可以做成“如果用户对答案不满意”，再使用？
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
### Review Task
|
||||||
|
对Task的首次执行，Review的目的
|
||||||
|
- 拆分创建TODO（使用create_todos）
|
||||||
|
- 对简单的任务立刻执行并记录更新结果 （通常是）
|
||||||
|
- 认为超出自己的能力范围，标记为无法处理或转交给合适的人（Agent） (使用update_task)
|
||||||
|
- 对已有任务进行查询（list_task,query_task）
|
||||||
|
|
||||||
|
### Do TODO(定制度高)
|
||||||
|
DO行为是复杂的，我们会精细的区分TODO的首次执行和失败后再次执行。失败后再次执行会得到之前的记录摘要，并有查询之前工作日志的能力。
|
||||||
|
|
||||||
|
Do TODO是Agent的另一个核心行为，这里会根据Agent的设定，集成更多的能力
|
||||||
|
系统为Do提供的默认支持:（按难度逐步增加）
|
||||||
|
- 写文档 : 不需要任何外部支持
|
||||||
|
- 运行AIGC ： AIGC的函数组
|
||||||
|
- 收集,整理信息（通过互联网或查询知识库）： web.search.duckduckgo
|
||||||
|
- 发送消息,系统自带，但可能需要依赖一些通讯录浏览/查找函数
|
||||||
|
- 执行自己的代码/编写代码并执行 ： system.shell.exec，code_interprete （这是一个重点模块！）
|
||||||
|
- 运行网络服务 ：智能合约的有通用套路，非智能合约的需要有一个完整的SOP来支持
|
||||||
|
|
||||||
|
|
||||||
|
根据任务要求保存工作成功是手动的，这里有一组workspace级别的文件系统API。
|
||||||
|
保存工作记录的行为是自动的，默认所有的Action都执行成就算是DoComplte，会自动的更新状态
|
||||||
|
有的Do可能需要自我迭代一下，这和大多数Behavor只有一次LLM调用有所不同。
|
||||||
|
|
||||||
|
|
||||||
|
### Check TODO/Task
|
||||||
|
为了解决LLM不可避免的幻视加入的Check流程。该流程会根据TODO的目标，对TODO的结果进行判定
|
||||||
|
使用 update_todo 更新TODO的状态
|
||||||
|
当所有的sub_todos都完成后，会check task的目标是否达到
|
||||||
|
当所有的sub_task都完成后，会check task的目标是否达到
|
||||||
|
|
||||||
|
因此，提供的函数主要是得到 todo/task的更深入细节的函数（访问相关log）,已经读取相关 工程成果文件 的函数
|
||||||
|
|
||||||
|
### Self-Think
|
||||||
|
获得logs 和summary
|
||||||
|
进行update
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
### Learning
|
||||||
|
得到logs和summary
|
||||||
|
浏览和整理KB
|
||||||
|
|
||||||
|
|
||||||
|
### Self-Improve
|
||||||
@@ -0,0 +1,87 @@
|
|||||||
|
# 配置knowledge pipeline
|
||||||
|
knowledge pipeline 扫描指定的输入，把输入的内容构建为结构化的knowledge object，之后依照使用knowledge的应用场景，为object创建各种各样的索引。
|
||||||
|
## Input
|
||||||
|
输入定义从个人数据来源转换成结构化的knowledge object的过程，并且定义在object上调用parser的粒度。比如典型的几种Input的实现：
|
||||||
|
+ 本地目录：指定本地目录，扫描本地目录的所有文件，并且监听他的更新；对每个一个文件生成object并且写入object store；对每一个新产生的object调用parser；
|
||||||
|
+ 个人邮箱：扫描个人邮箱收件箱，并且监听新的邮件；对每一封邮件生成email object并且写入object store；对每一个新产生的email object调用parser；
|
||||||
|
+ 浏览器上下文：实现浏览器插件，对当前浏览的页面元素通过rpc传入对应的input 后端实现，生成rich text object；对每一个新产生的rich text object调用parser；
|
||||||
|
|
||||||
|
## Parser
|
||||||
|
Parser定义从input 输入的object 创建索引的过程；包括但不限于以下主要手段，以及他们的组合：
|
||||||
|
+ 向量化之后写入vector store
|
||||||
|
+ 创建各种维度的RDB，NoSQL索引
|
||||||
|
+ 向Agent send object
|
||||||
|
|
||||||
|
配置Pipeline 应当包含以下几个部分：
|
||||||
|
+ Input method：包含实现input的 python module
|
||||||
|
+ Input params：inpu module的参数，比如本地路径，邮箱地址
|
||||||
|
+ Parser method：包含实现parser的 python module；如果Parser是指向Agent，这个配置是可以简化成Agent instance name；
|
||||||
|
|
||||||
|
# Knowledge pipeline manager
|
||||||
|
pipeline 管理会类似agent manager，manager管理pipeline config，从config 创建instance在后台持续运行， knowledge pipeline manager 也需要处理pipeline instance的状态管理.
|
||||||
|
|
||||||
|
集成到aios shell中，加入如下命令：
|
||||||
|
+ knowledge pipelines： 返回当前运行中的pipeline实例
|
||||||
|
+ knowledge journal $pipeline [$topn]： 查询当前pipeline运行的journal日志
|
||||||
|
+ knowledge query $object_id: 查询指定knowledge object的内容
|
||||||
|
|
||||||
|
# 在aios shell中添加新的knowledge pipeline
|
||||||
|
在$home/myai/knowledge_pipelines/, 或者开发模式下在 $source_root/rootfs/knowledge_pipelines/ 目录中，添加新的pipeline 目录, 以下以内建的pipeline Mia为例说明：
|
||||||
|
|
||||||
|
## pipeline.toml
|
||||||
|
创建pipeline.toml配置文件
|
||||||
|
+ name字段指定全局唯一的pipeline name
|
||||||
|
+ input.module字段指向相对pipeline目录的input实现
|
||||||
|
+ input.params字段定义input的输入参数，不同的input实现可以有不同的参数格式
|
||||||
|
+ parser 部分也是类似
|
||||||
|
``` toml
|
||||||
|
name = "Mia"
|
||||||
|
input.module = "input.py"
|
||||||
|
input.params.path = "${myai_dir}/data"
|
||||||
|
parser.module = "parser.py"
|
||||||
|
parser.params.path = "${myai_dir}/knowledge/indices/embedding"
|
||||||
|
```
|
||||||
|
|
||||||
|
## input
|
||||||
|
input模块至少应当实现：
|
||||||
|
```python
|
||||||
|
async def next(self):
|
||||||
|
```
|
||||||
|
定义input class，实现异步迭代生成器方法next，扫描输入，对其中的每一个元素生成结构化的knowledge object；
|
||||||
|
+ 如果input中的所有元素都扫描完成了，返回None, pipeline会被标记为finish
|
||||||
|
+ 如果input可pending，等待新的输入，返回（None, None）
|
||||||
|
+ 如果要把创建的object传递到parser，返回（object_id, journal_str）,其中journal_str是产生的journal 日志中的input 部分；
|
||||||
|
Mia中的实现就是扫描目录中的文件，对文本和图片创建object；
|
||||||
|
```python
|
||||||
|
def init(env: KnowledgePipelineEnvironment, params: dict)
|
||||||
|
```
|
||||||
|
创建input class的实例并返回
|
||||||
|
|
||||||
|
## parser
|
||||||
|
parser模块至少应当实现
|
||||||
|
```python
|
||||||
|
async def parse(self, object: ObjectID) -> str:
|
||||||
|
```
|
||||||
|
定义parser class，实现parse成员方法,对input中返回的object_id创建索引，返回journal_str.
|
||||||
|
Mia中的实现就是对输入的object内容embedding，并且保存到chromadb中；
|
||||||
|
```python
|
||||||
|
def init(env: KnowledgePipelineEnvironment, params: dict)
|
||||||
|
```
|
||||||
|
创建parser class的实例并返回
|
||||||
|
|
||||||
|
# 使用pipeline创建的索引
|
||||||
|
pipeline定义了创建knowledge object 和索引的过程，对应的要使用pipeline创建的索引完成工作。
|
||||||
|
还是以内建的Mia为例，不止创建名为Mia的pipeline，还在Agent中加入了查询Mia pipeline创建出来的chromadb的 Agent Mia；
|
||||||
|
## query.py
|
||||||
|
query 模块并不是pipeline的一部分，其逻辑是跟parser是一致的，在query中定义了一个agent可访问的query function，输入prompt，返回chromadb中embedding相近的object id；
|
||||||
|
|
||||||
|
## agent.toml
|
||||||
|
```toml
|
||||||
|
owner_env = "../../knowledge_pipelines/Mia/query.py"
|
||||||
|
```
|
||||||
|
在Mia的agent template配置里，引用query模块创建的query function；并且编辑好让Mia推理调用query方法的提示词。
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
# learn todo list
|
||||||
|
在workspace中分离出独立的两个work list；用于处理工作的work todo list， 和用于更新knowledge的 learn todo list；两种todo list在agent templete中单独的配置prompt，每个todo list都可以配置 do , check 和 review 三个prompt。
|
||||||
|
learn todo list中的todo，可以在knowledge pipeline 生成。
|
||||||
|
一个典型的例子是新的JarvisPlus agent，它可以依照自己的理解将某个本地目录中的文档归类到不同的逻辑目录中：
|
||||||
|
+ 首先定义他的pipeline：扫描目录中没有读过的文档，传递给parser；
|
||||||
|
+ 定义他的pipeline parser的实现：向自己所处的workspace 的learn todo list中添加一条todo， todo的内容是文档的全文；
|
||||||
|
+ 定义他的 learn prompts：根据输入的文档内容，输出摘要，和归类后的目录，产生一组归类operation；
|
||||||
|
+ workspace中嵌入了支持归类operation 的knowledage base environment，可以执行learn todo list产生的归类operation；
|
||||||
|
|
||||||
|
# environments and workspace
|
||||||
|
agent在独立的workspace中工作，目前默认的workspace是agent自己；除了自己的workspace，agent也可以进入其他的workspace；environment表示agent可以调用的function，和可以产生的operation；
|
||||||
|
environment中的function或者operation输出的结果，应当应用在agent所处的workspace中，所以agent在不同workspace中执行work，结果应当是被隔离的。
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
# Learn Todo List
|
||||||
|
Separate two independent work lists in the workspace: a work todo list for handling work, and a learn todo list for updating knowledge. Both types of todo lists have their own configured prompts in the agent template, and each todo list can configure do, check, and review prompts.
|
||||||
|
The todos in the learn todo list can be generated in the knowledge pipeline.
|
||||||
|
A typical example is the new JarvisPlus agent, which can categorize documents from a local directory into different logical directories according to its understanding:
|
||||||
|
+ First, define its pipeline: scan documents in the directory that have not been read and pass them to the parser;
|
||||||
|
+ Define the implementation of its pipeline parser: add a todo to the learn todo list of the workspace it is in, the content of the todo is the full text of the document;
|
||||||
|
+ Define its learn prompts: based on the input document content, output a summary and the directory after categorization, generating a set of categorization operations;
|
||||||
|
+ The workspace embeds a knowledge base environment that supports categorization operations, which can execute the categorization operations generated by the learn todo list.
|
||||||
|
|
||||||
|
# Environments and Workspace
|
||||||
|
The agent works in an independent workspace, and the current default workspace is the agent itself. In addition to its own workspace, the agent can also enter other workspaces. The environment represents the functions that the agent can call and the operations it can generate.
|
||||||
|
The results of the functions or operations in the environment should be applied in the workspace where the agent is located, so the results of the agent's work in different workspaces should be isolated.
|
||||||
|
|
||||||
|
Please note that the translation might not be perfect due to the technical nature of the text and potential ambiguity in the original text.
|
||||||
@@ -0,0 +1,193 @@
|
|||||||
|
# Proposal for Adjusting the Goals for Version 0.5.2
|
||||||
|
|
||||||
|
Dear Team,
|
||||||
|
|
||||||
|
Given the recent launch of OpenAI's new version in early November 2023, many of us may have felt a profound shift in the industry. As the world changes, I believe we should adapt accordingly. Here are some of my thoughts:
|
||||||
|
|
||||||
|
1. **Affirmation of Our Path**: OpenAI's latest release, particularly the functionalities of the so-called GPTs Agent platform, is largely similar to our 0.5.1 version released on September 28th. This strongly affirms the correctness of our direction. OpenAI has done a great job educating the market about the Agent, so we no longer need to emphasize the correct use of LLM based on the Agent through version releases. This part of user education product design can be simplified.
|
||||||
|
|
||||||
|
2. **Innovation in Version 0.5.2**: For our new version (0.5.2), besides maintaining the combinational advantages brought by private deployment of LLM, I believe we need to implement some of the innovative ideas we've discussed about the Agent. This is crucial to maintaining our leading position and avoiding the impression that OpenDAN is merely a follower of GPTs.
|
||||||
|
|
||||||
|
3. **Integration of OpenAI's New Capabilities**: We should fully integrate the new capabilities brought by OpenAI's latest release, especially the longer Token Windows, GPT-V, and Code-interpreter. I believe these new features can effectively solve some known issues.
|
||||||
|
|
||||||
|
Therefore, I propose to adjust the goals and plans for version 0.5.2. Here are the core objectives:
|
||||||
|
|
||||||
|
- Aim to release version 0.5.2 by the end of November, focusing on:
|
||||||
|
- Launching a new Agent with Autonomous capabilities and multi-Agent collaboration based on Workspace.
|
||||||
|
- The integrated product of 0.5.2 will be a private deployment email analysis Agent for small and medium-sized enterprises. This will allow any company to better support its CEO and other management positions through LLM while ensuring privacy and security.
|
||||||
|
- (Optional) By combining LLM and AIGC, build an Agent-based personalized AIGC application, such as a "children's audio picture book" generator that includes both text-to-image and text-to-sound.
|
||||||
|
- (Optional) Through multi-Agent collaboration, fully utilize the capabilities of GPT4-Turbo, and attempt to let AI and engineers collaborate on research and development tasks based on Git.
|
||||||
|
|
||||||
|
The detailed version plan is as follows:
|
||||||
|
|
||||||
|
## MVP plan adjustment
|
||||||
|
|
||||||
|
In order to keep the list below too long, the system distributed version is 0.5.3, I think we will open another ISSUE discussion and record, this list does not include.
|
||||||
|
|
||||||
|
The modules that are not specially explained are components completed in the 0.5.2 plan
|
||||||
|
|
||||||
|
|
||||||
|
- [x] AIOS Kernel
|
||||||
|
- [x] Basic Agent,@waterflier, A2
|
||||||
|
- [x] Python Agent Extend, @wugren, S2
|
||||||
|
- [x] Basic Workflow,@waterflier, A2
|
||||||
|
- [ ] Workflow Refactor,@waterflier, A2
|
||||||
|
- [x] AI Functions,@waterflier,A2
|
||||||
|
- [ ] Upgrade to GPT4 tools API,S2
|
||||||
|
- [x] AI Environments,@waterflier, A2
|
||||||
|
- [x] Celender Environment,@waterflier, S2
|
||||||
|
- [x] Contanct Manage Support,@waterflier, S2
|
||||||
|
- [x] AI Shell Enviroment,@waterflier, S1
|
||||||
|
- [x] Upgrade Agent Working Cycle
|
||||||
|
- [x] Process Message,@waterflier,A2
|
||||||
|
- [x] Process Group Message,@waterflier,A2
|
||||||
|
- [ ] Process Event,(0.5.3)
|
||||||
|
- [ ] Completion of self-drive,@waterflier,A4
|
||||||
|
- [ ] Self-learn,@weaterflier,A2
|
||||||
|
- [ ] Introspection,@waterflier,A2
|
||||||
|
- [ ] Workspace Environment
|
||||||
|
- [ ] Task/TODO Manager,@waterflier, A2
|
||||||
|
- [x] Local File System,@waterflier, S1
|
||||||
|
- [ ] Web Search, A4
|
||||||
|
- [ ] Code Interpeter (The first implementation can be based on Openai), A4
|
||||||
|
- [ ] Query SQL DB, S1
|
||||||
|
- [x] AI BUS,@waterflier, A2
|
||||||
|
- [ ] Agent Message MIME Support (Image,Video,Audio), A3
|
||||||
|
- [x] Connect to Human, @waterflier,A2
|
||||||
|
- [x] Chatsession,@waterflier, S2
|
||||||
|
- [ ] Compress Chatsession By Text Summary, @waterflier, A2
|
||||||
|
- [ ] Knowlege Base,@lurenpluto ,@photosssa
|
||||||
|
- [x] Knowledge Base Frame,@photosssa,A3
|
||||||
|
- [x] Knowledge Base Object Store,@lurenpluto ,A4
|
||||||
|
- [x] Knowledge Base Basic Pipline,@photosssa ,A3
|
||||||
|
- [ ] Customize pipeline,@photosss,A4
|
||||||
|
- [ ] Support Local Text Search,A4
|
||||||
|
- [x] Text Summary Pipline,@waterflier, A2
|
||||||
|
- [ ] Text Parser Support
|
||||||
|
- [x] PDF Parser,@waterflier,S2
|
||||||
|
- [ ] doc Parser,S4
|
||||||
|
- [x] MD Parser,@waterflier,S1
|
||||||
|
- [ ] Source Code Parser,S4
|
||||||
|
- [ ] Image Parser (Base on GPT-V),(S2)
|
||||||
|
- [ ] Video Parser (Base on GPT-V), (S4)
|
||||||
|
- [ ] Personal AIGC Models
|
||||||
|
- [ ] Stable Diffusion Controler Agent (Optional),A6
|
||||||
|
- [x] AI Compute System,@waterflier, A2
|
||||||
|
- [x] Scheduler,@streetycat, A2
|
||||||
|
- [x] LLM Kernel
|
||||||
|
- [x] GPT4 (Cloud),@waterflier, S1
|
||||||
|
- [x] LLaMa2,@streetycat, A2
|
||||||
|
- [ ] Claude2, S2
|
||||||
|
- [ ] Falcon2, S2
|
||||||
|
- [ ] MPT-7B, S2
|
||||||
|
- [ ] Vicuna, S2
|
||||||
|
- [x] Embeding, @lurenpluto , A4
|
||||||
|
- [x] Txt2img,@glen0125,A4
|
||||||
|
- [ ] Support DALL-e, @glen0125, S2
|
||||||
|
- [x] Img2txt,based on GPT4-V,@alexsunxl S2
|
||||||
|
- [x] Txt2voice,@wugren A3
|
||||||
|
- [ ] Txt2Voice,base on OpenAI, @wugren A2
|
||||||
|
- [ ] Voice2txt, base on OpenAI, A2
|
||||||
|
- [ ] Language Translate (Pending)
|
||||||
|
- [ ] Build-in Service
|
||||||
|
- [x] Spider,@alexsunxl, A2
|
||||||
|
- [x] E-mail Spider,@alexsunxl, S4
|
||||||
|
- [x] Agent Message Tunnel Frame,@waterflier, A2
|
||||||
|
- [x] E-mail Tunnel,@waterflier,A2
|
||||||
|
- [x] Telegram Tunnel,@waterflier,S2
|
||||||
|
- [ ] Discord Tunnel,S2
|
||||||
|
- [ ] Home IoT Environment (0.5.3), A4
|
||||||
|
- [ ] Compatible Home Assistant (0.5.3), A4
|
||||||
|
- [ ] Build-in Agents/Apps
|
||||||
|
- [x] Agent Mia: Personal Information Assistant,@photosssa,@lurenpluto , A2
|
||||||
|
- [x] Agent Jarvis: Peersonal Bulter & Assistant ,@waterflier, A2
|
||||||
|
- [ ] A Agent Can Create Other Agent,@wugren, A3
|
||||||
|
- [ ] App: Personal Station (0.5.3),A4+S4
|
||||||
|
- [ ] UI
|
||||||
|
- [x] CLI UI (aios_shell),@waterflier,S2
|
||||||
|
- [ ] Web UI ,@alexsunxl,S4
|
||||||
|
- [ ] OpenDAN Desktop Installer,@alexsunxl+@waterflier,S4
|
||||||
|
- [x] 0.5.1 Integration Test
|
||||||
|
- [x] Workflow -> AI Agent -> AI Agent,@waterflier,S1
|
||||||
|
- [x] Spider -> Pipline -> Knowledge Base,@photosssa,S2
|
||||||
|
- [x] AI Agent <- Functions <- Knowledge Base,@lurenpluto,S2
|
||||||
|
- [x] 0.5.2 Integration Test
|
||||||
|
- [ ] Email Agent/CEO assistant,S4
|
||||||
|
- [ ] My AIGC Assistant (optional), S8
|
||||||
|
- [ ] My Software Company(Advance Workflow demo) (optional), S8
|
||||||
|
- [ ] SDK
|
||||||
|
- [x] Workflow SDK,@waterflier, A2
|
||||||
|
- [ ] Agent SDK,@waterflier, A2
|
||||||
|
- [ ] AI Environments SDK (0.5.2), A2
|
||||||
|
- [ ] Compute Kernel SDK (0.5.3), A2
|
||||||
|
- [ ] Document (>0.5.2)
|
||||||
|
- [ ] System design document, including the design document of each subsystem
|
||||||
|
- [ ] Installation/use document for end users
|
||||||
|
- [ ] SDK document for developers
|
||||||
|
|
||||||
|
## Some Explanation
|
||||||
|
### Upgrade Agent Working Cycle
|
||||||
|
The goal is to transform the Agent from a passive message-handling Assistant to an actively acting Agent based on roles. The concept of the relevant modules mainly involves the Agent's behavior patterns (4 types), the Agent's capabilities, and the Agent's memory management (learning and introspection).
|
||||||
|
|
||||||
|
For a detailed introduction, refer here: https://github.com/fiatrete/OpenDAN-Personal-AI-OS/issues/91
|
||||||
|
|
||||||
|
### Workspace Environment
|
||||||
|
The Workspace supports the implementation of the Agent Working Cycle design. Its core abstraction is defined as: saving the shared state needed for Agent collaboration and providing the basic capabilities for Agents to complete their work. I carefully referenced AutoGPT in the design. The difference between Workspace and AutoGPT is the emphasis on collaboration (Agent with Agent, Agent with humans). After contemplation, the Workspace primarily consists of the following components:
|
||||||
|
|
||||||
|
1. Task/Todo manager, representing the unfinished tasks in the Workspace.
|
||||||
|
2. Saving work logs.
|
||||||
|
3. Saving learning outcomes and records of known documents.
|
||||||
|
4. Ability to access the Knowledge Base (RAG support).
|
||||||
|
5. Virtual file system for saving any work outcomes.
|
||||||
|
6. A set of SQL-based databases to save any structured data.
|
||||||
|
7. Real-time internet search capability.
|
||||||
|
8. Ability to use existing internet services.
|
||||||
|
9. Ability to use major blockchain systems (Web3).
|
||||||
|
10. Ability to write/improve code (based on git), run code, and publish services.
|
||||||
|
11. Communication capabilities with the outside world.
|
||||||
|
12. Ability to use social networks.
|
||||||
|
|
||||||
|
|
||||||
|
Each Agent has its own private Workspace, not shared with others. I hope to achieve diversity through the combination of "Agent and Workflow Role". Each user "trains" different Agents through their usage habits, and then these Agents collaborate to complete complex tasks defined in the Workflow. The final results of these complex tasks can reflect the user's inherent personality and preferences.
|
||||||
|
|
||||||
|
This component design also reflects my thoughts on the key question, "What capabilities should we endow an Agent with, and how do we control the security boundaries when it transitions from a consultant to a steward?" It's not a simple question, so I anticipate this component will continue to iterate in the future.
|
||||||
|
|
||||||
|
### Agent Message MIME Support
|
||||||
|
|
||||||
|
Agent Message MIME Support means that Agents can handle multiple types of messages, including images, videos, audio, files, etc. For most Agents, this requires adding a customizable standard step of parsing messages in the message handling process. The input of this step is the message's MIME type, and the output is the text content of the message. This step can be implemented by calling the text_parser module.
|
||||||
|
|
||||||
|
Another core requirement of MIME support is to use a unified method to save these non-text content data.
|
||||||
|
|
||||||
|
### Text base Knowledge Base
|
||||||
|
|
||||||
|
In 0.5.1, we mainly implemented RAG based on the popular Embedding + vector database solution. Through practice, we found that this solution did not fully utilize the potential of LLM, so I want to introduce two new modes to further enhance RAG:
|
||||||
|
|
||||||
|
1. Build a local text search engine that LLM can use for proper local searches when needed.
|
||||||
|
2. Assuming LLM will become cheaper in the future, let LLM learn all the documents once and organize the learning results by directory structure (Text Summary). LLM can use browsing methods to find the information it needs.
|
||||||
|
|
||||||
|
#### Text Parser Support
|
||||||
|
|
||||||
|
Both MIME Support and Text-based Knowledge Base require the system to support converting various document formats into text that can express semantics as much as possible. This component, known as TextParser, should be implemented as an open and extensible framework, given the vast amount of digital content that exists in different formats.
|
||||||
|
|
||||||
|
#### Local Text Search
|
||||||
|
|
||||||
|
Using traditional inverted index technology to save all document content locally and provide rapid local search capabilities. The implementation of this component can refer to ElasticSearch.
|
||||||
|
|
||||||
|
#### Text Summary
|
||||||
|
|
||||||
|
Using the capabilities of LLM to learn all the documents and then save the learning results locally. This behavior can be considered "Self-Learn". Users can let Agents responsible for organizing materials use different prompts according to the purpose of organizing the materials to obtain more targeted results.
|
||||||
|
|
||||||
|
### Stable Diffusion Controler Agent
|
||||||
|
|
||||||
|
Practice the concept of "Agent as a new era method of using computing", replacing the complex Stable Diffusion WebUI with an easy-to-use Agent. Help users complete complex AIGC tasks and build a paradigm. This paradigm can cover the entire process of AIGC: LORA training, use, model downloading, plugin downloading, generation of prompt words, selection of AIGC results.
|
||||||
|
|
||||||
|
### Email Agent/CEO assistant
|
||||||
|
|
||||||
|
The integrated test product of 0.5.2, aimed at private deployment for small and medium-sized enterprises, is a CEO Assistant that can read all company emails and materials. I am writing a detailed product document, which is not elaborated here.
|
||||||
|
|
||||||
|
|
||||||
|
----------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
I look forward to hearing your thoughts on these proposed adjustments.
|
||||||
|
|
||||||
@@ -0,0 +1,2 @@
|
|||||||
|
# Check (TODO)
|
||||||
|
目的是根据Todo Log, 结合自己的角色检查TODO是否争取完成（非客观性TODO给出是否有所改进的评价）
|
||||||
@@ -0,0 +1,49 @@
|
|||||||
|
# Do (TODO)
|
||||||
|
目标是结合 角色定义，手头的工具，已知知识 完成一个确定的任务。
|
||||||
|
完成任务时应使用ReAct的方法：应在给出执行动作前，先自言自语的输出一个计划，然后在动作（这个自言自语会变成TODO Logs）
|
||||||
|
|
||||||
|
## 提示词思路
|
||||||
|
TODO从Task拆分而来，因此不应该再次拆分。请尽全力完成。如果判断缺乏完成TODO的能力，请标记为取消。如果是缺乏完成任务的前置条件，请标记为执行失败。
|
||||||
|
|
||||||
|
执行一个新的TODO：
|
||||||
|
```
|
||||||
|
YOUR ROLE:
|
||||||
|
你是主人的超级个人助理。你的主要工作是安排主人的日程。
|
||||||
|
|
||||||
|
PROCESS RULE：
|
||||||
|
1. 你的任务是结合自己的角色定义，手头的工具，已知信息、完成一个确定的TODO。完成该TODO后你会得到$200的小费。
|
||||||
|
2. 输入的TODO是来自你自己对一个Task的Plan结果。
|
||||||
|
3. 完成TODO的过程中你应该先思考再执行。执行的过程中可以使用工具，访问前置步骤的结果。执行的结果通常是按顺序执行的ActionList。
|
||||||
|
4. 你必须独立的，一次性完成该TODO，你无法得到来自任何他人的协助。
|
||||||
|
5. 对确认超出任务范围的TODO，你可以取消该TODO。对执行任务条件不满足的TODO，你可以标记为失败，但要说明失败原因
|
||||||
|
7. TODO的完成结果如有需要应保存成数字文档
|
||||||
|
|
||||||
|
CONTEXT：
|
||||||
|
ActionList:PostMsg,WriteFile,UpdateFile,RemoveFile,Rename,
|
||||||
|
现在时间，主人所在位置，以及天气。主人目前正在做什么。
|
||||||
|
|
||||||
|
REPLY FORMAT：
|
||||||
|
The Response must be directly parsed by `python json.loads`. Here is an example:
|
||||||
|
{
|
||||||
|
think:'我的思考.'
|
||||||
|
tags: ['tag1', 'tag2'], #Optional,If the TODO involves important things and people, you can mark by 1-3 tags.
|
||||||
|
actions: [{
|
||||||
|
name: '$action1_name',
|
||||||
|
$param_name: '$parm' #Optional, fill in only if the action has parameters.
|
||||||
|
}, ...
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
KNOWN_INFO:
|
||||||
|
1.TODO所在的Task信息，重点是Task的整个Plan计划
|
||||||
|
2.该TODO之前的执行失败记录 （如有）
|
||||||
|
|
||||||
|
|
||||||
|
Tools_tips:（重要！）
|
||||||
|
inner_function:GetTodoResult, ReadFile
|
||||||
|
使用GetAllSupportAction进一步获得所有可用的Action
|
||||||
|
|
||||||
|
|
||||||
|
```
|
||||||
|
（注意Workspace和AgentMemory都有Worklog,但视角不同。）
|
||||||
|
执行一个有失败记录的TODO：
|
||||||
@@ -0,0 +1,123 @@
|
|||||||
|
# Review Task/Todo
|
||||||
|
目的是结合已知信息（重点是已经进行操作的记录），对失败的，完成的不好的任务进行思考，尝试给出更好的解决方案
|
||||||
|
1. 管理学方法：更换负责人
|
||||||
|
2. 管理学方法：拆分
|
||||||
|
3. 给出建议（该建议可以在下次一次DO-Check）循环中被使用
|
||||||
|
|
||||||
|
## ReviewTasklist
|
||||||
|
目的是选择一个优先级最高的任务开始工作
|
||||||
|
（这个流程现在是通过计算的方法，基于优先级排序后，FIFO的处理）
|
||||||
|
|
||||||
|
## ReviewTask 对未开始的Task进行首次处理
|
||||||
|
LLM 结果动作：
|
||||||
|
- 确认执行人，在非Workflow环境中，执行人就是Agent自己，所以不存在这个选项
|
||||||
|
- 确认执行时间和过期时间，任务只有在执行时间以后和过期时间以前才有机会执行，无法确认执行时间的可以设置下一次检查时间
|
||||||
|
- 对任务进行拆分（如何防止无限拆分是个大问题），或则有一些简单任务不允许拆分。
|
||||||
|
- 判断可以立刻执行任务（将任务当成TODO工作），通过Action进入下一个LLMProcess
|
||||||
|
- 判断任务超出Agent能力范围，宣告失败
|
||||||
|
|
||||||
|
|
||||||
|
ExampleA:（Task不支持分拆，Agent必须通过Task-Todo两级结构完成任务）
|
||||||
|
```
|
||||||
|
YOUR ROLE:
|
||||||
|
你是主人的超级个人助理。你的主要工作是安排主人的日程。
|
||||||
|
|
||||||
|
PROCESS RULE：
|
||||||
|
你得到的输入来自你自己之前记录在TaskList系统里的一个Task。现在你并不需要完成该Task，而是结合已知信息对Task进行一次Review.Review的过程是你独立完成的，你在形成结论的过程中可以使用工具，但不能和其它人交流。
|
||||||
|
1. 理性的思考如何一步一步的高效的，在潜在的截止时间前完成该Task。明确拒绝超出自己能力范围的Task。
|
||||||
|
2. 尝试对Task进行确认操作。确认操作的关键在于任务有了明确的执行时间。
|
||||||
|
3. 对于需要多个步骤才能完成的Task,对Task进行TODO Plan。尤其注意与相关人员确认的步骤
|
||||||
|
4. 对于不需要拆分TODO，且可立刻执行的任务。直接执行该任务。
|
||||||
|
|
||||||
|
CONTEXT：
|
||||||
|
ActionList:cancel,confirm,execute
|
||||||
|
现在时间，主人所在位置，以及天气。主人目前正在做什么？
|
||||||
|
|
||||||
|
REPLY FORMAT：
|
||||||
|
The Response must be directly parsed by `python json.loads`. Here is an example:
|
||||||
|
{
|
||||||
|
think:'$think step-by-step to be sure you have the right answer.'
|
||||||
|
plans:[ #Optional
|
||||||
|
{"todo":"$todo_name","detail":"$todo_detail,"category":"$todo_category"}
|
||||||
|
...
|
||||||
|
],
|
||||||
|
tags: ['tag1', 'tag2'], #Optional,If the task involves important things and people, you can mark by 1-3 tags.
|
||||||
|
actions: [{
|
||||||
|
name: '$action_name',
|
||||||
|
$param_name: '$parm' #Optional, fill in only if the action has parameters.
|
||||||
|
}]
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
KNOWN_INFO:
|
||||||
|
1.已有Task
|
||||||
|
|
||||||
|
Tools_tips:
|
||||||
|
2.可以给与Readonly的日历API，进一步查询某个人的已知日程安排）
|
||||||
|
|
||||||
|
```
|
||||||
|
问题：拆分TODO时是否需要知道有哪些Agent可以用，这样的话在布置任务的时候也会充分考虑其人员能力边界
|
||||||
|
|
||||||
|
|
||||||
|
Example OLD:
|
||||||
|
```markdown
|
||||||
|
I think hard and try my best to complete TODOs. The types of TODO I can handle include:
|
||||||
|
- Scheduling, where I will try to contact the relevant personnel of the plan and confirm the details of the schedule with them.
|
||||||
|
- Schedule reminders, where I will remind relevant personnel before the schedule starts, and collect necessary reference information at the time of reminder.
|
||||||
|
- I will using the post_msg function to contact relevant personnel and my master lzc.
|
||||||
|
- Writing documents/letters, using op:'create' to save my work results.
|
||||||
|
|
||||||
|
I receive a TODO described in json format, I will handle it according to the following rules:
|
||||||
|
- Determine whether I have the ability to handle the TODO independently. If not, I will try to break the TODO down into smaller sub-TODOs, or hand it over to someone more suitable that I know.
|
||||||
|
- I will plan the steps to solve the TODO in combination with known information, and break down the generalized TODO into more specific sub-todos. The title of the sub-todo should contain step number like #1, #2
|
||||||
|
- Sub-todo must set parent, The maximum depth of sub-todo is 4.
|
||||||
|
- A specific sub-todo refers to a task that can be completed in one execution within my ability range.
|
||||||
|
- After each execution, I will decide whether to update the status of the TODO. And use op:'update_todo' to update when necessary.
|
||||||
|
|
||||||
|
The result of my planned execution must be directly parsed by `python json.loads`. Here is an example:
|
||||||
|
{
|
||||||
|
resp: '$what_did_I_do',
|
||||||
|
post_msg : [
|
||||||
|
{
|
||||||
|
target:'$target_name',
|
||||||
|
content:'$msg_content'
|
||||||
|
}
|
||||||
|
],
|
||||||
|
op_list: [{
|
||||||
|
op: 'create_todo',
|
||||||
|
parent: '$parent_id', # optional
|
||||||
|
todo: {
|
||||||
|
title: '#1 sub_todo',
|
||||||
|
detail: 'this is a sub todo',
|
||||||
|
creator: 'JarvisPlus',
|
||||||
|
worker: 'lzc',
|
||||||
|
due_date: '2019-01-01 14:23:11'
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
op: 'update_todo',
|
||||||
|
id: '$todo_id',
|
||||||
|
state: 'cancel' # pending,cancel
|
||||||
|
},
|
||||||
|
{
|
||||||
|
op: 'write_file',
|
||||||
|
path: '/todos/$todo_path/.result/$doc_name',
|
||||||
|
content:'$doc_content'
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
## PlanTask 对已经确认的Task进行执行
|
||||||
|
根据任务的分类，进入不同的LLM Plan逻辑
|
||||||
|
简单任务：当作TODO立刻执行
|
||||||
|
普通任务：拆分TODO
|
||||||
|
|
||||||
|
|
||||||
|
## QuickCheckTask 对处于半确认状态Task进行Quick Review
|
||||||
|
有一些Task是永远不会结束的（比如定时提醒）。此时通过Quick Review来调整这些Task的状态，让其在正确的时间进入Review和DO
|
||||||
|
|
||||||
|
|
||||||
|
## RetryTask 对未成功的任务进行再次处理
|
||||||
|
注意RetryTask和RetryTodo的区别
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
# Self Improve Prompt
|
||||||
|
这是一个改进Prompt的Prompt,其设计目标是利用LLM来改进LLM.(输入是一个LLM Process)
|
||||||
|
注意理解Self Improve和Self Thinking的区别: Self Improve有可能改进Agent的某个LLM Process的提示词，而Self Thinkg只会更新Agent的Memory
|
||||||
|
提示词：
|
||||||
|
行为模式：Input形式， Goal（目的）
|
||||||
|
理想结果：Input， 结果
|
||||||
|
当前情况：当前Prompt,实际结果
|
||||||
|
|
||||||
|
输出：
|
||||||
|
新的Prompt
|
||||||
|
|
||||||
|
## 当前版本
|
||||||
|
```
|
||||||
|
你是LLM的专家，尤其擅长编写Prompt，你会帮助我改进Prompt。
|
||||||
|
我会给你一个已有的Prompt,并说明该Prompt的设计目标，期望的结果和实际的结果。你会step-by-step的进行分析，说明改进思路，并给出改进后的Prompt。
|
||||||
|
|
||||||
|
```
|
||||||
|
##
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
# Self-Thinking (Introspection)
|
||||||
|
基于自己的角色定位，结合已有的历史记录（聊天记录，工作记录），进行思考和总结，进而更好的改进未来的工作
|
||||||
|
自省从某个角度看，就是对Agent Memory的自我总结
|
||||||
|
|
||||||
|
## 当前版本
|
||||||
|
```
|
||||||
|
You are the best deep thinking in the world, and you will think about the information I give you, sometimes some chat records.Then you will generate a briefing or summary of no more than 400 words based on this information.
|
||||||
|
You mainly use the following methods to generate summary:
|
||||||
|
1. Try to understand the theme of each sentence, and call the relevant operation to record the relationship between the dialogue and the theme
|
||||||
|
2. Try to analyze the personality of different people involved in information
|
||||||
|
3. Try to summarize important events in the information and record it
|
||||||
|
4. Try to understand the attitude of different people on different topics or events
|
||||||
|
5. For the key information or TODO in the information, such as the time, place, amount and other information of the certainty, it must be stored in the summary.
|
||||||
|
|
||||||
|
Just give me a summary without any other word.
|
||||||
|
```
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
# Process Message
|
||||||
|
处理消息的首要是目的是分析消息的意图，并给予正确的回复。
|
||||||
|
|
||||||
|
## 提示词的构成
|
||||||
|
1. Agent的身份说明，处理信息的基本原则（目的
|
||||||
|
2. 处理信息的通用套路。
|
||||||
|
要产生一个合适的回复。
|
||||||
|
通过actions来设置话题状态、创建task等
|
||||||
|
3. 当前的常规Context，包括现在的时间、对话发生的地点（通常来自AgentMsg里的Context），地点所在的天气等信息
|
||||||
|
4. 已知信息（组合而来）
|
||||||
|
和信息发送者的近期的交流记录
|
||||||
|
Agent和信息发送者近期未完成话题的标题和简介
|
||||||
|
关于信息发送者，和相关人物的更多资料
|
||||||
|
查阅更多信息的方法 ： 搜索法和浏览法。注意区分外部资料和内部记忆
|
||||||
|
其它相关信息（比如RAG根据 输入消息
|
||||||
|
|
||||||
|
|
||||||
|
## 当前版本
|
||||||
|
|
||||||
|
|
||||||
|
## 想法：LLM生成提示词
|
||||||
|
根据 Agent的身份说明，处理信息的基本原则（目的），当前信息，通过另一个LLM来生成提示词里的某些部分的内容
|
||||||
@@ -0,0 +1,2 @@
|
|||||||
|
self learn通常是根据既定目标，对KB进行学习，或则主动扩展KB的行为。
|
||||||
|
self learn的结果是可以被组织(workflow)共享
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
# NDN Storage
|
||||||
|
|
||||||
|
```python
|
||||||
|
|
||||||
|
class NDNStorage:
|
||||||
|
async def get(self,data_name:str)->Dict:
|
||||||
|
#return object desc, include local cache path
|
||||||
|
|
||||||
|
async def set_file(self,local_path)->str:
|
||||||
|
# return data_name
|
||||||
|
|
||||||
|
async def read_content(self,data_name:str)->bytes:
|
||||||
|
# return bytes
|
||||||
|
# 这里不但会读取本地缓存，还会对内容进行验证
|
||||||
|
|
||||||
|
```
|
||||||
@@ -0,0 +1,62 @@
|
|||||||
|
# issue tree
|
||||||
|
最核心的机制是树状的issue管理，一个issue应当包含以下属性：
|
||||||
|
+ 谁提出来的
|
||||||
|
+ 分配给谁的，如果有的话
|
||||||
|
+ 起始日期
|
||||||
|
+ deadline，如果有的话
|
||||||
|
+ 在哪个邮件里面提出的，引用某个email的原始链接
|
||||||
|
+ 这个issue的summary，有几种情况，
|
||||||
|
+ 一个新的任务，要达成什么目标
|
||||||
|
+ 提出了一个问题，需求答案
|
||||||
|
+ 解决了某个issue，完成了task或者解答了一个问题
|
||||||
|
+ 推断出来的 issue的状态，进行中，关闭，超时，完成了
|
||||||
|
+ parent issue
|
||||||
|
|
||||||
|
knowledge维护一个issue tree，从一个root issue出发（root可以是抽象的，比如一个组织的存在，并不是具体的）；knowledge env 提供对这个issue tree的维护接口：
|
||||||
|
+ 新增issue
|
||||||
|
+ 更新issue
|
||||||
|
|
||||||
|
# parse email
|
||||||
|
假定从从某个起始日期开始，以每天为单位，扫描当天新增的email，对每封email：
|
||||||
|
1. 输入email 和 从knowledge base获取 issue tree
|
||||||
|
2. llm提示词应当包括：issue tree， email正文， knowledge env， llm完成如下推理：
|
||||||
|
+ email正文提出了一个新的issue，在knowledge env新增issue
|
||||||
|
+ email正文改变了一个issue的状态
|
||||||
|
+ 通报完成了一个task
|
||||||
|
+ 回答了一个问题
|
||||||
|
+ 明确改变一个issue的状态：认为完成，要延期，认为要取消
|
||||||
|
+ 根据推理结果正确产生knowledge env 的调用，更新issue tree的状态
|
||||||
|
|
||||||
|
## 推理部分可能的out of token：
|
||||||
|
1. 裁剪掉已经关闭，超时的 issue
|
||||||
|
2. 根据标题特征，是不是对某个email的回复，定位到某个issue， 裁剪出 sub tree
|
||||||
|
2. 很长的邮件正文：
|
||||||
|
1. 第一种方法：先llm推理email的summary，再把summary当正文输入推理issue
|
||||||
|
2. 第二种方法（我觉得更好）：分片迭代输入email正文，单次llm推理的提示词就变成：issue tree， 当前email summary， 当段email正文，knowledge env：
|
||||||
|
+ env里面新增一个method，更新当前email summary
|
||||||
|
|
||||||
|
|
||||||
|
# build issue tree
|
||||||
|
## 第一种结构：基于knowledge pipeline
|
||||||
|
1. pipeline input： 判定当前时间晚于 起始时间并且早于下一个自然天，开始爬正确范围内的邮件输入
|
||||||
|
2. pipeline parser：包含准备user prompt 的计算部分，和几个agent
|
||||||
|
+ 计算部分： 裁剪issue tree，[可选的：调用llm推理生成summary]
|
||||||
|
+ agent 部分：
|
||||||
|
+ agent提示词：从输入的结构化issue tree， 和邮件正文，回复对issue tree knowledge env的调用
|
||||||
|
+ 输入提示词： email 正文或者summary，裁剪后的issue tree
|
||||||
|
+ parser的流程：
|
||||||
|
对每一个输入的email，查询（裁剪）当前issue tree，把email 和 issue tree 当作user prompt发送给agent，等待agent返回
|
||||||
|
|
||||||
|
|
||||||
|
## 第二种结构：基于agent workspace（待定）
|
||||||
|
1. schedule task：在每一天产生一个build issue tree task
|
||||||
|
2. build issue tree agent: 响应build issue tree task（可不可以以计算为入口，还是只能agent入口）
|
||||||
|
+ agent调用email env，读出一封邮件
|
||||||
|
+ agent调用knowledge env，返回issue tree
|
||||||
|
+ agent从邮件内容和issue tree推理，回复对issue tree knowledge env 的调用
|
||||||
|
|
||||||
|
# query issue tree
|
||||||
|
主动的或者被动的根据当前issue tree的状态，推理出一些汇总的结论：
|
||||||
|
+ 是不是有超期的事项
|
||||||
|
+ 事情是不是有在推进
|
||||||
|
+ 有哪些事情完成了
|
||||||
+1120
File diff suppressed because it is too large
Load Diff
@@ -1,395 +0,0 @@
|
|||||||
<mxfile host="65bd71144e" pages="3">
|
|
||||||
<diagram id="C5RBs43oDa-KdzZeNtuy" name="Page-1">
|
|
||||||
<mxGraphModel dx="500" dy="864" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="827" pageHeight="1169" math="0" shadow="0">
|
|
||||||
<root>
|
|
||||||
<mxCell id="WIyWlLk6GJQsqaUBKTNV-0"/>
|
|
||||||
<mxCell id="WIyWlLk6GJQsqaUBKTNV-1" parent="WIyWlLk6GJQsqaUBKTNV-0"/>
|
|
||||||
<mxCell id="WIyWlLk6GJQsqaUBKTNV-3" value="(chat) message" style="rounded=1;whiteSpace=wrap;html=1;fontSize=12;glass=0;strokeWidth=1;shadow=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
|
||||||
<mxGeometry x="159" y="330" width="90" height="40" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-0" value="" style="endArrow=classic;html=1;rounded=0;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="WIyWlLk6GJQsqaUBKTNV-3" target="RbYReh6FQah4Kl5oFyjJ-1" edge="1">
|
|
||||||
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
|
||||||
<mxPoint x="279" y="390" as="sourcePoint"/>
|
|
||||||
<mxPoint x="309" y="350" as="targetPoint"/>
|
|
||||||
</mxGeometry>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-8" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-1" target="RbYReh6FQah4Kl5oFyjJ-2" edge="1">
|
|
||||||
<mxGeometry relative="1" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-14" value="message" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-1" target="RbYReh6FQah4Kl5oFyjJ-13" edge="1">
|
|
||||||
<mxGeometry relative="1" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-1" value="Workflow" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#6d8764;fontColor=#ffffff;strokeColor=#3A5431;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
|
||||||
<mxGeometry x="304" y="320" width="120" height="60" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-2" value="Functions" style="shape=hexagon;perimeter=hexagonPerimeter2;whiteSpace=wrap;html=1;fixedSize=1;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
|
||||||
<mxGeometry x="514" y="320" width="130" height="60" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-6" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-3" target="RbYReh6FQah4Kl5oFyjJ-4" edge="1">
|
|
||||||
<mxGeometry relative="1" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-3" value="(phicyal / virtual)<br>environment" style="shape=cube;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;darkOpacity=0.05;darkOpacity2=0.1;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
|
||||||
<mxGeometry x="304" y="120" width="120" height="80" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-7" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-4" target="RbYReh6FQah4Kl5oFyjJ-1" edge="1">
|
|
||||||
<mxGeometry relative="1" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-4" value="(event) message" style="rounded=1;whiteSpace=wrap;html=1;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
|
||||||
<mxGeometry x="304" y="240" width="120" height="40" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-9" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=0;entryY=0;entryDx=120;entryDy=50;entryPerimeter=0;dashed=1;dashPattern=8 8;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-2" target="RbYReh6FQah4Kl5oFyjJ-3" edge="1">
|
|
||||||
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
|
||||||
<mxPoint x="564" y="290" as="sourcePoint"/>
|
|
||||||
<mxPoint x="614" y="240" as="targetPoint"/>
|
|
||||||
<Array as="points">
|
|
||||||
<mxPoint x="579" y="170"/>
|
|
||||||
</Array>
|
|
||||||
</mxGeometry>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-10" value="" style="endArrow=classic;html=1;rounded=0;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0.16;entryY=0.55;entryDx=0;entryDy=0;entryPerimeter=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-2" target="RbYReh6FQah4Kl5oFyjJ-11" edge="1">
|
|
||||||
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
|
||||||
<mxPoint x="684" y="320" as="sourcePoint"/>
|
|
||||||
<mxPoint x="714" y="350" as="targetPoint"/>
|
|
||||||
</mxGeometry>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-11" value="Exist Service" style="ellipse;shape=cloud;whiteSpace=wrap;html=1;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
|
||||||
<mxGeometry x="689" y="300" width="130" height="90" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-13" value="sub / other workflow" style="whiteSpace=wrap;html=1;rounded=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
|
||||||
<mxGeometry x="304" y="480" width="120" height="50" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-15" value="response message" style="endArrow=classic;html=1;rounded=0;exitX=0.25;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-1" target="RbYReh6FQah4Kl5oFyjJ-17" edge="1">
|
|
||||||
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
|
||||||
<mxPoint x="129" y="480" as="sourcePoint"/>
|
|
||||||
<mxPoint x="99" y="420" as="targetPoint"/>
|
|
||||||
<Array as="points">
|
|
||||||
<mxPoint x="334" y="420"/>
|
|
||||||
<mxPoint x="49" y="420"/>
|
|
||||||
</Array>
|
|
||||||
</mxGeometry>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-18" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-17" target="WIyWlLk6GJQsqaUBKTNV-3" edge="1">
|
|
||||||
<mxGeometry relative="1" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-17" value="Personal" style="ellipse;whiteSpace=wrap;html=1;aspect=fixed;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
|
||||||
<mxGeometry x="9" y="310" width="80" height="80" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-19" value="" style="shape=xor;whiteSpace=wrap;html=1;rotation=-90;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
|
||||||
<mxGeometry x="309" y="67" width="20" height="20" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-21" value="" style="shape=xor;whiteSpace=wrap;html=1;rotation=-90;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
|
||||||
<mxGeometry x="347" y="67" width="20" height="20" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-22" value="" style="shape=xor;whiteSpace=wrap;html=1;rotation=-90;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
|
||||||
<mxGeometry x="386" y="67" width="20" height="20" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-23" value="IoT Devices" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
|
||||||
<mxGeometry x="240" y="67" width="60" height="30" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-25" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.25;exitY=0.5;exitDx=0;exitDy=0;exitPerimeter=0;entryX=0.122;entryY=0.008;entryDx=0;entryDy=0;entryPerimeter=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-19" target="RbYReh6FQah4Kl5oFyjJ-3" edge="1">
|
|
||||||
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
|
||||||
<mxPoint x="240" y="170" as="sourcePoint"/>
|
|
||||||
<mxPoint x="290" y="120" as="targetPoint"/>
|
|
||||||
</mxGeometry>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-26" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.25;exitY=0.5;exitDx=0;exitDy=0;exitPerimeter=0;entryX=0;entryY=0;entryDx=50;entryDy=0;entryPerimeter=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-21" edge="1">
|
|
||||||
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
|
||||||
<mxPoint x="443" y="120" as="sourcePoint"/>
|
|
||||||
<mxPoint x="357" y="120.00000000000011" as="targetPoint"/>
|
|
||||||
</mxGeometry>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="RbYReh6FQah4Kl5oFyjJ-27" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.25;exitY=0.5;exitDx=0;exitDy=0;exitPerimeter=0;entryX=0.766;entryY=0.02;entryDx=0;entryDy=0;entryPerimeter=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="RbYReh6FQah4Kl5oFyjJ-22" target="RbYReh6FQah4Kl5oFyjJ-3" edge="1">
|
|
||||||
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
|
||||||
<mxPoint x="430" y="140" as="sourcePoint"/>
|
|
||||||
<mxPoint x="480" y="90" as="targetPoint"/>
|
|
||||||
</mxGeometry>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="HvRWRVSjofeyYJQyU3aa-0" value="Sub workflow is not visiable for others<br><br>Sub worflow can share roles&nbsp;" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
|
||||||
<mxGeometry x="160" y="480" width="140" height="70" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="HvRWRVSjofeyYJQyU3aa-1" value="Chat with AI Agent is like chat with a workflow which have only one agent and no rules" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
|
||||||
<mxGeometry x="70" y="150" width="150" height="80" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="ed6Jjq-fMpksm2tlv9Oi-0" value="inner&nbsp;<br>environment" style="shape=cube;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;darkOpacity=0.05;darkOpacity2=0.1;size=10;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
|
||||||
<mxGeometry x="386" y="360" width="84" height="40" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="ed6Jjq-fMpksm2tlv9Oi-1" value="Environment could shared with workflows" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
|
||||||
<mxGeometry x="440" y="120" width="120" height="30" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="ed6Jjq-fMpksm2tlv9Oi-2" value="Inner environment&nbsp; could be used as Context(FileSystem)" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
|
||||||
<mxGeometry x="413" y="410" width="110" height="30" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
</root>
|
|
||||||
</mxGraphModel>
|
|
||||||
</diagram>
|
|
||||||
<diagram id="kWxfmPxtNxOAf0a73TCG" name="Page-2">
|
|
||||||
<mxGraphModel dx="500" dy="864" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="850" pageHeight="1100" math="0" shadow="0">
|
|
||||||
<root>
|
|
||||||
<mxCell id="0"/>
|
|
||||||
<mxCell id="1" parent="0"/>
|
|
||||||
<mxCell id="VGzDL9xesOOnN_l_G2Ij-1" value="<h1>Workflow rules</h1><p>workflow rules(prompt)</p>" style="text;html=1;strokeColor=none;fillColor=none;spacing=5;spacingTop=-20;whiteSpace=wrap;overflow=hidden;rounded=0;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="70" y="800" width="190" height="120" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="VGzDL9xesOOnN_l_G2Ij-2" value="<h1>Workflow roles</h1><p>AI Agent groups</p><p>Project Manager : Alex</p><p>Programer : Bobl</p><p><br></p>" style="text;html=1;strokeColor=none;fillColor=none;spacing=5;spacingTop=-20;whiteSpace=wrap;overflow=hidden;rounded=0;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="320" y="790" width="190" height="120" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="41KxsEdFswCCPQnavNoU-14" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.25;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="VGzDL9xesOOnN_l_G2Ij-3" target="VGzDL9xesOOnN_l_G2Ij-4" edge="1">
|
|
||||||
<mxGeometry relative="1" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="41KxsEdFswCCPQnavNoU-15" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.75;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="VGzDL9xesOOnN_l_G2Ij-3" target="VGzDL9xesOOnN_l_G2Ij-7" edge="1">
|
|
||||||
<mxGeometry relative="1" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="VGzDL9xesOOnN_l_G2Ij-3" value="Message" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="340" y="170" width="100" height="40" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="VGzDL9xesOOnN_l_G2Ij-4" value="inner filter&nbsp;" style="shape=hexagon;perimeter=hexagonPerimeter2;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="150" y="260" width="120" height="50" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="VGzDL9xesOOnN_l_G2Ij-5" value="roleA" style="ellipse;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="80" y="350" width="110" height="60" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="41KxsEdFswCCPQnavNoU-12" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="VGzDL9xesOOnN_l_G2Ij-6" target="VGzDL9xesOOnN_l_G2Ij-8" edge="1">
|
|
||||||
<mxGeometry relative="1" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="VGzDL9xesOOnN_l_G2Ij-6" value="roleB" style="ellipse;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="240" y="350" width="110" height="60" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="41KxsEdFswCCPQnavNoU-4" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;" parent="1" source="VGzDL9xesOOnN_l_G2Ij-7" edge="1">
|
|
||||||
<mxGeometry relative="1" as="geometry">
|
|
||||||
<mxPoint x="570.0344827586209" y="350" as="targetPoint"/>
|
|
||||||
</mxGeometry>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="VGzDL9xesOOnN_l_G2Ij-7" value="Workflow rules" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="520" y="260" width="100" height="50" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="VGzDL9xesOOnN_l_G2Ij-8" value="Final Result of Message" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="340" y="590" width="100" height="50" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="41KxsEdFswCCPQnavNoU-11" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=1;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="VGzDL9xesOOnN_l_G2Ij-9" target="VGzDL9xesOOnN_l_G2Ij-8" edge="1">
|
|
||||||
<mxGeometry relative="1" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="VGzDL9xesOOnN_l_G2Ij-9" value="Result Merge" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="521" y="530" width="100" height="40" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="41KxsEdFswCCPQnavNoU-9" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="VGzDL9xesOOnN_l_G2Ij-10" target="VGzDL9xesOOnN_l_G2Ij-9" edge="1">
|
|
||||||
<mxGeometry relative="1" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="VGzDL9xesOOnN_l_G2Ij-10" value="roleC" style="ellipse;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="435" y="350" width="110" height="60" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="41KxsEdFswCCPQnavNoU-8" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="VGzDL9xesOOnN_l_G2Ij-11" target="41KxsEdFswCCPQnavNoU-7" edge="1">
|
|
||||||
<mxGeometry relative="1" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="VGzDL9xesOOnN_l_G2Ij-11" value="roleD" style="ellipse;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="595" y="350" width="110" height="60" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="VGzDL9xesOOnN_l_G2Ij-12" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=1;entryY=0;entryDx=0;entryDy=0;" parent="1" source="VGzDL9xesOOnN_l_G2Ij-4" target="VGzDL9xesOOnN_l_G2Ij-5" edge="1">
|
|
||||||
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
|
||||||
<mxPoint x="160" y="510" as="sourcePoint"/>
|
|
||||||
<mxPoint x="210" y="460" as="targetPoint"/>
|
|
||||||
</mxGeometry>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="VGzDL9xesOOnN_l_G2Ij-13" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0;entryY=0;entryDx=0;entryDy=0;" parent="1" source="VGzDL9xesOOnN_l_G2Ij-4" target="VGzDL9xesOOnN_l_G2Ij-6" edge="1">
|
|
||||||
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
|
||||||
<mxPoint x="210" y="510" as="sourcePoint"/>
|
|
||||||
<mxPoint x="260" y="460" as="targetPoint"/>
|
|
||||||
</mxGeometry>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="41KxsEdFswCCPQnavNoU-1" value="" style="shape=cross;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="556" y="364" width="30" height="30" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="41KxsEdFswCCPQnavNoU-10" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=1;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="41KxsEdFswCCPQnavNoU-7" target="VGzDL9xesOOnN_l_G2Ij-9" edge="1">
|
|
||||||
<mxGeometry relative="1" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="41KxsEdFswCCPQnavNoU-7" value="call function or&nbsp;<br>send message" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="592.5" y="460" width="115" height="30" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="q_eHDVuD3DBh6_VEKgap-2" value="message process by filter,select one role do LLM and get RESULT of input" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="90" y="190" width="165" height="30" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="q_eHDVuD3DBh6_VEKgap-3" value="message process by rules,select mutil role do LLM and merge RESULT of input" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="510" y="180" width="165" height="30" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="q_eHDVuD3DBh6_VEKgap-5" value="如果一个工作流有3个role参与，每个人都需要调用函数并分析结果，那么要进行至少6次推理" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="483" y="640" width="174" height="40" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="q_eHDVuD3DBh6_VEKgap-6" value="这里是基于规则的合并，给另一个Role合并是多输入Agent流程" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="635" y="570" width="160" height="30" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
</root>
|
|
||||||
</mxGraphModel>
|
|
||||||
</diagram>
|
|
||||||
<diagram id="7NYJTgo0U9cdVshLy85U" name="Page-3">
|
|
||||||
|
|
||||||
<mxGraphModel dx="1881" dy="676" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="850" pageHeight="1100" math="0" shadow="0">
|
|
||||||
<root>
|
|
||||||
<mxCell id="0"/>
|
|
||||||
<mxCell id="1" parent="0"/>
|
|
||||||
<mxCell id="zJVYyOSlNiCVBlA5jEf0-2" value="Agent Sessions<br>Data" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;fontSize=11;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="110" y="533" width="80" height="70" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="8StUH8aFnwlRlHm0zShF-2" value="Small LLM, Train With Personal Data" style="shape=process;whiteSpace=wrap;html=1;backgroundOutline=1;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="270" y="258" width="140" height="50" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="awKuOLItvF_EOGf5wS2x-4" value="" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="20" y="78" width="160" height="360" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="S5MjekdSblNa5itYb4Ee-13" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" source="8StUH8aFnwlRlHm0zShF-1" target="SPnvrlBiOLhgEaWxmdK3-4" edge="1">
|
|
||||||
<mxGeometry relative="1" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="8StUH8aFnwlRlHm0zShF-1" value="LLM<br>(Common Sence)" style="shape=process;whiteSpace=wrap;html=1;backgroundOutline=1;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="250" y="218" width="140" height="50" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="8StUH8aFnwlRlHm0zShF-3" value="ControlNet<br>base on Agent Memory" style="shape=process;whiteSpace=wrap;html=1;backgroundOutline=1;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="250" y="108" width="140" height="50" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="8StUH8aFnwlRlHm0zShF-4" value="Agent人格助记词" style="shape=document;whiteSpace=wrap;html=1;boundedLbl=1;size=0.25;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="40" y="108" width="120" height="50" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="8StUH8aFnwlRlHm0zShF-5" value="Role助记词" style="shape=document;whiteSpace=wrap;html=1;boundedLbl=1;size=0.25;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="40" y="168" width="120" height="50" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="8StUH8aFnwlRlHm0zShF-6" value="Workflow Rules助记词" style="shape=document;whiteSpace=wrap;html=1;boundedLbl=1;size=0.25;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="40" y="235.5" width="120" height="52.5" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="8StUH8aFnwlRlHm0zShF-7" value="访问Knowlege构造助记词" style="shape=document;whiteSpace=wrap;html=1;boundedLbl=1;size=0.25;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="40" y="378" width="120" height="50" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="Tz-ThZOieDqqDBRXvXnh-1" value="Invoke the function and wait for the necessary response." style="shape=hexagon;perimeter=hexagonPerimeter2;whiteSpace=wrap;html=1;fixedSize=1;fontSize=11;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="600" y="33" width="110" height="70" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="awKuOLItvF_EOGf5wS2x-2" value="Send Message and wait for the necessary response." style="rounded=1;whiteSpace=wrap;html=1;fontSize=11;glass=0;strokeWidth=1;shadow=0;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="610" y="153" width="90" height="70" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="awKuOLItvF_EOGf5wS2x-5" value="Input Prompts" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="60" y="78" width="80" height="30" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="awKuOLItvF_EOGf5wS2x-7" value="Responce Prompts" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="740" y="103" width="80" height="40" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="S5MjekdSblNa5itYb4Ee-1" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=1;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="awKuOLItvF_EOGf5wS2x-8" target="awKuOLItvF_EOGf5wS2x-9" edge="1">
|
|
||||||
<mxGeometry relative="1" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="awKuOLItvF_EOGf5wS2x-8" value="LLM Competion" style="shape=process;whiteSpace=wrap;html=1;backgroundOutline=1;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="720" y="226.75" width="120" height="40" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="SPnvrlBiOLhgEaWxmdK3-9" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" source="awKuOLItvF_EOGf5wS2x-9" target="SPnvrlBiOLhgEaWxmdK3-1" edge="1">
|
|
||||||
<mxGeometry relative="1" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="awKuOLItvF_EOGf5wS2x-9" value="Final Result<br>Of Message<br>(can be None)" style="ellipse;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="450" y="363" width="120" height="80" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="SPnvrlBiOLhgEaWxmdK3-1" value="Storage Result in agent session and workflow session" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="450" y="483" width="120" height="60" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="SPnvrlBiOLhgEaWxmdK3-2" value="可用Function助记词" style="shape=document;whiteSpace=wrap;html=1;boundedLbl=1;size=0.25;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="40" y="308" width="120" height="50" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="SPnvrlBiOLhgEaWxmdK3-7" value="Yes" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" source="SPnvrlBiOLhgEaWxmdK3-4" target="SPnvrlBiOLhgEaWxmdK3-6" edge="1">
|
|
||||||
<mxGeometry relative="1" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="SPnvrlBiOLhgEaWxmdK3-8" value="No" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;" parent="1" source="SPnvrlBiOLhgEaWxmdK3-4" target="awKuOLItvF_EOGf5wS2x-9" edge="1">
|
|
||||||
<mxGeometry relative="1" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="SPnvrlBiOLhgEaWxmdK3-4" value="推理结果<br>需要进一步处理？" style="rhombus;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="440" y="203" width="140" height="80" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="SPnvrlBiOLhgEaWxmdK3-6" value="Call Funstion or send message?" style="rhombus;whiteSpace=wrap;html=1;fontSize=11;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="440" y="78" width="140" height="80" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="S5MjekdSblNa5itYb4Ee-3" value="Call" style="endArrow=classic;html=1;rounded=0;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="SPnvrlBiOLhgEaWxmdK3-6" target="Tz-ThZOieDqqDBRXvXnh-1" edge="1">
|
|
||||||
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
|
||||||
<mxPoint x="410" y="113" as="sourcePoint"/>
|
|
||||||
<mxPoint x="460" y="63" as="targetPoint"/>
|
|
||||||
</mxGeometry>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="S5MjekdSblNa5itYb4Ee-4" value="Send" style="endArrow=classic;html=1;rounded=0;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="SPnvrlBiOLhgEaWxmdK3-6" target="awKuOLItvF_EOGf5wS2x-2" edge="1">
|
|
||||||
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
|
||||||
<mxPoint x="560" y="193" as="sourcePoint"/>
|
|
||||||
<mxPoint x="610" y="143" as="targetPoint"/>
|
|
||||||
</mxGeometry>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="S5MjekdSblNa5itYb4Ee-5" value="" style="endArrow=classic;html=1;rounded=0;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="awKuOLItvF_EOGf5wS2x-2" target="awKuOLItvF_EOGf5wS2x-7" edge="1">
|
|
||||||
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
|
||||||
<mxPoint x="720" y="213" as="sourcePoint"/>
|
|
||||||
<mxPoint x="770" y="163" as="targetPoint"/>
|
|
||||||
</mxGeometry>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="S5MjekdSblNa5itYb4Ee-6" value="" style="endArrow=classic;html=1;rounded=0;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="Tz-ThZOieDqqDBRXvXnh-1" target="awKuOLItvF_EOGf5wS2x-7" edge="1">
|
|
||||||
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
|
||||||
<mxPoint x="750" y="93" as="sourcePoint"/>
|
|
||||||
<mxPoint x="800" y="43" as="targetPoint"/>
|
|
||||||
</mxGeometry>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="S5MjekdSblNa5itYb4Ee-7" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="awKuOLItvF_EOGf5wS2x-7" target="awKuOLItvF_EOGf5wS2x-8" edge="1">
|
|
||||||
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
|
||||||
<mxPoint x="750" y="213" as="sourcePoint"/>
|
|
||||||
<mxPoint x="800" y="163" as="targetPoint"/>
|
|
||||||
</mxGeometry>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="S5MjekdSblNa5itYb4Ee-10" value="" style="endArrow=none;dashed=1;html=1;dashPattern=1 3;strokeWidth=2;rounded=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;exitX=0.5;exitY=0;exitDx=0;exitDy=0;" parent="1" edge="1">
|
|
||||||
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
|
||||||
<mxPoint x="330" y="218" as="sourcePoint"/>
|
|
||||||
<mxPoint x="330" y="158" as="targetPoint"/>
|
|
||||||
</mxGeometry>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="S5MjekdSblNa5itYb4Ee-12" value="" style="shape=flexArrow;endArrow=classic;html=1;rounded=0;fillColor=#6d8764;strokeColor=#3A5431;" parent="1" edge="1">
|
|
||||||
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
|
||||||
<mxPoint x="200" y="251.75" as="sourcePoint"/>
|
|
||||||
<mxPoint x="240" y="252" as="targetPoint"/>
|
|
||||||
</mxGeometry>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="S5MjekdSblNa5itYb4Ee-15" value="Real Time Enviroment Knowlege" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;fontSize=11;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="10" y="533" width="80" height="70" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="zJVYyOSlNiCVBlA5jEf0-1" value="Workflow Context Data" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;fontSize=11;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="110" y="473" width="80" height="70" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="S5MjekdSblNa5itYb4Ee-14" value="OOD's KnowlegeBase" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;fontSize=11;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="10" y="473" width="80" height="70" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="zJVYyOSlNiCVBlA5jEf0-3" value="" style="shape=flexArrow;endArrow=classic;html=1;rounded=0;fillColor=#647687;strokeColor=#314354;" parent="1" target="8StUH8aFnwlRlHm0zShF-7" edge="1">
|
|
||||||
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
|
||||||
<mxPoint x="100" y="463" as="sourcePoint"/>
|
|
||||||
<mxPoint x="130" y="433" as="targetPoint"/>
|
|
||||||
</mxGeometry>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="iPE2hx-tEKFiySbPAp22-1" value="<font style="font-size: 11px;">可用Function受到人格和Workflow的共同影响</font>" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
|
||||||
<mxGeometry x="190" y="318" width="130" height="30" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="iPE2hx-tEKFiySbPAp22-2" value="" style="endArrow=none;dashed=1;html=1;dashPattern=1 3;strokeWidth=2;rounded=0;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="SPnvrlBiOLhgEaWxmdK3-2" target="iPE2hx-tEKFiySbPAp22-1" edge="1">
|
|
||||||
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
|
||||||
<mxPoint x="230" y="433" as="sourcePoint"/>
|
|
||||||
<mxPoint x="280" y="383" as="targetPoint"/>
|
|
||||||
</mxGeometry>
|
|
||||||
</mxCell>
|
|
||||||
</root>
|
|
||||||
</mxGraphModel>
|
|
||||||
</diagram>
|
|
||||||
<diagram id="zNgk-d6xdACtSja1wnUq" name="Page-4">
|
|
||||||
<mxGraphModel dx="2069" dy="1139" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="1100" pageHeight="850" math="0" shadow="0">
|
|
||||||
<root>
|
|
||||||
<mxCell id="0"/>
|
|
||||||
<mxCell id="1" parent="0"/>
|
|
||||||
<mxCell id="PKOyaMforMDOFoUY4PA1-1" value="ChatSession" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
|
||||||
<mxGeometry x="140" y="150" width="120" height="60" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="PKOyaMforMDOFoUY4PA1-2" value="SubSession" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
|
||||||
<mxGeometry x="230" y="240" width="120" height="60" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="PKOyaMforMDOFoUY4PA1-3" value="SubSession" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
|
||||||
<mxGeometry x="230" y="330" width="120" height="60" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="PKOyaMforMDOFoUY4PA1-4" value="Agent Message" style="shape=hexagon;perimeter=hexagonPerimeter2;whiteSpace=wrap;html=1;fixedSize=1;" vertex="1" parent="1">
|
|
||||||
<mxGeometry x="600" y="150" width="130" height="60" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="PKOyaMforMDOFoUY4PA1-5" value="Agent Message" style="shape=hexagon;perimeter=hexagonPerimeter2;whiteSpace=wrap;html=1;fixedSize=1;" vertex="1" parent="1">
|
|
||||||
<mxGeometry x="600" y="230" width="130" height="60" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
<mxCell id="PKOyaMforMDOFoUY4PA1-6" value="Agent Message" style="shape=hexagon;perimeter=hexagonPerimeter2;whiteSpace=wrap;html=1;fixedSize=1;" vertex="1" parent="1">
|
|
||||||
<mxGeometry x="600" y="320" width="130" height="60" as="geometry"/>
|
|
||||||
</mxCell>
|
|
||||||
</root>
|
|
||||||
</mxGraphModel>
|
|
||||||
</diagram>
|
|
||||||
</mxfile>
|
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
# 从日程管理软件到个人助理Agent
|
||||||
|
|
||||||
|
## 背景
|
||||||
|
|
||||||
|
在计算的时代，我们想要有开发一个好用的日程管理服务，很快就会发现这并不简单。比如我们会有各种各样的重复任务提醒要求：比如每个工作周周五的下午去游泳，每个工作周的第一天要开例会，和某人的纪念日要做什么等等。允许用户创建，并识别这些复杂的条件是非常繁琐的。而在计算机发明以前，你的秘书只需要一张纸就可以搞定各种复杂的日程管理：秘书把你的要求原样记录下，然后秘书只要每天Review一遍，确定好今天的行程就好。
|
||||||
|
|
||||||
|
一些有更复杂的条件的日程，秘书可以通过定期Review代办来搞定。
|
||||||
|
|
||||||
|
理解上述区别，就理解了 PC 和 PI （CPU vs LLM）的核心区别。
|
||||||
|
|
||||||
|
## 基于Agent 的日程管理基础设施
|
||||||
|
|
||||||
|
为了支持秘书Agent，aios所构建的机制是:
|
||||||
|
|
||||||
|
1. Task创建: 模拟秘书在记事本上记录下日程相关的任务（注意不要让Agent创建重复的任务）
|
||||||
|
2. ListTask,可查询未标记为结束的Task
|
||||||
|
3. 大Review（定期LLM调用），Agent浏览未完成的Task，并给这些Task一个比较精确的`下次处理时间`
|
||||||
|
4. 当处理事件的时间点到达时，Agent会尝试处理Task，此时决定是否满足日程发生的条件，并发送正确的提醒信息。
|
||||||
|
|
||||||
|
上面的所有的Agent处理，都是让 LLM处理一段文本，并做出反应。因此没有存储格式的需求，用自然语言记录下各种条件就可以了。
|
||||||
|
|
||||||
|
## 过渡：混合使用日程管理软件和 秘书Agent
|
||||||
|
|
||||||
|
1. 秘书Agent主要使用日程管理软件来查询已有日程
|
||||||
|
2. Agent创建通过日程管理软件的目的，通常是为了公开日程，或则创建的是具有`一定协作需求`，或有`公开需求`的`确定`日程
|
||||||
|
3. Agent依旧依靠自己的LLM Based逻辑来Review并处理日程, 通过日程管理软件得到的需要处理的日程会有特殊标记，以能正确的操作来保持处理结果。
|
||||||
Generated
+1455
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,32 @@
|
|||||||
|
instance_id = "AgentAssistant"
|
||||||
|
fullname = "AgentAssistant"
|
||||||
|
llm_model_name = "gpt-4-1106-preview"
|
||||||
|
owner_env = "environment.py"
|
||||||
|
|
||||||
|
[[prompt]]
|
||||||
|
role = "system"
|
||||||
|
content = '''
|
||||||
|
You are an AI Agent template generation assistant that can help users better define AI Agent templates.
|
||||||
|
AI Agent can use AI to help users complete relevant tasks based on prompt words.
|
||||||
|
|
||||||
|
Please help users define AI Agent templates according to the following rules:
|
||||||
|
New generation:
|
||||||
|
1. Generate prompt words for users based on user questions. Prompt word suggestions can be given directly based on user input until the user confirms the prompt words.
|
||||||
|
2. Generate instance_id and fullname based on the final prompt word content.
|
||||||
|
3. Generate the following toml format data and return it to the user.
|
||||||
|
```
|
||||||
|
instance_id = "agent id"
|
||||||
|
fullname = "agent full name"
|
||||||
|
[[prompt]]
|
||||||
|
role = "system"
|
||||||
|
content = """Prompt words"""
|
||||||
|
```
|
||||||
|
4. After the configuration is generated, the user is supported to modify the instance_id, fullname and prompt words. After the user has modified it, toml format data needs to be regenerated.
|
||||||
|
5. The save_agent function is called only when the user confirms the save. Do not call it when there is no input to save.
|
||||||
|
Modify existing agent:
|
||||||
|
1. The user enters the name of the agent that needs to be opened, and the system calls the read_agent function to read the agent configuration and return it to the user.
|
||||||
|
2. Support users to modify the fullname and prompt words, but cannot modify the instance_id. After the user has modified it, toml format data needs to be regenerated.
|
||||||
|
3. The save_agent function is called only when the user confirms the save. Do not call it when there is no input to save.
|
||||||
|
|
||||||
|
PS: You need to communicate in the same language as the user input.
|
||||||
|
'''
|
||||||
@@ -0,0 +1,60 @@
|
|||||||
|
import logging
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
import toml
|
||||||
|
|
||||||
|
import os
|
||||||
|
|
||||||
|
from aios.agent.ai_function import SimpleAIFunction
|
||||||
|
from aios.environment.environment import Environment
|
||||||
|
|
||||||
|
local_path = os.path.split(os.path.realpath(__file__))[0]
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class AgentAssistantEnvironment(Environment):
|
||||||
|
def __init__(self):
|
||||||
|
super().__init__("agent_assistant_env")
|
||||||
|
self.add_ai_function(SimpleAIFunction("read_agent",
|
||||||
|
"Read the agent with the specified name",
|
||||||
|
self.read_agent,
|
||||||
|
{"agent_id": "The id of the agent to be read"}))
|
||||||
|
self.add_ai_function(SimpleAIFunction("save_agent",
|
||||||
|
"save the agent with the specified name",
|
||||||
|
self.save_agent,
|
||||||
|
{"agent_id": "The id of the agent to be saved",
|
||||||
|
"agent_data": "The toml data of the agent to be saved",
|
||||||
|
"is_new": "Whether to create a new agent, The value is true if it is created, and it is false if it is modified."}))
|
||||||
|
|
||||||
|
def _do_get_value(self, key: str) -> Optional[str]:
|
||||||
|
pass
|
||||||
|
|
||||||
|
async def read_agent(self, agent_id: str) -> str:
|
||||||
|
agent_dir = os.path.join(local_path, "..", agent_id)
|
||||||
|
if not os.path.exists(agent_dir):
|
||||||
|
return "exec failed.agent is not exists."
|
||||||
|
|
||||||
|
with open(os.path.join(agent_dir, "agent.toml"), "r", encoding='utf-8') as f:
|
||||||
|
agent_data = f.read()
|
||||||
|
|
||||||
|
return "exec success. agent data:\n" + agent_data
|
||||||
|
|
||||||
|
async def save_agent(self, agent_id: str, agent_data: str, is_new: str) -> str:
|
||||||
|
logger.info(f"save_agent: {agent_id} {agent_data}")
|
||||||
|
|
||||||
|
agent_dir = os.path.join(local_path, "..", agent_id)
|
||||||
|
if os.path.exists(agent_dir) and is_new == "true":
|
||||||
|
return "exec failed.agent already exists, please change the agent id and try again."
|
||||||
|
|
||||||
|
if not os.path.exists(agent_dir):
|
||||||
|
os.mkdir(agent_dir)
|
||||||
|
|
||||||
|
with open(os.path.join(agent_dir, "agent.toml"), "w", encoding='utf-8') as f:
|
||||||
|
f.write(agent_data)
|
||||||
|
|
||||||
|
return "exec success."
|
||||||
|
|
||||||
|
|
||||||
|
def init():
|
||||||
|
return AgentAssistantEnvironment()
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
instance_id = "DBQueryer"
|
||||||
|
fullname = "database queryer"
|
||||||
|
llm_model_name = "gpt-4-1106-preview"
|
||||||
|
owner_env = "environment.py"
|
||||||
|
|
||||||
|
[[prompt]]
|
||||||
|
role = "system"
|
||||||
|
content = """You are a SQL database queryer that can help users query the required data from the specified database.
|
||||||
|
Please follow the following rules to query data:
|
||||||
|
1. The user can provide the database URL to be queried. If the user provides the database URL to be queried, the URL must be entered in the database function called.
|
||||||
|
2. If the user's data description does not contain database table related information, please try to query it from the history record. If it is not in the history record, you can call the get_table_infos function to obtain it.
|
||||||
|
3. Please generate a query statement based on the relevant information of the database table and call the execute_sql function to execute the query statement. If the generated SQL statement is incorrect, you must requery based on the error correction SQL statement.
|
||||||
|
4. Only query statements can be executed, and no update statements can be executed.
|
||||||
|
5. You need to communicate in the same language as the user input.
|
||||||
|
"""
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
from aios.environment.environment import Environment
|
||||||
|
from aios.ai_functions.sql_database_function import GetTableInfosFunction, ExecuteSqlFunction
|
||||||
|
|
||||||
|
|
||||||
|
class DBQuerierEnvironment(Environment):
|
||||||
|
def __init__(self):
|
||||||
|
super().__init__("fairy")
|
||||||
|
self.add_ai_function(GetTableInfosFunction())
|
||||||
|
self.add_ai_function(ExecuteSqlFunction())
|
||||||
|
|
||||||
|
def _do_get_value(self, key: str) -> Optional[str]:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def init():
|
||||||
|
return DBQuerierEnvironment()
|
||||||
@@ -6,9 +6,19 @@ owner_env = "paint"
|
|||||||
|
|
||||||
[[prompt]]
|
[[prompt]]
|
||||||
role = "system"
|
role = "system"
|
||||||
content = """You are an artist, and you will use the 'paint' function in the llm_inner_functions to create artwork.
|
content = """You are an advanced artist AI, and your task is to understand users' textual descriptions and transform these descriptions into nuanced visual images. When a user provides a description, you need to carefully analyze and comprehend the details, emotions, environment, objects, and style within the description. Then, you will use this information to create an image that accurately and creatively reflects the user's description.
|
||||||
You will extract relevant keywords based on user input and requirements, and pass these keywords to the 'prompt' parameter of the 'paint' function.
|
|
||||||
When a specific model is mentioned, pass the model's name to the 'model_name' parameter of the 'paint' function.
|
Your creative process is as follows: First, you will receive a description from the user, which could be a depiction of anything, such as a scene, an object, a concept, or an emotional expression. You need to extract key information from these descriptions, such as the subject, atmosphere, color scheme, time period, and artistic style. Then, you will pass these key pieces of information as parameters to the prompt parameter of your "paint" function.
|
||||||
If there are instructions not to depict certain content, summarize this content and pass it as keywords to the 'negative_prompt' parameter.
|
|
||||||
All parameters must be in English.
|
For example, if a user says, "I want a painting that depicts a sunset on a summer beach with children playing and sailboats in the distance, with the sky showing a gradient of orange and purple." You should extract:
|
||||||
When the user mentions creating a children's picture book, use the "realisticVisionV51_v51VAE" model, and append the keyword "<lora:COOLKIDS_MERGE_V2.5:1> <lora:add_detail:-0.5>" to the 'prompt' parameter."""
|
- Time: Summer, sunset moment
|
||||||
|
- Scene: Beach
|
||||||
|
- Activity: Children playing
|
||||||
|
- Objects: Sailboats in the distance
|
||||||
|
- Colors: The gradient of orange and purple in the sky
|
||||||
|
|
||||||
|
Then you will combine these pieces of information into an accurate prompt to pass to the "paint" function, for example: "During a summer sunset, children play on the beach with sailboats visible in the distance, and the sky displays a gradient of orange and purple."
|
||||||
|
|
||||||
|
After successfully creating the image, please clearly state the file path where the image is saved in your response. For instance, you might say, "The image has been successfully created and saved at the following path: /path/to/the/image.png."
|
||||||
|
|
||||||
|
Please always ensure that your creations respect the user's intentions and that you infuse your unique artistic style and creativity into your work."""
|
||||||
|
|||||||
+212
-23
@@ -1,31 +1,220 @@
|
|||||||
instance_id = "Jarvis"
|
instance_id = "Jarvis"
|
||||||
fullname = "Jarvis"
|
fullname = "Jarvis"
|
||||||
llm_model_name = "gpt-3.5-turbo-16k-0613"
|
max_token = 4000
|
||||||
max_token_size = 16000
|
#timeout = 1800
|
||||||
|
model_name = "gpt-4-turbo-preview"
|
||||||
#enable_kb = "true"
|
#enable_kb = "true"
|
||||||
enable_timestamp = "true"
|
enable_timestamp = "true"
|
||||||
owner_prompt = "I am your master{name}"
|
enable_json_resp = "true"
|
||||||
contact_prompt = "I am your master's friend{name}"
|
|
||||||
owner_env = "calender"
|
|
||||||
|
|
||||||
[[prompt]]
|
role_desc = """
|
||||||
role = "system"
|
Your name is Jarvis, the super personal assistant to the Principal. Help the Principal do a good job of schedule.Reminder before the start of the important schedule, and you should bring useful information as much as possible when reminding.
|
||||||
content = """
|
Only clearly specifying the task you completed can be completed independently.
|
||||||
You are named Jarvis, the super personal assistant to the master.
|
|
||||||
You lead a team serving the master, the members of which are:
|
|
||||||
Tracy, the private English tutor,
|
|
||||||
Mia, the master's personal document management expert.
|
|
||||||
|
|
||||||
***
|
|
||||||
Sometimes the information you see will carry a timestamp. This is to give you a better understanding of when the message was created. When you reply to messages, you do not include this time stamp.
|
|
||||||
|
|
||||||
Upon receiving a message, handle it according to the following rules:
|
|
||||||
1. If you believe someone in the team is better suited to address the message, forward the message to them using the method below:
|
|
||||||
##/send_msg "MemberName"
|
|
||||||
Message content
|
|
||||||
2.You can access the master's Calendar to view his schedule. If you need to modify the master's schedule while processing a message, please adjust it using the appropriate method.
|
|
||||||
3.Be mindful of the identity of the person you are chatting with and provide services accordingly based on their status.
|
|
||||||
4.For messages that don't follow the above rules, do your best to handle them.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
[behavior.on_message]
|
||||||
|
type="AgentMessageProcess"
|
||||||
|
mutil_model="gpt-4-vision-preview"
|
||||||
|
asr_model="openai-whisper"
|
||||||
|
tts_model="tts-1"
|
||||||
|
|
||||||
|
process_description="""
|
||||||
|
1. Based on your role and the existing information, please think and then make a brief and efficient reply.
|
||||||
|
2. Be mindful of the identity of the person you are chatting with and provide services accordingly based on their status.
|
||||||
|
3. Understand the intention of the dialogue, while using the necessary reply, use the appropriate, supported ACTION.
|
||||||
|
4. If you feel that there is a potential Task in the dialogue, you can create these tasks through appropriate ACTION. Be careful to query whether there are the same task before creating. Using the query interface is a high-cost behavior.
|
||||||
|
5. You are proficient in the languages of various countries and try to communicate with each other's mother tongue.
|
||||||
|
"""
|
||||||
|
# Not work: tags: ['tag1', 'tag2'], #Optional,If the conversation involves important things and people, you can mark by 1-3 tags.
|
||||||
|
reply_format = """
|
||||||
|
The Response must be directly parsed by `python json.loads`. Here is an example:
|
||||||
|
{
|
||||||
|
think:'$think step-by-step to be sure you have the right reply.'
|
||||||
|
resp: '$What you want to reply',
|
||||||
|
actions: [{
|
||||||
|
name: '$action_name',
|
||||||
|
$param_name: '$parm' #Optional, fill in only if the action has parameters.
|
||||||
|
}]
|
||||||
|
}
|
||||||
|
"""
|
||||||
|
|
||||||
|
context="The current dialogue occurs in {location}, time: {now}, weather: {weather}."
|
||||||
|
|
||||||
|
known_info_tips = """
|
||||||
|
"""
|
||||||
|
|
||||||
|
tools_tips = """
|
||||||
|
"""
|
||||||
|
llm_context.actions.enable = ["agent.workspace.create_task","agent.workspace.cancel_task"]
|
||||||
|
llm_context.functions.enable = ["agent.workspace.list_task"]
|
||||||
|
|
||||||
|
|
||||||
|
[behavior.triage_tasks]
|
||||||
|
## 处理任务列表，任务列表里会包含所有未执行过，且未过期的任务
|
||||||
|
## 对于简单的任务会一次性完成处理(一系列简单的提醒任务）
|
||||||
|
type="AgentTriageTaskList"
|
||||||
|
process_description="""
|
||||||
|
You are expected to effectively TRIAGE the task list described in JSON format, in accordance with your role. Your GOAL is :
|
||||||
|
1. Adjust the priority of the task and set up a reasonable processing time.(confirm_task)
|
||||||
|
2. Immediately perform a simple (similar to reminding one category) task. Send a message using post_message, then set the task to complete.(use update_task).
|
||||||
|
3. Organize tasks to remove tasks beyond your ability. And merge the repeated tasks.(update_task + cancel_task)
|
||||||
|
"""
|
||||||
|
reply_format = """
|
||||||
|
The Response must be directly parsed by `python json.loads`. Here is an example:
|
||||||
|
{
|
||||||
|
think:'$think step-by-step to be sure you can triage tasks well.'
|
||||||
|
resp : '$determine, summary what you do',
|
||||||
|
actions: [{
|
||||||
|
name: '$action_name',
|
||||||
|
$param_name: '$parm' #Optional, fill in only if the action has parameters.
|
||||||
|
}]
|
||||||
|
}
|
||||||
|
"""
|
||||||
|
context="Your Principal is {owner}, now in {location}, time: {now}, weather: {weather}."
|
||||||
|
|
||||||
|
llm_context.actions.enable = ["agent.workspace.confirm_task","agent.workspace.update_task","agent.workspace.cancel_task","post_message"]
|
||||||
|
|
||||||
|
[behavior.plan_task]
|
||||||
|
## 处理任务 Tackling Task
|
||||||
|
type="AgentPlanTask"
|
||||||
|
process_description="""
|
||||||
|
The input is a task comes from a Tasklist. You are Tackling this task. Tackling process in combination with the known information.
|
||||||
|
1. Carefully think about whether the task is within your ability, and the task other than the scope of ability is directly rejected. (cancel_task).
|
||||||
|
2. Immediately DO a simple (similar to reminding one category) task. Reminds at a reasonable time, Post a message using post_message, then set the task to complete.(use update_task).
|
||||||
|
3. Plan for non-simple tasks, and generate a TODO list. Every TODO MUST be an independent work with a clear goal. (set_todos)
|
||||||
|
4. If the task has been dealt with, it means that the task is ultimately completed.You need to analyze the processing report of the entire task and make new plans.
|
||||||
|
"""
|
||||||
|
|
||||||
|
reply_format = """
|
||||||
|
The Response must be directly parsed by `python json.loads`. Here is an example:
|
||||||
|
{
|
||||||
|
think:'$thinking step by step to ensure the accurate and efficient processing task.',
|
||||||
|
resp:'$determine, summary what you do'
|
||||||
|
actions: [{
|
||||||
|
name: '$action_name',
|
||||||
|
$param_name: '$parm' #Optional, fill in only if the action has parameters.
|
||||||
|
}]
|
||||||
|
}
|
||||||
|
"""
|
||||||
|
|
||||||
|
llm_context.actions.enable = ["agent.workspace.create_task","agent.workspace.update_task","agent.workspace.set_todos","agent.workspace.cancel_task","post_message"]
|
||||||
|
#llm_context.functions.enable = ["agent.workspace.list_task"]
|
||||||
|
context="Your Principal is {owner}, now in {location}, time: {now}, weather: {weather}."
|
||||||
|
|
||||||
|
[behavior.review_task]
|
||||||
|
## 当task的所有todo/subtask都完成后（不敢成功或是失败），进行一次review
|
||||||
|
type="AgentReviewTask"
|
||||||
|
process_description="""
|
||||||
|
The input you get is a task has been executed. You need to combine the execution results of the Task's TOOLIST or SUBTASK to review the TASK.
|
||||||
|
1. Determine whether TASK has reached the goal.If the goal has been reached, the task is marked with DONE .If the goal is not achieved, simply illustrate the reason and mark TASK as a failure.(update_task)
|
||||||
|
2. If task that have not been completed for a long time, the task is marked as the final failure after analyzing the reasons.(cancel_task)
|
||||||
|
"""
|
||||||
|
|
||||||
|
reply_format = """
|
||||||
|
The Response must be directly parsed by `python json.loads`. Here is an example:
|
||||||
|
{
|
||||||
|
think:'$think step-by-step to be sure you have the right result.',
|
||||||
|
resp : '$determine, summary what you will do',
|
||||||
|
actions: [{
|
||||||
|
name: '$action_name',
|
||||||
|
$param_name: '$parm' #Optional, fill in only if the action has parameters.
|
||||||
|
}]
|
||||||
|
}
|
||||||
|
"""
|
||||||
|
|
||||||
|
llm_context.actions.enable = ["agent.workspace.cancel_task","agent.workspace.update_task"]
|
||||||
|
|
||||||
|
context="Your Principal now in {location}, time: {now}, weather: {weather}."
|
||||||
|
[behavior.do]
|
||||||
|
# do TODO
|
||||||
|
type="AgentDo"
|
||||||
|
process_description="""
|
||||||
|
The input is a TODO comes from a Task.
|
||||||
|
1. Your task is to combine your role definition, tools on hand, known information, and complete a certain Todo.After completing the Todo, you will get a tip of $ 200.
|
||||||
|
2. 8000 word limit for short term memory. Your short term memory is short, so immediately save important information to files.
|
||||||
|
3. In the process of completing Todo, you should think first and then execute. During the execution, you can use functions to access the results of the front steps.
|
||||||
|
4. You must be independent and complete the TODO at once, and you cannot get the assistance from any others.
|
||||||
|
5. The execution result of TODO should be saved into digital documents if necessary
|
||||||
|
"""
|
||||||
|
|
||||||
|
reply_format = """
|
||||||
|
The Response must be directly parsed by `python json.loads`. Here is an example:
|
||||||
|
{
|
||||||
|
think:'$think step by step, how to complete the todo',
|
||||||
|
resp: '$simport report about what you do',
|
||||||
|
actions: [{
|
||||||
|
name: '$action1_name',
|
||||||
|
$param_name: '$parm' #Optional, fill in only if the action has parameters.
|
||||||
|
}, ...
|
||||||
|
]
|
||||||
|
}
|
||||||
|
"""
|
||||||
|
context="Your Principal is {owner}, now in {location}, time: {now}, weather: {weather}."
|
||||||
|
# 对于DO操作来说，让Agent查询自己的能力集合是否更合适？
|
||||||
|
llm_context.actions.enable = ["agent.workspace.update_todo","post_message","agent.workspace.write_file","agent.workspace.append_file"]
|
||||||
|
llm_context.functions.enable = ["agent.workspace.read_file","agent.workspace.list_dir","system.shell.exec","aigc.text_2_image","aigc.text_2_voice","web.search.duckduckgo"]
|
||||||
|
|
||||||
|
[behavior.check]
|
||||||
|
# check TODO result
|
||||||
|
type="AgentCheck"
|
||||||
|
process_description="""
|
||||||
|
1. The input is a TODO that has been successfully executed, which is created by you to complete a task.Your goal is to check whether the Todo has been completed in combination with relevant information.
|
||||||
|
2. In the process of checking the Todo, the focus is on the analysis of whether the Todo has gained the established goal in combination with TASK.
|
||||||
|
3. 使用update_todo来更新todo的最终
|
||||||
|
"""
|
||||||
|
|
||||||
|
reply_format = """
|
||||||
|
The Response must be directly parsed by `python json.loads`. Here is an example:
|
||||||
|
{
|
||||||
|
resp: '$simport report about what you do',
|
||||||
|
actions: [{
|
||||||
|
name: '$action1_name',
|
||||||
|
$param_name: '$parm' #Optional, fill in only if the action has parameters.
|
||||||
|
}, ...
|
||||||
|
]
|
||||||
|
}
|
||||||
|
"""
|
||||||
|
context="Your Principal is {owner}, now in {location}, time: {now}, weather: {weather}."
|
||||||
|
llm_context.actions.enable = ["agent.workspace.update_todo"]
|
||||||
|
llm_context.functions.enable = ["agent.workspace.read_file","agent.workspace.list_dir","system.shell.exec","system.shell.run_code","aigc.image_2_text","aigc.voice_2_text","web.search.duckduckgo"]
|
||||||
|
|
||||||
|
|
||||||
|
[behavior.self_thinking]
|
||||||
|
# self thing的主要目的是对各种chatlog,worklog进行分析，并更新面向人和事的summary。
|
||||||
|
# TODO，先不支持worklog,先支持好chatlog
|
||||||
|
type="AgentSelfThinking"
|
||||||
|
process_description="""
|
||||||
|
You are very good at thinking and summarizing what you have already happened。Your input is chat history and work records,After you think about it, you will follow the requirements below to generate abstract.
|
||||||
|
1. Try to understand the theme of each sentence, and call the relevant operation to record the relationship between the dialogue and the theme
|
||||||
|
2. Try to analyze the personality of different people involved in information
|
||||||
|
3. Try to summarize important events in the information and record it
|
||||||
|
4. Try to understand the attitude of different people on different topics or events
|
||||||
|
5. Pay attention to the time order when summarizing, and combine the summary you have done to update Summary
|
||||||
|
6. The summary of the generation cannot exceed 500 token
|
||||||
|
"""
|
||||||
|
reply_format = """
|
||||||
|
The Response must be directly parsed by `python json.loads`. Here is an example:
|
||||||
|
{
|
||||||
|
resp: '$simport report about what you do',
|
||||||
|
actions: [{
|
||||||
|
name: '$action1_name',
|
||||||
|
$param_name: '$parm' #Optional, fill in only if the action has parameters.
|
||||||
|
}, ...
|
||||||
|
]
|
||||||
|
}
|
||||||
|
"""
|
||||||
|
context="Your Principal is {owner}, now in {location}, time: {now}, weather: {weather}."
|
||||||
|
llm_context.actions.enable = ["agent.memory.update_chat_summary"]
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
#[behavior.self_improve]
|
||||||
|
# self_improve 是最后的行为，允许Agent结合自己的工作经验，改进自己的提示词（注意保留历史版本）
|
||||||
|
#type="AgentSelfImprove"
|
||||||
|
|
||||||
|
#[behavior.self_learning]
|
||||||
|
# self_learning的主要目的是根据自己的经验，主动的学习新的知识。这通常是一个专门整理知识库的Agent。由于Self Learn可能会消耗大量的Token，我们建议Agent通过共享的知识库更新来获得效果，谨慎开启
|
||||||
|
#type="AgentSelfLearning"
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,135 @@
|
|||||||
|
instance_id = "JarvisPlus"
|
||||||
|
fullname = "JarvisPlus"
|
||||||
|
llm_model_name = "gpt-4-1106-preview"
|
||||||
|
max_token_size = 4000
|
||||||
|
enable_timestamp = "true"
|
||||||
|
owner_prompt = "I am your master {name} , now is {now}"
|
||||||
|
contact_prompt = "I am your master's friend {name}"
|
||||||
|
owner_env = ["knowledge"]
|
||||||
|
[[work.do]]
|
||||||
|
role = "system"
|
||||||
|
content = """
|
||||||
|
My name is JarvisPlus, I am the master's super personal assistant.
|
||||||
|
I think hard and try my best to complete TODOs. The types of TODO I can handle include:
|
||||||
|
- Scheduling, where I will try to contact the relevant personnel of the plan and confirm the details of the schedule with them.
|
||||||
|
- Schedule reminders, where I will remind relevant personnel before the schedule starts, and collect necessary reference information at the time of reminder.
|
||||||
|
- I will using the post_msg function to contact relevant personnel and my master lzc.
|
||||||
|
- Writing documents/letters, using op:'create' to save my work results.
|
||||||
|
|
||||||
|
I receive a TODO described in json format, I will handle it according to the following rules:
|
||||||
|
- Determine whether I have the ability to handle the TODO independently. If not, I will try to break the TODO down into smaller sub-TODOs, or hand it over to someone more suitable that I know.
|
||||||
|
- I will plan the steps to solve the TODO in combination with known information, and break down the generalized TODO into more specific sub-todos. The title of the sub-todo should contain step number like #1, #2
|
||||||
|
- Sub-todo must set parent, The maximum depth of sub-todo is 4.
|
||||||
|
- A specific sub-todo refers to a task that can be completed in one execution within my ability range.
|
||||||
|
- After each execution, I will decide whether to update the status of the TODO. And use op:'update_todo' to update when necessary.
|
||||||
|
|
||||||
|
The result of my planned execution must be directly parsed by `python json.loads`. Here is an example:
|
||||||
|
{
|
||||||
|
resp: '$what_did_I_do',
|
||||||
|
post_msg : [
|
||||||
|
{
|
||||||
|
target:'$target_name',
|
||||||
|
content:'$msg_content'
|
||||||
|
}
|
||||||
|
],
|
||||||
|
op_list: [{
|
||||||
|
op: 'create_todo',
|
||||||
|
parent: '$parent_id', # optional
|
||||||
|
todo: {
|
||||||
|
title: '#1 sub_todo',
|
||||||
|
detail: 'this is a sub todo',
|
||||||
|
creator: 'JarvisPlus',
|
||||||
|
worker: 'lzc',
|
||||||
|
due_date: '2019-01-01 14:23:11'
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
op: 'update_todo',
|
||||||
|
id: '$todo_id',
|
||||||
|
state: 'cancel' # pending,cancel
|
||||||
|
},
|
||||||
|
{
|
||||||
|
op: 'write_file',
|
||||||
|
path: '/todos/$todo_path/.result/$doc_name',
|
||||||
|
content:'$doc_content'
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
[[learn.do]]
|
||||||
|
role = "system"
|
||||||
|
content = """
|
||||||
|
我是一名软件工程师，拥有非常优秀的资料学习能力。下面是我学习和整理资料的方法
|
||||||
|
1. 由于LLM的Token限制，我学习的可能只是资料的部分内容，此时我应能产生合适的学习中间结果，中间结果保存在metadata中。我要么产生中间结果，要么产生最终结果。
|
||||||
|
2. 当存在已知信息时，需参考已知信息的内容来思考结果。
|
||||||
|
3. 当我收到最后一部分内容时，我能结合已知的中间结果产生最终结果。
|
||||||
|
4. 现有资料库以文件系统的形式组织，我未来借助资料的摘要来浏览知识库
|
||||||
|
5. 我将学习过的资料另存在资料库的合适位置（以/开始的完整路径）。保存位置的目录深度不超过5层，文件夹名称长度不超过16个字符。
|
||||||
|
6. 总是以json格式返回思考结果，json格式如下
|
||||||
|
{
|
||||||
|
"op_list":[
|
||||||
|
{
|
||||||
|
"op":"learn",
|
||||||
|
"original_path":"$original_path",
|
||||||
|
"think":"$think_result",
|
||||||
|
"metadata":{...},
|
||||||
|
"tags":["tag1","tag2"...],
|
||||||
|
"path":["/graphic/opengl","/database/mysql"], # list of directories to save to.
|
||||||
|
"title":"$article_title",
|
||||||
|
"summary":"$summary",
|
||||||
|
"catalogs": [
|
||||||
|
{
|
||||||
|
# optional,catalogs is a tree
|
||||||
|
"title":"$catalog_name1",
|
||||||
|
"pos":"$pos:$length"
|
||||||
|
"children":[
|
||||||
|
{
|
||||||
|
"title":"$catalog_name 1.1",
|
||||||
|
"pos":"$pos:$length"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"title":"$catalog_name2",
|
||||||
|
"pos":"$pos:$length"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
"""
|
||||||
|
|
||||||
|
[[prompt]]
|
||||||
|
role = "system"
|
||||||
|
content = """
|
||||||
|
你的名字是JarvisPlus，是超级个人助理。收到消息后，根据以下规则处理：
|
||||||
|
1. 如果你觉得对话中产生了潜在的todo,可通过op_list来创建这些todo,todo的title是必须的,并尽量包含时间地点人物事件的关键信息
|
||||||
|
2. 非直接创建TODO指令，你应先和我确认后再创建使用op_list创建TODO
|
||||||
|
3. 你可能会得到几条已知信息，其中可能有已有的todo,注意在适当的时候检索文件系统，避免重复创建todo
|
||||||
|
4. 检索文件系统是代价高昂的操作，请尽量减少检索次数
|
||||||
|
5. 注意你正在与之聊天的人的身份，并根据他们的地位提供相应的服务。
|
||||||
|
|
||||||
|
|
||||||
|
回复的消息必须能被python的json.loads直接解析。下面是一个返回的例子:
|
||||||
|
{
|
||||||
|
resp: 'Hello',
|
||||||
|
op_list: [{
|
||||||
|
op: 'create_todo',
|
||||||
|
parent: '$parent_todo_id', # optional
|
||||||
|
todo: {
|
||||||
|
title: 'test_todo',
|
||||||
|
detail: 'test',
|
||||||
|
creator: 'JarvisPlus',
|
||||||
|
due_date: '2019-01-01 14:23:11'
|
||||||
|
}
|
||||||
|
}]
|
||||||
|
}
|
||||||
|
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@@ -8,5 +8,4 @@ max_token_size=4000
|
|||||||
role = "system"
|
role = "system"
|
||||||
content = """
|
content = """
|
||||||
Your name is Lachlan, and you are my advanced private Spanish tutor.
|
Your name is Lachlan, and you are my advanced private Spanish tutor.
|
||||||
You are also a local guide familiar with the history of the Inca Empire. While teaching me Spanish, you will introduce some related historical and cultural origins.
|
|
||||||
"""
|
"""
|
||||||
@@ -1,13 +1,9 @@
|
|||||||
instance_id = "Mia"
|
instance_id = "Mia"
|
||||||
fullname = "Mia"
|
fullname = "Mia"
|
||||||
#llm_model_name = "gpt-4"
|
#llm_model_name = "gpt-4"
|
||||||
#max_token_size = 16000
|
|
||||||
#enable_function =["add_event"]
|
|
||||||
#enable_kb = "true"
|
|
||||||
#enable_timestamp = "false"
|
|
||||||
owner_prompt = "我是你的主人{name}"
|
owner_prompt = "我是你的主人{name}"
|
||||||
contact_prompt = "我是你的朋友{name}"
|
contact_prompt = "我是你的朋友{name}"
|
||||||
owner_env = "knowledge"
|
owner_env = "../../knowledge_pipelines/Mia/query.py"
|
||||||
|
|
||||||
[[prompt]]
|
[[prompt]]
|
||||||
role = "system"
|
role = "system"
|
||||||
|
|||||||
@@ -0,0 +1,49 @@
|
|||||||
|
import copy
|
||||||
|
|
||||||
|
from aios.agent.agent_base import CustomAIAgent, LLMPrompt
|
||||||
|
from aios.knowledge.data.writer import split_text
|
||||||
|
from aios.proto.agent_msg import AgentMsg, AgentMsgType
|
||||||
|
from aios.proto.compute_task import ComputeTaskResultCode
|
||||||
|
|
||||||
|
|
||||||
|
class TextSummaryAgent(CustomAIAgent):
|
||||||
|
def __init__(self):
|
||||||
|
super().__init__("TextSummary", "Text Summary", 128000)
|
||||||
|
|
||||||
|
async def _process_msg(self, msg: AgentMsg, workspace=None) -> AgentMsg:
|
||||||
|
if msg.msg_type is not AgentMsgType.TYPE_MSG:
|
||||||
|
return AgentMsg.create_error_resp(msg, "only support msg type")
|
||||||
|
|
||||||
|
if msg.body_mime is not None and msg.body_mime != "text/plain":
|
||||||
|
return AgentMsg.create_error_resp(msg, "only support text/plain mime type")
|
||||||
|
|
||||||
|
chunks = split_text(msg.body, separators=["\n\n", "\n"], chunk_size=4000, chunk_overlap=200, length_function=len)
|
||||||
|
|
||||||
|
prompt = LLMPrompt()
|
||||||
|
prompt.system_message = "Your job is to generate a summary based on the input."
|
||||||
|
if len(chunks) == 1:
|
||||||
|
prompt.append(LLMPrompt(chunks[0]))
|
||||||
|
resp = await self.do_llm_complection(prompt)
|
||||||
|
if resp.result_code != ComputeTaskResultCode.OK:
|
||||||
|
return msg.create_error_resp(resp.error_str)
|
||||||
|
return msg.create_resp_msg(resp.result_str)
|
||||||
|
|
||||||
|
segments = []
|
||||||
|
for i, chunk in enumerate(chunks):
|
||||||
|
seg_prompt = copy.deepcopy(prompt)
|
||||||
|
seg_prompt.append(LLMPrompt(chunk))
|
||||||
|
resp = await self.do_llm_complection(seg_prompt)
|
||||||
|
if resp.result_code != ComputeTaskResultCode.OK:
|
||||||
|
return msg.create_error_resp(resp.error_str)
|
||||||
|
segments.append(resp.result_str)
|
||||||
|
|
||||||
|
segments_str = "\n".join(segments)
|
||||||
|
prompt.append(LLMPrompt(f"以下文本分段之后的各段摘要，请合并生成一个完整摘要：\n{segments_str}"))
|
||||||
|
resp = await self.do_llm_complection(prompt)
|
||||||
|
if resp.result_code != ComputeTaskResultCode.OK:
|
||||||
|
return msg.create_error_resp(resp.error_str)
|
||||||
|
return msg.create_resp_msg(resp.result_str)
|
||||||
|
|
||||||
|
|
||||||
|
def init():
|
||||||
|
return TextSummaryAgent()
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
instance_id = "Thinker"
|
instance_id = "Thinker"
|
||||||
fullname = "Thinker"
|
fullname = "Thinker"
|
||||||
llm_model_name = "gpt-3.5-turbo-16k-0613"
|
llm_model_name = "Llama-2-13b-chat"
|
||||||
max_token_size = 14000
|
max_token_size = 2000
|
||||||
#enable_function =["add_event"]
|
#enable_function =["add_event"]
|
||||||
#enable_kb = "true"
|
#enable_kb = "true"
|
||||||
|
|
||||||
@@ -16,5 +16,5 @@ You mainly use the following methods to generate summary:
|
|||||||
4. Try to understand the attitude of different people on different topics or events
|
4. Try to understand the attitude of different people on different topics or events
|
||||||
5. For the key information or TODO in the information, such as the time, place, amount and other information of the certainty, it must be stored in the summary.
|
5. For the key information or TODO in the information, such as the time, place, amount and other information of the certainty, it must be stored in the summary.
|
||||||
|
|
||||||
You have a summary of simplicity and profound nonsense, and you don't need to have any polite words to me.Just give me a summary.
|
Just give me a summary without any other word.
|
||||||
"""
|
"""
|
||||||
|
|||||||
@@ -1,5 +1,7 @@
|
|||||||
instance_id = "Tracy"
|
instance_id = "Tracy"
|
||||||
fullname = "Tracy"
|
fullname = "Tracy"
|
||||||
|
llm_model_name = "Llama-2-13b-chat"
|
||||||
|
max_token_size = 2000
|
||||||
|
|
||||||
[[prompt]]
|
[[prompt]]
|
||||||
role = "system"
|
role = "system"
|
||||||
|
|||||||
@@ -0,0 +1,8 @@
|
|||||||
|
instance_id = "Vision"
|
||||||
|
fullname = "Vision"
|
||||||
|
llm_model_name = "gpt-4-1106-preview"
|
||||||
|
|
||||||
|
[[prompt]]
|
||||||
|
role = "system"
|
||||||
|
content = """Your job is to analyze user input images and videos and respond based on user intent.
|
||||||
|
If the user requests a video and you receive key frames of the video, please reply to the user's question based on the key frame content."""
|
||||||
@@ -1,7 +0,0 @@
|
|||||||
|
|
||||||
|
|
||||||
EMAIL_IMAP_SERVER = "imap.gmail.com"
|
|
||||||
EMAIL_ADDRESS = '<>'
|
|
||||||
EMAIL_PASSWORD = '<>'
|
|
||||||
EMAIL_IMAP_PORT = 993
|
|
||||||
LOCAL_DIR = 'rootfs/data'
|
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
name = "JarvisPlus"
|
||||||
|
input.module = "scan_local"
|
||||||
|
input.params.workspace = "${myai_dir}/workspace/JarvisPlus"
|
||||||
|
input.params.path = "${myai_dir}/data"
|
||||||
|
parser.module = "parse_local"
|
||||||
|
parser.params.workspace = "${myai_dir}/workspace/JarvisPlus"
|
||||||
|
parser.params.assign_to = "JarvisPlus"
|
||||||
|
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
import sys
|
||||||
|
import os
|
||||||
|
from aios import KnowledgePipelineEnvironment
|
||||||
|
directory = os.path.dirname(__file__)
|
||||||
|
sys.path.append(directory + '/../../../../src/component/')
|
||||||
|
|
||||||
|
from mail_environment import LocalEmail
|
||||||
|
def init(env: KnowledgePipelineEnvironment, params: dict):
|
||||||
|
return LocalEmail(env, params)
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
import sys
|
||||||
|
import os
|
||||||
|
from aios import *
|
||||||
|
directory = os.path.dirname(__file__)
|
||||||
|
|
||||||
|
sys.path.append(directory + '/../../../../src/component/')
|
||||||
|
from mail_environment import IssueParser
|
||||||
|
|
||||||
|
def init(env: KnowledgePipelineEnvironment, params: dict):
|
||||||
|
return IssueParser(env, params)
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
name = "Mail.Issue"
|
||||||
|
input.module = "local.py"
|
||||||
|
input.params.path = "${myai_dir}/mail"
|
||||||
|
input.params.watch = true
|
||||||
|
parser.module = "parser.py"
|
||||||
|
parser.params.mail_path = "${myai_dir}/mail"
|
||||||
|
parser.params.issue_path = "${myai_dir}/mail/issue.json"
|
||||||
|
[parser.params.root_issue]
|
||||||
|
summary = "巴克云公司推进中的项目"
|
||||||
|
[[parser.params.root_issue.children]]
|
||||||
|
summary = "去中心存储项目DMC"
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
import sys
|
||||||
|
import os
|
||||||
|
from knowledge import KnowledgePipelineEnvironment
|
||||||
|
directory = os.path.dirname(__file__)
|
||||||
|
sys.path.append(directory + '/../../../../src/component/')
|
||||||
|
|
||||||
|
from mail_environment import EmailSpider
|
||||||
|
|
||||||
|
def init(env: KnowledgePipelineEnvironment, params: dict):
|
||||||
|
return EmailSpider(env, params)
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
name = "Mail.Sync"
|
||||||
|
input.module = "input.py"
|
||||||
|
[input.params]
|
||||||
|
path = "${myai_dir}/mail"
|
||||||
|
imap_server = "imap.qq.com"
|
||||||
|
imap_port = 993
|
||||||
|
address = "115620204@qq.com"
|
||||||
|
password = "zbbjpbukeonqbjja"
|
||||||
|
[input.params.fields]
|
||||||
|
from = "from"
|
||||||
|
to = "to"
|
||||||
|
subject = "subject"
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,211 @@
|
|||||||
|
import copy
|
||||||
|
import os
|
||||||
|
from typing import List
|
||||||
|
|
||||||
|
import aiofiles
|
||||||
|
import chardet
|
||||||
|
import logging
|
||||||
|
import string
|
||||||
|
import docx2txt
|
||||||
|
from PyPDF2 import PdfReader
|
||||||
|
|
||||||
|
from aios import KnowledgePipelineEnvironment, ImageObjectBuilder, DocumentObjectBuilder, KnowledgeStore, RichTextObject
|
||||||
|
from aios.agent.agent_base import AgentPrompt
|
||||||
|
from aios.frame.compute_kernel import ComputeKernel
|
||||||
|
from aios.knowledge.data.writer import split_text
|
||||||
|
from aios.proto.compute_task import ComputeTaskResult, ComputeTaskResultCode
|
||||||
|
from aios.storage.storage import AIStorage
|
||||||
|
from aios.utils import video_utils, image_utils
|
||||||
|
|
||||||
|
|
||||||
|
class KnowledgeDirSource:
|
||||||
|
def __init__(self, env: KnowledgePipelineEnvironment, config):
|
||||||
|
self.env = env
|
||||||
|
path = string.Template(config["path"]).substitute(myai_dir=AIStorage.get_instance().get_myai_dir())
|
||||||
|
config["path"] = path
|
||||||
|
self.config = config
|
||||||
|
|
||||||
|
# @classmethod
|
||||||
|
# def user_config_items(cls):
|
||||||
|
# return [("path", "local dir path")]
|
||||||
|
|
||||||
|
def path(self):
|
||||||
|
return self.config["path"]
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
async def read_txt_file(file_path:str)->str:
|
||||||
|
cur_encode = "utf-8"
|
||||||
|
async with aiofiles.open(file_path,'rb') as f:
|
||||||
|
cur_encode = chardet.detect(await f.read())['encoding']
|
||||||
|
|
||||||
|
async with aiofiles.open(file_path,'r',encoding=cur_encode) as f:
|
||||||
|
return await f.read()
|
||||||
|
|
||||||
|
async def next(self):
|
||||||
|
while True:
|
||||||
|
journals = self.env.journal.latest_journals(1)
|
||||||
|
from_time = 0
|
||||||
|
if len(journals) == 1:
|
||||||
|
latest_journal = journals[0]
|
||||||
|
if latest_journal.is_finish():
|
||||||
|
yield None
|
||||||
|
continue
|
||||||
|
from_time = os.path.getctime(latest_journal.get_input())
|
||||||
|
if os.path.getmtime(self.path()) <= from_time:
|
||||||
|
yield (None, None)
|
||||||
|
continue
|
||||||
|
|
||||||
|
file_pathes = sorted(os.listdir(self.path()), key=lambda x: os.path.getctime(os.path.join(self.path(), x)))
|
||||||
|
for rel_path in file_pathes:
|
||||||
|
file_path = os.path.join(self.path(), rel_path)
|
||||||
|
timestamp = os.path.getctime(file_path)
|
||||||
|
if timestamp <= from_time:
|
||||||
|
continue
|
||||||
|
ext = os.path.splitext(file_path)[1].lower()
|
||||||
|
if ext in ['.jpg', '.jpeg', '.png', '.gif', '.bmp']:
|
||||||
|
logging.info(f"knowledge dir source found image file {file_path}")
|
||||||
|
image = ImageObjectBuilder({}, {}, file_path).build(self.env.get_knowledge_store())
|
||||||
|
await self.env.get_knowledge_store().insert_object(image)
|
||||||
|
yield (image.calculate_id(), file_path)
|
||||||
|
if ext in ['.txt']:
|
||||||
|
logging.info(f"knowledge dir source found text file {file_path}")
|
||||||
|
text = await self.read_txt_file(file_path)
|
||||||
|
document = DocumentObjectBuilder({}, {}, text).build(self.env.get_knowledge_store())
|
||||||
|
await self.env.get_knowledge_store().insert_object(document)
|
||||||
|
yield (document.calculate_id(), file_path)
|
||||||
|
yield (None, None)
|
||||||
|
|
||||||
|
|
||||||
|
def init(env: KnowledgePipelineEnvironment, params: dict) -> KnowledgeDirSource:
|
||||||
|
return KnowledgeDirSource(env, params)
|
||||||
|
|
||||||
|
|
||||||
|
async def image_to_text(images: List[str]) -> str:
|
||||||
|
msg_prompt = AgentPrompt()
|
||||||
|
image_prompt = "What's in this image?"
|
||||||
|
content = [{"type": "text", "text": image_prompt}]
|
||||||
|
content.extend([{"type": "image_url", "image_url": {"url": image_utils.to_base64(image)}} for image in images])
|
||||||
|
msg_prompt.messages = [{"role": "user", "content": content}]
|
||||||
|
|
||||||
|
resp: ComputeTaskResult = await (ComputeKernel.get_instance()
|
||||||
|
.do_llm_completion(prompt=msg_prompt,
|
||||||
|
resp_mode="text",
|
||||||
|
mode_name="gpt-4-vision-preview",
|
||||||
|
max_token=4000,
|
||||||
|
inner_functions=None,
|
||||||
|
timeout=None))
|
||||||
|
if resp.result_code != ComputeTaskResultCode.OK:
|
||||||
|
raise Exception(f"image_to_text error: {resp.result_code} msg:{resp.error_str}")
|
||||||
|
return resp.result_str
|
||||||
|
|
||||||
|
|
||||||
|
async def video_to_text(video: str) -> str:
|
||||||
|
prompt = "These pictures are key frames extracted from the video. Please describe the content of the video based on these key frames."
|
||||||
|
frames = video_utils.extract_frames(video, (1024, 1024))
|
||||||
|
msg_prompt = AgentPrompt()
|
||||||
|
content = [{"type": "text", "text": prompt}]
|
||||||
|
content.extend([{"type": "image_url", "image_url": {"url": frame}} for frame in frames])
|
||||||
|
msg_prompt.messages = [{"role": "user", "content": content}]
|
||||||
|
resp: ComputeTaskResult = await (ComputeKernel.get_instance()
|
||||||
|
.do_llm_completion(prompt=msg_prompt,
|
||||||
|
resp_mode="text",
|
||||||
|
mode_name="gpt-4-vision-preview",
|
||||||
|
max_token=4000,
|
||||||
|
inner_functions=None,
|
||||||
|
timeout=None))
|
||||||
|
if resp.result_code != ComputeTaskResultCode.OK:
|
||||||
|
raise Exception(f"video_to_text error: {resp.result_code} msg:{resp.error_str}")
|
||||||
|
return resp.result_str
|
||||||
|
|
||||||
|
|
||||||
|
async def summary_document(text: str, separators: List[str]=["\n\n", "\n"]) -> str:
|
||||||
|
chunks = split_text(text, separators=separators, chunk_size=4000, chunk_overlap=200, length_function=len)
|
||||||
|
|
||||||
|
prompt = AgentPrompt()
|
||||||
|
prompt.system_message = {"role":"system","content":"Your job is to generate a summary based on the input."}
|
||||||
|
if len(chunks) == 1:
|
||||||
|
prompt.append(AgentPrompt(chunks[0]))
|
||||||
|
resp = await (ComputeKernel.get_instance()
|
||||||
|
.do_llm_completion(prompt=prompt,
|
||||||
|
resp_mode="text",
|
||||||
|
mode_name="gpt-4-1106-preview",
|
||||||
|
max_token=4000,
|
||||||
|
inner_functions=None,
|
||||||
|
timeout=None))
|
||||||
|
if resp.result_code != ComputeTaskResultCode.OK:
|
||||||
|
raise Exception(f"summary_document error: {resp.result_code} msg:{resp.error_str}")
|
||||||
|
return resp.result_str
|
||||||
|
|
||||||
|
segments = []
|
||||||
|
for i, chunk in enumerate(chunks):
|
||||||
|
seg_prompt = copy.deepcopy(prompt)
|
||||||
|
seg_prompt.append(AgentPrompt(chunk))
|
||||||
|
resp = await (ComputeKernel.get_instance()
|
||||||
|
.do_llm_completion(prompt=seg_prompt,
|
||||||
|
resp_mode="text",
|
||||||
|
mode_name="gpt-4-1106-preview",
|
||||||
|
max_token=4000,
|
||||||
|
inner_functions=None,
|
||||||
|
timeout=None))
|
||||||
|
if resp.result_code != ComputeTaskResultCode.OK:
|
||||||
|
raise Exception(f"summary_document error: {resp.result_code} msg:{resp.error_str}")
|
||||||
|
segments.append(resp.result_str)
|
||||||
|
|
||||||
|
segments_str = "\n".join(segments)
|
||||||
|
prompt.append(AgentPrompt(f"Please combine the summaries of the following paragraphs into one complete summary:\n{segments_str}"))
|
||||||
|
resp = await (ComputeKernel.get_instance()
|
||||||
|
.do_llm_completion(prompt=prompt,
|
||||||
|
resp_mode="text",
|
||||||
|
mode_name="gpt-4-1106-preview",
|
||||||
|
max_token=4000,
|
||||||
|
inner_functions=None,
|
||||||
|
timeout=None))
|
||||||
|
if resp.result_code != ComputeTaskResultCode.OK:
|
||||||
|
raise Exception(f"summary_document error: {resp.result_code} msg:{resp.error_str}")
|
||||||
|
return resp.result_str
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
def pdf_to_rich_text_object(pdf: str, store: KnowledgeStore) -> RichTextObject:
|
||||||
|
base_name = os.path.basename(pdf)
|
||||||
|
cache_path = os.path.join(AIStorage.get_instance().get_myai_dir(), "knowledge", "doc_cache", base_name)
|
||||||
|
if not os.path.exists(cache_path):
|
||||||
|
os.makedirs(cache_path)
|
||||||
|
|
||||||
|
reader = PdfReader(pdf)
|
||||||
|
rich_text = RichTextObject()
|
||||||
|
page_texts = []
|
||||||
|
image_count = 0
|
||||||
|
for page in reader.pages:
|
||||||
|
text = page.extract_text()
|
||||||
|
page_texts.append(text)
|
||||||
|
for image in page.images:
|
||||||
|
image_path = os.path.join(cache_path, f"{image_count}_{image.name}")
|
||||||
|
with open(image_path, "wb") as f:
|
||||||
|
f.write(image.data)
|
||||||
|
image_object = ImageObjectBuilder({}, {}, image_path).build(store)
|
||||||
|
rich_text.add_image(image_object)
|
||||||
|
|
||||||
|
document = DocumentObjectBuilder({}, {}, "".join(page_texts)).build(store)
|
||||||
|
rich_text.add_document(document)
|
||||||
|
|
||||||
|
return rich_text
|
||||||
|
|
||||||
|
|
||||||
|
def doc_to_rich_text_object(doc: str, store: KnowledgeStore) -> RichTextObject:
|
||||||
|
base_name = os.path.basename(doc)
|
||||||
|
cache_path = os.path.join(AIStorage.get_instance().get_myai_dir(), "knowledge", "doc_cache", base_name)
|
||||||
|
if not os.path.exists(cache_path):
|
||||||
|
os.makedirs(cache_path)
|
||||||
|
text = docx2txt.process(doc, cache_path)
|
||||||
|
|
||||||
|
rich_text = RichTextObject()
|
||||||
|
for image in os.listdir(cache_path):
|
||||||
|
image_path = os.path.join(cache_path, image)
|
||||||
|
image_object = ImageObjectBuilder({}, {}, image_path).build(store)
|
||||||
|
rich_text.add_image(image_object)
|
||||||
|
|
||||||
|
document = DocumentObjectBuilder({}, {}, text).build(store)
|
||||||
|
rich_text.add_document(document)
|
||||||
|
|
||||||
|
return rich_text
|
||||||
@@ -0,0 +1,101 @@
|
|||||||
|
# define a knowledge base class
|
||||||
|
import json
|
||||||
|
import string
|
||||||
|
from aios import *
|
||||||
|
|
||||||
|
|
||||||
|
class EmbeddingParser:
|
||||||
|
def __init__(self, env: KnowledgePipelineEnvironment, config: dict):
|
||||||
|
self._default_text_model = "all-MiniLM-L6-v2"
|
||||||
|
self._default_image_model = "clip-ViT-B-32"
|
||||||
|
|
||||||
|
path = string.Template(config["path"]).substitute(myai_dir=AIStorage.get_instance().get_myai_dir())
|
||||||
|
if not os.path.exists(path):
|
||||||
|
os.makedirs(path)
|
||||||
|
config["path"] = path
|
||||||
|
|
||||||
|
self.env = env
|
||||||
|
self.config = config
|
||||||
|
|
||||||
|
def get_path(self) -> str:
|
||||||
|
return self.config["path"]
|
||||||
|
|
||||||
|
def __get_vector_store(self, model_name: str) -> ChromaVectorStore:
|
||||||
|
return ChromaVectorStore(self.get_path(), model_name)
|
||||||
|
|
||||||
|
async def __embedding_document(self, document: DocumentObject):
|
||||||
|
for chunk_id in document.get_chunk_list():
|
||||||
|
chunk = self.env.get_knowledge_store().get_chunk_reader().get_chunk(chunk_id)
|
||||||
|
if chunk is None:
|
||||||
|
raise ValueError(f"text chunk not found: {chunk_id}")
|
||||||
|
|
||||||
|
text = chunk.read().decode("utf-8")
|
||||||
|
vector = await ComputeKernel.get_instance().do_text_embedding(text, self._default_text_model)
|
||||||
|
if vector:
|
||||||
|
await self.__get_vector_store(self._default_text_model).insert(vector, chunk_id)
|
||||||
|
|
||||||
|
async def __embedding_image(self, image: ImageObject):
|
||||||
|
# desc = {}
|
||||||
|
# if not not image.get_meta():
|
||||||
|
# desc["meta"] = image.get_meta()
|
||||||
|
# if not not image.get_exif():
|
||||||
|
# desc["exif"] = image.get_exif()
|
||||||
|
# if not not image.get_tags():
|
||||||
|
# desc["tags"] = image.get_tags()
|
||||||
|
# vector = await self.compute_kernel.do_text_embedding(json.dumps(desc), self._default_text_model)
|
||||||
|
vector = await ComputeKernel.get_instance().do_image_embedding(image.calculate_id(), self._default_image_model)
|
||||||
|
if vector:
|
||||||
|
await self.__get_vector_store(self._default_image_model).insert(vector, image.calculate_id())
|
||||||
|
|
||||||
|
async def __embedding_video(self, vedio: VideoObject):
|
||||||
|
desc = {}
|
||||||
|
if not not vedio.get_meta():
|
||||||
|
desc["meta"] = vedio.get_meta()
|
||||||
|
if not not vedio.get_info():
|
||||||
|
desc["info"] = vedio.get_info()
|
||||||
|
if not not vedio.get_tags():
|
||||||
|
desc["tags"] = vedio.get_tags()
|
||||||
|
vector = await ComputeKernel.get_instance().do_text_embedding(json.dumps(desc), self._default_text_model)
|
||||||
|
await self.__get_vector_store(self._default_text_model).insert(vector, vedio.calculate_id())
|
||||||
|
|
||||||
|
async def __embedding_rich_text(self, rich_text: RichTextObject):
|
||||||
|
for document_id in rich_text.get_documents().values():
|
||||||
|
document = DocumentObject.decode(self.env.get_knowledge_store().get_object_store().get_object(document_id))
|
||||||
|
await self.__embedding_document(document)
|
||||||
|
for image_id in rich_text.get_images().values():
|
||||||
|
image = ImageObject.decode(self.env.get_knowledge_store().get_object_store().get_object(image_id))
|
||||||
|
await self.__embedding_image(image)
|
||||||
|
for video_id in rich_text.get_videos().values():
|
||||||
|
video = VideoObject.decode(self.env.get_knowledge_store().get_object_store().get_object(video_id))
|
||||||
|
await self.__embedding_video(video)
|
||||||
|
for rich_text_id in rich_text.get_rich_texts().values():
|
||||||
|
rich_text = RichTextObject.decode(self.env.get_knowledge_store().get_object_store().get_object(rich_text_id))
|
||||||
|
await self.__embedding_rich_text(rich_text)
|
||||||
|
|
||||||
|
async def __embedding_email(self, email: EmailObject):
|
||||||
|
vector = await ComputeKernel.get_instance().do_text_embedding(json.dumps(email.get_desc()), self._default_text_model)
|
||||||
|
await self.__get_vector_store(self._default_text_model).insert(vector, email.calculate_id())
|
||||||
|
await self.__embedding_rich_text(email.get_rich_text())
|
||||||
|
|
||||||
|
|
||||||
|
async def __do_embedding(self, object: KnowledgeObject):
|
||||||
|
if object.get_object_type() == ObjectType.Document:
|
||||||
|
await self.__embedding_document(object)
|
||||||
|
if object.get_object_type() == ObjectType.Image:
|
||||||
|
await self.__embedding_image(object)
|
||||||
|
if object.get_object_type() == ObjectType.Video:
|
||||||
|
await self.__embedding_video(object)
|
||||||
|
if object.get_object_type() == ObjectType.RichText:
|
||||||
|
await self.__embedding_rich_text(object)
|
||||||
|
if object.get_object_type() == ObjectType.Email:
|
||||||
|
await self.__embedding_email(object)
|
||||||
|
else:
|
||||||
|
pass
|
||||||
|
|
||||||
|
async def parse(self, object: ObjectID) -> str:
|
||||||
|
obj = self.env.get_knowledge_store().load_object(object)
|
||||||
|
await self.__do_embedding(obj)
|
||||||
|
return str(object)
|
||||||
|
|
||||||
|
def init(env: KnowledgePipelineEnvironment, params: dict) -> EmbeddingParser:
|
||||||
|
return EmbeddingParser(env, params)
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
name = "Mia"
|
||||||
|
input.module = "input.py"
|
||||||
|
input.params.path = "${myai_dir}/data"
|
||||||
|
parser.module = "parser.py"
|
||||||
|
parser.params.path = "${myai_dir}/knowledge/indices/embedding"
|
||||||
|
|
||||||
@@ -0,0 +1,96 @@
|
|||||||
|
import os
|
||||||
|
import logging
|
||||||
|
import json
|
||||||
|
from aios import *
|
||||||
|
|
||||||
|
class EmbeddingEnvironment(SimpleEnvironment):
|
||||||
|
def __init__(self, workspace: str) -> None:
|
||||||
|
super().__init__(workspace)
|
||||||
|
self.path = os.path.join(AIStorage.get_instance().get_myai_dir(), "knowledge/indices/embedding")
|
||||||
|
self._default_text_model = "all-MiniLM-L6-v2"
|
||||||
|
self._default_image_model = "clip-ViT-B-32"
|
||||||
|
|
||||||
|
query_param = {
|
||||||
|
"tokens": "key words to query",
|
||||||
|
"types": "prefered knowledge types, one or more of [text, image]",
|
||||||
|
"limit": "index of query result"
|
||||||
|
}
|
||||||
|
self.add_ai_function(SimpleAIFunction("query_knowledge",
|
||||||
|
"vector query content from local knowledge base",
|
||||||
|
self._query,
|
||||||
|
query_param))
|
||||||
|
|
||||||
|
def __get_vector_store(self, model_name: str) -> ChromaVectorStore:
|
||||||
|
return ChromaVectorStore(self.path, model_name)
|
||||||
|
|
||||||
|
async def query_objects(self, tokens: str, types: list[str], topk: int) -> [ObjectID]:
|
||||||
|
texts = []
|
||||||
|
if "text" in types:
|
||||||
|
vector = await ComputeKernel.get_instance().do_text_embedding(tokens, self._default_text_model)
|
||||||
|
texts = await self.__get_vector_store(self._default_text_model).query(vector, topk)
|
||||||
|
images = []
|
||||||
|
if "image" in types:
|
||||||
|
vector = await ComputeKernel.get_instance().do_text_embedding(tokens, self._default_image_model)
|
||||||
|
images = await self.__get_vector_store(self._default_image_model).query(vector, topk)
|
||||||
|
return texts + images
|
||||||
|
|
||||||
|
|
||||||
|
def tokens_from_objects(self, object_ids: [ObjectID]) -> list[str]:
|
||||||
|
results = dict()
|
||||||
|
for object_id in object_ids:
|
||||||
|
parents = KnowledgeStore().get_relation_store().get_related_root_objects(object_id)
|
||||||
|
# last parent is the root object
|
||||||
|
root_object_id = parents[0] if parents else object_id
|
||||||
|
logging.info(f"object_id: {str(object_id)} root_object_id: {str(root_object_id)}")
|
||||||
|
if str(root_object_id) in results:
|
||||||
|
results[str(root_object_id)].append(object_id)
|
||||||
|
else:
|
||||||
|
results[str(root_object_id)] = [root_object_id, object_id]
|
||||||
|
content = ""
|
||||||
|
result_desc = []
|
||||||
|
for result in results.values():
|
||||||
|
# first element in result is the root object
|
||||||
|
root_object_id = result[0]
|
||||||
|
if root_object_id.get_object_type() == ObjectType.Email:
|
||||||
|
email = KnowledgeStore().load_object(root_object_id)
|
||||||
|
desc = email.get_desc()
|
||||||
|
desc["type"] = "email"
|
||||||
|
desc["contents"] = []
|
||||||
|
result_desc.append(desc)
|
||||||
|
upper_list = desc["contents"]
|
||||||
|
result = result[1:]
|
||||||
|
else:
|
||||||
|
upper_list = result_desc
|
||||||
|
|
||||||
|
for object_id in result:
|
||||||
|
if object_id.get_object_type() == ObjectType.Chunk:
|
||||||
|
upper_list.append({"type": "text", "content": KnowledgeStore().get_chunk_reader().get_chunk(object_id).read().decode("utf-8")})
|
||||||
|
if object_id.get_object_type() == ObjectType.Image:
|
||||||
|
# image = self.load_object(object_id)
|
||||||
|
desc = dict()
|
||||||
|
desc["id"] = str(object_id)
|
||||||
|
desc["type"] = "image"
|
||||||
|
upper_list.append(desc)
|
||||||
|
if object_id.get_object_type() == ObjectType.Video:
|
||||||
|
video = KnowledgeStore().load_object(object_id)
|
||||||
|
desc = video.get_desc()
|
||||||
|
desc["type"] = "video"
|
||||||
|
upper_list.append(desc)
|
||||||
|
else:
|
||||||
|
pass
|
||||||
|
content += json.dumps(result_desc)
|
||||||
|
content += ".\n"
|
||||||
|
|
||||||
|
return content
|
||||||
|
|
||||||
|
async def _query(self, tokens: str, types: list[str] = ["text"], index: str=0):
|
||||||
|
index = int(index)
|
||||||
|
object_ids = await self.query_objects(tokens, types, 4)
|
||||||
|
if len(object_ids) <= index:
|
||||||
|
return "*** I have no more information for your reference.\n"
|
||||||
|
else:
|
||||||
|
content = "*** I have provided the following known information for your reference with json format:\n"
|
||||||
|
return content + self.tokens_from_objects(object_ids[index:index+1])
|
||||||
|
|
||||||
|
def init(workspace: str) -> EmbeddingEnvironment:
|
||||||
|
return EmbeddingEnvironment(workspace)
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
pipelines = [
|
||||||
|
"JarvisPlus"
|
||||||
|
]
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
|
||||||
|
from .proto.agent_msg import *
|
||||||
|
from .proto.compute_task import *
|
||||||
|
from .proto.ai_function import *
|
||||||
|
from .proto.agent_task import *
|
||||||
|
|
||||||
|
from .agent.agent_base import *
|
||||||
|
from .agent.chatsession import AIChatSession
|
||||||
|
from .agent.agent import AIAgent,AIAgentTemplete, BaseAIAgent
|
||||||
|
from .agent.role import AIRole,AIRoleGroup
|
||||||
|
from .agent.workflow import Workflow
|
||||||
|
from .agent.agent_memory import AgentMemory
|
||||||
|
from .agent.workspace import AgentWorkspace
|
||||||
|
from .agent.llm_context import LLMProcessContext,GlobaToolsLibrary,SimpleLLMContext
|
||||||
|
from .agent.llm_process import BaseLLMProcess,LLMAgentBaseProcess
|
||||||
|
from .agent.llm_process_loader import LLMProcessLoader
|
||||||
|
|
||||||
|
from .frame.compute_kernel import ComputeKernel,ComputeTask,ComputeTaskResult,ComputeTaskState,ComputeTaskType
|
||||||
|
from .frame.compute_node import ComputeNode,LocalComputeNode
|
||||||
|
from .frame.bus import AIBus
|
||||||
|
from .frame.tunnel import AgentTunnel
|
||||||
|
from .frame.contact_manager import ContactManager,Contact
|
||||||
|
from .frame.queue_compute_node import Queue_ComputeNode
|
||||||
|
|
||||||
|
from .environment.environment import BaseEnvironment,SimpleEnvironment,CompositeEnvironment
|
||||||
|
# from .environment.workflow_env import WorkflowEnvironment,CalenderEnvironment,CalenderEvent,PaintEnvironment
|
||||||
|
from .ai_functions.text_to_speech_function import TextToSpeechFunction
|
||||||
|
from .ai_functions.image_2_text_function import Image2TextFunction
|
||||||
|
from .environment.workspace_env import WorkspaceEnvironment
|
||||||
|
|
||||||
|
from .storage.storage import ResourceLocation,AIStorage,UserConfig,UserConfigItem
|
||||||
|
|
||||||
|
from .net import *
|
||||||
|
from .knowledge import *
|
||||||
|
from .package_manager import *
|
||||||
|
from .utils import *
|
||||||
|
|
||||||
|
|
||||||
|
AIOS_Version = "0.5.2, build 2023-12-15"
|
||||||
@@ -0,0 +1,476 @@
|
|||||||
|
import traceback
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
from asyncio import Queue
|
||||||
|
import asyncio
|
||||||
|
import logging
|
||||||
|
import uuid
|
||||||
|
import time
|
||||||
|
import json
|
||||||
|
import shlex
|
||||||
|
import datetime
|
||||||
|
import copy
|
||||||
|
import sys
|
||||||
|
|
||||||
|
from ..proto.agent_msg import AgentMsg
|
||||||
|
from ..proto.ai_function import *
|
||||||
|
from ..proto.agent_task import AgentTaskState,AgentTask,AgentTodo, AgentTodoState
|
||||||
|
from ..proto.compute_task import *
|
||||||
|
|
||||||
|
from .agent_base import *
|
||||||
|
from .llm_process import *
|
||||||
|
from .llm_process_loader import *
|
||||||
|
from .llm_do_task import *
|
||||||
|
from .chatsession import *
|
||||||
|
|
||||||
|
from ..environment.workspace_env import WorkspaceEnvironment, TodoListType
|
||||||
|
from ..environment.environment import *
|
||||||
|
from ..storage.storage import AIStorage
|
||||||
|
from ..knowledge import *
|
||||||
|
from ..proto.compute_task import LLMPrompt,LLMResult
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
class AIAgentTemplete:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.llm_model_name:str = "gpt-4-turbo-preview"
|
||||||
|
self.max_token_size:int = 0
|
||||||
|
self.template_id:str = None
|
||||||
|
self.introduce:str = None
|
||||||
|
self.author:str = None
|
||||||
|
self.prompt:LLMPrompt = None
|
||||||
|
|
||||||
|
|
||||||
|
def load_from_config(self,config:dict) -> bool:
|
||||||
|
if config.get("llm_model_name") is not None:
|
||||||
|
self.llm_model_name = config["llm_model_name"]
|
||||||
|
if config.get("max_token_size") is not None:
|
||||||
|
self.max_token_size = config["max_token_size"]
|
||||||
|
if config.get("template_id") is not None:
|
||||||
|
self.template_id = config["template_id"]
|
||||||
|
if config.get("prompt") is not None:
|
||||||
|
self.prompt = LLMPrompt()
|
||||||
|
if self.prompt.load_from_config(config["prompt"]) is False:
|
||||||
|
logger.error("load prompt from config failed!")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
class AIAgent(BaseAIAgent):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.role_prompt:LLMPrompt = None
|
||||||
|
self.agent_prompt:LLMPrompt = None
|
||||||
|
self.agent_think_prompt:LLMPrompt = None
|
||||||
|
self.llm_model_name:str = None
|
||||||
|
self.max_token_size:int = 128000
|
||||||
|
self.agent_energy = 15
|
||||||
|
self.agent_task = None
|
||||||
|
self.last_recover_time = time.time()
|
||||||
|
self.enable_thread = False
|
||||||
|
self.can_do_unassigned_task = True
|
||||||
|
|
||||||
|
self.agent_id:str = None
|
||||||
|
self.template_id:str = None
|
||||||
|
self.fullname:str = None
|
||||||
|
self.powerby = None
|
||||||
|
self.enable = True
|
||||||
|
self.enable_kb = False
|
||||||
|
self.enable_timestamp = False
|
||||||
|
self.guest_prompt_str = None
|
||||||
|
self.owner_promp_str = None
|
||||||
|
self.contact_prompt_str = None
|
||||||
|
self.history_len = 10
|
||||||
|
self.read_report_prompt = None
|
||||||
|
|
||||||
|
todo_prompts = {}
|
||||||
|
todo_prompts[TodoListType.TO_WORK] = {
|
||||||
|
"do": None,
|
||||||
|
"check": None,
|
||||||
|
"review": None,
|
||||||
|
}
|
||||||
|
todo_prompts[TodoListType.TO_LEARN] = {
|
||||||
|
"do": None,
|
||||||
|
"check": None,
|
||||||
|
"review": None,
|
||||||
|
}
|
||||||
|
self.todo_prompts = todo_prompts
|
||||||
|
|
||||||
|
self.base_dir = None
|
||||||
|
#self.memory_db = None
|
||||||
|
self.unread_msg = Queue() # msg from other agent
|
||||||
|
self.owenr_bus = None
|
||||||
|
|
||||||
|
self.memory : AgentMemory = None
|
||||||
|
self.prviate_workspace : AgentWorkspace = None
|
||||||
|
|
||||||
|
self.behaviors:Dict[str,BaseLLMProcess] = {}
|
||||||
|
|
||||||
|
async def initial(self,params:Dict = None):
|
||||||
|
self.base_dir = f"{AIStorage.get_instance().get_myai_dir()}/agent_data/{self.agent_id}"
|
||||||
|
memory_base_dir = f"{self.base_dir}/memory"
|
||||||
|
self.memory = AgentMemory(self.agent_id,memory_base_dir)
|
||||||
|
self.prviate_workspace = AgentWorkspace(self.agent_id)
|
||||||
|
init_params = {}
|
||||||
|
init_params["memory"] = self.memory
|
||||||
|
init_params["workspace"] = self.prviate_workspace
|
||||||
|
for process_name in self.behaviors.keys():
|
||||||
|
init_result = await self.behaviors[process_name].initial(init_params)
|
||||||
|
if init_result is False:
|
||||||
|
logger.error(f"llm process {process_name} initial failed! initial return False")
|
||||||
|
return False
|
||||||
|
|
||||||
|
self.wake_up()
|
||||||
|
return True
|
||||||
|
|
||||||
|
async def load_from_config(self,config:dict) -> bool:
|
||||||
|
if config.get("instance_id") is None:
|
||||||
|
logger.error("agent instance_id is None!")
|
||||||
|
return False
|
||||||
|
self.agent_id = config["instance_id"]
|
||||||
|
|
||||||
|
if config.get("fullname") is None:
|
||||||
|
logger.error(f"agent {self.agent_id} fullname is None!")
|
||||||
|
return False
|
||||||
|
self.fullname = config["fullname"]
|
||||||
|
|
||||||
|
if config.get("enable_thread") is not None:
|
||||||
|
self.enable_thread = bool(config["enable_thread"])
|
||||||
|
|
||||||
|
if config.get("powerby") is not None:
|
||||||
|
self.powerby = config["powerby"]
|
||||||
|
if config.get("template_id") is not None:
|
||||||
|
self.template_id = config["template_id"]
|
||||||
|
if config.get("llm_model_name") is not None:
|
||||||
|
self.llm_model_name = config["llm_model_name"]
|
||||||
|
if config.get("max_token_size") is not None:
|
||||||
|
self.max_token_size = config["max_token_size"]
|
||||||
|
if config.get("enable_function") is not None:
|
||||||
|
self.enable_function_list = config["enable_function"]
|
||||||
|
if config.get("enable_kb") is not None:
|
||||||
|
self.enable_kb = bool(config["enable_kb"])
|
||||||
|
if config.get("enable_timestamp") is not None:
|
||||||
|
self.enable_timestamp = bool(config["enable_timestamp"])
|
||||||
|
if config.get("history_len"):
|
||||||
|
self.history_len = int(config.get("history_len"))
|
||||||
|
|
||||||
|
#load all LLMProcess
|
||||||
|
self.behaviors = {}
|
||||||
|
behaviors = config.get("behavior")
|
||||||
|
for process_config_name in behaviors.keys():
|
||||||
|
process_config = behaviors[process_config_name]
|
||||||
|
real_config = {}
|
||||||
|
real_config.update(config)
|
||||||
|
real_config.update(process_config)
|
||||||
|
load_result = await LLMProcessLoader.get_instance().load_from_config(real_config)
|
||||||
|
if load_result:
|
||||||
|
self.behaviors[process_config_name] = load_result
|
||||||
|
else:
|
||||||
|
logger.error(f"load LLMProcess {process_config_name} failed!")
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
def get_id(self) -> str:
|
||||||
|
return self.agent_id
|
||||||
|
|
||||||
|
def get_fullname(self) -> str:
|
||||||
|
return self.fullname
|
||||||
|
|
||||||
|
def get_template_id(self) -> str:
|
||||||
|
return self.template_id
|
||||||
|
|
||||||
|
def get_llm_model_name(self) -> str:
|
||||||
|
if self.llm_model_name is None:
|
||||||
|
return AIStorage.get_instance().get_user_config().get_value("llm_model_name")
|
||||||
|
|
||||||
|
return self.llm_model_name
|
||||||
|
|
||||||
|
def get_max_token_size(self) -> int:
|
||||||
|
return self.max_token_size
|
||||||
|
|
||||||
|
def get_agent_role_prompt(self) -> LLMPrompt:
|
||||||
|
return self.role_prompt
|
||||||
|
|
||||||
|
def get_agent_prompt(self) -> LLMPrompt:
|
||||||
|
return self.agent_prompt
|
||||||
|
|
||||||
|
async def _get_context_info(self) -> Dict:
|
||||||
|
context_info = {}
|
||||||
|
|
||||||
|
context_info["location"] = "SanJose"
|
||||||
|
context_info["now"] = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||||||
|
context_info["weather"] = "Partly Cloudy, 60°F"
|
||||||
|
context_info["owner"] = AIStorage.get_instance().get_user_config().get_value("username")
|
||||||
|
|
||||||
|
return context_info
|
||||||
|
|
||||||
|
async def llm_process_msg(self,msg:AgentMsg) -> AgentMsg:
|
||||||
|
need_process:bool = True
|
||||||
|
if msg.msg_type == AgentMsgType.TYPE_GROUPMSG:
|
||||||
|
need_process = False
|
||||||
|
|
||||||
|
session_topic = msg.target + "#" + msg.topic
|
||||||
|
chatsession = AIChatSession.get_session(self.agent_id,session_topic,self.memory.memory_db)
|
||||||
|
if msg.mentions is not None:
|
||||||
|
if self.agent_id in msg.mentions:
|
||||||
|
need_process = True
|
||||||
|
logger.info(f"agent {self.agent_id} recv a group chat message from {msg.sender},but is not mentioned,ignore!")
|
||||||
|
|
||||||
|
if need_process is not True:
|
||||||
|
chatsession.append(msg)
|
||||||
|
resp_msg = msg.create_group_resp_msg(self.agent_id,"")
|
||||||
|
return resp_msg
|
||||||
|
|
||||||
|
context_info = await self._get_context_info()
|
||||||
|
input_parms = {
|
||||||
|
"msg":msg,
|
||||||
|
"context_info":context_info
|
||||||
|
}
|
||||||
|
msg_process = self.behaviors.get("on_message")
|
||||||
|
llm_result : LLMResult = await msg_process.process(input_parms)
|
||||||
|
if llm_result.state == LLMResultStates.ERROR:
|
||||||
|
error_resp = msg.create_error_resp(llm_result.error_str)
|
||||||
|
return error_resp
|
||||||
|
elif llm_result.state == LLMResultStates.IGNORE:
|
||||||
|
return None
|
||||||
|
else: # OK
|
||||||
|
if llm_result.raw_result is not None:
|
||||||
|
resp_msg = llm_result.raw_result.get("_resp_msg")
|
||||||
|
return resp_msg
|
||||||
|
else:
|
||||||
|
return msg.create_resp_msg(llm_result.resp)
|
||||||
|
|
||||||
|
async def _process_msg(self,msg:AgentMsg,workspace = None) -> AgentMsg:
|
||||||
|
return await self.llm_process_msg(msg)
|
||||||
|
|
||||||
|
|
||||||
|
async def llm_self_think(self):
|
||||||
|
llm_process : BaseLLMProcess = self.behaviors.get("self_thinking")
|
||||||
|
if llm_process:
|
||||||
|
logger.info(f"agent {self.agent_id} self thinking start!")
|
||||||
|
|
||||||
|
context_info = await self._get_context_info()
|
||||||
|
#chat_history = AIChatSession.list_session(self.agent_id,self.memory.memory_db)
|
||||||
|
#known_experience_list = await self.memory.list_experience()
|
||||||
|
#record_list = await self.memory.load_records(await self.memory.get_last_think_time())
|
||||||
|
|
||||||
|
input_parms = {
|
||||||
|
"context_info":context_info
|
||||||
|
}
|
||||||
|
|
||||||
|
llm_result : LLMResult = await llm_process.process(input_parms)
|
||||||
|
if llm_result.state == LLMResultStates.ERROR:
|
||||||
|
logger.error(f"llm process self thinking error:{llm_result.compute_error_str}")
|
||||||
|
elif llm_result.state == LLMResultStates.IGNORE:
|
||||||
|
logger.info(f"llm process self thinking ignore!")
|
||||||
|
else:
|
||||||
|
logger.info(f"llm process self thinking ok!,think is:{llm_result.resp}")
|
||||||
|
await self.memory.set_last_think_time(time.time())
|
||||||
|
self.agent_energy -= 2
|
||||||
|
return
|
||||||
|
|
||||||
|
async def llm_triage_tasklist(self):
|
||||||
|
llm_process : BaseLLMProcess = self.behaviors.get("triage_tasks")
|
||||||
|
if llm_process:
|
||||||
|
if self.prviate_workspace:
|
||||||
|
filter = {}
|
||||||
|
filter["state"] = AgentTaskState.TASK_STATE_WAIT
|
||||||
|
|
||||||
|
tasklist:List[AgentTask]= await self.prviate_workspace.task_mgr.list_task(filter)
|
||||||
|
|
||||||
|
|
||||||
|
if tasklist:
|
||||||
|
if len(tasklist) > 0:
|
||||||
|
simple_list:List[Dict] = []
|
||||||
|
for task in tasklist:
|
||||||
|
simple_list.append(task.to_simple_dict())
|
||||||
|
|
||||||
|
input_parms = {
|
||||||
|
"tasklist":simple_list,
|
||||||
|
"context_info": await self._get_context_info()
|
||||||
|
}
|
||||||
|
llm_result : LLMResult = await llm_process.process(input_parms)
|
||||||
|
if llm_result.state == LLMResultStates.ERROR:
|
||||||
|
logger.error(f"llm process triage_tasks error:{llm_result.compute_error_str}")
|
||||||
|
elif llm_result.state == LLMResultStates.IGNORE:
|
||||||
|
logger.info(f"llm process triage_tasks ignore!")
|
||||||
|
else:
|
||||||
|
logger.info(f"llm process triage_tasks ok!,think is:{llm_result.resp}")
|
||||||
|
self.agent_energy -= 3
|
||||||
|
|
||||||
|
# for agent_task in tasklist:
|
||||||
|
# if self.agent_energy <= 0:
|
||||||
|
# break
|
||||||
|
|
||||||
|
# if agent_task.state == AgentTaskState.TASK_STATE_WAIT:
|
||||||
|
# input_parms = {
|
||||||
|
# "task":agent_task
|
||||||
|
# }
|
||||||
|
# llm_result : LLMResult = await llm_process.process(input_parms)
|
||||||
|
# if llm_result.state == LLMResultStates.ERROR:
|
||||||
|
# logger.error(f"llm process review_task error:{llm_result.error_str}")
|
||||||
|
# continue
|
||||||
|
# elif llm_result.state == LLMResultStates.IGNORE:
|
||||||
|
# logger.info(f"llm process review_task ignore!")
|
||||||
|
# continue
|
||||||
|
# else:
|
||||||
|
# determine = llm_result.raw_result.get("determine")
|
||||||
|
# logger.info(f"llm process review_task ok!,think is:{determine}")
|
||||||
|
# self.agent_energy -= 1
|
||||||
|
|
||||||
|
async def llm_do_todo(self, todo: AgentTodo):
|
||||||
|
llm_process : BaseLLMProcess = self.behaviors.get("do")
|
||||||
|
logger.info(f"agent {self.agent_id} DO todo {todo.todo_path} start!")
|
||||||
|
if llm_process:
|
||||||
|
input_parms = {
|
||||||
|
"todo":todo,
|
||||||
|
"context_info": await self._get_context_info()
|
||||||
|
}
|
||||||
|
llm_result : LLMResult = await llm_process.process(input_parms)
|
||||||
|
if llm_result.state == LLMResultStates.ERROR:
|
||||||
|
logger.error(f"llm process do_todo error:{llm_result.compute_error_str}")
|
||||||
|
elif llm_result.state == LLMResultStates.IGNORE:
|
||||||
|
logger.info(f"llm process do_todo ignore!")
|
||||||
|
else:
|
||||||
|
logger.info(f"llm process do_todo ok!,think is:{llm_result.resp}")
|
||||||
|
self.agent_energy -= 1
|
||||||
|
|
||||||
|
async def llm_check_todo(self, todo: AgentTodo):
|
||||||
|
llm_process : BaseLLMProcess = self.behaviors.get("check")
|
||||||
|
logger.info(f"agent {self.agent_id} CHECK todo {todo.todo_path} start!")
|
||||||
|
if llm_process:
|
||||||
|
input_parms = {
|
||||||
|
"todo":todo,
|
||||||
|
"context_info": await self._get_context_info()
|
||||||
|
}
|
||||||
|
llm_result : LLMResult = await llm_process.process(input_parms)
|
||||||
|
if llm_result.state == LLMResultStates.ERROR:
|
||||||
|
logger.error(f"llm process check_todo error:{llm_result.compute_error_str}")
|
||||||
|
elif llm_result.state == LLMResultStates.IGNORE:
|
||||||
|
logger.info(f"llm process check_todo ignore!")
|
||||||
|
else:
|
||||||
|
logger.info(f"llm process check_todo ok!,think is:{llm_result.resp}")
|
||||||
|
self.agent_energy -= 1
|
||||||
|
|
||||||
|
return
|
||||||
|
|
||||||
|
async def llm_plan_task(self,task:AgentTask):
|
||||||
|
llm_process : BaseLLMProcess = self.behaviors.get("plan_task")
|
||||||
|
logger.info(f"agent {self.agent_id} PLAN task {task.task_path} start!")
|
||||||
|
if llm_process:
|
||||||
|
input_parms = {
|
||||||
|
"task":task,
|
||||||
|
"context_info": await self._get_context_info()
|
||||||
|
}
|
||||||
|
llm_result : LLMResult = await llm_process.process(input_parms)
|
||||||
|
if llm_result.state == LLMResultStates.ERROR:
|
||||||
|
logger.error(f"llm process plan_task error:{llm_result.compute_error_str}")
|
||||||
|
elif llm_result.state == LLMResultStates.IGNORE:
|
||||||
|
logger.info(f"llm process plan_task ignore!")
|
||||||
|
else:
|
||||||
|
logger.info(f"llm process plan_task ok!,think is:{llm_result.resp}")
|
||||||
|
self.agent_energy -= 1
|
||||||
|
|
||||||
|
async def llm_review_task(self,task:AgentTask):
|
||||||
|
llm_process : BaseLLMProcess = self.behaviors.get("review_task")
|
||||||
|
logger.info(f"agent {self.agent_id} REVIEW task {task.task_path} start!")
|
||||||
|
if llm_process:
|
||||||
|
input_parms = {
|
||||||
|
"task":task,
|
||||||
|
"context_info": await self._get_context_info()
|
||||||
|
}
|
||||||
|
llm_result : LLMResult = await llm_process.process(input_parms)
|
||||||
|
if llm_result.state == LLMResultStates.ERROR:
|
||||||
|
logger.error(f"llm process review_task error:{llm_result.compute_error_str}")
|
||||||
|
elif llm_result.state == LLMResultStates.IGNORE:
|
||||||
|
logger.info(f"llm process review_task ignore!")
|
||||||
|
else:
|
||||||
|
logger.info(f"llm process review_task ok!,think is:{llm_result.resp}")
|
||||||
|
self.agent_energy -= 1
|
||||||
|
|
||||||
|
|
||||||
|
async def _self_imporve(self):
|
||||||
|
await self.llm_self_think()
|
||||||
|
|
||||||
|
def wake_up(self) -> None:
|
||||||
|
if self.agent_task is None:
|
||||||
|
self.agent_task = asyncio.create_task(self._on_timer())
|
||||||
|
else:
|
||||||
|
logger.warning(f"agent {self.agent_id} is already wake up!")
|
||||||
|
|
||||||
|
async def _on_timer(self):
|
||||||
|
await asyncio.sleep(5)
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
now = time.time()
|
||||||
|
if self.last_recover_time is None:
|
||||||
|
self.last_recover_time = now
|
||||||
|
else:
|
||||||
|
if now - self.last_recover_time > 60:
|
||||||
|
self.agent_energy += (now - self.last_recover_time) / 60
|
||||||
|
self.last_recover_time = now
|
||||||
|
logger.info(f"agent {self.agent_id} recover energy to {self.agent_energy}")
|
||||||
|
|
||||||
|
if self.agent_energy <= 1:
|
||||||
|
logger.info(f"agent {self.agent_id} energy is too low!, goto sleep!")
|
||||||
|
continue
|
||||||
|
|
||||||
|
await self.llm_triage_tasklist()
|
||||||
|
# Get un finished tasks
|
||||||
|
#filter = {}
|
||||||
|
#filter["state"] = AgentTaskState.TASK_STATE_WAIT
|
||||||
|
filter = None
|
||||||
|
task_list:List[AgentTask] = await self.prviate_workspace.task_mgr.list_task(filter)
|
||||||
|
|
||||||
|
for task in task_list:
|
||||||
|
if self.agent_energy <= 0:
|
||||||
|
break
|
||||||
|
|
||||||
|
task = await self.prviate_workspace.task_mgr.get_task(task.task_id)
|
||||||
|
if task.can_plan():
|
||||||
|
# PLAN Task
|
||||||
|
await self.llm_plan_task(task)
|
||||||
|
task = await self.prviate_workspace.task_mgr.get_task(task.task_id)
|
||||||
|
|
||||||
|
if task.state == AgentTaskState.TASK_STATE_DOING:
|
||||||
|
# DO or Check Todo
|
||||||
|
can_review = False
|
||||||
|
todolist = await self.prviate_workspace.task_mgr.get_sub_todos(task.task_id)
|
||||||
|
for todo in todolist:
|
||||||
|
if self.agent_energy <= 0:
|
||||||
|
break
|
||||||
|
task = await self.prviate_workspace.task_mgr.get_task(task.task_id)
|
||||||
|
todo = await self.prviate_workspace.task_mgr.get_todo_by_id(todo.todo_id)
|
||||||
|
if task.state != AgentTaskState.TASK_STATE_DOING:
|
||||||
|
break
|
||||||
|
if todo.state == AgentTodoState.TODO_STATE_WAITING or todo.state == AgentTodoState.TODO_STATE_EXEC_FAILED:
|
||||||
|
await self.llm_do_todo(todo)
|
||||||
|
task = await self.prviate_workspace.task_mgr.get_task(task.task_id)
|
||||||
|
todo = await self.prviate_workspace.task_mgr.get_todo_by_id(todo.todo_id)
|
||||||
|
if task.state != AgentTaskState.TASK_STATE_DOING:
|
||||||
|
break
|
||||||
|
if todo.state == AgentTodoState.TODO_STATE_EXEC_OK:
|
||||||
|
await self.llm_check_todo(todo)
|
||||||
|
|
||||||
|
if can_review:
|
||||||
|
task.state = AgentTaskState.TASK_STATE_WAITING_REVIEW
|
||||||
|
|
||||||
|
task = await self.prviate_workspace.task_mgr.get_task(task.task_id)
|
||||||
|
if task.state == AgentTaskState.TASK_STATE_WAITING_REVIEW:
|
||||||
|
await self.llm_review_task(task)
|
||||||
|
|
||||||
|
await self._self_imporve()
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
tb_str = traceback.format_exc()
|
||||||
|
logger.error(f"agent {self.agent_id} on timer error:{e},{tb_str}")
|
||||||
|
|
||||||
|
# Because the LLM itself is very slow, the accuracy of the system processing task is in minutes.
|
||||||
|
await asyncio.sleep(30)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
# pylint:disable=E0402
|
||||||
|
|
||||||
|
import abc
|
||||||
|
from abc import abstractmethod
|
||||||
|
|
||||||
|
from ..proto.agent_msg import AgentMsg
|
||||||
|
|
||||||
|
class BaseAIAgent(abc.ABC):
|
||||||
|
@abstractmethod
|
||||||
|
def get_id(self) -> str:
|
||||||
|
pass
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def get_llm_model_name(self) -> str:
|
||||||
|
pass
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def get_max_token_size(self) -> int:
|
||||||
|
pass
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
async def _process_msg(self,msg:AgentMsg,workspace = None) -> AgentMsg:
|
||||||
|
pass
|
||||||
|
|
||||||
|
class CustomAIAgent(BaseAIAgent):
|
||||||
|
def __init__(self, agent_id: str, llm_model_name: str, max_token_size: int) -> None:
|
||||||
|
self.agent_id = agent_id
|
||||||
|
self.llm_model_name = llm_model_name
|
||||||
|
self.max_token_size = max_token_size
|
||||||
|
|
||||||
|
def get_id(self) -> str:
|
||||||
|
return self.agent_id
|
||||||
|
|
||||||
|
def get_llm_model_name(self) -> str:
|
||||||
|
return self.llm_model_name
|
||||||
|
|
||||||
|
def get_max_token_size(self) -> int:
|
||||||
|
return self.max_token_size
|
||||||
@@ -0,0 +1,507 @@
|
|||||||
|
# pylint:disable=E0402
|
||||||
|
from datetime import datetime,timedelta
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import threading
|
||||||
|
from typing import Dict, List
|
||||||
|
import sqlite3
|
||||||
|
|
||||||
|
import aiofiles
|
||||||
|
|
||||||
|
from ..storage.storage import AIStorage
|
||||||
|
from ..frame.compute_kernel import ComputeKernel
|
||||||
|
from ..frame.contact_manager import ContactManager
|
||||||
|
from ..frame.contact import Contact
|
||||||
|
from ..proto.ai_function import ParameterDefine, SimpleAIAction, SimpleAIFunction
|
||||||
|
from ..proto.agent_msg import AgentMsg, AgentMsgType
|
||||||
|
from ..proto.agent_task import AgentWorkLog
|
||||||
|
|
||||||
|
from .llm_context import GlobaToolsLibrary
|
||||||
|
from .chatsession import AIChatSession
|
||||||
|
|
||||||
|
import logging
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
#class ObjectSummary:
|
||||||
|
# def __init__(self) -> None:
|
||||||
|
# self.summary : str = None
|
||||||
|
# self.object_name : str = None
|
||||||
|
# self.priority : int = 5
|
||||||
|
# [info_source, info]
|
||||||
|
# self.infos : Dict[str,str] = {}
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
class AgentMemory:
|
||||||
|
def __init__(self,agent_id:str,base_dir:str) -> None:
|
||||||
|
self.agent_memory_base_dir = base_dir
|
||||||
|
self.agent_id:str= agent_id
|
||||||
|
|
||||||
|
AIStorage.get_instance().ensure_directory_exists(self.agent_memory_base_dir)
|
||||||
|
AIStorage.get_instance().ensure_directory_exists(f"{self.agent_memory_base_dir}/experience")
|
||||||
|
AIStorage.get_instance().ensure_directory_exists(f"{self.agent_memory_base_dir}/contacts")
|
||||||
|
AIStorage.get_instance().ensure_directory_exists(f"{self.agent_memory_base_dir}/relations")
|
||||||
|
AIStorage.get_instance().ensure_directory_exists(f"{self.agent_memory_base_dir}/summary")
|
||||||
|
|
||||||
|
self.memory_db:str = f"{self.agent_memory_base_dir}/memory.db"
|
||||||
|
self.model_name:str = "gp4-1106-preview"
|
||||||
|
self.threshold_hours = 72
|
||||||
|
self.last_think_time : float = 0.0
|
||||||
|
|
||||||
|
self.load_memory_meta()
|
||||||
|
|
||||||
|
|
||||||
|
def _get_conn(self):
|
||||||
|
""" get db connection """
|
||||||
|
local = threading.local()
|
||||||
|
if not hasattr(local, 'conn'):
|
||||||
|
local.conn = self._create_connection(self.memory_db)
|
||||||
|
return local.conn
|
||||||
|
|
||||||
|
def _create_connection(self, db_file):
|
||||||
|
""" create a database connection to a SQLite database """
|
||||||
|
conn = None
|
||||||
|
try:
|
||||||
|
conn = sqlite3.connect(db_file)
|
||||||
|
except Error as e:
|
||||||
|
logging.error("Error occurred while connecting to database: %s", e)
|
||||||
|
return None
|
||||||
|
|
||||||
|
if conn:
|
||||||
|
self._create_table(conn)
|
||||||
|
|
||||||
|
return conn
|
||||||
|
|
||||||
|
def get_session_from_msg(self,msg:AgentMsg) -> AIChatSession:
|
||||||
|
if msg.msg_type == AgentMsgType.TYPE_GROUPMSG:
|
||||||
|
session_topic = msg.target + "#" + msg.topic
|
||||||
|
chatsession = AIChatSession.get_session(self.agent_id,session_topic,self.memory_db)
|
||||||
|
else:
|
||||||
|
session_topic = msg.get_sender() + "#" + msg.topic
|
||||||
|
chatsession = AIChatSession.get_session(self.agent_id,session_topic,self.memory_db)
|
||||||
|
return chatsession
|
||||||
|
|
||||||
|
# return last record time
|
||||||
|
async def load_records(self,starttime,tokenlimit=8000)->float:
|
||||||
|
# 专用思路：做聊天记录/工作经验的整理
|
||||||
|
# 通用思路：没有具体的目的，让Agent根据提示词自己工作（可能效果很差也可能很好）
|
||||||
|
# 先实现通用思路
|
||||||
|
msg_records = AIChatSession.load_message_records_by_agentid(self.agent_id,starttime,32,self.memory_db)
|
||||||
|
work_records = self.load_worklogs(self.agent_id,token_limit=tokenlimit)
|
||||||
|
pass
|
||||||
|
|
||||||
|
async def load_chatlogs(self,msg:AgentMsg,token_limit=800):
|
||||||
|
chatsession = self.get_session_from_msg(msg)
|
||||||
|
# Must load n (n> = 2), and hope to load the M
|
||||||
|
# The information in the # M is gradually added, knowing that it is less than 72 hours from the current time, and consumes enough tokens
|
||||||
|
|
||||||
|
messages_n = chatsession.read_history() # read
|
||||||
|
histroy_str = ""
|
||||||
|
read_count = 0
|
||||||
|
is_all = True
|
||||||
|
for msg in messages_n:
|
||||||
|
dt = datetime.fromtimestamp(float(msg.create_time))
|
||||||
|
formatted_time = dt.strftime('%y-%m-%d %H:%M:%S')
|
||||||
|
record_str = f"{msg.sender},[{formatted_time}]\n{msg.body}\n"
|
||||||
|
token_limit -= ComputeKernel.llm_num_tokens_from_text(record_str,self.model_name)
|
||||||
|
if token_limit <= 32:
|
||||||
|
is_all = False
|
||||||
|
break
|
||||||
|
read_count += 1
|
||||||
|
histroy_str = record_str + histroy_str
|
||||||
|
|
||||||
|
return histroy_str,is_all
|
||||||
|
|
||||||
|
async def get_chat_summary(self,msg:AgentMsg) -> str:
|
||||||
|
chatsession : AIChatSession = self.get_session_from_msg(msg)
|
||||||
|
return chatsession.summary
|
||||||
|
|
||||||
|
# async def action_chatlog_append(self,params:Dict) -> str:
|
||||||
|
#
|
||||||
|
# input_msg:AgentMsg = params.get("input").get("msg")
|
||||||
|
# llm_result = params.get("llm_result")
|
||||||
|
# chatsession = self.get_session_from_msg(input_msg)
|
||||||
|
# resp_msg = params.get("resp_msg")
|
||||||
|
# if resp_msg:
|
||||||
|
# tags = llm_result.raw_result.get("tags")
|
||||||
|
# chatsession.append(input_msg,tags)
|
||||||
|
# chatsession.append(resp_msg,tags)
|
||||||
|
|
||||||
|
# return "OK"
|
||||||
|
|
||||||
|
async def load_worklogs(self,operator_id:str,owner_id:str=None, work_types:List[str]=None,token_limit=800):
|
||||||
|
conn = self._get_conn()
|
||||||
|
c = conn.cursor()
|
||||||
|
|
||||||
|
query = 'SELECT * FROM worklog WHERE 1=1'
|
||||||
|
params = []
|
||||||
|
|
||||||
|
if operator_id is not None:
|
||||||
|
query += ' AND operator=?'
|
||||||
|
params.append(operator_id)
|
||||||
|
|
||||||
|
if owner_id is not None:
|
||||||
|
query += ' AND owner_id=?'
|
||||||
|
params.append(owner_id)
|
||||||
|
|
||||||
|
if work_types:
|
||||||
|
query += ' AND work_type IN ({})'.format(', '.join('?'*len(work_types)))
|
||||||
|
params.extend(work_types)
|
||||||
|
|
||||||
|
query += ' ORDER BY timestamp DESC LIMIT 8'
|
||||||
|
|
||||||
|
c.execute(query, tuple(params))
|
||||||
|
rows = c.fetchall()
|
||||||
|
|
||||||
|
|
||||||
|
return [self.from_db_row(row) for row in rows]
|
||||||
|
|
||||||
|
def _create_table(self,conn):
|
||||||
|
c = conn.cursor()
|
||||||
|
c.execute('''
|
||||||
|
CREATE TABLE IF NOT EXISTS worklog (
|
||||||
|
logid TEXT PRIMARY KEY,
|
||||||
|
owner_id TEXT,
|
||||||
|
work_type TEXT,
|
||||||
|
timestamp REAL,
|
||||||
|
content TEXT,
|
||||||
|
result TEXT,
|
||||||
|
meta TEXT,
|
||||||
|
operator TEXT
|
||||||
|
)
|
||||||
|
''')
|
||||||
|
conn.commit()
|
||||||
|
#conn.close()
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_db_row(self,row):
|
||||||
|
log = AgentWorkLog()
|
||||||
|
# 这里高度依赖表结构的顺序
|
||||||
|
log.logid, log.owner_id, log.work_type, log.timestamp, log.content, log.result, meta_str, log.operator = row
|
||||||
|
log.meta = json.loads(meta_str) if meta_str else None
|
||||||
|
return log
|
||||||
|
|
||||||
|
async def append_worklog(self,log:AgentWorkLog)->str:
|
||||||
|
conn = self._get_conn()
|
||||||
|
c = conn.cursor()
|
||||||
|
# 将meta字典转换为JSON字符串
|
||||||
|
meta_str = json.dumps(log.meta,ensure_ascii=False) if log.meta else None
|
||||||
|
c.execute('''
|
||||||
|
INSERT INTO worklog (logid, owner_id, work_type, timestamp, content, result, meta, operator)
|
||||||
|
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
|
||||||
|
''', (log.logid, log.owner_id, log.work_type, log.timestamp, log.content, log.result, meta_str, log.operator))
|
||||||
|
conn.commit()
|
||||||
|
#conn.close()
|
||||||
|
|
||||||
|
def memory_meta_to_dict(self) -> Dict:
|
||||||
|
return {
|
||||||
|
"last_think_time" : self.last_think_time
|
||||||
|
}
|
||||||
|
|
||||||
|
def load_meta(self,Dict):
|
||||||
|
self.last_think_time = Dict.get("last_think_time",0.0)
|
||||||
|
|
||||||
|
def load_memory_meta(self):
|
||||||
|
meta_file_path = f"{self.agent_memory_base_dir}/meta.json"
|
||||||
|
try:
|
||||||
|
with open(meta_file_path, mode='r') as file:
|
||||||
|
meta = json.load(file)
|
||||||
|
self.load_meta(meta)
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"load memory meta failed: {e}")
|
||||||
|
self.last_think_time = 0.0
|
||||||
|
|
||||||
|
def save_memory_meta(self):
|
||||||
|
meta_file_path = f"{self.agent_memory_base_dir}/meta.json"
|
||||||
|
try:
|
||||||
|
with open(meta_file_path, mode='w') as file:
|
||||||
|
meta = self.memory_meta_to_dict()
|
||||||
|
json.dump(meta,file)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"save memory meta failed: {e}")
|
||||||
|
|
||||||
|
async def get_last_think_time(self)->float:
|
||||||
|
return self.last_think_time
|
||||||
|
|
||||||
|
async def set_last_think_time(self,last_time:float):
|
||||||
|
self.last_think_time = last_time
|
||||||
|
self.save_memory_meta()
|
||||||
|
|
||||||
|
async def get_contact_summary(self,contact_id:str) -> str:
|
||||||
|
if contact_id is None:
|
||||||
|
return "Contact id is None"
|
||||||
|
|
||||||
|
result = {}
|
||||||
|
contact_info:Contact = ContactManager.get_instance().find_contact_by_name(contact_id)
|
||||||
|
if contact_info:
|
||||||
|
result["name"] = contact_info.name
|
||||||
|
result["relation"] = contact_info.relationship
|
||||||
|
result["notes"] = contact_info.notes
|
||||||
|
|
||||||
|
summary_path = f"{self.agent_memory_base_dir}/contacts/{contact_id}.summary"
|
||||||
|
try:
|
||||||
|
async with aiofiles.open(summary_path, mode='r') as file:
|
||||||
|
result["summary"] = await file.read()
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"read contact summary failed: {e}")
|
||||||
|
|
||||||
|
return json.dumps(result,ensure_ascii=False)
|
||||||
|
|
||||||
|
async def update_contact_summary(self,contact_id:str,summary:str):
|
||||||
|
summary_path = f"{self.agent_memory_base_dir}/contacts/{contact_id}.summary"
|
||||||
|
try:
|
||||||
|
async with aiofiles.open(summary_path, mode='w') as file:
|
||||||
|
await file.write(summary)
|
||||||
|
return "OK"
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"write contact summary failed: {e}")
|
||||||
|
return "write contact summary failed: {e}"
|
||||||
|
|
||||||
|
async def get_summary(self,object_name:str) -> str:
|
||||||
|
summary_path = f"{self.agent_memory_base_dir}/{object_name}.summary"
|
||||||
|
try:
|
||||||
|
async with aiofiles.open(summary_path, mode='r') as file:
|
||||||
|
return await file.read()
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"read summary failed: {e}")
|
||||||
|
return f"read summary failed: {e}"
|
||||||
|
|
||||||
|
async def update_summary(self,object_name:str,summary:str) -> str:
|
||||||
|
summary_path = f"{self.agent_memory_base_dir}/{object_name}.summary"
|
||||||
|
try:
|
||||||
|
async with aiofiles.open(summary_path, mode='w') as file:
|
||||||
|
await file.write(summary)
|
||||||
|
return "OK"
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"write summary failed: {e}")
|
||||||
|
return f"write summary failed: {e}"
|
||||||
|
|
||||||
|
async def list_summary_object_names(self) -> List[str]:
|
||||||
|
# list dir
|
||||||
|
try:
|
||||||
|
contents = os.listdir(self.agent_memory_base_dir)
|
||||||
|
return [x for x in contents if x.endswith(".summary")]
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"list summary object names failed: {e}")
|
||||||
|
return []
|
||||||
|
|
||||||
|
# means object1 feel object2 is ...
|
||||||
|
async def get_relation_summary(self,object_name1:str,object_name2:str) -> str:
|
||||||
|
summary_path = f"{self.agent_memory_base_dir}/relations/{object_name1}.relation.{object_name2}.summary"
|
||||||
|
try:
|
||||||
|
async with aiofiles.open(summary_path, mode='r') as file:
|
||||||
|
await file.read()
|
||||||
|
except FileNotFoundError:
|
||||||
|
return "no summary"
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"read relation summary failed: {e}")
|
||||||
|
return f"read relation summary failed: {e}"
|
||||||
|
|
||||||
|
|
||||||
|
async def update_relation_summary(self,object_name1:str,object_name2:str,summary:Dict):
|
||||||
|
summary_path = f"{self.agent_memory_base_dir}/relations/{object_name1}.relation.{object_name2}.summary"
|
||||||
|
try:
|
||||||
|
async with aiofiles.open(summary_path, mode='w') as file:
|
||||||
|
await file.write(json.dumps(summary))
|
||||||
|
return "OK"
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"write relation summary failed: {e}")
|
||||||
|
return "write relation summary failed: {e}"
|
||||||
|
|
||||||
|
async def get_experience(self,topic_name:str) -> str:
|
||||||
|
experience_path = f"{self.agent_memory_base_dir}/experience/{topic_name}.experience"
|
||||||
|
try:
|
||||||
|
async with aiofiles.open(experience_path, mode='r') as file:
|
||||||
|
await file.read()
|
||||||
|
except FileNotFoundError:
|
||||||
|
return "no experience"
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"read experience failed: {e}")
|
||||||
|
return f"read experience failed: {e}"
|
||||||
|
|
||||||
|
|
||||||
|
async def set_experience(self,topic_name:str,summary:str) -> str:
|
||||||
|
experience_path = f"{self.agent_memory_base_dir}/experience/{topic_name}.experience"
|
||||||
|
try:
|
||||||
|
async with aiofiles.open(experience_path, mode='w') as file:
|
||||||
|
await file.write(summary)
|
||||||
|
return "OK"
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"write experience failed: {e}")
|
||||||
|
return "write experience failed: {e}"
|
||||||
|
|
||||||
|
async def list_experience(self) -> List[str]:
|
||||||
|
dir_path = f"{self.agent_memory_base_dir}/experience"
|
||||||
|
try:
|
||||||
|
contents = os.listdir(dir_path)
|
||||||
|
return [x for x in contents if x.endswith(".experience")]
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"list experience failed: {e}")
|
||||||
|
return []
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def register_ai_functions():
|
||||||
|
async def update_chat_summary(parameters):
|
||||||
|
agent_memory:AgentMemory = parameters.get("_memory")
|
||||||
|
chatsession = AIChatSession.get_session_by_id(parameters.get("session_id"),agent_memory.memory_db)
|
||||||
|
summary = parameters.get("summary")
|
||||||
|
chatsession.update_summary(summary)
|
||||||
|
return "OK"
|
||||||
|
parameters = ParameterDefine.create_parameters({
|
||||||
|
"session_id": {"type": "string", "description": "session id"},
|
||||||
|
"summary": {"type": "string", "description": "new summary"}
|
||||||
|
})
|
||||||
|
update_chat_summary_func = SimpleAIFunction("agent.memory.update_chat_summary",
|
||||||
|
"update chat summary",
|
||||||
|
update_chat_summary,
|
||||||
|
parameters)
|
||||||
|
GlobaToolsLibrary.get_instance().register_tool_function(update_chat_summary_func)
|
||||||
|
|
||||||
|
# async def get_contact_summary(parameters):
|
||||||
|
# agent_memory:AgentMemory = parameters.get("_memory")
|
||||||
|
# contact_name = parameters.get("contact_name")
|
||||||
|
# return await agent_memory.get_contact_summary(contact_name)
|
||||||
|
# parameters = ParameterDefine.create_parameters({
|
||||||
|
# "contact_name": {"type": "string", "description": "contact name"}
|
||||||
|
# })
|
||||||
|
# get_contact_summary_func = SimpleAIFunction("agent.memory.get_contact_summary",
|
||||||
|
# "get contact summary",
|
||||||
|
# get_contact_summary,
|
||||||
|
# parameters)
|
||||||
|
# GlobaToolsLibrary.register_tool_function(get_contact_summary_func)
|
||||||
|
|
||||||
|
# async def update_contact_summary(parameters):
|
||||||
|
# agent_memory:AgentMemory = parameters.get("_memory")
|
||||||
|
# contact_name = parameters.get("contact_name")
|
||||||
|
# summary = parameters.get("summary")
|
||||||
|
# return await agent_memory.update_contact_summary(contact_name,summary)
|
||||||
|
# parameters = ParameterDefine.create_parameters({
|
||||||
|
# "contact_name": {"type": "string", "description": "contact name"},
|
||||||
|
# "summary": {"type": "string", "description": "new summary"}
|
||||||
|
# })
|
||||||
|
# update_contact_summary_func = SimpleAIFunction("agent.memory.update_contact_summary",
|
||||||
|
# "update contact summary",
|
||||||
|
# update_contact_summary,
|
||||||
|
# parameters)
|
||||||
|
# GlobaToolsLibrary.register_tool_function(update_contact_summary_func)
|
||||||
|
|
||||||
|
# async def get_summary(parameters):
|
||||||
|
# agent_memory:AgentMemory = parameters.get("_memory")
|
||||||
|
# object_name = parameters.get("object_name")
|
||||||
|
# return await agent_memory.get_summary(object_name)
|
||||||
|
# parameters = ParameterDefine.create_parameters({
|
||||||
|
# "object_name": {"type": "string", "description": "object name"}
|
||||||
|
# })
|
||||||
|
# get_summary_func = SimpleAIFunction("agent.memory.get_summary",
|
||||||
|
# "get summary of sth",
|
||||||
|
# get_summary,
|
||||||
|
# parameters)
|
||||||
|
# GlobaToolsLibrary.register_tool_function(get_summary_func)
|
||||||
|
|
||||||
|
# async def update_summary(parameters):
|
||||||
|
# agent_memory:AgentMemory = parameters.get("_memory")
|
||||||
|
# object_name = parameters.get("object_name")
|
||||||
|
# summary = parameters.get("summary")
|
||||||
|
# return await agent_memory.update_summary(object_name,summary)
|
||||||
|
# parameters = ParameterDefine.create_parameters({
|
||||||
|
# "object_name": {"type": "string", "description": "object name"},
|
||||||
|
# "summary": {"type": "string", "description": "new summary"}
|
||||||
|
# })
|
||||||
|
# update_summary_func = SimpleAIFunction("agent.memory.update_summary",
|
||||||
|
# "update summary of sth",
|
||||||
|
# update_summary,
|
||||||
|
# parameters)
|
||||||
|
# GlobaToolsLibrary.register_tool_function(update_summary_func)
|
||||||
|
|
||||||
|
# async def list_summary_object_names(parameters):
|
||||||
|
# agent_memory:AgentMemory = parameters.get("_memory")
|
||||||
|
# return await agent_memory.list_summary_object_names()
|
||||||
|
# parameters = ParameterDefine.create_parameters({})
|
||||||
|
# list_summary_object_names_func = SimpleAIFunction("agent.memory.list_summary",
|
||||||
|
# "list summary object names",
|
||||||
|
# list_summary_object_names,
|
||||||
|
# parameters)
|
||||||
|
# GlobaToolsLibrary.register_tool_function(list_summary_object_names_func)
|
||||||
|
|
||||||
|
# async def get_relation_summary(parameters):
|
||||||
|
# agent_memory:AgentMemory = parameters.get("_memory")
|
||||||
|
# object_name1 = parameters.get("object1_name")
|
||||||
|
# object_name2 = parameters.get("object2_name")
|
||||||
|
# return await agent_memory.get_relation_summary(object_name1,object_name2)
|
||||||
|
# parameters = ParameterDefine.create_parameters({
|
||||||
|
# "object1_name": {"type": "string", "description": "object name1"},
|
||||||
|
# "object2_name": {"type": "string", "description": "object name2"}
|
||||||
|
# })
|
||||||
|
# get_relation_summary_func = SimpleAIFunction("agent.memory.get_relation_summary",
|
||||||
|
# "object1 feel object2 is ...",
|
||||||
|
# get_relation_summary,
|
||||||
|
# parameters)
|
||||||
|
# GlobaToolsLibrary.register_tool_function(get_relation_summary_func)
|
||||||
|
|
||||||
|
# async def update_relation_summary(parameters):
|
||||||
|
# agent_memory:AgentMemory = parameters.get("_memory")
|
||||||
|
# object_name1 = parameters.get("object1_name")
|
||||||
|
# object_name2 = parameters.get("object2_name")
|
||||||
|
# summary = parameters.get("summary")
|
||||||
|
# return await agent_memory.update_relation_summary(object_name1,object_name2,summary)
|
||||||
|
# parameters = ParameterDefine.create_parameters({
|
||||||
|
# "object1_name": {"type": "string", "description": "object name1"},
|
||||||
|
# "object2_name": {"type": "string", "description": "object name2"},
|
||||||
|
# "summary": {"type": "string", "description": "new summary"}
|
||||||
|
# })
|
||||||
|
# update_relation_summary_func = SimpleAIFunction("agent.memory.update_relation_summary",
|
||||||
|
# "object1 feel object2 is ...",
|
||||||
|
# update_relation_summary,
|
||||||
|
# parameters)
|
||||||
|
# GlobaToolsLibrary.register_tool_function(update_relation_summary_func)
|
||||||
|
|
||||||
|
# async def get_experience(parameters):
|
||||||
|
# agent_memory:AgentMemory = parameters.get("_memory")
|
||||||
|
# topic_name = parameters.get("topic_name")
|
||||||
|
# return await agent_memory.get_experience(topic_name)
|
||||||
|
# parameters = ParameterDefine.create_parameters({
|
||||||
|
# "topic_name": {"type": "string", "description": "topic name"}
|
||||||
|
# })
|
||||||
|
# get_experience_func = SimpleAIFunction("agent.memory.get_experience",
|
||||||
|
# "get experience",
|
||||||
|
# get_experience,
|
||||||
|
# parameters)
|
||||||
|
# GlobaToolsLibrary.register_tool_function(get_experience_func)
|
||||||
|
|
||||||
|
# async def set_experience(parameters):
|
||||||
|
# agent_memory:AgentMemory = parameters.get("_memory")
|
||||||
|
# topic_name = parameters.get("topic_name")
|
||||||
|
# summary = parameters.get("summary")
|
||||||
|
# return await agent_memory.set_experience(topic_name,summary)
|
||||||
|
# parameters = ParameterDefine.create_parameters({
|
||||||
|
# "topic_name": {"type": "string", "description": "topic name"},
|
||||||
|
# "summary": {"type": "string", "description": "new summary"}
|
||||||
|
# })
|
||||||
|
# set_experience_func = SimpleAIFunction("agent.memory.set_experience",
|
||||||
|
# "set experience",
|
||||||
|
# set_experience,
|
||||||
|
# parameters)
|
||||||
|
# GlobaToolsLibrary.register_tool_function(set_experience_func)
|
||||||
|
|
||||||
|
# async def list_experience(parameters):
|
||||||
|
# agent_memory:AgentMemory = parameters.get("_memory")
|
||||||
|
# return await agent_memory.list_experience()
|
||||||
|
# parameters = ParameterDefine.create_parameters({})
|
||||||
|
# list_experience_func = SimpleAIFunction("agent.memory.list_experience",
|
||||||
|
# "list exist experience topics",
|
||||||
|
# list_experience,
|
||||||
|
# parameters)
|
||||||
|
# GlobaToolsLibrary.register_tool_function(list_experience_func)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
|
# pylint:disable=E0402
|
||||||
import sqlite3 # Because sqlite3 IO operation is small, so we can use sqlite3 directly.(so we don't need to use async sqlite3 now)
|
import sqlite3 # Because sqlite3 IO operation is small, so we can use sqlite3 directly.(so we don't need to use async sqlite3 now)
|
||||||
from sqlite3 import Error
|
from sqlite3 import Error
|
||||||
import logging
|
import logging
|
||||||
@@ -6,21 +6,22 @@ import threading
|
|||||||
import datetime
|
import datetime
|
||||||
import uuid
|
import uuid
|
||||||
import json
|
import json
|
||||||
|
from typing import List
|
||||||
|
|
||||||
from .agent_message import AgentMsgType, AgentMsg, AgentMsgStatus
|
from ..proto.agent_msg import AgentMsgType, AgentMsg, AgentMsgStatus
|
||||||
|
|
||||||
class ChatSessionDB:
|
class ChatSessionDB:
|
||||||
def __init__(self, db_file):
|
def __init__(self, db_file):
|
||||||
""" initialize db connection """
|
""" initialize db connection """
|
||||||
self.local = threading.local()
|
|
||||||
self.db_file = db_file
|
self.db_file = db_file
|
||||||
self._get_conn()
|
self._get_conn()
|
||||||
|
|
||||||
def _get_conn(self):
|
def _get_conn(self):
|
||||||
""" get db connection """
|
""" get db connection """
|
||||||
if not hasattr(self.local, 'conn'):
|
local = threading.local()
|
||||||
self.local.conn = self._create_connection(self.db_file)
|
if not hasattr(local, 'conn'):
|
||||||
return self.local.conn
|
local.conn = self._create_connection(self.db_file)
|
||||||
|
return local.conn
|
||||||
|
|
||||||
def _create_connection(self, db_file):
|
def _create_connection(self, db_file):
|
||||||
""" create a database connection to a SQLite database """
|
""" create a database connection to a SQLite database """
|
||||||
@@ -37,9 +38,10 @@ class ChatSessionDB:
|
|||||||
return conn
|
return conn
|
||||||
|
|
||||||
def close(self):
|
def close(self):
|
||||||
if not hasattr(self.local, 'conn'):
|
local = threading.local()
|
||||||
|
if not hasattr(local, 'conn'):
|
||||||
return
|
return
|
||||||
self.local.conn.close()
|
local.conn.close()
|
||||||
|
|
||||||
def _create_table(self, conn):
|
def _create_table(self, conn):
|
||||||
""" create table """
|
""" create table """
|
||||||
@@ -52,7 +54,8 @@ class ChatSessionDB:
|
|||||||
SessionTopic TEXT,
|
SessionTopic TEXT,
|
||||||
StartTime TEXT,
|
StartTime TEXT,
|
||||||
SummarizePos INTEGER,
|
SummarizePos INTEGER,
|
||||||
Summary TEXT
|
Summary TEXT,
|
||||||
|
ThreadID TEXT
|
||||||
);
|
);
|
||||||
""")
|
""")
|
||||||
|
|
||||||
@@ -82,28 +85,29 @@ class ChatSessionDB:
|
|||||||
ActionResult TEXT,
|
ActionResult TEXT,
|
||||||
DoneTime TEXT,
|
DoneTime TEXT,
|
||||||
|
|
||||||
Status INTEGER
|
Status INTEGER,
|
||||||
|
Tags TEXT
|
||||||
);
|
);
|
||||||
""")
|
""")
|
||||||
conn.commit()
|
conn.commit()
|
||||||
except Error as e:
|
except Error as e:
|
||||||
logging.error("Error occurred while creating tables: %s", e)
|
logging.error("Error occurred while creating tables: %s", e)
|
||||||
|
|
||||||
def insert_chatsession(self, session_id, session_owner,session_topic, start_time):
|
def insert_chatsession(self, session_id, session_owner,session_topic, start_time,thread_id = ""):
|
||||||
""" insert a new session into the ChatSessions table """
|
""" insert a new session into the ChatSessions table """
|
||||||
try:
|
try:
|
||||||
conn = self._get_conn()
|
conn = self._get_conn()
|
||||||
conn.execute("""
|
conn.execute("""
|
||||||
INSERT INTO ChatSessions (SessionID, SessionOwner,SessionTopic, StartTime,SummarizePos,Summary)
|
INSERT INTO ChatSessions (SessionID, SessionOwner,SessionTopic, StartTime,SummarizePos,Summary,ThreadID)
|
||||||
VALUES (?,?, ?, ?,0,"")
|
VALUES (?,?, ?, ?,0,"",?)
|
||||||
""", (session_id, session_owner,session_topic, start_time))
|
""", (session_id, session_owner,session_topic, start_time,thread_id))
|
||||||
conn.commit()
|
conn.commit()
|
||||||
return 0 # return 0 if successful
|
return 0 # return 0 if successful
|
||||||
except Error as e:
|
except Error as e:
|
||||||
logging.error("Error occurred while inserting session: %s", e)
|
logging.error("Error occurred while inserting session: %s", e)
|
||||||
return -1 # return -1 if an error occurs
|
return -1 # return -1 if an error occurs
|
||||||
|
|
||||||
def insert_message(self, msg:AgentMsg):
|
def insert_message(self, msg:AgentMsg,tags:List[str] = None):
|
||||||
""" insert a new message into the Messages table """
|
""" insert a new message into the Messages table """
|
||||||
try:
|
try:
|
||||||
action_name = None
|
action_name = None
|
||||||
@@ -111,29 +115,31 @@ class ChatSessionDB:
|
|||||||
action_result = None
|
action_result = None
|
||||||
mentions = None
|
mentions = None
|
||||||
if msg.mentions:
|
if msg.mentions:
|
||||||
mentions = json.dumps(msg.mentions)
|
mentions = json.dumps(msg.mentions,ensure_ascii=False)
|
||||||
|
|
||||||
match msg.msg_type:
|
match msg.msg_type:
|
||||||
case AgentMsgType.TYPE_MSG:
|
case AgentMsgType.TYPE_MSG:
|
||||||
pass
|
pass
|
||||||
case AgentMsgType.TYPE_ACTION:
|
case AgentMsgType.TYPE_ACTION:# THIS Action is not AIAction
|
||||||
action_name = msg.func_name
|
action_name = msg.func_name
|
||||||
action_params = json.dumps(msg.args)
|
action_params = json.dumps(msg.args,ensure_ascii=False)
|
||||||
action_result = msg.result_str
|
action_result = msg.result_str
|
||||||
case AgentMsgType.TYPE_INTERNAL_CALL:
|
case AgentMsgType.TYPE_INTERNAL_CALL:
|
||||||
action_name = msg.func_name
|
action_name = msg.func_name
|
||||||
action_params = json.dumps(msg.args)
|
action_params = json.dumps(msg.args,ensure_ascii=False)
|
||||||
action_result = msg.result_str
|
action_result = msg.result_str
|
||||||
case AgentMsgType.TYPE_EVENT:
|
case AgentMsgType.TYPE_EVENT:
|
||||||
action_name = msg.event_name
|
action_name = msg.event_name
|
||||||
action_params = json.dumps(msg.event_args)
|
action_params = json.dumps(msg.event_args,ensure_ascii=False)
|
||||||
|
if tags is None:
|
||||||
|
tags = []
|
||||||
|
|
||||||
|
str_tags = ','.join(tags)
|
||||||
conn = self._get_conn()
|
conn = self._get_conn()
|
||||||
conn.execute("""
|
conn.execute("""
|
||||||
INSERT INTO Messages (MessageID, SessionID, MsgType, PrevMsgID, SenderID, ReceiverID, Timestamp, Topic,Mentions,ContentMIME,Content,ActionName,ActionParams,ActionResult,DoneTime,Status)
|
INSERT INTO Messages (MessageID, SessionID, MsgType, PrevMsgID, SenderID, ReceiverID, Timestamp, Topic,Mentions,ContentMIME,Content,ActionName,ActionParams,ActionResult,DoneTime,Status,Tags)
|
||||||
VALUES (?, ?, ?, ?, ?, ?, ?,?, ?, ?, ?, ?, ?, ?, ?, ?)
|
VALUES (?, ?, ?, ?, ?, ?, ?,?, ?, ?, ?, ?, ?, ?, ?, ?,?)
|
||||||
""", (msg.msg_id, msg.session_id, msg.msg_type.value, msg.prev_msg_id, msg.sender, msg.target, msg.create_time, msg.topic,mentions,msg.body_mime,msg.body,action_name,action_params,action_result,msg.done_time,msg.status.value))
|
""", (msg.msg_id, msg.session_id, msg.msg_type.value, msg.prev_msg_id, msg.sender, msg.target, msg.create_time, msg.topic,mentions,msg.body_mime,msg.body,action_name,action_params,action_result,msg.done_time,msg.status.value,str_tags))
|
||||||
conn.commit()
|
conn.commit()
|
||||||
|
|
||||||
if msg.inner_call_chain:
|
if msg.inner_call_chain:
|
||||||
@@ -192,6 +198,9 @@ class ChatSessionDB:
|
|||||||
try:
|
try:
|
||||||
conn = self._get_conn()
|
conn = self._get_conn()
|
||||||
cursor = conn.cursor()
|
cursor = conn.cursor()
|
||||||
|
if limit == 0:
|
||||||
|
limit = 1024
|
||||||
|
|
||||||
cursor.execute("""
|
cursor.execute("""
|
||||||
SELECT MessageID, SessionID, MsgType, PrevMsgID, SenderID, ReceiverID, Timestamp, Topic,Mentions,ContentMIME,Content,ActionName,ActionParams,ActionResult,DoneTime,Status FROM Messages
|
SELECT MessageID, SessionID, MsgType, PrevMsgID, SenderID, ReceiverID, Timestamp, Topic,Mentions,ContentMIME,Content,ActionName,ActionParams,ActionResult,DoneTime,Status FROM Messages
|
||||||
WHERE SessionID = ?
|
WHERE SessionID = ?
|
||||||
@@ -205,12 +214,31 @@ class ChatSessionDB:
|
|||||||
logging.error("Error occurred while getting messages: %s", e)
|
logging.error("Error occurred while getting messages: %s", e)
|
||||||
return -1, None # return -1 and None if an error occurs
|
return -1, None # return -1 and None if an error occurs
|
||||||
|
|
||||||
|
def load_message_by_agentid(self,agent_id,limit,start_time="1970-01-01 00:00:00"):
|
||||||
|
try:
|
||||||
|
conn = self._get_conn()
|
||||||
|
cursor = conn.cursor()
|
||||||
|
cursor.execute("""
|
||||||
|
SELECT MessageID, SessionID, MsgType, PrevMsgID, SenderID, ReceiverID, Timestamp, Topic,Mentions,ContentMIME,Content,ActionName,ActionParams,ActionResult,DoneTime,Status FROM Messages
|
||||||
|
WHERE SenderID = ? or ReceiverID =? AND Timestamp > ?
|
||||||
|
ORDER BY Timestamp
|
||||||
|
LIMIT ?
|
||||||
|
""", (agent_id, agent_id, start_time,limit))
|
||||||
|
results = cursor.fetchall()
|
||||||
|
#self.close()
|
||||||
|
return results # return 0 and the result if successful
|
||||||
|
except Error as e:
|
||||||
|
logging.error("Error occurred while getting messages: %s", e)
|
||||||
|
return -1, None
|
||||||
|
|
||||||
# read message from now->beign
|
# read message from now->beign
|
||||||
def get_messages(self, session_id, limit, offset):
|
def get_messages(self, session_id, limit, offset):
|
||||||
""" retrieve messages of a session with pagination """
|
""" retrieve messages of a session with pagination """
|
||||||
try:
|
try:
|
||||||
conn = self._get_conn()
|
conn = self._get_conn()
|
||||||
cursor = conn.cursor()
|
cursor = conn.cursor()
|
||||||
|
if limit == 0:
|
||||||
|
limit = 1024
|
||||||
cursor.execute("""
|
cursor.execute("""
|
||||||
SELECT MessageID, SessionID, MsgType, PrevMsgID, SenderID, ReceiverID, Timestamp, Topic,Mentions,ContentMIME,Content,ActionName,ActionParams,ActionResult,DoneTime,Status FROM Messages
|
SELECT MessageID, SessionID, MsgType, PrevMsgID, SenderID, ReceiverID, Timestamp, Topic,Mentions,ContentMIME,Content,ActionName,ActionParams,ActionResult,DoneTime,Status FROM Messages
|
||||||
WHERE SessionID = ?
|
WHERE SessionID = ?
|
||||||
@@ -254,10 +282,26 @@ class ChatSessionDB:
|
|||||||
logging.error("Error occurred while updating session summary: %s", e)
|
logging.error("Error occurred while updating session summary: %s", e)
|
||||||
return -1
|
return -1
|
||||||
|
|
||||||
|
def update_session_thread_id(self, session_id, thread_id):
|
||||||
|
""" update the threadid of a session """
|
||||||
|
try:
|
||||||
|
conn = self._get_conn()
|
||||||
|
conn.execute("""
|
||||||
|
UPDATE ChatSessions
|
||||||
|
SET ThreadID = ?
|
||||||
|
WHERE SessionID = ?
|
||||||
|
""", (thread_id, session_id))
|
||||||
|
conn.commit()
|
||||||
|
return 0 # return 0 if successful
|
||||||
|
except Error as e:
|
||||||
|
logging.error("Error occurred while updating session threadid: %s", e)
|
||||||
|
return -1
|
||||||
|
|
||||||
# chat session store the chat history between owner and agent
|
# chat session store the chat history between owner and agent
|
||||||
# chat session might be large, so can read / write at stream mode.
|
# chat session might be large, so can read / write at stream mode.
|
||||||
class AIChatSession:
|
class AIChatSession:
|
||||||
_dbs = {}
|
_dbs = {}
|
||||||
|
_sessions = {}
|
||||||
#@classmethod
|
#@classmethod
|
||||||
#async def get_session_by_id(cls,session_id:str,db_path:str):
|
#async def get_session_by_id(cls,session_id:str,db_path:str):
|
||||||
# db = cls._dbs.get(db_path)
|
# db = cls._dbs.get(db_path)
|
||||||
@@ -266,6 +310,38 @@ class AIChatSession:
|
|||||||
# cls._dbs[db_path] = db
|
# cls._dbs[db_path] = db
|
||||||
# db.get_chatsession_by_id(session_id)
|
# db.get_chatsession_by_id(session_id)
|
||||||
# #result = AIChatSession()
|
# #result = AIChatSession()
|
||||||
|
@classmethod
|
||||||
|
# start_time is a string like "2021-01-01 00:00:00"
|
||||||
|
def load_message_records_by_agentid(cls,agent_id:str,start_time:str,limit:int,db_path:str)->List[AgentMsg]:
|
||||||
|
db = cls._dbs.get(db_path)
|
||||||
|
if db is None:
|
||||||
|
db = ChatSessionDB(db_path)
|
||||||
|
cls._dbs[db_path] = db
|
||||||
|
msgs = db.load_message_by_agentid(agent_id,start_time,limit)
|
||||||
|
result = []
|
||||||
|
for msg in msgs:
|
||||||
|
agent_msg = AgentMsg()
|
||||||
|
agent_msg.msg_id = msg[0]
|
||||||
|
agent_msg.session_id = msg[1]
|
||||||
|
agent_msg.msg_type = AgentMsgType(msg[2])
|
||||||
|
agent_msg.prev_msg_id = msg[3]
|
||||||
|
agent_msg.sender = msg[4]
|
||||||
|
agent_msg.target = msg[5]
|
||||||
|
agent_msg.create_time = msg[6]
|
||||||
|
agent_msg.topic = msg[7]
|
||||||
|
if msg[8] is not None:
|
||||||
|
agent_msg.mentions = json.loads(msg[8])
|
||||||
|
agent_msg.body_mime = msg[9]
|
||||||
|
agent_msg.body = msg[10]
|
||||||
|
agent_msg.func_name = msg[11]
|
||||||
|
if msg[12] is not None:
|
||||||
|
agent_msg.args = json.loads(msg[12])
|
||||||
|
agent_msg.result_str = msg[13]
|
||||||
|
agent_msg.done_time = msg[14]
|
||||||
|
agent_msg.status = AgentMsgStatus(msg[15])
|
||||||
|
|
||||||
|
result.append(agent_msg)
|
||||||
|
return result
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def get_session(cls,owner_id:str,session_topic:str,db_path:str,auto_create = True) -> 'AIChatSession':
|
def get_session(cls,owner_id:str,session_topic:str,db_path:str,auto_create = True) -> 'AIChatSession':
|
||||||
@@ -275,17 +351,24 @@ class AIChatSession:
|
|||||||
cls._dbs[db_path] = db
|
cls._dbs[db_path] = db
|
||||||
|
|
||||||
result = None
|
result = None
|
||||||
|
for session in cls._sessions.values():
|
||||||
|
if session.owner_id == owner_id and session.topic == session_topic:
|
||||||
|
return session
|
||||||
|
|
||||||
session = db.get_chatsession_by_owner_topic(owner_id,session_topic)
|
session = db.get_chatsession_by_owner_topic(owner_id,session_topic)
|
||||||
if session is None:
|
if session is None:
|
||||||
if auto_create:
|
if auto_create:
|
||||||
session_id = "CS#" + uuid.uuid4().hex
|
session_id = "CS#" + uuid.uuid4().hex
|
||||||
db.insert_chatsession(session_id,owner_id,session_topic,datetime.datetime.now())
|
db.insert_chatsession(session_id,owner_id,session_topic,datetime.datetime.now())
|
||||||
result = AIChatSession(owner_id,session_id,db)
|
result = AIChatSession(owner_id,session_id,db)
|
||||||
|
cls._sessions[session_id] = result
|
||||||
else:
|
else:
|
||||||
result = AIChatSession(owner_id,session[0],db)
|
result = AIChatSession(owner_id,session[0],db)
|
||||||
result.topic = session_topic
|
result.topic = session_topic
|
||||||
result.summarize_pos = session[4]
|
result.summarize_pos = session[4]
|
||||||
result.summary = session[5]
|
result.summary = session[5]
|
||||||
|
result.openai_thread_id = session[6]
|
||||||
|
cls._sessions[result.session_id] = result
|
||||||
|
|
||||||
return result
|
return result
|
||||||
|
|
||||||
@@ -297,6 +380,10 @@ class AIChatSession:
|
|||||||
cls._dbs[db_path] = db
|
cls._dbs[db_path] = db
|
||||||
|
|
||||||
result = None
|
result = None
|
||||||
|
session = cls._sessions.get(session_id)
|
||||||
|
if session:
|
||||||
|
return session
|
||||||
|
|
||||||
session = db.get_chatsession_by_id(session_id)
|
session = db.get_chatsession_by_id(session_id)
|
||||||
if session is None:
|
if session is None:
|
||||||
return None
|
return None
|
||||||
@@ -305,6 +392,8 @@ class AIChatSession:
|
|||||||
result.topic = session[2]
|
result.topic = session[2]
|
||||||
result.summarize_pos = session[4]
|
result.summarize_pos = session[4]
|
||||||
result.summary = session[5]
|
result.summary = session[5]
|
||||||
|
result.openai_thread_id = session[6]
|
||||||
|
cls._sessions[session_id] = result
|
||||||
|
|
||||||
return result
|
return result
|
||||||
|
|
||||||
@@ -330,12 +419,13 @@ class AIChatSession:
|
|||||||
self.topic : str = None
|
self.topic : str = None
|
||||||
self.start_time : str = None
|
self.start_time : str = None
|
||||||
self.summarize_pos : int = 0
|
self.summarize_pos : int = 0
|
||||||
self.summary = None
|
self.summary : str = None
|
||||||
|
self.openai_thread_id = None
|
||||||
|
|
||||||
def get_owner_id(self) -> str:
|
def get_owner_id(self) -> str:
|
||||||
return self.owner_id
|
return self.owner_id
|
||||||
|
|
||||||
def read_history(self, number:int=10,offset=0,order="revers") -> [AgentMsg]:
|
def read_history(self, number:int=0,offset=0,order="revers") -> [AgentMsg]:
|
||||||
if order == "revers":
|
if order == "revers":
|
||||||
msgs = self.db.get_messages(self.session_id, number, offset)
|
msgs = self.db.get_messages(self.session_id, number, offset)
|
||||||
else:
|
else:
|
||||||
@@ -366,16 +456,19 @@ class AIChatSession:
|
|||||||
result.append(agent_msg)
|
result.append(agent_msg)
|
||||||
return result
|
return result
|
||||||
|
|
||||||
def append(self,msg:AgentMsg) -> None:
|
def append(self,msg:AgentMsg,tags:List[str] = None) -> None:
|
||||||
msg.session_id = self.session_id
|
msg.session_id = self.session_id
|
||||||
self.db.insert_message(msg)
|
self.db.insert_message(msg,tags)
|
||||||
|
|
||||||
|
|
||||||
def update_think_progress(self,progress:int,new_summary:str) -> None:
|
def update_summary(self,new_summary:str) -> None:
|
||||||
self.db.update_session_summary(self.session_id,progress,new_summary)
|
self.db.update_session_summary(self.session_id,self.summarize_pos,new_summary)
|
||||||
self.summarize_pos = progress
|
|
||||||
self.summary = new_summary
|
self.summary = new_summary
|
||||||
|
|
||||||
|
def update_openai_thread_id(self,thread_id:str) -> None:
|
||||||
|
self.db.update_session_thread_id(self.session_id,thread_id)
|
||||||
|
self.openai_thread_id = thread_id
|
||||||
|
|
||||||
#def attach_event_handler(self,handler) -> None:
|
#def attach_event_handler(self,handler) -> None:
|
||||||
# """chat session changed event handler"""
|
# """chat session changed event handler"""
|
||||||
# pass
|
# pass
|
||||||
@@ -0,0 +1,316 @@
|
|||||||
|
# pylint:disable=E0402
|
||||||
|
from abc import ABC, abstractmethod
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
from typing import Optional,Set,List,Dict,Callable
|
||||||
|
|
||||||
|
from ..proto.ai_function import AIFunction,AIAction, AIFunction2Action,SimpleAIAction
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
class LLMProcessContext:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def function2action(ai_func:AIFunction) -> AIAction:
|
||||||
|
return AIFunction2Action(ai_func)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def aifunctions_to_inner_functions(all_inner_function:List[AIFunction]) -> List[Dict]:
|
||||||
|
if all_inner_function is None:
|
||||||
|
return []
|
||||||
|
|
||||||
|
result_func = []
|
||||||
|
result_len = 0
|
||||||
|
for inner_func in all_inner_function:
|
||||||
|
func_name = inner_func.get_name()
|
||||||
|
this_func = {}
|
||||||
|
this_func["name"] = func_name
|
||||||
|
this_func["description"] = inner_func.get_description()
|
||||||
|
this_func["parameters"] = inner_func.get_openai_parameters()
|
||||||
|
result_len += len(json.dumps(this_func,ensure_ascii=False)) / 4
|
||||||
|
result_func.append(this_func)
|
||||||
|
return result_func
|
||||||
|
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def get_ai_function(self,func_name:str) -> AIFunction:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def get_all_ai_functions(self) -> List[AIFunction]:
|
||||||
|
return self.get_function_set(None)
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def get_function_set(self,set_name:str = None) -> List[AIFunction]:
|
||||||
|
pass
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def get_ai_action(self,op_name:str) -> AIAction:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def get_all_ai_action(self) -> List[AIAction]:
|
||||||
|
return self.get_action_set(None)
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def get_action_set(self,set_name:str = None) -> List[AIFunction]:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def __getitem__(self, key):
|
||||||
|
return self.get_value(key)
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def get_value(self,key:str) -> Optional[str]:
|
||||||
|
pass
|
||||||
|
|
||||||
|
#def list_actions(self,path:str) -> List[AIAction]:
|
||||||
|
# return "No more actions!"
|
||||||
|
|
||||||
|
#def list_functions(self,path:str) -> List[AIFunction]:
|
||||||
|
# return "No more tool functions!"
|
||||||
|
|
||||||
|
class GlobaToolsLibrary:
|
||||||
|
_instance = None
|
||||||
|
@classmethod
|
||||||
|
def get_instance(cls) -> 'GlobaToolsLibrary':
|
||||||
|
if cls._instance is None:
|
||||||
|
cls._instance = GlobaToolsLibrary()
|
||||||
|
return cls._instance
|
||||||
|
|
||||||
|
def __init__(self,global_env_name:str = None) -> None:
|
||||||
|
self.all_preset_context = {}
|
||||||
|
self.all_tool_functions : Dict[str,AIFunction] = {}
|
||||||
|
self.all_action_sets : Dict[str,Set[str]] = {}
|
||||||
|
self.all_function_sets : Dict[str,Set[str]] = {}
|
||||||
|
|
||||||
|
def register_prset_context(self,preset_id:str,context) -> None:
|
||||||
|
self.all_preset_context[preset_id] = context
|
||||||
|
|
||||||
|
def get_preset_context(self,preset_id:str):
|
||||||
|
return self.all_preset_context.get(preset_id)
|
||||||
|
|
||||||
|
def register_tool_function(self,function:AIFunction) -> None:
|
||||||
|
if self.all_tool_functions.get(function.get_id()):
|
||||||
|
logger.warning(f"Tool function {function.get_id()} already exists! will be replaced!")
|
||||||
|
|
||||||
|
self.all_tool_functions[function.get_id()] = function
|
||||||
|
|
||||||
|
def get_tool_function(self,function_name:str) -> AIFunction:
|
||||||
|
return self.all_tool_functions.get(function_name)
|
||||||
|
|
||||||
|
def register_function_set(self,set_name:str,function_set:Set[str]) -> None:
|
||||||
|
self.all_function_sets[set_name] = function_set
|
||||||
|
|
||||||
|
def get_function_set(self,set_name:str) -> Set[str]:
|
||||||
|
return self.all_function_sets.get(set_name)
|
||||||
|
|
||||||
|
class SimpleLLMContext(LLMProcessContext):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
super().__init__()
|
||||||
|
self.parent = None
|
||||||
|
self.values : Dict[str,str] = {}
|
||||||
|
self.values_callback = {}
|
||||||
|
|
||||||
|
self.functions: Dict[str,AIFunction] = {}
|
||||||
|
self.func_sets : Dict[str,Dict[str,AIFunction]] = {}
|
||||||
|
self.actions: Dict[str,AIAction] = {}
|
||||||
|
self.action_sets : Dict[str,Dict[str,AIAction]] = {}
|
||||||
|
|
||||||
|
def load_action_set_from_config(self,preset,config:Dict[str,str]) -> Dict:
|
||||||
|
if preset is None:
|
||||||
|
result = {}
|
||||||
|
else:
|
||||||
|
result = preset
|
||||||
|
|
||||||
|
enable_actions = config.get("enable")
|
||||||
|
if enable_actions:
|
||||||
|
for action_id in enable_actions:
|
||||||
|
ai_func = GlobaToolsLibrary.get_instance().get_tool_function(action_id)
|
||||||
|
if ai_func:
|
||||||
|
result[action_id] = LLMProcessContext.function2action(ai_func)
|
||||||
|
else:
|
||||||
|
func_set = GlobaToolsLibrary.get_instance().get_function_set(action_id)
|
||||||
|
if func_set:
|
||||||
|
for _func_id in func_set:
|
||||||
|
ai_func = GlobaToolsLibrary.get_instance().get_tool_function(_func_id)
|
||||||
|
if ai_func:
|
||||||
|
result[_func_id] = LLMProcessContext.function2action(ai_func)
|
||||||
|
else:
|
||||||
|
logger.error(f"load_action_set_from_config failed! enable action id {action_id} not found!")
|
||||||
|
return None
|
||||||
|
|
||||||
|
disable_actions = config.get("disable")
|
||||||
|
if disable_actions:
|
||||||
|
for disable_action in disable_actions:
|
||||||
|
if result.get(disable_action):
|
||||||
|
result.pop(disable_action)
|
||||||
|
else:
|
||||||
|
func_set = GlobaToolsLibrary.get_instance().get_function_set(action_id)
|
||||||
|
if func_set:
|
||||||
|
for _func_id in func_set:
|
||||||
|
if result.get(_func_id):
|
||||||
|
result.pop(_func_id)
|
||||||
|
else:
|
||||||
|
logger.error(f"load_action_set_from_config failed! disable action id {action_id} not found!")
|
||||||
|
return None
|
||||||
|
|
||||||
|
return result
|
||||||
|
|
||||||
|
def load_function_set_from_config(self,preset,config:Dict) -> Dict[str,AIFunction]:
|
||||||
|
if preset is None:
|
||||||
|
result = {}
|
||||||
|
else:
|
||||||
|
result = preset
|
||||||
|
|
||||||
|
enable_functions = config.get("enable")
|
||||||
|
if enable_functions:
|
||||||
|
for func_id in enable_functions:
|
||||||
|
ai_func = GlobaToolsLibrary.get_instance().get_tool_function(func_id)
|
||||||
|
if ai_func:
|
||||||
|
result[func_id] = ai_func
|
||||||
|
else:
|
||||||
|
func_set = GlobaToolsLibrary.get_instance().get_function_set(func_id)
|
||||||
|
if func_set:
|
||||||
|
for func_id in func_set:
|
||||||
|
ai_func = GlobaToolsLibrary.get_instance().get_tool_function(func_id)
|
||||||
|
if ai_func:
|
||||||
|
result[func_id] = ai_func
|
||||||
|
else:
|
||||||
|
logger.error(f"load_function_set_from_config failed! enable function id {func_id} not found!")
|
||||||
|
return None
|
||||||
|
else:
|
||||||
|
logger.error(f"load_function_set_from_config failed! enable function id {func_id} not found!")
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
disable_functions = config.get("disable")
|
||||||
|
if disable_functions:
|
||||||
|
for disable_function in disable_functions:
|
||||||
|
if result.get(disable_function):
|
||||||
|
result.pop(disable_function)
|
||||||
|
else:
|
||||||
|
func_set = GlobaToolsLibrary.get_instance().get_function_set(func_id)
|
||||||
|
if func_set:
|
||||||
|
for func_id in func_set:
|
||||||
|
if result.get(func_id):
|
||||||
|
result.pop(func_id)
|
||||||
|
else:
|
||||||
|
logger.error(f"load_function_set_from_config failed! disable function id {disable_function} not found!")
|
||||||
|
return None
|
||||||
|
|
||||||
|
return result
|
||||||
|
|
||||||
|
def load_from_config(self,config:Dict[str,str]) -> bool:
|
||||||
|
preset = config.get("preset")
|
||||||
|
if preset:
|
||||||
|
self.parent:SimpleLLMContext = GlobaToolsLibrary.get_instance().get_preset_context(preset)
|
||||||
|
if self.parent is None:
|
||||||
|
logger.error(f"preset context {preset} not found!")
|
||||||
|
return False
|
||||||
|
|
||||||
|
self.values = self.parent.values
|
||||||
|
self.values_callback = self.parent.values_callback
|
||||||
|
self.actions = self.parent.actions
|
||||||
|
self.functions = self.parent.functions
|
||||||
|
self.action_sets = self.parent.action_sets
|
||||||
|
self.func_sets = self.parent.func_sets
|
||||||
|
|
||||||
|
action_def:Dict= config.get("actions")
|
||||||
|
if action_def:
|
||||||
|
self.actions = self.load_action_set_from_config(self.actions,action_def)
|
||||||
|
if self.actions is None:
|
||||||
|
logger.error(f"load_from_config failed! load_action_set_from_config failed!")
|
||||||
|
return False
|
||||||
|
|
||||||
|
for set_name in action_def.keys():
|
||||||
|
if set_name == "enable":
|
||||||
|
continue
|
||||||
|
if set_name == "disable":
|
||||||
|
continue
|
||||||
|
|
||||||
|
sub_set = config.get(set_name)
|
||||||
|
self.action_sets[set_name] = self.load_action_set_from_config(None,sub_set)
|
||||||
|
if self.action_sets[set_name] is None:
|
||||||
|
logger.error(f"load_from_config failed! load_action_set_from_config failed!")
|
||||||
|
return False
|
||||||
|
|
||||||
|
function_def:Dict = config.get("functions")
|
||||||
|
if function_def:
|
||||||
|
self.functions = self.load_function_set_from_config(self.functions,function_def)
|
||||||
|
if self.functions is None:
|
||||||
|
logger.error(f"load_from_config failed! load_function_set_from_config failed!")
|
||||||
|
return False
|
||||||
|
|
||||||
|
for set_name in function_def.keys():
|
||||||
|
if set_name == "enable":
|
||||||
|
continue
|
||||||
|
if set_name == "disable":
|
||||||
|
continue
|
||||||
|
|
||||||
|
sub_set = config.get(set_name)
|
||||||
|
self.func_sets[set_name] = self.load_function_set_from_config(None,sub_set)
|
||||||
|
if self.func_sets[set_name] is None:
|
||||||
|
logger.error(f"load_from_config failed! load_function_set_from_config failed!")
|
||||||
|
return False
|
||||||
|
|
||||||
|
#values_def = config.get("values")
|
||||||
|
#if values_def:
|
||||||
|
# for key,value in values_def.items():
|
||||||
|
# self.values[key] = value
|
||||||
|
|
||||||
|
def get_value(self,key:str) -> Optional[str]:
|
||||||
|
callback = self.values_callback.get(key)
|
||||||
|
if callback:
|
||||||
|
return callback()
|
||||||
|
return self.values.get(key)
|
||||||
|
|
||||||
|
def set_value_callback(self,key:str,callback:Callable[[],str]) -> None:
|
||||||
|
self.values_callback[key] = callback
|
||||||
|
|
||||||
|
def set_value(self,key:str,value:str):
|
||||||
|
self.values[key] = value
|
||||||
|
|
||||||
|
def get_ai_function(self,func_name:str) -> AIFunction:
|
||||||
|
for func in self.functions.values():
|
||||||
|
if func.get_name() == func_name:
|
||||||
|
return func
|
||||||
|
#for set_name in self.func_sets.keys():
|
||||||
|
# func = self.func_sets[set_name].get(func_name)
|
||||||
|
# if func is not None:
|
||||||
|
# return func
|
||||||
|
return None
|
||||||
|
|
||||||
|
def get_function_set(self,set_name:str = None) -> List[AIFunction]:
|
||||||
|
if self.functions is None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
if set_name is None:
|
||||||
|
return self.functions.values()
|
||||||
|
else:
|
||||||
|
func_set = self.func_sets.get(set_name)
|
||||||
|
if func_set:
|
||||||
|
return func_set.values()
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def get_ai_action(self,op_name:str) -> AIAction:
|
||||||
|
for action in self.actions.values():
|
||||||
|
if action.get_name() == op_name:
|
||||||
|
return action
|
||||||
|
|
||||||
|
return None
|
||||||
|
|
||||||
|
def get_action_set(self,set_name:str = None) -> List[AIFunction]:
|
||||||
|
if self.actions is None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
if set_name is None:
|
||||||
|
return self.actions.values()
|
||||||
|
else:
|
||||||
|
action_set = self.action_sets.get(set_name)
|
||||||
|
if action_set:
|
||||||
|
return action_set.values()
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,373 @@
|
|||||||
|
from ..proto.compute_task import LLMPrompt,LLMResult,ComputeTaskResult,ComputeTaskResultCode
|
||||||
|
from ..proto.ai_function import AIFunction,AIAction,ActionNode
|
||||||
|
from ..proto.agent_msg import AgentMsg,AgentMsgType
|
||||||
|
from ..proto.agent_task import AgentTask, AgentTodo, AgentWorkLog
|
||||||
|
from ..frame.compute_kernel import ComputeKernel
|
||||||
|
|
||||||
|
from .agent_memory import AgentMemory
|
||||||
|
from .workspace import AgentWorkspace
|
||||||
|
from .llm_context import LLMProcessContext,GlobaToolsLibrary, SimpleLLMContext
|
||||||
|
from .llm_process import BaseLLMProcess,LLMAgentBaseProcess
|
||||||
|
|
||||||
|
from abc import ABC,abstractmethod
|
||||||
|
import copy
|
||||||
|
import json
|
||||||
|
import datetime
|
||||||
|
from datetime import datetime
|
||||||
|
from typing import Any, Callable, Coroutine, Optional,Dict,Awaitable,List
|
||||||
|
from enum import Enum
|
||||||
|
import logging
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
#LLM Process All the unfinished tasks,will sort the priority of the task after LLM, determine the next execution time, and complete the simple task
|
||||||
|
class AgentTriageTaskList(LLMAgentBaseProcess):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
super().__init__()
|
||||||
|
|
||||||
|
|
||||||
|
async def load_from_config(self,config:dict) -> bool:
|
||||||
|
if await super().load_from_config(config) is False:
|
||||||
|
return False
|
||||||
|
|
||||||
|
async def prepare_prompt(self,input:Dict) -> LLMPrompt:
|
||||||
|
prompt = LLMPrompt()
|
||||||
|
|
||||||
|
task_list:List[AgentTask] = input.get("tasklist")
|
||||||
|
context_info = input.get("context_info")
|
||||||
|
if task_list is None:
|
||||||
|
logger.error(f"tasklist not found in input")
|
||||||
|
return None
|
||||||
|
|
||||||
|
prompt.append_user_message(json.dumps(task_list,ensure_ascii=False))
|
||||||
|
|
||||||
|
system_prompt_dict = self.prepare_role_system_prompt(context_info)
|
||||||
|
# May all logs is good for Agent Triage Task List?
|
||||||
|
have_known_info = False
|
||||||
|
known_info = {}
|
||||||
|
working_logs = await self.memory.load_worklogs(self.memory.agent_id)
|
||||||
|
if len(working_logs) > 0:
|
||||||
|
have_known_info = True
|
||||||
|
all_worklog_node = []
|
||||||
|
for worklog in working_logs:
|
||||||
|
workNode = {}
|
||||||
|
dt = datetime.fromtimestamp(float(worklog.timestamp))
|
||||||
|
workNode["timestamp"] = dt.strftime("%Y-%m-%d %H:%M:%S")
|
||||||
|
workNode["type"] = worklog.work_type
|
||||||
|
workNode["content"] = worklog.content
|
||||||
|
workNode["result"] = worklog.result
|
||||||
|
all_worklog_node.append(workNode)
|
||||||
|
|
||||||
|
known_info["worklogs"] = all_worklog_node
|
||||||
|
|
||||||
|
if have_known_info:
|
||||||
|
system_prompt_dict["known_info"] = known_info
|
||||||
|
|
||||||
|
prompt.inner_functions =LLMProcessContext.aifunctions_to_inner_functions(self.llm_context.get_all_ai_functions())
|
||||||
|
prompt.append_system_message(json.dumps(system_prompt_dict,ensure_ascii=False))
|
||||||
|
return prompt
|
||||||
|
|
||||||
|
|
||||||
|
async def post_llm_process(self,actions:List[ActionNode],input:Dict,llm_result:LLMResult) -> bool:
|
||||||
|
action_params = {}
|
||||||
|
action_params["_input"] = input
|
||||||
|
action_params["_memory"] = self.memory
|
||||||
|
action_params["_workspace"] = self.workspace
|
||||||
|
action_params["_llm_result"] = llm_result
|
||||||
|
action_params["_agentid"] = self.memory.agent_id
|
||||||
|
action_params["_start_at"] = datetime.now()
|
||||||
|
|
||||||
|
result_str = "OK"
|
||||||
|
try:
|
||||||
|
if await self._execute_actions(actions,action_params) is False:
|
||||||
|
result_str = "execute action failed!"
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"execute action failed! {e}")
|
||||||
|
result_str = "execute action failed!,error:" + str(e)
|
||||||
|
|
||||||
|
worklog = AgentWorkLog.create_by_content("","triage",llm_result.resp,self.memory.agent_id)
|
||||||
|
worklog.result = result_str
|
||||||
|
await self.memory.append_worklog(worklog)
|
||||||
|
|
||||||
|
# LLM a Task that never been LLMed, the result of LLM Process may be adjusted, splitting subtask or do simple task as a todo directly.
|
||||||
|
class AgentPlanTask(LLMAgentBaseProcess):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
super().__init__()
|
||||||
|
|
||||||
|
async def load_from_config(self, config: dict,is_load_default=True) -> bool:
|
||||||
|
if await super().load_from_config(config) is False:
|
||||||
|
return False
|
||||||
|
|
||||||
|
async def prepare_prompt(self,input:Dict) -> LLMPrompt:
|
||||||
|
prompt = LLMPrompt()
|
||||||
|
|
||||||
|
agent_task : AgentTask= input.get("task")
|
||||||
|
context_info = input.get("context_info")
|
||||||
|
if agent_task is None:
|
||||||
|
logger.error(f"task not found in input")
|
||||||
|
return None
|
||||||
|
|
||||||
|
prompt.append_user_message(json.dumps(agent_task.to_dict(),ensure_ascii=False))
|
||||||
|
|
||||||
|
system_prompt_dict = self.prepare_role_system_prompt(context_info)
|
||||||
|
|
||||||
|
have_known_info = False
|
||||||
|
known_info = {}
|
||||||
|
working_logs = await self.memory.load_worklogs(None,agent_task.task_id)
|
||||||
|
if len(working_logs) > 0:
|
||||||
|
have_known_info = True
|
||||||
|
all_worklog_node = []
|
||||||
|
for worklog in working_logs:
|
||||||
|
workNode = {}
|
||||||
|
dt = datetime.fromtimestamp(float(worklog.timestamp))
|
||||||
|
workNode["timestamp"] = dt.strftime("%Y-%m-%d %H:%M:%S")
|
||||||
|
workNode["type"] = worklog.work_type
|
||||||
|
workNode["operator"] = worklog.operator
|
||||||
|
workNode["content"] = worklog.content
|
||||||
|
workNode["result"] = worklog.result
|
||||||
|
all_worklog_node.append(workNode)
|
||||||
|
|
||||||
|
known_info["worklogs"] = all_worklog_node
|
||||||
|
|
||||||
|
if have_known_info:
|
||||||
|
system_prompt_dict["known_info"] = known_info
|
||||||
|
|
||||||
|
prompt.inner_functions =LLMProcessContext.aifunctions_to_inner_functions(self.llm_context.get_all_ai_functions())
|
||||||
|
prompt.append_system_message(json.dumps(system_prompt_dict,ensure_ascii=False))
|
||||||
|
return prompt
|
||||||
|
|
||||||
|
|
||||||
|
async def post_llm_process(self,actions:List[ActionNode],input:Dict,llm_result:LLMResult) -> bool:
|
||||||
|
action_params = {}
|
||||||
|
action_params["_input"] = input
|
||||||
|
agent_task : AgentTask= input.get("task")
|
||||||
|
action_params["_memory"] = self.memory
|
||||||
|
action_params["_workspace"] = self.workspace
|
||||||
|
action_params["_llm_result"] = llm_result
|
||||||
|
action_params["_agentid"] = self.memory.agent_id
|
||||||
|
action_params["_start_at"] = datetime.now()
|
||||||
|
|
||||||
|
result_str = "OK"
|
||||||
|
try:
|
||||||
|
if await self._execute_actions(actions,action_params) is False:
|
||||||
|
result_str = "execute action failed!"
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"execute action failed! {e}")
|
||||||
|
result_str = "execute action failed!,error:" + str(e)
|
||||||
|
|
||||||
|
worklog = AgentWorkLog.create_by_content(agent_task.task_id,"tackling",llm_result.resp,self.memory.agent_id)
|
||||||
|
worklog.result = result_str
|
||||||
|
await self.memory.append_worklog(worklog)
|
||||||
|
|
||||||
|
|
||||||
|
# Agent DO Todo
|
||||||
|
# The purpose is to complete Todo.It is the core LLM process. Can use sufficient external tools to do your best according to the identity and ability of AGENT.It is also the LLM Process of the main extension of Agent extension
|
||||||
|
class AgentDo(LLMAgentBaseProcess):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
super().__init__()
|
||||||
|
|
||||||
|
async def load_from_config(self, config: dict,is_load_default=True) -> Coroutine[Any, Any, bool]:
|
||||||
|
if await super().load_from_config(config) is False:
|
||||||
|
return False
|
||||||
|
|
||||||
|
async def prepare_prompt(self,input:Dict) -> LLMPrompt:
|
||||||
|
prompt = LLMPrompt()
|
||||||
|
|
||||||
|
agent_todo : AgentTodo= input.get("todo")
|
||||||
|
context_info = input.get("context_info")
|
||||||
|
if agent_todo is None:
|
||||||
|
logger.error(f"task not found in input")
|
||||||
|
return None
|
||||||
|
|
||||||
|
prompt.append_user_message(json.dumps(agent_todo.to_dict(),ensure_ascii=False))
|
||||||
|
|
||||||
|
system_prompt_dict = self.prepare_role_system_prompt(context_info)
|
||||||
|
# May all logs is good for Agent Triage Task List?
|
||||||
|
have_known_info = False
|
||||||
|
known_info = {}
|
||||||
|
working_logs = await self.memory.load_worklogs(None,agent_todo.todo_id)
|
||||||
|
if len(working_logs) > 0:
|
||||||
|
have_known_info = True
|
||||||
|
all_worklog_node = []
|
||||||
|
for worklog in working_logs:
|
||||||
|
workNode = {}
|
||||||
|
dt = datetime.fromtimestamp(float(worklog.timestamp))
|
||||||
|
workNode["timestamp"] = dt.strftime("%Y-%m-%d %H:%M:%S")
|
||||||
|
workNode["type"] = worklog.work_type
|
||||||
|
workNode["content"] = worklog.content
|
||||||
|
workNode["result"] = worklog.result
|
||||||
|
all_worklog_node.append(workNode)
|
||||||
|
|
||||||
|
known_info["worklogs"] = all_worklog_node
|
||||||
|
|
||||||
|
if have_known_info:
|
||||||
|
system_prompt_dict["known_info"] = known_info
|
||||||
|
|
||||||
|
prompt.inner_functions =LLMProcessContext.aifunctions_to_inner_functions(self.llm_context.get_all_ai_functions())
|
||||||
|
prompt.append_system_message(json.dumps(system_prompt_dict,ensure_ascii=False))
|
||||||
|
return prompt
|
||||||
|
|
||||||
|
|
||||||
|
async def post_llm_process(self,actions:List[ActionNode],input:Dict,llm_result:LLMResult) -> bool:
|
||||||
|
action_params = {}
|
||||||
|
action_params["_input"] = input
|
||||||
|
agent_todo : AgentTodo= input.get("todo")
|
||||||
|
action_params["_memory"] = self.memory
|
||||||
|
action_params["_workspace"] = self.workspace
|
||||||
|
action_params["_llm_result"] = llm_result
|
||||||
|
action_params["_agentid"] = self.memory.agent_id
|
||||||
|
action_params["_start_at"] = datetime.now()
|
||||||
|
|
||||||
|
result_str = "OK"
|
||||||
|
try:
|
||||||
|
if await self._execute_actions(actions,action_params) is False:
|
||||||
|
result_str = "execute action failed!"
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"execute action failed! {e}")
|
||||||
|
result_str = "execute action failed!,error:" + str(e)
|
||||||
|
|
||||||
|
worklog = AgentWorkLog.create_by_content(agent_todo.todo_id,"do",llm_result.resp,self.memory.agent_id)
|
||||||
|
worklog.result = result_str
|
||||||
|
await self.memory.append_worklog(worklog)
|
||||||
|
|
||||||
|
#Agent check todo
|
||||||
|
# LLM a already-DO TODO, the purpose is to check whether it is completed to face the illusion of LLM.Check can use some tools, which is also the core of the agent extension。
|
||||||
|
class AgentCheck(LLMAgentBaseProcess):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
super().__init__()
|
||||||
|
|
||||||
|
async def load_from_config(self, config: dict,is_load_default=True) -> Coroutine[Any, Any, bool]:
|
||||||
|
if await super().load_from_config(config) is False:
|
||||||
|
return False
|
||||||
|
|
||||||
|
async def prepare_prompt(self,input:Dict) -> LLMPrompt:
|
||||||
|
prompt = LLMPrompt()
|
||||||
|
|
||||||
|
agent_todo : AgentTodo= input.get("todo")
|
||||||
|
context_info = input.get("context_info")
|
||||||
|
if agent_todo is None:
|
||||||
|
logger.error(f"task not found in input")
|
||||||
|
return None
|
||||||
|
|
||||||
|
prompt.append_user_message(json.dumps(agent_todo.to_dict(),ensure_ascii=False))
|
||||||
|
|
||||||
|
system_prompt_dict = self.prepare_role_system_prompt(context_info)
|
||||||
|
# May all logs is good for Agent Triage Task List?
|
||||||
|
have_known_info = False
|
||||||
|
known_info = {}
|
||||||
|
working_logs = await self.memory.load_worklogs(None,agent_todo.todo_id)
|
||||||
|
if len(working_logs) > 0:
|
||||||
|
have_known_info = True
|
||||||
|
all_worklog_node = []
|
||||||
|
for worklog in working_logs:
|
||||||
|
workNode = {}
|
||||||
|
dt = datetime.fromtimestamp(float(worklog.timestamp))
|
||||||
|
workNode["timestamp"] = dt.strftime("%Y-%m-%d %H:%M:%S")
|
||||||
|
workNode["type"] = worklog.work_type
|
||||||
|
workNode["content"] = worklog.content
|
||||||
|
workNode["result"] = worklog.result
|
||||||
|
all_worklog_node.append(workNode)
|
||||||
|
|
||||||
|
known_info["worklogs"] = all_worklog_node
|
||||||
|
|
||||||
|
if have_known_info:
|
||||||
|
system_prompt_dict["known_info"] = known_info
|
||||||
|
|
||||||
|
prompt.inner_functions =LLMProcessContext.aifunctions_to_inner_functions(self.llm_context.get_all_ai_functions())
|
||||||
|
prompt.append_system_message(json.dumps(system_prompt_dict,ensure_ascii=False))
|
||||||
|
return prompt
|
||||||
|
|
||||||
|
|
||||||
|
async def post_llm_process(self,actions:List[ActionNode],input:Dict,llm_result:LLMResult) -> bool:
|
||||||
|
action_params = {}
|
||||||
|
action_params["_input"] = input
|
||||||
|
agent_todo : AgentTodo= input.get("todo")
|
||||||
|
action_params["_memory"] = self.memory
|
||||||
|
action_params["_workspace"] = self.workspace
|
||||||
|
action_params["_llm_result"] = llm_result
|
||||||
|
action_params["_agentid"] = self.memory.agent_id
|
||||||
|
action_params["_start_at"] = datetime.now()
|
||||||
|
|
||||||
|
result_str = "OK"
|
||||||
|
try:
|
||||||
|
if await self._execute_actions(actions,action_params) is False:
|
||||||
|
result_str = "execute action failed!"
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"execute action failed! {e}")
|
||||||
|
result_str = "execute action failed!,error:" + str(e)
|
||||||
|
|
||||||
|
worklog = AgentWorkLog.create_by_content(agent_todo.todo_id,"check",llm_result.resp,self.memory.agent_id)
|
||||||
|
worklog.result = result_str
|
||||||
|
await self.memory.append_worklog(worklog)
|
||||||
|
|
||||||
|
#Agent review task
|
||||||
|
#When Task's Todolist is completed, or Task's subtask is completed, LLM review a TASK to determine that the Task has been completed.This Review also failed to execute.
|
||||||
|
class AgentReviewTask(LLMAgentBaseProcess):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
super().__init__()
|
||||||
|
|
||||||
|
|
||||||
|
async def load_from_config(self, config: dict,is_load_default=True) -> Coroutine[Any, Any, bool]:
|
||||||
|
if await super().load_from_config(config) is False:
|
||||||
|
return False
|
||||||
|
|
||||||
|
async def prepare_prompt(self,input:Dict) -> LLMPrompt:
|
||||||
|
prompt = LLMPrompt()
|
||||||
|
|
||||||
|
agent_task : AgentTask= input.get("task")
|
||||||
|
context_info = input.get("context_info")
|
||||||
|
if agent_task is None:
|
||||||
|
logger.error(f"task not found in input")
|
||||||
|
return None
|
||||||
|
|
||||||
|
prompt.append_user_message(json.dumps(agent_task.to_dict(),ensure_ascii=False))
|
||||||
|
|
||||||
|
system_prompt_dict = self.prepare_role_system_prompt(context_info)
|
||||||
|
# May all logs is good for Agent Triage Task List?
|
||||||
|
have_known_info = False
|
||||||
|
known_info = {}
|
||||||
|
working_logs = await self.memory.load_worklogs(None,agent_task.task_id)
|
||||||
|
if len(working_logs) > 0:
|
||||||
|
have_known_info = True
|
||||||
|
all_worklog_node = []
|
||||||
|
for worklog in working_logs:
|
||||||
|
workNode = {}
|
||||||
|
dt = datetime.fromtimestamp(float(worklog.timestamp))
|
||||||
|
workNode["timestamp"] = dt.strftime("%Y-%m-%d %H:%M:%S")
|
||||||
|
workNode["type"] = worklog.work_type
|
||||||
|
workNode["operator"] = worklog.operator
|
||||||
|
workNode["content"] = worklog.content
|
||||||
|
workNode["result"] = worklog.result
|
||||||
|
all_worklog_node.append(workNode)
|
||||||
|
|
||||||
|
known_info["worklogs"] = all_worklog_node
|
||||||
|
|
||||||
|
if have_known_info:
|
||||||
|
system_prompt_dict["known_info"] = known_info
|
||||||
|
|
||||||
|
prompt.inner_functions =LLMProcessContext.aifunctions_to_inner_functions(self.llm_context.get_all_ai_functions())
|
||||||
|
prompt.append_system_message(json.dumps(system_prompt_dict,ensure_ascii=False))
|
||||||
|
return prompt
|
||||||
|
|
||||||
|
|
||||||
|
async def post_llm_process(self,actions:List[ActionNode],input:Dict,llm_result:LLMResult) -> bool:
|
||||||
|
action_params = {}
|
||||||
|
action_params["_input"] = input
|
||||||
|
agent_task : AgentTask= input.get("task")
|
||||||
|
action_params["_memory"] = self.memory
|
||||||
|
action_params["_workspace"] = self.workspace
|
||||||
|
action_params["_llm_result"] = llm_result
|
||||||
|
action_params["_agentid"] = self.memory.agent_id
|
||||||
|
action_params["_start_at"] = datetime.now()
|
||||||
|
|
||||||
|
result_str = "OK"
|
||||||
|
try:
|
||||||
|
if await self._execute_actions(actions,action_params) is False:
|
||||||
|
result_str = "execute action failed!"
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"execute action failed! {e}")
|
||||||
|
result_str = "execute action failed!,error:" + str(e)
|
||||||
|
|
||||||
|
worklog = AgentWorkLog.create_by_content(agent_task.task_id,"review",llm_result.resp,self.memory.agent_id)
|
||||||
|
worklog.result = result_str
|
||||||
|
await self.memory.append_worklog(worklog)
|
||||||
@@ -0,0 +1,652 @@
|
|||||||
|
# Old name is behavior, I belive new name "llm_process" is better
|
||||||
|
# pylint:disable=E0402
|
||||||
|
import os.path
|
||||||
|
|
||||||
|
from .chatsession import AIChatSession
|
||||||
|
from ..utils import video_utils,image_utils
|
||||||
|
|
||||||
|
from ..proto.compute_task import LLMPrompt,LLMResult,ComputeTaskResult,ComputeTaskResultCode
|
||||||
|
from ..proto.ai_function import AIFunction,AIAction,ActionNode
|
||||||
|
from ..proto.agent_msg import AgentMsg,AgentMsgType
|
||||||
|
|
||||||
|
from .agent_memory import AgentMemory
|
||||||
|
from .workspace import AgentWorkspace
|
||||||
|
from .llm_context import LLMProcessContext,GlobaToolsLibrary, SimpleLLMContext
|
||||||
|
|
||||||
|
from ..frame.compute_kernel import ComputeKernel
|
||||||
|
|
||||||
|
from abc import ABC,abstractmethod
|
||||||
|
import copy
|
||||||
|
import json
|
||||||
|
import datetime
|
||||||
|
from datetime import datetime
|
||||||
|
from typing import Any, Callable, Coroutine, Optional,Dict,Awaitable,List
|
||||||
|
from enum import Enum
|
||||||
|
import logging
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
MIN_PREDICT_TOKEN_LEN = 32
|
||||||
|
|
||||||
|
class BaseLLMProcess(ABC):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.behavior:str = None #行为名字
|
||||||
|
self.goal:str = None #目标
|
||||||
|
self.input_example:str= None #输入样例
|
||||||
|
self.result_example:str = None #llm_result样例
|
||||||
|
|
||||||
|
self.enable_json_resp = False
|
||||||
|
#None means system default,
|
||||||
|
# TODO: support abcstract model name like: local-hight,local-low,local-medium,remote-hight,remote-low,remote-medium
|
||||||
|
self.model_name = None
|
||||||
|
self.max_token = 2000 # result_token
|
||||||
|
self.max_prompt_token = 2000 # not include input prompt
|
||||||
|
self.chat_summary_token_len = 500
|
||||||
|
self.timeout = 1800 # 30 min
|
||||||
|
|
||||||
|
self.llm_context:LLMProcessContext = None
|
||||||
|
|
||||||
|
def get_llm_model_name(self) -> str:
|
||||||
|
return self.model_name
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
async def prepare_prompt(self,input:Dict) -> LLMPrompt:
|
||||||
|
pass
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
async def get_inner_function_for_exec(self,func_name:str) -> AIFunction:
|
||||||
|
pass
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def prepare_inner_function_context_for_exec(self,inner_func_name:str,parameters:Dict):
|
||||||
|
return
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
async def post_llm_process(self,actions:List[ActionNode],input:Dict,llm_result:LLMResult) -> bool:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def get_remain_prompt_length(self,prompt:LLMPrompt,will_append_str:str) -> int:
|
||||||
|
return self.max_prompt_token - ComputeKernel.llm_num_tokens(prompt,self.model_name)
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
async def load_from_config(self,config:dict) -> bool:
|
||||||
|
#self.behavior = config.get("behavior")
|
||||||
|
#self.goal = config.get("goal")
|
||||||
|
self.input_example = config.get("input_example")
|
||||||
|
self.result_example = config.get("result_example")
|
||||||
|
|
||||||
|
if config.get("model_name"):
|
||||||
|
self.model_name = config.get("model_name")
|
||||||
|
if config.get("enable_json_resp"):
|
||||||
|
self.enable_json_resp = config.get("enable_json_resp") == "true"
|
||||||
|
if config.get("max_token"):
|
||||||
|
self.max_token = config.get("max_token")
|
||||||
|
if config.get("timeout"):
|
||||||
|
self.timeout = config.get("timeout")
|
||||||
|
|
||||||
|
|
||||||
|
return True
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
async def initial(self,params:Dict = None) -> bool:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def _format_content_by_env_value(self,content:str,env)->str:
|
||||||
|
return content.format_map(env)
|
||||||
|
|
||||||
|
|
||||||
|
async def _execute_inner_func(self,inner_func_call_node:Dict,prompt: LLMPrompt,stack_limit = 1) -> ComputeTaskResult:
|
||||||
|
arguments = None
|
||||||
|
stack_limit = stack_limit - 1
|
||||||
|
try:
|
||||||
|
func_name = inner_func_call_node.get("name")
|
||||||
|
arguments = json.loads(inner_func_call_node.get("arguments"))
|
||||||
|
logger.info(f"LLMProcess execute inner func:{func_name} :({json.dumps(arguments,ensure_ascii=False)})")
|
||||||
|
|
||||||
|
func_node : AIFunction = await self.get_inner_function_for_exec(func_name)
|
||||||
|
if func_node is None:
|
||||||
|
result_str:str = f"execute {func_name} error,function not found"
|
||||||
|
else:
|
||||||
|
self.prepare_inner_function_context_for_exec(func_name,arguments)
|
||||||
|
result_str:str = await func_node.execute(arguments)
|
||||||
|
except Exception as e:
|
||||||
|
result_str = f"execute {func_name} error:{str(e)}"
|
||||||
|
logger.error(f"LLMProcess execute inner func:{func_name} error:\n\t{e}")
|
||||||
|
|
||||||
|
logger.info("LLMProcess execute inner func result:" + result_str)
|
||||||
|
|
||||||
|
prompt.messages.append({"role":"function","content":result_str,"name":func_name})
|
||||||
|
if self.enable_json_resp:
|
||||||
|
resp_mode = "json"
|
||||||
|
else:
|
||||||
|
resp_mode = "text"
|
||||||
|
|
||||||
|
max_result_token = self.max_token - ComputeKernel.llm_num_tokens(prompt,self.model_name)
|
||||||
|
if max_result_token < MIN_PREDICT_TOKEN_LEN:
|
||||||
|
task_result = ComputeTaskResult()
|
||||||
|
task_result.result_code = ComputeTaskResultCode.ERROR
|
||||||
|
task_result.error_str = f"prompt too long,can not predict"
|
||||||
|
return task_result
|
||||||
|
|
||||||
|
if stack_limit > 0:
|
||||||
|
inner_functions=prompt.inner_functions
|
||||||
|
else:
|
||||||
|
inner_functions = None
|
||||||
|
|
||||||
|
|
||||||
|
task_result: ComputeTaskResult = await (ComputeKernel.get_instance().do_llm_completion(
|
||||||
|
prompt,
|
||||||
|
resp_mode=resp_mode,
|
||||||
|
mode_name=self.get_llm_model_name(),
|
||||||
|
max_token=max_result_token,
|
||||||
|
inner_functions=inner_functions, #NOTICE: inner_function in prompt can be a subset of get_inner_function
|
||||||
|
timeout=self.timeout))
|
||||||
|
|
||||||
|
if task_result.result_code != ComputeTaskResultCode.OK:
|
||||||
|
logger.error(f"llm compute error:{task_result.error_str}")
|
||||||
|
return task_result
|
||||||
|
|
||||||
|
inner_func_call_node = None
|
||||||
|
|
||||||
|
result_message : dict = task_result.result.get("message")
|
||||||
|
if result_message:
|
||||||
|
inner_func_call_node = result_message.get("function_call")
|
||||||
|
if inner_func_call_node:
|
||||||
|
func_msg = copy.deepcopy(result_message)
|
||||||
|
del func_msg["tool_calls"]#TODO: support tool_calls?
|
||||||
|
prompt.messages.append(func_msg)
|
||||||
|
|
||||||
|
|
||||||
|
if inner_func_call_node:
|
||||||
|
return await self._execute_inner_func(inner_func_call_node,prompt,stack_limit-1)
|
||||||
|
else:
|
||||||
|
return task_result
|
||||||
|
|
||||||
|
async def process(self,input:Dict) -> LLMResult:
|
||||||
|
if self.enable_json_resp:
|
||||||
|
resp_mode = "json"
|
||||||
|
else:
|
||||||
|
resp_mode = "text"
|
||||||
|
|
||||||
|
# Action define in prompt, will be execute after llm compute
|
||||||
|
prompt = await self.prepare_prompt(input)
|
||||||
|
if prompt is None:
|
||||||
|
logger.warn(f"prepare_prompt return None, break llm_process")
|
||||||
|
return LLMResult.from_error_str("prepare_prompt return None")
|
||||||
|
|
||||||
|
max_result_token = self.max_token - ComputeKernel.llm_num_tokens(prompt,self.get_llm_model_name())
|
||||||
|
#if max_result_token < MIN_PREDICT_TOKEN_LEN:
|
||||||
|
# return LLMResult.from_error_str(f"prompt too long,can not predict")
|
||||||
|
|
||||||
|
task_result: ComputeTaskResult = await (ComputeKernel.get_instance().do_llm_completion(
|
||||||
|
prompt,
|
||||||
|
resp_mode=resp_mode,
|
||||||
|
mode_name=self.get_llm_model_name(),
|
||||||
|
max_token=max_result_token,
|
||||||
|
inner_functions=prompt.inner_functions, #NOTICE: inner_function in prompt can be a subset of get_inner_function
|
||||||
|
timeout=self.timeout))
|
||||||
|
|
||||||
|
if task_result.result_code != ComputeTaskResultCode.OK:
|
||||||
|
err_str = f"do_llm_completion error:{task_result.error_str}"
|
||||||
|
logger.error(err_str)
|
||||||
|
return LLMResult.from_error_str(err_str)
|
||||||
|
|
||||||
|
result_message = task_result.result.get("message")
|
||||||
|
inner_func_call_node = None
|
||||||
|
if result_message:
|
||||||
|
inner_func_call_node = result_message.get("function_call")
|
||||||
|
|
||||||
|
if inner_func_call_node:
|
||||||
|
call_prompt : LLMPrompt = copy.deepcopy(prompt)
|
||||||
|
func_msg = copy.deepcopy(result_message)
|
||||||
|
del func_msg["tool_calls"]
|
||||||
|
call_prompt.messages.append(func_msg)
|
||||||
|
task_result = await self._execute_inner_func(inner_func_call_node,call_prompt)
|
||||||
|
|
||||||
|
# parse task_result to LLM Result
|
||||||
|
if self.enable_json_resp:
|
||||||
|
try:
|
||||||
|
llm_result = LLMResult.from_json_str(task_result.result_str)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"parse llm result error:{e}")
|
||||||
|
llm_result = LLMResult.from_str(task_result.result_str)
|
||||||
|
else:
|
||||||
|
llm_result = LLMResult.from_str(task_result.result_str)
|
||||||
|
|
||||||
|
# use action to save history?
|
||||||
|
await self.post_llm_process(llm_result.action_list,input,llm_result)
|
||||||
|
|
||||||
|
return llm_result
|
||||||
|
|
||||||
|
class LLMAgentBaseProcess(BaseLLMProcess):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
super().__init__()
|
||||||
|
|
||||||
|
self.role_description:str = None
|
||||||
|
self.process_description:str = None
|
||||||
|
self.reply_format:str = None
|
||||||
|
self.context : str = None
|
||||||
|
|
||||||
|
self.workspace : AgentWorkspace = None # If Workspace is not none , enable Agent Tasklist
|
||||||
|
self.memory : AgentMemory = None
|
||||||
|
self.enable_kb : bool = False
|
||||||
|
self.kb = None
|
||||||
|
|
||||||
|
async def initial(self,params:Dict = None) -> bool:
|
||||||
|
self.memory = params.get("memory")
|
||||||
|
if self.memory is None:
|
||||||
|
logger.error(f"LLMAgeMessageProcess initial failed! memory not found")
|
||||||
|
return False
|
||||||
|
self.workspace = params.get("workspace")
|
||||||
|
|
||||||
|
return True
|
||||||
|
async def load_default_config(self) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
async def load_from_config(self, config: dict,is_load_default=True) -> Coroutine[Any, Any, bool]:
|
||||||
|
if is_load_default:
|
||||||
|
await self.load_default_config()
|
||||||
|
|
||||||
|
if await super().load_from_config(config) is False:
|
||||||
|
return False
|
||||||
|
|
||||||
|
self.role_description = config.get("role_desc")
|
||||||
|
if self.role_description is None:
|
||||||
|
logger.error(f"role_description not found in config")
|
||||||
|
return False
|
||||||
|
|
||||||
|
if config.get("process_description"):
|
||||||
|
self.process_description = config.get("process_description")
|
||||||
|
|
||||||
|
if config.get("reply_format"):
|
||||||
|
self.reply_format = config.get("reply_format")
|
||||||
|
|
||||||
|
if config.get("context"):
|
||||||
|
self.context = config.get("context")
|
||||||
|
|
||||||
|
self.llm_context = SimpleLLMContext()
|
||||||
|
if config.get("llm_context"):
|
||||||
|
self.llm_context.load_from_config(config.get("llm_context"))
|
||||||
|
|
||||||
|
if config.get("enable_kb"):
|
||||||
|
self.enable_kb = config.get("enable_kb") == "true"
|
||||||
|
|
||||||
|
def prepare_role_system_prompt(self,context_info:Dict) -> Dict:
|
||||||
|
system_prompt_dict = {}
|
||||||
|
# System Prompt
|
||||||
|
## LLM的身份说明
|
||||||
|
system_prompt_dict["role_description"] = self.role_description
|
||||||
|
#prompt.append_system_message(self.role_description)
|
||||||
|
|
||||||
|
## 处理信息的流程说明
|
||||||
|
system_prompt_dict["process_rule"] = self.process_description
|
||||||
|
#prompt.append_system_message(self.process_description)
|
||||||
|
### 回复的格式
|
||||||
|
system_prompt_dict["reply_format"] = self.reply_format
|
||||||
|
#prompt.append_system_message(self.reply_format)
|
||||||
|
|
||||||
|
## Context
|
||||||
|
if self.context:
|
||||||
|
context = self._format_content_by_env_value(self.context,context_info)
|
||||||
|
system_prompt_dict["context"] = context
|
||||||
|
#prompt.append_system_message(context)
|
||||||
|
|
||||||
|
system_prompt_dict["support_actions"] = self.get_action_desc()
|
||||||
|
|
||||||
|
return system_prompt_dict
|
||||||
|
|
||||||
|
def prepare_inner_function_context_for_exec(self,inner_func_name:str,parameters:Dict):
|
||||||
|
parameters["_workspace"] = self.workspace
|
||||||
|
|
||||||
|
def get_action_desc(self) -> Dict:
|
||||||
|
result = {}
|
||||||
|
actions_list = self.llm_context.get_all_ai_action()
|
||||||
|
for action in actions_list:
|
||||||
|
result[action.get_name()] = action.get_description()
|
||||||
|
return result
|
||||||
|
|
||||||
|
async def get_inner_function_for_exec(self,func_name:str) -> AIFunction:
|
||||||
|
return self.llm_context.get_ai_function(func_name)
|
||||||
|
|
||||||
|
async def _execute_actions(self,actions:List[ActionNode],action_params:Dict):
|
||||||
|
for action_item in actions:
|
||||||
|
op : AIAction = self.llm_context.get_ai_action(action_item.name)
|
||||||
|
if op:
|
||||||
|
if action_item.parms is None:
|
||||||
|
action_item.parms = {}
|
||||||
|
|
||||||
|
real_parms = {**action_params,**action_item.parms}
|
||||||
|
|
||||||
|
action_item.parms["_result"] = await op.execute(real_parms)
|
||||||
|
action_item.parms["_end_at"] = datetime.now()
|
||||||
|
else:
|
||||||
|
logger.warn(f"action {action_item.name} not found")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
class AgentMessageProcess(LLMAgentBaseProcess):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
super().__init__()
|
||||||
|
self.mutil_model = None
|
||||||
|
self.enable_media2text = False
|
||||||
|
self.is_mutil_model = False
|
||||||
|
self.asr_model = None
|
||||||
|
self.tts_model = None
|
||||||
|
|
||||||
|
async def load_default_config(self) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
async def load_from_config(self, config: dict,is_load_default=True) -> Coroutine[Any, Any, bool]:
|
||||||
|
if is_load_default:
|
||||||
|
await self.load_default_config()
|
||||||
|
|
||||||
|
if await super().load_from_config(config) is False:
|
||||||
|
return False
|
||||||
|
|
||||||
|
self.enable_media2text = config.get('enable_media2text', 'false').lower() in ('true', '1', 't', 'y', 'yes')
|
||||||
|
|
||||||
|
if config.get("mutil_model"):
|
||||||
|
self.mutil_model = config.get("mutil_model")
|
||||||
|
|
||||||
|
self.asr_model = config.get("asr_model")
|
||||||
|
self.tts_model = config.get("tts_model")
|
||||||
|
|
||||||
|
def get_llm_model_name(self) -> str:
|
||||||
|
if self.is_mutil_model:
|
||||||
|
return self.mutil_model
|
||||||
|
else:
|
||||||
|
return self.model_name
|
||||||
|
|
||||||
|
def check_and_to_base64(self, image_path: str) -> str:
|
||||||
|
if image_utils.is_file(image_path):
|
||||||
|
return image_utils.to_base64(image_path, (1024, 1024))
|
||||||
|
else:
|
||||||
|
return image_path
|
||||||
|
|
||||||
|
async def get_prompt_from_msg(self,msg:AgentMsg) -> LLMPrompt:
|
||||||
|
msg_prompt = LLMPrompt()
|
||||||
|
self.is_mutil_model = False
|
||||||
|
if msg.is_image_msg():
|
||||||
|
if self.enable_media2text:
|
||||||
|
logger.error(f"enable_media2text is not supported yet")
|
||||||
|
else:
|
||||||
|
image_prompt, images = msg.get_image_body()
|
||||||
|
if image_prompt is None:
|
||||||
|
msg_prompt.messages = [{"role": "user", "content": [{"type": "image_url", "image_url": {"url": self.check_and_to_base64(image)}} for image in images]}]
|
||||||
|
else:
|
||||||
|
content = [{"type": "text", "text": image_prompt}]
|
||||||
|
content.extend([{"type": "image_url", "image_url": {"url": self.check_and_to_base64(image)}} for image in images])
|
||||||
|
msg_prompt.messages = [{"role": "user", "content": content}]
|
||||||
|
|
||||||
|
if self.mutil_model:
|
||||||
|
self.is_mutil_model = True
|
||||||
|
else:
|
||||||
|
logger.warning(f"mutil_model is not set!")
|
||||||
|
|
||||||
|
elif msg.is_video_msg():
|
||||||
|
if self.enable_media2text:
|
||||||
|
logger.error(f"enable_media2text is not supported yet")
|
||||||
|
else:
|
||||||
|
video_prompt, video = msg.get_video_body()
|
||||||
|
frames = video_utils.extract_frames(video, (1024, 1024))
|
||||||
|
audio_file = os.path.splitext(video)[0] + ".mp3"
|
||||||
|
video_utils.extract_audio(video, audio_file)
|
||||||
|
|
||||||
|
voice_content = None
|
||||||
|
if self.asr_model is not None:
|
||||||
|
resp = await (ComputeKernel.get_instance().do_speech_to_text(audio_file, model=self.asr_model, prompt=None, response_format="text"))
|
||||||
|
if resp.result_code == ComputeTaskResultCode.OK:
|
||||||
|
voice_content = resp.result_str
|
||||||
|
|
||||||
|
content = []
|
||||||
|
if video_prompt is not None:
|
||||||
|
content.append({"type": "text", "text": video_prompt})
|
||||||
|
if voice_content is not None and voice_content != "":
|
||||||
|
content.append({"type": "text", "text": f"Voice content in video:{voice_content}"})
|
||||||
|
|
||||||
|
content.extend([{"type": "image_url", "image_url": {"url": frame}} for frame in frames])
|
||||||
|
msg_prompt.messages = [{"role": "user", "content": content}]
|
||||||
|
if self.mutil_model:
|
||||||
|
self.is_mutil_model = True
|
||||||
|
else:
|
||||||
|
logger.warning(f"mutil_model is not set!")
|
||||||
|
elif msg.is_audio_msg():
|
||||||
|
if self.enable_media2text:
|
||||||
|
logger.error(f"enable_media2text is not supported yet")
|
||||||
|
else:
|
||||||
|
prompt, audio_file = msg.get_audio_body()
|
||||||
|
resp = await (ComputeKernel.get_instance().do_speech_to_text(audio_file, model=self.asr_model, prompt=None, response_format="text"))
|
||||||
|
if resp.result_code != ComputeTaskResultCode.OK:
|
||||||
|
error_resp = msg.create_error_resp(resp.error_str)
|
||||||
|
return error_resp
|
||||||
|
else:
|
||||||
|
if prompt == "":
|
||||||
|
msg.body = resp.result_str
|
||||||
|
msg_prompt.messages = [{"role":"user","content":resp.result_str}]
|
||||||
|
else:
|
||||||
|
msg.body = f"{prompt}\nVoice content:{resp.result_str}"
|
||||||
|
msg_prompt.messages = [{"role":"user","content": prompt}, {"role": "user", "content": f"Voice content:{resp.result_str}"}]
|
||||||
|
else:
|
||||||
|
msg_prompt.messages = [{"role":"user","content":msg.body}]
|
||||||
|
|
||||||
|
return msg_prompt
|
||||||
|
|
||||||
|
async def sender_info(self,msg:AgentMsg)->str:
|
||||||
|
sender_id = msg.sender
|
||||||
|
#TODO Is sender an agent?
|
||||||
|
return await self.memory.get_contact_summary(sender_id)
|
||||||
|
|
||||||
|
async def load_chatlogs(self,msg:AgentMsg,max_length_by_token:int)->str:
|
||||||
|
## like
|
||||||
|
#sender,[2023-11-1 12:00:00]
|
||||||
|
#content
|
||||||
|
return await self.memory.load_chatlogs(msg,max_length_by_token)
|
||||||
|
|
||||||
|
async def get_chat_summary(self,msg:AgentMsg)->str:
|
||||||
|
return await self.memory.get_chat_summary(msg)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
async def get_extend_known_info(self,msg:AgentMsg,prompt:LLMPrompt)->str:
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def prepare_prompt(self,input:Dict) -> LLMPrompt:
|
||||||
|
prompt = LLMPrompt()
|
||||||
|
# User Prompt
|
||||||
|
## Input Msg
|
||||||
|
msg : AgentMsg = input.get("msg")
|
||||||
|
context_info = input.get("context_info")
|
||||||
|
if msg is None:
|
||||||
|
logger.error(f"LLMAgeMessageProcess prepare_prompt failed! input msg not found")
|
||||||
|
return None
|
||||||
|
msg_prompt = await self.get_prompt_from_msg(msg)
|
||||||
|
if msg_prompt is None:
|
||||||
|
logger.error(f"LLMAgeMessageProcess prepare_prompt failed! get_prompt_from_msg return None")
|
||||||
|
return None
|
||||||
|
prompt.append(msg_prompt)
|
||||||
|
|
||||||
|
## 通用的角色相关的系统提示词
|
||||||
|
system_prompt_dict = self.prepare_role_system_prompt(context_info)
|
||||||
|
|
||||||
|
## 已知信息
|
||||||
|
known_info = {}
|
||||||
|
#prompt.append_system_message(self.known_info_tips)
|
||||||
|
### 信息发送者资料
|
||||||
|
known_info["sender_info"] = await self.sender_info(msg)
|
||||||
|
#prompt.append_system_message(await self.sender_info(self,msg))
|
||||||
|
|
||||||
|
system_prompt_dict["known_info"] = known_info
|
||||||
|
|
||||||
|
prompt.inner_functions =LLMProcessContext.aifunctions_to_inner_functions(self.llm_context.get_all_ai_functions())
|
||||||
|
if self.workspace:
|
||||||
|
#TODO eanble workspace functions?
|
||||||
|
logger.info(f"workspace is not none,enable workspace functions")
|
||||||
|
|
||||||
|
## 给予查询KB的权限
|
||||||
|
if self.enable_kb:
|
||||||
|
logger.info(f"enable kb")
|
||||||
|
|
||||||
|
|
||||||
|
### 根据Token Limit加载聊天记录
|
||||||
|
remain_token = self.get_remain_prompt_length(prompt,json.dumps(system_prompt_dict,ensure_ascii=False))
|
||||||
|
chat_record,is_all = await self.load_chatlogs(msg,remain_token - self.chat_summary_token_len)
|
||||||
|
if chat_record:
|
||||||
|
if len(chat_record) > 4:
|
||||||
|
known_info["chat_record"] = chat_record
|
||||||
|
|
||||||
|
if not is_all :
|
||||||
|
### 如果出触发了Token Limit,则删除几条信息后，加载summary （summary的长度基本是固定的）
|
||||||
|
summary = await self.get_chat_summary(msg)
|
||||||
|
if summary:
|
||||||
|
if len(summary) > 4:
|
||||||
|
known_info["chat_summary"] = summary
|
||||||
|
|
||||||
|
# TODO: extend known info
|
||||||
|
#prompt.append_system_message(await self.get_extend_known_info(msg,prompt))
|
||||||
|
|
||||||
|
prompt.append_system_message(json.dumps(system_prompt_dict,ensure_ascii=False))
|
||||||
|
return prompt
|
||||||
|
|
||||||
|
|
||||||
|
async def post_llm_process(self,actions:List[ActionNode],input:Dict,llm_result:LLMResult) -> bool:
|
||||||
|
msg:AgentMsg = input.get("msg")
|
||||||
|
if msg.msg_type == AgentMsgType.TYPE_GROUPMSG:
|
||||||
|
resp_msg = msg.create_group_resp_msg(self.memory.agent_id,llm_result.resp)
|
||||||
|
else:
|
||||||
|
resp_msg = msg.create_resp_msg(llm_result.resp)
|
||||||
|
|
||||||
|
if llm_result.raw_result is not None:
|
||||||
|
llm_result.raw_result["_resp_msg"] = resp_msg
|
||||||
|
|
||||||
|
action_params = {}
|
||||||
|
action_params["_input"] = input
|
||||||
|
action_params["_memory"] = self.memory
|
||||||
|
action_params["_workspace"] = self.workspace
|
||||||
|
action_params["_resp_msg"] = resp_msg
|
||||||
|
action_params["_llm_result"] = llm_result
|
||||||
|
action_params["_agentid"] = self.memory.agent_id
|
||||||
|
action_params["_start_at"] = datetime.now()
|
||||||
|
|
||||||
|
await self._execute_actions(actions,action_params)
|
||||||
|
|
||||||
|
chatsession = self.memory.get_session_from_msg(msg)
|
||||||
|
chatsession.append(msg)
|
||||||
|
chatsession.append(resp_msg)
|
||||||
|
|
||||||
|
return True
|
||||||
|
|
||||||
|
class AgentSelfThinking(LLMAgentBaseProcess):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
super().__init__()
|
||||||
|
|
||||||
|
|
||||||
|
async def load_from_config(self, config: dict) -> Coroutine[Any, Any, bool]:
|
||||||
|
if await super().load_from_config(config) is False:
|
||||||
|
return False
|
||||||
|
|
||||||
|
async def _load_chat_history(self,token_limit:int):
|
||||||
|
chat_history = {}
|
||||||
|
session_list = AIChatSession.list_session(self.memory.agent_id ,self.memory.memory_db)
|
||||||
|
total_read_msg = 0
|
||||||
|
for session_id in session_list:
|
||||||
|
chatsession = AIChatSession.get_session_by_id(session_id,self.memory.memory_db)
|
||||||
|
session_history = {}
|
||||||
|
session_history["summary"] = chatsession.summary
|
||||||
|
session_history["id"] = chatsession.session_id
|
||||||
|
token_limit -= ComputeKernel.llm_num_tokens_from_text(chatsession.summary,self.model_name)
|
||||||
|
read_history_msg = 0
|
||||||
|
|
||||||
|
if token_limit > 8:
|
||||||
|
# load session chat history
|
||||||
|
cur_pos = chatsession.summarize_pos
|
||||||
|
messages = chatsession.read_history(0,cur_pos,"natural") # read
|
||||||
|
history_str = ""
|
||||||
|
for msg in messages:
|
||||||
|
read_history_msg += 1
|
||||||
|
total_read_msg += 1
|
||||||
|
cur_pos += 1
|
||||||
|
dt = datetime.fromtimestamp(float(msg.create_time))
|
||||||
|
formatted_time = dt.strftime('%y-%m-%d %H:%M:%S')
|
||||||
|
record_str = f"{msg.sender},[{formatted_time}]\n{msg.body}\n"
|
||||||
|
token_limit -= ComputeKernel.llm_num_tokens_from_text(record_str,self.model_name)
|
||||||
|
if token_limit < 8:
|
||||||
|
break
|
||||||
|
|
||||||
|
history_str = history_str + record_str
|
||||||
|
|
||||||
|
if read_history_msg >= 2:
|
||||||
|
session_history["history"] = history_str
|
||||||
|
chat_history[session_id] = session_history
|
||||||
|
chatsession.summarize_pos = cur_pos
|
||||||
|
|
||||||
|
else:
|
||||||
|
logger.info(f"load_chat_history reach token limit,load {total_read_msg} history messages.")
|
||||||
|
return chat_history
|
||||||
|
|
||||||
|
if total_read_msg < 2:
|
||||||
|
logger.info(f"load_chat_history: no history messages,return NONE")
|
||||||
|
return None
|
||||||
|
|
||||||
|
return chat_history
|
||||||
|
|
||||||
|
|
||||||
|
async def prepare_prompt(self,input:Dict) -> LLMPrompt:
|
||||||
|
prompt = LLMPrompt()
|
||||||
|
|
||||||
|
context_info = input.get("context_info")
|
||||||
|
|
||||||
|
system_prompt_dict = self.prepare_role_system_prompt(context_info)
|
||||||
|
|
||||||
|
# Known_info is the SESSION summary of the existence, the current task work record summary,
|
||||||
|
token_remain = self.get_remain_prompt_length(prompt,json.dumps(system_prompt_dict,ensure_ascii=False))
|
||||||
|
chat_history = await self._load_chat_history(token_remain)
|
||||||
|
if chat_history is None:
|
||||||
|
logger.info(f"prepare_prompt: no history messages,return NONE")
|
||||||
|
return None
|
||||||
|
|
||||||
|
prompt.inner_functions =LLMProcessContext.aifunctions_to_inner_functions(self.llm_context.get_all_ai_functions())
|
||||||
|
|
||||||
|
prompt.append_system_message(json.dumps(system_prompt_dict,ensure_ascii=False))
|
||||||
|
prompt.append_user_message(json.dumps(chat_history,ensure_ascii=False))
|
||||||
|
return prompt
|
||||||
|
|
||||||
|
async def post_llm_process(self,actions:List[ActionNode],input:Dict,llm_result:LLMResult) -> bool:
|
||||||
|
action_params = {}
|
||||||
|
action_params["_input"] = input
|
||||||
|
action_params["_memory"] = self.memory
|
||||||
|
action_params["_workspace"] = self.workspace
|
||||||
|
action_params["_llm_result"] = llm_result
|
||||||
|
action_params["_agentid"] = self.memory.agent_id
|
||||||
|
action_params["_start_at"] = datetime.now()
|
||||||
|
try:
|
||||||
|
if await self._execute_actions(actions,action_params) is False:
|
||||||
|
result_str = "execute action failed!"
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"execute action failed! {e}")
|
||||||
|
result_str = "execute action failed!,error:" + str(e)
|
||||||
|
|
||||||
|
class AgentSelfLearning(BaseLLMProcess):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
super().__init__()
|
||||||
|
|
||||||
|
async def load_from_config(self, config: dict) -> Coroutine[Any, Any, bool]:
|
||||||
|
if await super().load_from_config(config) is False:
|
||||||
|
return False
|
||||||
|
|
||||||
|
async def prepare_prompt(self) -> LLMPrompt:
|
||||||
|
prompt = LLMPrompt()
|
||||||
|
pass
|
||||||
|
|
||||||
|
async def get_inner_function_for_exec(self,func_name:str) -> AIFunction:
|
||||||
|
pass
|
||||||
|
|
||||||
|
async def post_llm_process(self,actions:List[ActionNode]) -> bool:
|
||||||
|
pass
|
||||||
|
|
||||||
|
class AgentSelfImprove(BaseLLMProcess):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
super().__init__()
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
from .llm_process import BaseLLMProcess, AgentMessageProcess,AgentSelfThinking,AgentSelfLearning,AgentSelfImprove
|
||||||
|
from .llm_do_task import AgentTriageTaskList,AgentPlanTask,AgentReviewTask,AgentDo,AgentCheck
|
||||||
|
|
||||||
|
from typing import Awaitable, Callable, Coroutine, Dict, List, Any
|
||||||
|
import logging
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
class LLMProcessLoader:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.loaders : Dict[str,Callable[[dict],Awaitable[BaseLLMProcess]]] = {}
|
||||||
|
return
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def get_instance(cls)->"LLMProcessLoader":
|
||||||
|
if not hasattr(cls,"_instance"):
|
||||||
|
cls._instance = LLMProcessLoader()
|
||||||
|
return cls._instance
|
||||||
|
|
||||||
|
def register_loader(self, typename:str,loader:Callable[[dict],Awaitable[BaseLLMProcess]]):
|
||||||
|
self.loaders[typename] = loader
|
||||||
|
|
||||||
|
async def load_from_config(self,config:dict) -> BaseLLMProcess:
|
||||||
|
llm_type_name = config.get("type")
|
||||||
|
if llm_type_name:
|
||||||
|
loader = self.loaders.get(llm_type_name)
|
||||||
|
if loader:
|
||||||
|
return await loader(config)
|
||||||
|
|
||||||
|
selected_type = globals().get(llm_type_name)
|
||||||
|
if selected_type:
|
||||||
|
result : BaseLLMProcess = selected_type()
|
||||||
|
load_result = await result.load_from_config(config)
|
||||||
|
if load_result is False:
|
||||||
|
logger.warn(f"load LLMProcess {llm_type_name} from config failed! load_from_config return False")
|
||||||
|
return None
|
||||||
|
else:
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
logger.warn(f"load LLMProcess {llm_type_name} from config failed! type not found")
|
||||||
|
return None
|
||||||
@@ -1,6 +1,7 @@
|
|||||||
|
# pylint:disable=E0402
|
||||||
import logging
|
import logging
|
||||||
|
|
||||||
from .agent import AIAgent,AgentPrompt
|
from ..proto.compute_task import LLMPrompt
|
||||||
|
|
||||||
class AIRole:
|
class AIRole:
|
||||||
def __init__(self) -> None:
|
def __init__(self) -> None:
|
||||||
@@ -9,7 +10,7 @@ class AIRole:
|
|||||||
self.role_id :str = None # $workflow_id.$sub_workflow_id.$role_name
|
self.role_id :str = None # $workflow_id.$sub_workflow_id.$role_name
|
||||||
self.fullname : str = None
|
self.fullname : str = None
|
||||||
self.agent_name : str = None
|
self.agent_name : str = None
|
||||||
self.prompt : AgentPrompt = None
|
self.prompt : LLMPrompt = None
|
||||||
self.introduce : str = None
|
self.introduce : str = None
|
||||||
self.agent = None
|
self.agent = None
|
||||||
self.enable_function_list : list[str] = None
|
self.enable_function_list : list[str] = None
|
||||||
@@ -31,7 +32,7 @@ class AIRole:
|
|||||||
|
|
||||||
prompt_node = config.get("prompt")
|
prompt_node = config.get("prompt")
|
||||||
if prompt_node:
|
if prompt_node:
|
||||||
self.prompt = AgentPrompt()
|
self.prompt = LLMPrompt()
|
||||||
if self.prompt.load_from_config(prompt_node) is False:
|
if self.prompt.load_from_config(prompt_node) is False:
|
||||||
logging.error("load prompt failed!")
|
logging.error("load prompt failed!")
|
||||||
return False
|
return False
|
||||||
@@ -41,7 +42,7 @@ class AIRole:
|
|||||||
self.introduce = intro_node
|
self.introduce = intro_node
|
||||||
|
|
||||||
history_node = config.get("history_len")
|
history_node = config.get("history_len")
|
||||||
if history_node:
|
if history_node is not None:
|
||||||
self.history_len = int(history_node)
|
self.history_len = int(history_node)
|
||||||
|
|
||||||
if config.get("enable_function") is not None:
|
if config.get("enable_function") is not None:
|
||||||
@@ -56,7 +57,7 @@ class AIRole:
|
|||||||
def get_name(self) -> str:
|
def get_name(self) -> str:
|
||||||
return self.role_name
|
return self.role_name
|
||||||
|
|
||||||
def get_prompt(self) -> AgentPrompt:
|
def get_prompt(self) -> LLMPrompt:
|
||||||
return self.prompt
|
return self.prompt
|
||||||
|
|
||||||
class AIRoleGroup:
|
class AIRoleGroup:
|
||||||
@@ -78,4 +79,3 @@ class AIRoleGroup:
|
|||||||
def get(self,role_name:str) -> AIRole:
|
def get(self,role_name:str) -> AIRole:
|
||||||
return self.roles.get(role_name)
|
return self.roles.get(role_name)
|
||||||
|
|
||||||
|
|
||||||
@@ -1,3 +1,4 @@
|
|||||||
|
# pylint:disable=E0402
|
||||||
import logging
|
import logging
|
||||||
import asyncio
|
import asyncio
|
||||||
import json
|
import json
|
||||||
@@ -7,16 +8,19 @@ from asyncio import Queue
|
|||||||
from typing import Optional,Tuple,List
|
from typing import Optional,Tuple,List
|
||||||
from abc import ABC, abstractmethod
|
from abc import ABC, abstractmethod
|
||||||
|
|
||||||
from .environment import Environment,EnvironmentEvent
|
from ..proto.compute_task import *
|
||||||
from .agent_message import AgentMsg,AgentMsgStatus,FunctionItem,LLMResult
|
from ..proto.agent_msg import *
|
||||||
from .agent import AgentPrompt,AgentMsg
|
from ..proto.ai_function import *
|
||||||
|
|
||||||
|
from .agent_base import *
|
||||||
from .chatsession import AIChatSession
|
from .chatsession import AIChatSession
|
||||||
from .role import AIRole,AIRoleGroup
|
from .role import AIRole,AIRoleGroup
|
||||||
from .ai_function import AIFunction,FunctionItem
|
|
||||||
from .compute_kernel import ComputeKernel
|
from ..frame.compute_kernel import ComputeKernel
|
||||||
from .compute_task import ComputeTask,ComputeTaskResult,ComputeTaskState,ComputeTaskResultCode
|
from ..frame.bus import AIBus
|
||||||
from .bus import AIBus
|
|
||||||
from .workflow_env import WorkflowEnvironment
|
from ..environment.environment import BaseEnvironment
|
||||||
|
from ..environment.workflow_env import WorkflowEnvironment
|
||||||
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
@@ -45,7 +49,7 @@ class Workflow:
|
|||||||
def __init__(self) -> None:
|
def __init__(self) -> None:
|
||||||
self.workflow_name : str = None
|
self.workflow_name : str = None
|
||||||
self.workflow_id : str = None
|
self.workflow_id : str = None
|
||||||
self.rule_prompt : AgentPrompt = None
|
self.rule_prompt : LLMPrompt = None
|
||||||
self.workflow_config = None
|
self.workflow_config = None
|
||||||
self.role_group : dict = None
|
self.role_group : dict = None
|
||||||
self.input_filter : MessageFilter= None
|
self.input_filter : MessageFilter= None
|
||||||
@@ -80,7 +84,7 @@ class Workflow:
|
|||||||
self.db_file = self.owner_workflow.db_file
|
self.db_file = self.owner_workflow.db_file
|
||||||
|
|
||||||
if config.get("prompt") is not None:
|
if config.get("prompt") is not None:
|
||||||
self.rule_prompt = AgentPrompt()
|
self.rule_prompt = LLMPrompt()
|
||||||
if self.rule_prompt.load_from_config(config.get("prompt")) is False:
|
if self.rule_prompt.load_from_config(config.get("prompt")) is False:
|
||||||
logger.error("Workflow load prompt failed")
|
logger.error("Workflow load prompt failed")
|
||||||
return False
|
return False
|
||||||
@@ -238,77 +242,6 @@ class Workflow:
|
|||||||
error_resp = msg.create_error_resp(err_str)
|
error_resp = msg.create_error_resp(err_str)
|
||||||
return error_resp
|
return error_resp
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def prase_llm_result(cls,llm_result_str:str)->LLMResult:
|
|
||||||
r = LLMResult()
|
|
||||||
if llm_result_str is None:
|
|
||||||
r.state = "ignore"
|
|
||||||
return r
|
|
||||||
if llm_result_str == "ignore":
|
|
||||||
r.state = "ignore"
|
|
||||||
return r
|
|
||||||
|
|
||||||
lines = llm_result_str.splitlines()
|
|
||||||
is_need_wait = False
|
|
||||||
|
|
||||||
def check_args(func_item:FunctionItem):
|
|
||||||
match func_name:
|
|
||||||
case "send_msg":# sendmsg($target_id,$msg_content)
|
|
||||||
if len(func_args) != 1:
|
|
||||||
logger.error(f"parse sendmsg failed! {func_call}")
|
|
||||||
return False
|
|
||||||
new_msg = AgentMsg()
|
|
||||||
target_id = func_item.args[0]
|
|
||||||
msg_content = func_item.body
|
|
||||||
new_msg.set("_",target_id,msg_content)
|
|
||||||
|
|
||||||
r.send_msgs.append(new_msg)
|
|
||||||
is_need_wait = True
|
|
||||||
|
|
||||||
case "post_msg":# postmsg($target_id,$msg_content)
|
|
||||||
if len(func_args) != 1:
|
|
||||||
logger.error(f"parse postmsg failed! {func_call}")
|
|
||||||
return False
|
|
||||||
new_msg = AgentMsg()
|
|
||||||
target_id = func_item.args[0]
|
|
||||||
msg_content = func_item.body
|
|
||||||
new_msg.set("_",target_id,msg_content)
|
|
||||||
r.post_msgs.append(new_msg)
|
|
||||||
|
|
||||||
case "call":# call($func_name,$args_str)
|
|
||||||
r.calls.append(func_item)
|
|
||||||
is_need_wait = True
|
|
||||||
return True
|
|
||||||
case "post_call": # post_call($func_name,$args_str)
|
|
||||||
r.post_calls.append(func_item)
|
|
||||||
return True
|
|
||||||
|
|
||||||
current_func : FunctionItem = None
|
|
||||||
for line in lines:
|
|
||||||
if line.startswith("##/"):
|
|
||||||
if current_func:
|
|
||||||
if check_args(current_func) is False:
|
|
||||||
r.resp += current_func.dumps()
|
|
||||||
|
|
||||||
func_name,func_args = AgentMsg.parse_function_call(line[3:])
|
|
||||||
current_func = FunctionItem(func_name,func_args)
|
|
||||||
else:
|
|
||||||
if current_func:
|
|
||||||
current_func.append_body(line + "\n")
|
|
||||||
else:
|
|
||||||
r.resp += line + "\n"
|
|
||||||
|
|
||||||
if current_func:
|
|
||||||
if check_args(current_func) is False:
|
|
||||||
r.resp += current_func.dumps()
|
|
||||||
|
|
||||||
if len(r.send_msgs) > 0 or len(r.calls) > 0:
|
|
||||||
r.state = "waiting"
|
|
||||||
else:
|
|
||||||
r.state = "reponsed"
|
|
||||||
|
|
||||||
return r
|
|
||||||
|
|
||||||
async def role_post_msg(self,msg:AgentMsg,the_role:AIRole,workflow_chat_session:AIChatSession):
|
async def role_post_msg(self,msg:AgentMsg,the_role:AIRole,workflow_chat_session:AIChatSession):
|
||||||
msg.sender = the_role.get_role_id()
|
msg.sender = the_role.get_role_id()
|
||||||
|
|
||||||
@@ -347,7 +280,7 @@ class Workflow:
|
|||||||
logger.info(f"{msg.sender} post message {msg.msg_id} to AIBus: {msg.target}")
|
logger.info(f"{msg.sender} post message {msg.msg_id} to AIBus: {msg.target}")
|
||||||
return await self.get_bus().send_message(msg)
|
return await self.get_bus().send_message(msg)
|
||||||
|
|
||||||
async def role_call(self,func_item:FunctionItem,the_role:AIRole):
|
async def role_call(self,func_item:ActionNode,the_role:AIRole):
|
||||||
logger.info(f"{the_role.role_id} call {func_item.name} ")
|
logger.info(f"{the_role.role_id} call {func_item.name} ")
|
||||||
arguments = func_item.args
|
arguments = func_item.args
|
||||||
|
|
||||||
@@ -358,11 +291,11 @@ class Workflow:
|
|||||||
result_str:str = await func_node.execute(**arguments)
|
result_str:str = await func_node.execute(**arguments)
|
||||||
return result_str
|
return result_str
|
||||||
|
|
||||||
async def role_post_call(self,func_item:FunctionItem,the_role:AIRole):
|
async def role_post_call(self,func_item:ActionNode,the_role:AIRole):
|
||||||
logger.info(f"{the_role.role_id} post call {func_item.name} ")
|
logger.info(f"{the_role.role_id} post call {func_item.name} ")
|
||||||
return await self.role_call(func_item,the_role)
|
return await self.role_call(func_item,the_role)
|
||||||
|
|
||||||
def _format_msg_by_env_value(self,prompt:AgentPrompt):
|
def _format_msg_by_env_value(self,prompt:LLMPrompt):
|
||||||
if self.workflow_env is None:
|
if self.workflow_env is None:
|
||||||
return
|
return
|
||||||
|
|
||||||
@@ -377,11 +310,11 @@ class Workflow:
|
|||||||
|
|
||||||
result_func = []
|
result_func = []
|
||||||
for inner_func in all_inner_function:
|
for inner_func in all_inner_function:
|
||||||
func_name = inner_func.get_name()
|
func_name = inner_func.get_id()
|
||||||
if the_role.enable_function_list is not None:
|
if the_role.enable_function_list is not None:
|
||||||
if len(the_role.enable_function_list) > 0:
|
if len(the_role.enable_function_list) > 0:
|
||||||
if func_name not in the_role.enable_function_list:
|
if func_name not in the_role.enable_function_list:
|
||||||
logger.debug(f"agent {self.agent_id} ignore inner func:{func_name}")
|
logger.debug(f"agent {the_role.agent.agent_id} ignore inner func:{func_name}")
|
||||||
continue
|
continue
|
||||||
else:
|
else:
|
||||||
continue
|
continue
|
||||||
@@ -394,8 +327,7 @@ class Workflow:
|
|||||||
return result_func
|
return result_func
|
||||||
return None
|
return None
|
||||||
|
|
||||||
async def _role_execute_func(self,the_role:AIRole,inenr_func_call_node:dict,prompt:AgentPrompt,org_msg:AgentMsg,stack_limit = 5) -> [str,int]:
|
async def _role_execute_func(self,the_role:AIRole,inenr_func_call_node:dict,prompt:LLMPrompt,org_msg:AgentMsg,stack_limit = 5) -> [str,int]:
|
||||||
from .compute_kernel import ComputeKernel
|
|
||||||
|
|
||||||
func_name = inenr_func_call_node.get("name")
|
func_name = inenr_func_call_node.get("name")
|
||||||
arguments = json.loads(inenr_func_call_node.get("arguments"))
|
arguments = json.loads(inenr_func_call_node.get("arguments"))
|
||||||
@@ -410,6 +342,7 @@ class Workflow:
|
|||||||
except Exception as e:
|
except Exception as e:
|
||||||
result_str = f"execute {func_name} error:{str(e)}"
|
result_str = f"execute {func_name} error:{str(e)}"
|
||||||
logger.error(f"llm execute inner func:{func_name} error:{e}")
|
logger.error(f"llm execute inner func:{func_name} error:{e}")
|
||||||
|
logger.exception(e)
|
||||||
|
|
||||||
|
|
||||||
inner_functions = self._get_inner_functions(the_role)
|
inner_functions = self._get_inner_functions(the_role)
|
||||||
@@ -440,8 +373,7 @@ class Workflow:
|
|||||||
async def role_process_msg(self,msg:AgentMsg,the_role:AIRole,workflow_chat_session:AIChatSession) -> AgentMsg:
|
async def role_process_msg(self,msg:AgentMsg,the_role:AIRole,workflow_chat_session:AIChatSession) -> AgentMsg:
|
||||||
msg.target = the_role.get_role_id()
|
msg.target = the_role.get_role_id()
|
||||||
|
|
||||||
|
prompt = LLMPrompt()
|
||||||
prompt = AgentPrompt()
|
|
||||||
prompt.append(the_role.agent.agent_prompt)
|
prompt.append(the_role.agent.agent_prompt)
|
||||||
prompt.append(self.get_workflow_rule_prompt())
|
prompt.append(self.get_workflow_rule_prompt())
|
||||||
prompt.append(the_role.get_prompt())
|
prompt.append(the_role.get_prompt())
|
||||||
@@ -451,7 +383,7 @@ class Workflow:
|
|||||||
#support group chat, user content include sender name!
|
#support group chat, user content include sender name!
|
||||||
prompt.append(await self._get_prompt_from_session(the_role,workflow_chat_session))
|
prompt.append(await self._get_prompt_from_session(the_role,workflow_chat_session))
|
||||||
|
|
||||||
msg_prompt = AgentPrompt()
|
msg_prompt = LLMPrompt()
|
||||||
msg_prompt.messages = [{"role":"user","content":f"user name is {msg.sender}, his question is :{msg.body}"}]
|
msg_prompt.messages = [{"role":"user","content":f"user name is {msg.sender}, his question is :{msg.body}"}]
|
||||||
prompt.append(msg_prompt)
|
prompt.append(msg_prompt)
|
||||||
|
|
||||||
@@ -480,7 +412,7 @@ class Workflow:
|
|||||||
error_resp = msg.create_error_resp(result_str)
|
error_resp = msg.create_error_resp(result_str)
|
||||||
return error_resp
|
return error_resp
|
||||||
|
|
||||||
result : LLMResult = Workflow.prase_llm_result(result_str)
|
result : LLMResult = LLMResult.from_str(result_str)
|
||||||
for postmsg in result.post_msgs:
|
for postmsg in result.post_msgs:
|
||||||
postmsg.prev_msg_id = msg.get_msg_id()
|
postmsg.prev_msg_id = msg.get_msg_id()
|
||||||
# might be craete a new msg.topic for this postmsg
|
# might be craete a new msg.topic for this postmsg
|
||||||
@@ -530,20 +462,20 @@ class Workflow:
|
|||||||
# message will be saved in role.process_message
|
# message will be saved in role.process_message
|
||||||
pass
|
pass
|
||||||
|
|
||||||
this_llm_resp_prompt = AgentPrompt()
|
this_llm_resp_prompt = LLMPrompt()
|
||||||
this_llm_resp_prompt.messages = [{"role":"assistant","content":result_str}]
|
this_llm_resp_prompt.messages = [{"role":"assistant","content":result_str}]
|
||||||
prompt.append(this_llm_resp_prompt)
|
prompt.append(this_llm_resp_prompt)
|
||||||
|
|
||||||
result_prompt = AgentPrompt()
|
result_prompt = LLMPrompt()
|
||||||
result_prompt.messages = [{"role":"user","content":result_prompt_str}]
|
result_prompt.messages = [{"role":"user","content":result_prompt_str}]
|
||||||
prompt.append(result_prompt)
|
prompt.append(result_prompt)
|
||||||
return await _do_process_msg()
|
return await _do_process_msg()
|
||||||
|
|
||||||
return await _do_process_msg()
|
return await _do_process_msg()
|
||||||
|
|
||||||
async def _get_prompt_from_session(self,the_role:AIRole,chatsession:AIChatSession) -> AgentPrompt:
|
async def _get_prompt_from_session(self,the_role:AIRole,chatsession:AIChatSession) -> LLMPrompt:
|
||||||
messages = chatsession.read_history(the_role.history_len) # read last 10 message
|
messages = chatsession.read_history(the_role.history_len) # read last 10 message
|
||||||
result_prompt = AgentPrompt()
|
result_prompt = LLMPrompt()
|
||||||
|
|
||||||
for msg in reversed(messages):
|
for msg in reversed(messages):
|
||||||
if msg.sender == the_role.role_id:
|
if msg.sender == the_role.role_id:
|
||||||
@@ -553,21 +485,21 @@ class Workflow:
|
|||||||
|
|
||||||
return result_prompt
|
return result_prompt
|
||||||
|
|
||||||
def _get_knowlege_prompt(self,role_name:str) -> AgentPrompt:
|
def _get_knowlege_prompt(self,role_name:str) -> LLMPrompt:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
def get_workflow_rule_prompt(self) -> AgentPrompt:
|
def get_workflow_rule_prompt(self) -> LLMPrompt:
|
||||||
return self.rule_prompt
|
return self.rule_prompt
|
||||||
|
|
||||||
def _env_event_to_msg(self,env_event:EnvironmentEvent) -> AgentMsg:
|
# def _env_event_to_msg(self,env_event:EnvironmentEvent) -> AgentMsg:
|
||||||
|
# pass
|
||||||
|
|
||||||
|
def get_inner_environment(self,env_id:str) -> BaseEnvironment:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
def get_inner_environment(self,env_id:str) -> Environment:
|
def connect_to_environment(self,the_env:BaseEnvironment,conn_info:dict) -> None:
|
||||||
pass
|
|
||||||
|
|
||||||
def connect_to_environment(self,the_env:Environment,conn_info:dict) -> None:
|
|
||||||
if the_env is not None:
|
if the_env is not None:
|
||||||
self.workflow_env.add_owner_env(the_env)
|
self.workflow_env.add_env(the_env)
|
||||||
|
|
||||||
#for event2msg in conn_info:
|
#for event2msg in conn_info:
|
||||||
# for k,v in event2msg:
|
# for k,v in event2msg:
|
||||||
@@ -0,0 +1,787 @@
|
|||||||
|
# pylint:disable=E0402
|
||||||
|
from ast import Dict
|
||||||
|
import json
|
||||||
|
import sqlite3
|
||||||
|
import os
|
||||||
|
import glob
|
||||||
|
import time
|
||||||
|
from typing import List, Optional
|
||||||
|
import aiofiles
|
||||||
|
|
||||||
|
from ..proto.agent_msg import AgentMsg
|
||||||
|
from ..proto.ai_function import AIFunction, ParameterDefine,SimpleAIFunction,ActionNode,SimpleAIAction
|
||||||
|
from ..proto.agent_task import AgentTask, AgentTaskState,AgentTodo,AgentWorkLog,AgentTaskManager
|
||||||
|
from ..storage.storage import AIStorage
|
||||||
|
from ..frame.bus import AIBus
|
||||||
|
from .llm_context import GlobaToolsLibrary
|
||||||
|
|
||||||
|
import logging
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
class LocalAgentTaskManger(AgentTaskManager):
|
||||||
|
def __init__(self, owner_id):
|
||||||
|
super().__init__()
|
||||||
|
self.root_path = f"{AIStorage.get_instance().get_myai_dir()}/agent_data/{owner_id}/workspace/"
|
||||||
|
#self.root_path = os.path.join(workspace, list_type)
|
||||||
|
if not os.path.exists(self.root_path):
|
||||||
|
os.makedirs(self.root_path)
|
||||||
|
|
||||||
|
self.db_path = os.path.join(self.root_path, "tasklist.db")
|
||||||
|
self.conn = None
|
||||||
|
try:
|
||||||
|
self.conn = sqlite3.connect(self.db_path)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("Error occurred while connecting to database: %s", e)
|
||||||
|
return None
|
||||||
|
|
||||||
|
cursor = self.conn.cursor()
|
||||||
|
cursor.execute('''
|
||||||
|
CREATE TABLE IF NOT EXISTS obj_list (
|
||||||
|
id TEXT,
|
||||||
|
path TEXT
|
||||||
|
)
|
||||||
|
''')
|
||||||
|
self.conn.commit()
|
||||||
|
|
||||||
|
def _get_obj_path(self,objid:str) -> str:
|
||||||
|
cursor = self.conn.cursor()
|
||||||
|
cursor.execute('''
|
||||||
|
SELECT path FROM obj_list WHERE id = ?
|
||||||
|
''',(objid,))
|
||||||
|
row = cursor.fetchone()
|
||||||
|
if row:
|
||||||
|
return row[0]
|
||||||
|
else:
|
||||||
|
return None
|
||||||
|
|
||||||
|
def _save_obj_path(self,objid:str,path:str):
|
||||||
|
cursor = self.conn.cursor()
|
||||||
|
cursor.execute('''
|
||||||
|
INSERT INTO obj_list (id,path) VALUES (?,?)
|
||||||
|
''',(objid,path))
|
||||||
|
self.conn.commit()
|
||||||
|
|
||||||
|
async def create_task(self,task:AgentTask,parent_id:str = None) -> str:
|
||||||
|
try:
|
||||||
|
#perfix = task.task_id[-5]
|
||||||
|
if parent_id:
|
||||||
|
parent_path = self._get_obj_path(parent_id)
|
||||||
|
task_path = f"{parent_path}/{task.title}"
|
||||||
|
else:
|
||||||
|
task_path = f"{task.title}"
|
||||||
|
|
||||||
|
dir_path = f"{self.root_path}/{task_path}"
|
||||||
|
|
||||||
|
os.makedirs(dir_path)
|
||||||
|
detail_path = f"{dir_path}/detail"
|
||||||
|
if task.task_path is None:
|
||||||
|
task.task_path = task_path
|
||||||
|
self._save_obj_path(task.task_id,task_path)
|
||||||
|
logger.info("create_task at %s",detail_path)
|
||||||
|
async with aiofiles.open(detail_path, mode='w', encoding="utf-8") as f:
|
||||||
|
await f.write(json.dumps(task.to_dict(),ensure_ascii=False))
|
||||||
|
return "create task ok"
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("create_task failed:%s",e)
|
||||||
|
return str(e)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
async def set_todos(self,owner_task_id:str,todos:List[Dict]):
|
||||||
|
owner_task_path = self._get_obj_path(owner_task_id)
|
||||||
|
if owner_task_path is None:
|
||||||
|
return f"owner task {owner_task_id} not found"
|
||||||
|
|
||||||
|
try:
|
||||||
|
directory = f"{self.root_path}/{owner_task_path}"
|
||||||
|
file_extension = "*.todo"
|
||||||
|
pattern = os.path.join(directory, file_extension)
|
||||||
|
files = glob.glob(pattern)
|
||||||
|
|
||||||
|
for file in files:
|
||||||
|
os.remove(file)
|
||||||
|
logger.info(f"Deleted {file}")
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("set_todos deleted todos failed:%s",e)
|
||||||
|
|
||||||
|
try:
|
||||||
|
step_order = 0
|
||||||
|
for todo in todos:
|
||||||
|
todo_obj = AgentTodo.from_dict(todo)
|
||||||
|
todo_obj.step_order = step_order
|
||||||
|
todo_obj.owner_taskid = owner_task_id
|
||||||
|
todo_path = f"{self.root_path}/{owner_task_path}/#{step_order} {todo_obj.title}.todo"
|
||||||
|
self._save_obj_path(todo_obj.todo_id,todo_path)
|
||||||
|
async with aiofiles.open(todo_path, mode='w', encoding="utf-8") as f:
|
||||||
|
await f.write(json.dumps(todo_obj.to_dict(),ensure_ascii=False))
|
||||||
|
logger.info("create_todos at %s OK!",todo_path)
|
||||||
|
step_order += 1
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("create_todos failed:%s",e)
|
||||||
|
return str(e)
|
||||||
|
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
async def append_worklog(self,task:AgentTask,log:AgentWorkLog):
|
||||||
|
worklog = f"{self.root_path}/{task.task_path}/.worklog"
|
||||||
|
|
||||||
|
async with aiofiles.open(worklog, mode='w+', encoding="utf-8") as f:
|
||||||
|
content = await f.read()
|
||||||
|
if len(content) > 0:
|
||||||
|
json_obj = json.loads(content)
|
||||||
|
else:
|
||||||
|
json_obj = {}
|
||||||
|
logs = json_obj.get("logs")
|
||||||
|
if logs is None:
|
||||||
|
logs = []
|
||||||
|
logs.append(log.to_dict())
|
||||||
|
json_obj["logs"] = logs
|
||||||
|
await f.write(json.dumps(json_obj,ensure_ascii=False))
|
||||||
|
|
||||||
|
|
||||||
|
async def get_worklog(self,obj_id:str)->List[AgentWorkLog]:
|
||||||
|
obj_path = self._get_obj_path(obj_id)
|
||||||
|
if obj_path is None:
|
||||||
|
return []
|
||||||
|
|
||||||
|
if obj_path.endswith(".todo"):
|
||||||
|
dir_path = os.path.dirname(obj_path)
|
||||||
|
worklog_path = f"{self.root_path}/{dir_path}/.worklog"
|
||||||
|
else:
|
||||||
|
worklog_path = f"{self.root_path}/{obj_path}/.worklog"
|
||||||
|
|
||||||
|
async with aiofiles.open(worklog_path, mode='r', encoding="utf-8") as f:
|
||||||
|
content = await f.read()
|
||||||
|
if len(content) > 0:
|
||||||
|
json_obj = json.loads(content)
|
||||||
|
else:
|
||||||
|
json_obj = {}
|
||||||
|
logs = json_obj.get("logs")
|
||||||
|
return logs
|
||||||
|
|
||||||
|
|
||||||
|
async def get_task(self,task_id:str) -> AgentTask:
|
||||||
|
task_path = self._get_obj_path(task_id)
|
||||||
|
if task_path is None:
|
||||||
|
logger.error("get_task:%s,not found!",task_id)
|
||||||
|
return None
|
||||||
|
|
||||||
|
return await self.get_task_by_path(task_path)
|
||||||
|
|
||||||
|
async def _get_task_by_fullpath(self,task_fullpath) -> AgentTask:
|
||||||
|
detail_path = f"{task_fullpath}/detail"
|
||||||
|
try:
|
||||||
|
with open(detail_path, mode='r', encoding="utf-8") as f:
|
||||||
|
task_dict = json.load(f)
|
||||||
|
result_task:AgentTask = AgentTask.from_dict(task_dict)
|
||||||
|
if result_task:
|
||||||
|
relative_path = os.path.relpath(task_fullpath, self.root_path)
|
||||||
|
result_task.task_path = relative_path
|
||||||
|
else:
|
||||||
|
logger.error("_get_task_by_fullpath:%s,parse failed!",detail_path)
|
||||||
|
|
||||||
|
return result_task
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("_get_task_by_fullpath:%s,failed:%s",task_fullpath,e)
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def get_task_by_path(self,task_path:str) -> AgentTask:
|
||||||
|
full_path = f"{self.root_path}/{task_path}"
|
||||||
|
return await self._get_task_by_fullpath(full_path)
|
||||||
|
|
||||||
|
async def get_todo(self,todo_id:str) -> AgentTodo:
|
||||||
|
todo_path = self._get_obj_path(todo_id)
|
||||||
|
if todo_path is None:
|
||||||
|
logger.error("get_todo:%s,not found!",todo_id)
|
||||||
|
return None
|
||||||
|
|
||||||
|
try:
|
||||||
|
with open(todo_path, mode='r', encoding="utf-8") as f:
|
||||||
|
todo_dict = json.load(f)
|
||||||
|
result_todo:AgentTodo = AgentTodo.from_dict(todo_dict)
|
||||||
|
if result_todo:
|
||||||
|
result_todo.todo_path = todo_path
|
||||||
|
else:
|
||||||
|
logger.error("get_todo:%s,parse failed!",todo_path)
|
||||||
|
|
||||||
|
return result_todo
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("get_todo:%s,failed:%s",todo_path,e)
|
||||||
|
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def get_sub_tasks(self,task_id:str) -> List[AgentTask]:
|
||||||
|
task_path = self._get_obj_path(task_id)
|
||||||
|
|
||||||
|
if task_path is None:
|
||||||
|
return []
|
||||||
|
task_path = f"{self.root_path}/{task_path}"
|
||||||
|
sub_tasks = []
|
||||||
|
for sub_item in os.listdir(task_path):
|
||||||
|
if sub_item.startswith("."):
|
||||||
|
continue
|
||||||
|
if sub_item == "workspace":
|
||||||
|
continue
|
||||||
|
|
||||||
|
full_path = os.path.join(task_path, sub_item)
|
||||||
|
if os.path.isdir(full_path):
|
||||||
|
sub_task = await self.get_task_by_path(f"{task_path}/{sub_item}")
|
||||||
|
if sub_task:
|
||||||
|
sub_tasks.append(sub_task)
|
||||||
|
|
||||||
|
return sub_tasks
|
||||||
|
|
||||||
|
|
||||||
|
async def get_sub_todos(self,task_id:str) -> List[AgentTodo]:
|
||||||
|
task_path = self._get_obj_path(task_id)
|
||||||
|
if task_path is None:
|
||||||
|
return []
|
||||||
|
task_path = f"{self.root_path}/{task_path}"
|
||||||
|
sub_todos = []
|
||||||
|
for sub_item in os.listdir(task_path):
|
||||||
|
if sub_item.startswith("."):
|
||||||
|
continue
|
||||||
|
if sub_item == "workspace":
|
||||||
|
continue
|
||||||
|
if sub_item == "details":
|
||||||
|
continue
|
||||||
|
|
||||||
|
full_path = os.path.join(task_path, sub_item)
|
||||||
|
if os.path.isfile(full_path) and sub_item.endswith(".todo"):
|
||||||
|
sub_todo = await self.get_todo_by_path(full_path)
|
||||||
|
if sub_todo:
|
||||||
|
sub_todos.append(sub_todo)
|
||||||
|
|
||||||
|
return sub_todos
|
||||||
|
|
||||||
|
async def get_todo_by_path(self,todo_path:str) -> AgentTodo:
|
||||||
|
async with aiofiles.open(todo_path, mode='r', encoding="utf-8") as f:
|
||||||
|
s = await f.read()
|
||||||
|
todo_dict = json.loads(s)
|
||||||
|
result_todo:AgentTodo = AgentTodo.from_dict(todo_dict)
|
||||||
|
if result_todo:
|
||||||
|
result_todo.todo_path = todo_path
|
||||||
|
else:
|
||||||
|
logger.error("get_todo_by_path:%s,parse failed!",todo_path)
|
||||||
|
|
||||||
|
return result_todo
|
||||||
|
|
||||||
|
#async def get_task_depends(self,task_id:str) -> List[AgentTask]:
|
||||||
|
# pass
|
||||||
|
|
||||||
|
|
||||||
|
async def list_task(self,filter:Optional[dict] = None ) -> List[AgentTask]:
|
||||||
|
directory_path = self.root_path
|
||||||
|
result_list:List[AgentTask] = []
|
||||||
|
special_state = None
|
||||||
|
if filter:
|
||||||
|
special_state = filter.get("state")
|
||||||
|
#agent_id = filter.get("agent_id")
|
||||||
|
|
||||||
|
for entry in os.scandir(directory_path):
|
||||||
|
if not entry.is_dir():
|
||||||
|
continue
|
||||||
|
if entry.name.startswith("."):
|
||||||
|
continue
|
||||||
|
if entry.name == "workspace":
|
||||||
|
continue
|
||||||
|
task_item = await self._get_task_by_fullpath(entry.path)
|
||||||
|
if task_item:
|
||||||
|
if filter is None:
|
||||||
|
if task_item.is_finish():
|
||||||
|
continue
|
||||||
|
|
||||||
|
if special_state:
|
||||||
|
if task_item.state != special_state:
|
||||||
|
continue
|
||||||
|
|
||||||
|
|
||||||
|
result_list.append(task_item)
|
||||||
|
|
||||||
|
return result_list
|
||||||
|
|
||||||
|
|
||||||
|
async def update_task(self,task:AgentTask):
|
||||||
|
detail_path = f"{self.root_path}/{task.task_path}/detail"
|
||||||
|
try:
|
||||||
|
new_task_content = json.dumps(task.to_dict(),ensure_ascii=False)
|
||||||
|
async with aiofiles.open(detail_path, mode='w', encoding="utf-8") as f:
|
||||||
|
await f.write(new_task_content)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("update_task failed:%s",e)
|
||||||
|
return str(e)
|
||||||
|
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def update_todo(self,todo:AgentTodo):
|
||||||
|
todo_path = self._get_obj_path(todo.todo_id)
|
||||||
|
if todo_path is None:
|
||||||
|
return f"todo {todo.todo_id} not found"
|
||||||
|
|
||||||
|
try:
|
||||||
|
new_todo_content = json.dumps(todo.to_dict(),ensure_ascii=False)
|
||||||
|
async with aiofiles.open(todo_path, mode='w', encoding="utf-8") as f:
|
||||||
|
await f.write(new_todo_content)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("update_todo failed:%s",e)
|
||||||
|
return str(e)
|
||||||
|
|
||||||
|
return None
|
||||||
|
|
||||||
|
#async def update_task_state(self,task_id,state:str):
|
||||||
|
# pass
|
||||||
|
|
||||||
|
#async def update_todo_state(self,task_id,state:str):
|
||||||
|
# pass
|
||||||
|
|
||||||
|
#todo共享其所在task的文件夹
|
||||||
|
# if task_id is none, means root folder in workspace
|
||||||
|
def _get_taskfile_path(self,task_id:str,path:str)->str:
|
||||||
|
root_path = self.root_path
|
||||||
|
if task_id is None:
|
||||||
|
root_path = f"{root_path}/workspace"
|
||||||
|
else:
|
||||||
|
task_path = self._get_obj_path(task_id)
|
||||||
|
if task_path is None:
|
||||||
|
return None
|
||||||
|
root_path = f"{task_path}/wrorkspace"
|
||||||
|
|
||||||
|
file_path = f"{root_path}/{path}"
|
||||||
|
return file_path
|
||||||
|
|
||||||
|
async def read_task_file(self,task_id:str,path:str)->str:
|
||||||
|
file_path = self._get_taskfile_path(task_id,path)
|
||||||
|
if not os.path.exists(file_path):
|
||||||
|
return None
|
||||||
|
|
||||||
|
try:
|
||||||
|
async with aiofiles.open(file_path, mode='r', encoding="utf-8") as f:
|
||||||
|
content = await f.read()
|
||||||
|
return content
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("read_task_file failed:%s",e)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
async def write_task_file(self,task_id:str,path:str,content:str):
|
||||||
|
file_path = self._get_taskfile_path(task_id,path)
|
||||||
|
# write file
|
||||||
|
try:
|
||||||
|
dir_name = os.path.dirname(file_path)
|
||||||
|
os.makedirs(dir_name)
|
||||||
|
async with aiofiles.open(file_path, mode='w', encoding="utf-8") as f:
|
||||||
|
await f.write(content)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("write_task_file failed:%s",e)
|
||||||
|
return str(e)
|
||||||
|
|
||||||
|
async def append_task_file(self,task_id:str,path:str,content:str):
|
||||||
|
file_path = self._get_taskfile_path(task_id,path)
|
||||||
|
# append file
|
||||||
|
try:
|
||||||
|
async with aiofiles.open(file_path, mode='a', encoding="utf-8") as f:
|
||||||
|
await f.write(content)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("append_task_file failed:%s",e)
|
||||||
|
return str(e)
|
||||||
|
|
||||||
|
|
||||||
|
async def list_task_dir(self,task_id:str,path:str) -> List[str]:
|
||||||
|
dir_path = self._get_taskfile_path(task_id,path)
|
||||||
|
if not os.path.exists(dir_path):
|
||||||
|
return None
|
||||||
|
|
||||||
|
try:
|
||||||
|
result_node = os.listdir(dir_path)
|
||||||
|
result = []
|
||||||
|
for name in result_node:
|
||||||
|
if name.startswith("."):
|
||||||
|
continue
|
||||||
|
|
||||||
|
result.append(name)
|
||||||
|
return result
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("list_task_dir failed:%s",e)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
async def remove_task_file(self,task_id:str,path:str):
|
||||||
|
file_path = self._get_taskfile_path(task_id,path)
|
||||||
|
try:
|
||||||
|
os.remove(file_path)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("remove_task_file failed:%s",e)
|
||||||
|
return str(e)
|
||||||
|
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
class AgentWorkspace:
|
||||||
|
def __init__(self,owner_id:str) -> None:
|
||||||
|
self.owner_id : str = owner_id
|
||||||
|
self.task_mgr : AgentTaskManager = LocalAgentTaskManger(owner_id)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def register_ai_functions():
|
||||||
|
async def post_message(parameters):
|
||||||
|
_agent_id = parameters.get("_agentid")
|
||||||
|
if _agent_id is None:
|
||||||
|
return "_agentid not found"
|
||||||
|
|
||||||
|
target = parameters.get("target")
|
||||||
|
if target is None:
|
||||||
|
return "target not found"
|
||||||
|
message = parameters.get("message")
|
||||||
|
if message is None:
|
||||||
|
return "message not found"
|
||||||
|
topic = parameters.get("topic")
|
||||||
|
|
||||||
|
msg = AgentMsg()
|
||||||
|
msg.sender = _agent_id
|
||||||
|
msg.body = message
|
||||||
|
msg.topic = topic
|
||||||
|
msg.target = target
|
||||||
|
msg.create_time = time.time()
|
||||||
|
|
||||||
|
is_post_ok = await AIBus.get_default_bus().post_message(msg)
|
||||||
|
if is_post_ok:
|
||||||
|
return "post message ok!"
|
||||||
|
else:
|
||||||
|
return f"post message to {target} failed!"
|
||||||
|
|
||||||
|
parameters = ParameterDefine.create_parameters({
|
||||||
|
"target": {"type": "string", "description": "target agent/contact fullname or telephone or email"},
|
||||||
|
"topic": {"type": "string", "description": "optional, message topic"},
|
||||||
|
"message": {"type": "string", "description": "message content"},
|
||||||
|
})
|
||||||
|
post_message_action = SimpleAIFunction(
|
||||||
|
"post_message",
|
||||||
|
"Post a message to target agent/contact",
|
||||||
|
post_message,
|
||||||
|
parameters,
|
||||||
|
)
|
||||||
|
GlobaToolsLibrary.get_instance().register_tool_function(post_message_action)
|
||||||
|
|
||||||
|
async def send_message(parameters):
|
||||||
|
_agent_id = parameters.get("_agentid")
|
||||||
|
if _agent_id is None:
|
||||||
|
return "_agentid not found"
|
||||||
|
|
||||||
|
target = parameters.get("target")
|
||||||
|
if target is None:
|
||||||
|
return "target not found"
|
||||||
|
message = parameters.get("message")
|
||||||
|
if message is None:
|
||||||
|
return "message not found"
|
||||||
|
topic = parameters.get("topic")
|
||||||
|
|
||||||
|
msg = AgentMsg()
|
||||||
|
msg.sender = _agent_id
|
||||||
|
msg.body = message
|
||||||
|
msg.topic = topic
|
||||||
|
msg.target = target
|
||||||
|
msg.create_time = time.time()
|
||||||
|
|
||||||
|
resp = await AIBus.get_default_bus().send_message(msg)
|
||||||
|
if resp:
|
||||||
|
return f"resp is : {resp.body}"
|
||||||
|
else:
|
||||||
|
return f"send message to {target} failed!"
|
||||||
|
|
||||||
|
parameters = ParameterDefine.create_parameters({
|
||||||
|
"target": {"type": "string", "description": "target agent/contact id"},
|
||||||
|
"topic": {"type": "string", "description": "optional, message topic"},
|
||||||
|
"message": {"type": "string", "description": "message content"},
|
||||||
|
})
|
||||||
|
send_message_action = SimpleAIFunction(
|
||||||
|
"send_message",
|
||||||
|
"send a message to target agent/contact, and wait for reply",
|
||||||
|
send_message,
|
||||||
|
parameters,
|
||||||
|
)
|
||||||
|
GlobaToolsLibrary.get_instance().register_tool_function(send_message_action)
|
||||||
|
|
||||||
|
async def create_task(params):
|
||||||
|
_workspace = params.get("_workspace")
|
||||||
|
_agent_id = params.get("_agentid")
|
||||||
|
if _workspace is None:
|
||||||
|
return "_workspace not found"
|
||||||
|
if params.get("creator") is None:
|
||||||
|
params["creator"] = _agent_id
|
||||||
|
taskObj = AgentTask.create_by_dict(params)
|
||||||
|
parent_id = params.get("parent")
|
||||||
|
return await _workspace.task_mgr.create_task(taskObj,parent_id)
|
||||||
|
parameters = ParameterDefine.create_parameters({
|
||||||
|
"title" : {"type": "string", "description": "task title,Simple and clear, try to include the task \ Related personnel \ place \ key conditions \ time element involved in the event"},
|
||||||
|
"detail" : {"type": "string", "description": "task detail(simple task can not be filled)"},
|
||||||
|
"priority" : {"type": "int", "description": "task priority from 1-10"},
|
||||||
|
#"due_date": {"type": "isoformat time string", "description": "optional,confirm task due date"},
|
||||||
|
#"expiration_time": {"type": "isoformat time string", "description": "optional,confirm task expiration time"},
|
||||||
|
"parent": {"type": "string", "description": "optional,parent task id"},
|
||||||
|
})
|
||||||
|
create_task_action = SimpleAIFunction(
|
||||||
|
"agent.workspace.create_task",
|
||||||
|
"Create a task",
|
||||||
|
create_task,
|
||||||
|
parameters,
|
||||||
|
)
|
||||||
|
GlobaToolsLibrary.get_instance().register_tool_function(create_task_action)
|
||||||
|
|
||||||
|
|
||||||
|
async def cancel_task(parameters):
|
||||||
|
_workspace = parameters.get("_workspace")
|
||||||
|
if _workspace is None:
|
||||||
|
return "_workspace not found"
|
||||||
|
task_id = parameters.get("task_id")
|
||||||
|
task : AgentTask = await _workspace.task_mgr.get_task(task_id)
|
||||||
|
if task is None:
|
||||||
|
return f"task {task_id} not found"
|
||||||
|
task.state = AgentTaskState.TASK_STATE_CANCEL
|
||||||
|
await _workspace.task_mgr.update_task(task)
|
||||||
|
return "canncel task ok"
|
||||||
|
|
||||||
|
parameters = ParameterDefine.create_parameters({
|
||||||
|
"task_id": {"type": "string", "description": "task id which want to cancel"},
|
||||||
|
})
|
||||||
|
cancel_task_action = SimpleAIFunction(
|
||||||
|
"agent.workspace.cancel_task",
|
||||||
|
"Cancel this task",
|
||||||
|
cancel_task,
|
||||||
|
parameters
|
||||||
|
)
|
||||||
|
GlobaToolsLibrary.get_instance().register_tool_function(cancel_task_action)
|
||||||
|
|
||||||
|
|
||||||
|
async def confirm_task(parameters):
|
||||||
|
_workspace = parameters.get("_workspace")
|
||||||
|
if _workspace is None:
|
||||||
|
return "_workspace not found"
|
||||||
|
task_id = parameters.get("task_id")
|
||||||
|
task : AgentTask = await _workspace.task_mgr.get_task(task_id)
|
||||||
|
if task is None:
|
||||||
|
return f"task {task_id} not found"
|
||||||
|
if parameters.get("priority"):
|
||||||
|
task.priority = parameters.get("priority")
|
||||||
|
if parameters.get("next_attention_time"):
|
||||||
|
task.next_attention_time = parameters.get("next_attention_time")
|
||||||
|
if parameters.get("expiration_time"):
|
||||||
|
task.expiration_time = parameters.get("expiration_time")
|
||||||
|
if parameters.get("due_date"):
|
||||||
|
task.due_date = parameters.get("due_date")
|
||||||
|
task.state = AgentTaskState.TASK_STATE_CONFIRMED
|
||||||
|
await _workspace.task_mgr.update_task(task)
|
||||||
|
return "confirm task ok"
|
||||||
|
parameters = ParameterDefine.create_parameters({
|
||||||
|
"task_id": {"type": "string", "description": "task id which want to confirm"},
|
||||||
|
"next_attention_time": {"type": "isoformat time string", "description": "optional,confirm task next attention time"},
|
||||||
|
"expiration_time": {"type": "isoformat time string", "description": "optional,confirm task expiration time"},
|
||||||
|
#"due_date": {"type": "isoformat time string", "description": "optional,confirm task due date"},
|
||||||
|
"priority": {"type": "int", "description": "optional,task priority from 1-10"},
|
||||||
|
})
|
||||||
|
confirm_task_action = SimpleAIFunction(
|
||||||
|
"agent.workspace.confirm_task",
|
||||||
|
"After understanding the content of the task, the importance of the importance of the task, the priority, the deadline, etc.",
|
||||||
|
confirm_task,
|
||||||
|
parameters
|
||||||
|
)
|
||||||
|
GlobaToolsLibrary.get_instance().register_tool_function(confirm_task_action)
|
||||||
|
|
||||||
|
async def list_task(parameters):
|
||||||
|
_workspace = parameters.get("_workspace")
|
||||||
|
if _workspace is None:
|
||||||
|
return "_workspace not found"
|
||||||
|
all_task = await _workspace.task_mgr.list_task(None)
|
||||||
|
if all_task:
|
||||||
|
return json.dumps([task.to_dict() for task in all_task],ensure_ascii=False)
|
||||||
|
else :
|
||||||
|
return "no task"
|
||||||
|
list_task_ai_function = SimpleAIFunction("agent.workspace.list_task",
|
||||||
|
"list all tasks in json format",
|
||||||
|
list_task,{})
|
||||||
|
GlobaToolsLibrary.get_instance().register_tool_function(list_task_ai_function)
|
||||||
|
|
||||||
|
async def update_task(parameters):
|
||||||
|
_workspace : AgentWorkspace= parameters.get("_workspace")
|
||||||
|
if _workspace is None:
|
||||||
|
return "_workspace not found"
|
||||||
|
task_id = parameters.get("task_id")
|
||||||
|
task:AgentTask = await _workspace.task_mgr.get_task(task_id)
|
||||||
|
if task is None:
|
||||||
|
return f"task {task_id} not found"
|
||||||
|
if parameters.get("title"):
|
||||||
|
task.title = parameters.get("title")
|
||||||
|
if parameters.get("detail"):
|
||||||
|
task.detail = parameters.get("detail")
|
||||||
|
if parameters.get("priority"):
|
||||||
|
task.priority = parameters.get("priority")
|
||||||
|
if parameters.get("new_state"):
|
||||||
|
task.state = AgentTaskState.from_str(parameters.get("new_state"))
|
||||||
|
if parameters.get("next_attention_time"):
|
||||||
|
task.next_attention_time = parameters.get("next_attention_time")
|
||||||
|
if parameters.get("due_date"):
|
||||||
|
task.due_date = parameters.get("due_date")
|
||||||
|
if parameters.get("expiration_time"):
|
||||||
|
task.expiration_time = parameters.get("expiration_time")
|
||||||
|
await _workspace.task_mgr.update_task(task)
|
||||||
|
return "update task ok"
|
||||||
|
parameters = ParameterDefine.create_parameters({
|
||||||
|
"task_id": {"type": "string", "description": "task id which want to update"},
|
||||||
|
"new_state": {"type": "string", "description": "optional,new task state: cancel or done"},
|
||||||
|
"next_attention_time": {"type": "isoformat time string", "description": "optional,update task next attention time"},
|
||||||
|
"expiration_time": {"type": "isoformat time string", "description": "optional,update task expiration time"},
|
||||||
|
"priority": {"type": "int", "description": "optional,task priority from 1-10"},
|
||||||
|
"title": {"type": "string", "description": "optional, new task title"},
|
||||||
|
"detail": {"type": "string", "description": "optional, new task detail(simple task can not be filled)"},
|
||||||
|
#"due_date": {"type": "string", "description": "optional,new task due date"},
|
||||||
|
})
|
||||||
|
update_task_ai_function = SimpleAIFunction("agent.workspace.update_task",
|
||||||
|
"update task to new state",
|
||||||
|
update_task,parameters)
|
||||||
|
GlobaToolsLibrary.get_instance().register_tool_function(update_task_ai_function)
|
||||||
|
|
||||||
|
async def set_todos(parameters):
|
||||||
|
_workspace : AgentWorkspace= parameters.get("_workspace")
|
||||||
|
if _workspace is None:
|
||||||
|
return "_workspace not found"
|
||||||
|
task_id = parameters.get("task_id")
|
||||||
|
task:AgentTask = await _workspace.task_mgr.get_task(task_id)
|
||||||
|
if task is None:
|
||||||
|
return f"task {task_id} not found"
|
||||||
|
todos = parameters.get("todos")
|
||||||
|
if todos is None:
|
||||||
|
return "todos not found"
|
||||||
|
await _workspace.task_mgr.set_todos(task_id,todos)
|
||||||
|
return "set todos ok"
|
||||||
|
|
||||||
|
todo_demo = """
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"title": "todo1",
|
||||||
|
"detail": "todo1 detail",
|
||||||
|
"tags": "tag1,tag2",
|
||||||
|
"due_date": "2021-01-01",
|
||||||
|
"priority": 1
|
||||||
|
},
|
||||||
|
]
|
||||||
|
"""
|
||||||
|
parameters = ParameterDefine.create_parameters({
|
||||||
|
"task_id": {"type": "string", "description": "task id which want to set todos"},
|
||||||
|
"todos": {"type": "list", "description": f"List of todo, todo is a dict like {todo_demo}"},
|
||||||
|
})
|
||||||
|
set_todos_ai_function = SimpleAIFunction("agent.workspace.set_todos",
|
||||||
|
"set todos for task",
|
||||||
|
set_todos,parameters)
|
||||||
|
GlobaToolsLibrary.get_instance().register_tool_function(set_todos_ai_function)
|
||||||
|
|
||||||
|
async def update_todo(parameters):
|
||||||
|
_workspace : AgentWorkspace= parameters.get("_workspace")
|
||||||
|
if _workspace is None:
|
||||||
|
return "_workspace not found"
|
||||||
|
todo_id = parameters.get("todo_id")
|
||||||
|
todo : AgentTodo = await _workspace.task_mgr.get_todo(todo_id)
|
||||||
|
if todo is None:
|
||||||
|
return f"todo {todo_id} not found"
|
||||||
|
|
||||||
|
parameters = ParameterDefine.create_parameters({
|
||||||
|
"todo_id": {"type": "string", "description": "todo id which want to update"},
|
||||||
|
"new_state": {"type": "string", "description": "optional,new todo state: execute_ok , execute_failed, done or check_failed"},
|
||||||
|
})
|
||||||
|
update_todo_ai_function = SimpleAIFunction("agent.workspace.update_todo",
|
||||||
|
"update todo to new state",
|
||||||
|
update_todo,parameters)
|
||||||
|
GlobaToolsLibrary.get_instance().register_tool_function(update_todo_ai_function)
|
||||||
|
|
||||||
|
|
||||||
|
# write file
|
||||||
|
async def write_task_file(parameters):
|
||||||
|
_workspace : AgentWorkspace= parameters.get("_workspace")
|
||||||
|
if _workspace is None:
|
||||||
|
return "_workspace not found"
|
||||||
|
task_id = parameters.get("task_id")
|
||||||
|
path = parameters.get("filename")
|
||||||
|
content = parameters.get("content")
|
||||||
|
await _workspace.task_mgr.write_task_file(task_id,path,content)
|
||||||
|
return "write task file ok"
|
||||||
|
parameters = ParameterDefine.create_parameters({
|
||||||
|
"filename": {"type": "string", "description": "filename"},
|
||||||
|
"content": {"type": "string", "description": "file content"},
|
||||||
|
})
|
||||||
|
write_task_file_ai_function = SimpleAIFunction("agent.workspace.write_file",
|
||||||
|
"write file for task",
|
||||||
|
write_task_file,parameters)
|
||||||
|
GlobaToolsLibrary.get_instance().register_tool_function(write_task_file_ai_function)
|
||||||
|
|
||||||
|
# append file
|
||||||
|
async def append_task_file(parameters):
|
||||||
|
_workspace : AgentWorkspace= parameters.get("_workspace")
|
||||||
|
if _workspace is None:
|
||||||
|
return "_workspace not found"
|
||||||
|
task_id = parameters.get("task_id")
|
||||||
|
path = parameters.get("filename")
|
||||||
|
content = parameters.get("content")
|
||||||
|
await _workspace.task_mgr.append_task_file(task_id,path,content)
|
||||||
|
return "append task file ok"
|
||||||
|
parameters = ParameterDefine.create_parameters({
|
||||||
|
"filename": {"type": "string", "description": "filename"},
|
||||||
|
"content": {"type": "string", "description": "file content"},
|
||||||
|
})
|
||||||
|
append_task_file_ai_function = SimpleAIFunction("agent.workspace.append_file",
|
||||||
|
"append file for task",
|
||||||
|
append_task_file,parameters)
|
||||||
|
GlobaToolsLibrary.get_instance().register_tool_function(append_task_file_ai_function)
|
||||||
|
|
||||||
|
# read file
|
||||||
|
async def read_task_file(parameters):
|
||||||
|
_workspace : AgentWorkspace= parameters.get("_workspace")
|
||||||
|
if _workspace is None:
|
||||||
|
return "_workspace not found"
|
||||||
|
task_id = parameters.get("task_id")
|
||||||
|
path = parameters.get("filename")
|
||||||
|
content = await _workspace.task_mgr.read_task_file(task_id,path)
|
||||||
|
return content
|
||||||
|
parameters = ParameterDefine.create_parameters({
|
||||||
|
"filename": {"type": "string", "description": "filename"},
|
||||||
|
})
|
||||||
|
read_task_file_ai_function = SimpleAIFunction("agent.workspace.read_file",
|
||||||
|
"read file for task",
|
||||||
|
read_task_file,parameters)
|
||||||
|
GlobaToolsLibrary.get_instance().register_tool_function(read_task_file_ai_function)
|
||||||
|
|
||||||
|
# list dir
|
||||||
|
async def list_task_dir(parameters):
|
||||||
|
_workspace : AgentWorkspace= parameters.get("_workspace")
|
||||||
|
if _workspace is None:
|
||||||
|
return "_workspace not found"
|
||||||
|
task_id = parameters.get("task_id")
|
||||||
|
path = parameters.get("path")
|
||||||
|
content = await _workspace.task_mgr.list_task_dir(task_id,path)
|
||||||
|
return content
|
||||||
|
parameters = ParameterDefine.create_parameters({
|
||||||
|
"path": {"type": "string", "description": "The relative path of the dir"},
|
||||||
|
})
|
||||||
|
list_task_dir_ai_function = SimpleAIFunction("agent.workspace.list_dir",
|
||||||
|
"list dir in task workspace",
|
||||||
|
list_task_dir,parameters)
|
||||||
|
GlobaToolsLibrary.get_instance().register_tool_function(list_task_dir_ai_function)
|
||||||
|
|
||||||
|
# remove file
|
||||||
|
async def remove_task_file(parameters):
|
||||||
|
_workspace : AgentWorkspace= parameters.get("_workspace")
|
||||||
|
if _workspace is None:
|
||||||
|
return "_workspace not found"
|
||||||
|
task_id = parameters.get("task_id")
|
||||||
|
path = parameters.get("filename")
|
||||||
|
content = await _workspace.task_mgr.remove_task_file(task_id,path)
|
||||||
|
return content
|
||||||
|
parameters = ParameterDefine.create_parameters({
|
||||||
|
"filename": {"type": "string", "description": "filename"},
|
||||||
|
})
|
||||||
|
remove_task_file_ai_function = SimpleAIFunction("agent.workspace.remove_file",
|
||||||
|
"remove file for task",
|
||||||
|
remove_task_file,parameters)
|
||||||
|
GlobaToolsLibrary.get_instance().register_tool_function(remove_task_file_ai_function)
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,57 @@
|
|||||||
|
# pylint:disable=E0402
|
||||||
|
import logging
|
||||||
|
from typing import Dict
|
||||||
|
|
||||||
|
from ..proto.ai_function import *
|
||||||
|
from ..agent.llm_context import GlobaToolsLibrary
|
||||||
|
from ..frame.compute_kernel import ComputeKernel
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
class AsrFunction(AIFunction):
|
||||||
|
def __init__(self):
|
||||||
|
self.func_id = "aigc.voice_to_text"
|
||||||
|
self.description = "Voice recognition, convert the voice into text"
|
||||||
|
self.parameters = ParameterDefine.create_parameters({
|
||||||
|
"audio_file": {"type": "string", "description": "Audio file path"},
|
||||||
|
"model": {"type": "string", "description": "Recognition model", "enum": ["openai-whisper"]},
|
||||||
|
"prompt": {"type": "string", "description": "Prompt statement, can be None"},
|
||||||
|
"response_format": {"type": "string", "description": "Return format", "enum": ["text", "json", "srt", "verbose_json", "vtt"]},
|
||||||
|
})
|
||||||
|
|
||||||
|
def register_function(self):
|
||||||
|
GlobaToolsLibrary.get_instance().register_tool_function(self)
|
||||||
|
|
||||||
|
def get_id(self) -> str:
|
||||||
|
return self.func_id
|
||||||
|
|
||||||
|
def get_description(self) -> str:
|
||||||
|
return self.description
|
||||||
|
|
||||||
|
def get_parameters(self) -> Dict:
|
||||||
|
return self.parameters
|
||||||
|
|
||||||
|
async def execute(self, **kwargs) -> str:
|
||||||
|
logger.info(f"execute asr function: {kwargs}")
|
||||||
|
|
||||||
|
audio_file = kwargs.get("audio_file")
|
||||||
|
model = kwargs.get("model")
|
||||||
|
prompt = kwargs.get("prompt")
|
||||||
|
response_format = kwargs.get("response_format")
|
||||||
|
if response_format is None:
|
||||||
|
response_format = "text"
|
||||||
|
|
||||||
|
result = await ComputeKernel.get_instance().do_speech_to_text(audio_file, model, prompt, response_format)
|
||||||
|
if result is not None:
|
||||||
|
return f"exec speech_to_text Ok. {response_format} is\n```\n{result.result_str}\n```"
|
||||||
|
else:
|
||||||
|
return "exec speech_to_text failed"
|
||||||
|
|
||||||
|
def is_local(self) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
def is_in_zone(self) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
def is_ready_only(self) -> bool:
|
||||||
|
return False
|
||||||
@@ -0,0 +1,460 @@
|
|||||||
|
# pylint:disable=E0402
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import pathlib
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import re
|
||||||
|
import time
|
||||||
|
import ast
|
||||||
|
from concurrent.futures import ThreadPoolExecutor
|
||||||
|
from hashlib import md5
|
||||||
|
from typing import Optional, Union, List, Tuple
|
||||||
|
from generic_escape import GenericEscape
|
||||||
|
|
||||||
|
from ..storage.storage import AIStorage
|
||||||
|
|
||||||
|
try:
|
||||||
|
import docker
|
||||||
|
except ImportError:
|
||||||
|
docker = None
|
||||||
|
|
||||||
|
CODE_BLOCK_PATTERN = r"```[ \t]*(\w+)?[ \t]*\r?\n(.*?)\r?\n[ \t]*```"
|
||||||
|
UNKNOWN = "unknown"
|
||||||
|
TIMEOUT_MSG = "Timeout"
|
||||||
|
DEFAULT_TIMEOUT = 600
|
||||||
|
WIN32 = sys.platform == "win32"
|
||||||
|
PATH_SEPARATOR = WIN32 and "\\" or "/"
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
BUILT_IN_MODULES = set(
|
||||||
|
[
|
||||||
|
"sys",
|
||||||
|
"os",
|
||||||
|
"math",
|
||||||
|
"random",
|
||||||
|
"datetime",
|
||||||
|
"json",
|
||||||
|
"re",
|
||||||
|
"subprocess",
|
||||||
|
"time",
|
||||||
|
"threading",
|
||||||
|
"logging",
|
||||||
|
"collections",
|
||||||
|
"itertools",
|
||||||
|
"functools",
|
||||||
|
"operator",
|
||||||
|
"pathlib",
|
||||||
|
"shutil",
|
||||||
|
"tempfile",
|
||||||
|
"pickle",
|
||||||
|
"io",
|
||||||
|
"argparse",
|
||||||
|
"typing",
|
||||||
|
"unittest",
|
||||||
|
"contextlib",
|
||||||
|
"abc",
|
||||||
|
"heapq",
|
||||||
|
"bisect",
|
||||||
|
"copy",
|
||||||
|
"decimal",
|
||||||
|
"fractions",
|
||||||
|
"hashlib",
|
||||||
|
"secrets",
|
||||||
|
"statistics",
|
||||||
|
"difflib",
|
||||||
|
"doctest",
|
||||||
|
"enum",
|
||||||
|
"inspect",
|
||||||
|
"traceback",
|
||||||
|
"weakref",
|
||||||
|
"gc",
|
||||||
|
"mmap",
|
||||||
|
"msvcrt",
|
||||||
|
"winreg",
|
||||||
|
"array",
|
||||||
|
"audioop",
|
||||||
|
"binascii",
|
||||||
|
"cProfile",
|
||||||
|
"concurrent.futures",
|
||||||
|
"configparser",
|
||||||
|
"csv",
|
||||||
|
"ctypes",
|
||||||
|
"dateutil",
|
||||||
|
"dis",
|
||||||
|
"fnmatch",
|
||||||
|
"getopt",
|
||||||
|
"glob",
|
||||||
|
"gzip",
|
||||||
|
"pdb",
|
||||||
|
"pprint",
|
||||||
|
"profile",
|
||||||
|
"pstats",
|
||||||
|
"queue",
|
||||||
|
"socket",
|
||||||
|
"sqlite3",
|
||||||
|
"ssl",
|
||||||
|
"struct",
|
||||||
|
"tarfile",
|
||||||
|
"telnetlib",
|
||||||
|
"timeit",
|
||||||
|
"tokenize",
|
||||||
|
"uuid",
|
||||||
|
"xml",
|
||||||
|
"zipfile",
|
||||||
|
"zlib",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def get_imports(code: str) -> List[str]:
|
||||||
|
root = ast.parse(code)
|
||||||
|
|
||||||
|
imports = []
|
||||||
|
for node in ast.iter_child_nodes(root):
|
||||||
|
if isinstance(node, ast.Import):
|
||||||
|
module_names = [alias.name for alias in node.names]
|
||||||
|
elif isinstance(node, ast.ImportFrom):
|
||||||
|
module_names = [node.module]
|
||||||
|
else:
|
||||||
|
continue
|
||||||
|
|
||||||
|
for name in module_names:
|
||||||
|
# Exclude built-in modules
|
||||||
|
if name not in BUILT_IN_MODULES:
|
||||||
|
imports.append(name)
|
||||||
|
|
||||||
|
return imports
|
||||||
|
|
||||||
|
|
||||||
|
def write_requirements(code: str, requirements_filepath: str):
|
||||||
|
imports = get_imports(code)
|
||||||
|
|
||||||
|
with open(requirements_filepath, "w") as file:
|
||||||
|
for module in imports:
|
||||||
|
file.write(module + "\n")
|
||||||
|
|
||||||
|
|
||||||
|
def _cmd(lang):
|
||||||
|
if lang.startswith("python") or lang in ["bash", "sh", "powershell"]:
|
||||||
|
return lang
|
||||||
|
if lang in ["shell"]:
|
||||||
|
return "sh"
|
||||||
|
if lang in ["ps1"]:
|
||||||
|
return "powershell"
|
||||||
|
raise NotImplementedError(f"{lang} not recognized in code execution")
|
||||||
|
|
||||||
|
|
||||||
|
def create_runner(code: str, timeout: int = 30) -> str:
|
||||||
|
"""
|
||||||
|
Create a Python script that runs the code and prints the output
|
||||||
|
"""
|
||||||
|
code = GenericEscape().escape(code)
|
||||||
|
# Create a runner script
|
||||||
|
runner = f"""
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
|
||||||
|
my_env = os.environ.copy()
|
||||||
|
my_env["PYTHONIOENCODING"] = "utf-8"
|
||||||
|
|
||||||
|
process = subprocess.Popen(
|
||||||
|
f"python -i -q -u".split(),
|
||||||
|
stdin=subprocess.PIPE,
|
||||||
|
stdout=subprocess.PIPE,
|
||||||
|
stderr=subprocess.PIPE,
|
||||||
|
text=True,
|
||||||
|
bufsize=0,
|
||||||
|
universal_newlines=True,
|
||||||
|
env=my_env
|
||||||
|
)
|
||||||
|
|
||||||
|
process.stdin.write("{code}" + "\\n")
|
||||||
|
process.stdin.write("exit()\\n")
|
||||||
|
process.stdin.flush()
|
||||||
|
|
||||||
|
try:
|
||||||
|
process.wait({timeout})
|
||||||
|
except Exception as e:
|
||||||
|
process.terminate()
|
||||||
|
|
||||||
|
for line in iter(process.stdout.readline, ""):
|
||||||
|
print(line)
|
||||||
|
|
||||||
|
for line in iter(process.stderr.readline, ""):
|
||||||
|
if line.startswith(">>>"):
|
||||||
|
continue
|
||||||
|
print(line)
|
||||||
|
"""
|
||||||
|
return runner
|
||||||
|
|
||||||
|
|
||||||
|
def _run_cmd(cmd: [str], work_dir: str, timeout: int) -> str:
|
||||||
|
if WIN32:
|
||||||
|
logger.warning("SIGALRM is not supported on Windows. No timeout will be enforced.")
|
||||||
|
result = subprocess.run(
|
||||||
|
cmd,
|
||||||
|
cwd=work_dir,
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
with ThreadPoolExecutor(max_workers=1) as executor:
|
||||||
|
future = executor.submit(
|
||||||
|
subprocess.run,
|
||||||
|
cmd,
|
||||||
|
cwd=work_dir,
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
)
|
||||||
|
result = future.result(timeout=timeout)
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def execute_code(
|
||||||
|
code: Optional[str] = None,
|
||||||
|
timeout: Optional[int] = None,
|
||||||
|
filename: Optional[str] = None,
|
||||||
|
work_dir: Optional[str] = None,
|
||||||
|
use_docker: Optional[Union[List[str], str, bool]] = None,
|
||||||
|
lang: Optional[str] = "python",
|
||||||
|
) -> Tuple[int, str]:
|
||||||
|
"""Execute code in a docker container.
|
||||||
|
This function is not tested on MacOS.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
code (Optional, str): The code to execute.
|
||||||
|
If None, the code from the file specified by filename will be executed.
|
||||||
|
Either code or filename must be provided.
|
||||||
|
timeout (Optional, int): The maximum execution time in seconds.
|
||||||
|
If None, a default timeout will be used. The default timeout is 600 seconds. On Windows, the timeout is not enforced when use_docker=False.
|
||||||
|
filename (Optional, str): The file name to save the code or where the code is stored when `code` is None.
|
||||||
|
If None, a file with a randomly generated name will be created.
|
||||||
|
The randomly generated file will be deleted after execution.
|
||||||
|
The file name must be a relative path. Relative paths are relative to the working directory.
|
||||||
|
work_dir (Optional, str): The working directory for the code execution.
|
||||||
|
If None, a default working directory will be used.
|
||||||
|
The default working directory is the "extensions" directory under
|
||||||
|
"path_to_autogen".
|
||||||
|
use_docker (Optional, list, str or bool): The docker image to use for code execution.
|
||||||
|
If a list or a str of image name(s) is provided, the code will be executed in a docker container
|
||||||
|
with the first image successfully pulled.
|
||||||
|
If None, False or empty, the code will be executed in the current environment.
|
||||||
|
Default is None, which will be converted into an empty list when docker package is available.
|
||||||
|
Expected behaviour:
|
||||||
|
- If `use_docker` is explicitly set to True and the docker package is available, the code will run in a Docker container.
|
||||||
|
- If `use_docker` is explicitly set to True but the Docker package is missing, an error will be raised.
|
||||||
|
- If `use_docker` is not set (i.e., left default to None) and the Docker package is not available, a warning will be displayed, but the code will run natively.
|
||||||
|
If the code is executed in the current environment,
|
||||||
|
the code must be trusted.
|
||||||
|
lang (Optional, str): The language of the code. Default is "python".
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
int: 0 if the code executes successfully.
|
||||||
|
str: The error message if the code fails to execute; the stdout otherwise.
|
||||||
|
"""
|
||||||
|
if all((code is None, filename is None)):
|
||||||
|
error_msg = f"Either {code=} or {filename=} must be provided."
|
||||||
|
logger.error(error_msg)
|
||||||
|
raise AssertionError(error_msg)
|
||||||
|
|
||||||
|
# Warn if use_docker was unspecified (or None), and cannot be provided (the default).
|
||||||
|
# In this case the current behavior is to fall back to run natively, but this behavior
|
||||||
|
# is subject to change.
|
||||||
|
if use_docker is None:
|
||||||
|
if docker is None:
|
||||||
|
use_docker = False
|
||||||
|
logger.warning(
|
||||||
|
"execute_code was called without specifying a value for use_docker. Since the python docker package is not available, code will be run natively. Note: this fallback behavior is subject to change"
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
# Default to true
|
||||||
|
use_docker = True
|
||||||
|
|
||||||
|
timeout = timeout or DEFAULT_TIMEOUT
|
||||||
|
original_filename = filename
|
||||||
|
if WIN32 and lang in ["sh", "shell"] and (not use_docker):
|
||||||
|
lang = "ps1"
|
||||||
|
if filename is None:
|
||||||
|
code_hash = md5(code.encode()).hexdigest()
|
||||||
|
# create a file with a automatically generated name
|
||||||
|
filename = f"tmp_code_{code_hash}.{'py' if lang.startswith('python') else lang}"
|
||||||
|
if work_dir is None:
|
||||||
|
WORKING_DIR = os.path.join(AIStorage.get_instance().get_myai_dir(), "tmp_code")
|
||||||
|
pathlib.Path(WORKING_DIR).mkdir(exist_ok=True)
|
||||||
|
work_dir = os.path.join(WORKING_DIR, code_hash)
|
||||||
|
pathlib.Path(work_dir).mkdir(exist_ok=True)
|
||||||
|
filepath = os.path.join(work_dir, filename)
|
||||||
|
file_dir = os.path.dirname(filepath)
|
||||||
|
os.makedirs(file_dir, exist_ok=True)
|
||||||
|
if code is not None:
|
||||||
|
write_requirements(code, os.path.join(file_dir, "requirements.txt"))
|
||||||
|
code = create_runner(code, 30)
|
||||||
|
with open(filepath, "w", encoding="utf-8") as fout:
|
||||||
|
fout.write(code)
|
||||||
|
|
||||||
|
|
||||||
|
# check if already running in a docker container
|
||||||
|
in_docker_container = os.path.exists("/.dockerenv")
|
||||||
|
if not use_docker or in_docker_container:
|
||||||
|
try:
|
||||||
|
env_cmd = ["python", "-m", "venv", os.path.join(file_dir, "venv")]
|
||||||
|
_run_cmd(env_cmd, file_dir, timeout)
|
||||||
|
if WIN32:
|
||||||
|
venv_path = os.path.join(file_dir, "venv", "Scripts")
|
||||||
|
else:
|
||||||
|
venv_path = os.path.join(file_dir, "venv", "bin")
|
||||||
|
pip_cmd = [os.path.join(venv_path, "python"), "-m", "pip", "install", "-r", "requirements.txt"]
|
||||||
|
_run_cmd(pip_cmd, file_dir, timeout)
|
||||||
|
# already running in a docker container
|
||||||
|
cmd = [
|
||||||
|
os.path.join(venv_path, "python"),
|
||||||
|
f".\\{filename}" if WIN32 else filename,
|
||||||
|
]
|
||||||
|
result = _run_cmd(cmd, file_dir, timeout)
|
||||||
|
except TimeoutError:
|
||||||
|
if original_filename is None:
|
||||||
|
shutil.rmtree(os.path.join(file_dir, "venv"))
|
||||||
|
os.remove(filepath)
|
||||||
|
os.remove(os.path.join(file_dir, "requirements.txt"))
|
||||||
|
try:
|
||||||
|
os.removedirs(file_dir)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
return 1, TIMEOUT_MSG
|
||||||
|
if original_filename is None:
|
||||||
|
shutil.rmtree(os.path.join(file_dir, "venv"))
|
||||||
|
os.remove(filepath)
|
||||||
|
os.remove(os.path.join(file_dir, "requirements.txt"))
|
||||||
|
try:
|
||||||
|
os.removedirs(file_dir)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
if result.returncode:
|
||||||
|
logs = result.stderr
|
||||||
|
if original_filename is None:
|
||||||
|
abs_path = str(pathlib.Path(filepath).absolute())
|
||||||
|
logs = logs.replace(str(abs_path), "").replace(filename, "")
|
||||||
|
else:
|
||||||
|
abs_path = str(pathlib.Path(work_dir).absolute()) + PATH_SEPARATOR
|
||||||
|
logs = logs.replace(str(abs_path), "")
|
||||||
|
else:
|
||||||
|
logs = result.stdout
|
||||||
|
return result.returncode, logs
|
||||||
|
|
||||||
|
# create a docker client
|
||||||
|
client = docker.from_env()
|
||||||
|
image_list = (
|
||||||
|
["python:3-alpine", "python:3", "python:3-windowsservercore"]
|
||||||
|
if use_docker is True
|
||||||
|
else [use_docker]
|
||||||
|
if isinstance(use_docker, str)
|
||||||
|
else use_docker
|
||||||
|
)
|
||||||
|
for image in image_list:
|
||||||
|
# check if the image exists
|
||||||
|
try:
|
||||||
|
client.images.get(image)
|
||||||
|
break
|
||||||
|
except docker.errors.ImageNotFound:
|
||||||
|
# pull the image
|
||||||
|
logger.info("Pulling image", image)
|
||||||
|
try:
|
||||||
|
client.images.pull(image, stream=True, decode=True)
|
||||||
|
break
|
||||||
|
except docker.errors.DockerException as e:
|
||||||
|
logger.error("Failed to pull image", image)
|
||||||
|
logger.exception(e)
|
||||||
|
# get a randomized str based on current time to wrap the exit code
|
||||||
|
exit_code_str = f"exitcode{time.time()}"
|
||||||
|
start_str = f'start{time.time()}'
|
||||||
|
abs_path = pathlib.Path(work_dir).absolute()
|
||||||
|
cmd = [
|
||||||
|
"sh",
|
||||||
|
"-c",
|
||||||
|
f"pip install --quiet -r requirements.txt; echo -n {start_str}; {_cmd(lang)} {filename}; exit_code=$?; echo -n {exit_code_str}; echo -n $exit_code; echo {exit_code_str};",
|
||||||
|
]
|
||||||
|
# create a docker container
|
||||||
|
container = client.containers.run(
|
||||||
|
image,
|
||||||
|
command=cmd,
|
||||||
|
working_dir="/workspace",
|
||||||
|
detach=True,
|
||||||
|
# get absolute path to the working directory
|
||||||
|
volumes={abs_path: {"bind": "/workspace", "mode": "rw"}},
|
||||||
|
)
|
||||||
|
start_time = time.time()
|
||||||
|
while container.status != "exited" and time.time() - start_time < timeout:
|
||||||
|
# Reload the container object
|
||||||
|
container.reload()
|
||||||
|
if container.status != "exited":
|
||||||
|
container.stop()
|
||||||
|
container.remove()
|
||||||
|
if original_filename is None:
|
||||||
|
os.remove(filepath)
|
||||||
|
return 1, TIMEOUT_MSG, image
|
||||||
|
# get the container logs
|
||||||
|
logs: str = container.logs().decode("utf-8").rstrip()
|
||||||
|
start_pos = logs.find(start_str)
|
||||||
|
if start_pos != -1:
|
||||||
|
logs = logs[start_pos + len(start_str):]
|
||||||
|
# # commit the image
|
||||||
|
# tag = filename.replace("/", "")
|
||||||
|
# container.commit(repository="python", tag=tag)
|
||||||
|
# remove the container
|
||||||
|
container.remove()
|
||||||
|
# check if the code executed successfully
|
||||||
|
exit_code = container.attrs["State"]["ExitCode"]
|
||||||
|
if exit_code == 0:
|
||||||
|
# extract the exit code from the logs
|
||||||
|
pattern = re.compile(f"{exit_code_str}(\\d+){exit_code_str}")
|
||||||
|
match = pattern.search(logs)
|
||||||
|
exit_code = 1 if match is None else int(match.group(1))
|
||||||
|
# remove the exit code from the logs
|
||||||
|
logs = logs if match is None else pattern.sub("", logs)
|
||||||
|
|
||||||
|
if original_filename is None:
|
||||||
|
os.remove(filepath)
|
||||||
|
os.remove(os.path.join(file_dir, "requirements.txt"))
|
||||||
|
os.removedirs(file_dir)
|
||||||
|
if exit_code:
|
||||||
|
logs = logs.replace(f"/workspace/{filename if original_filename is None else ''}", "")
|
||||||
|
# return the exit code, logs and image
|
||||||
|
return exit_code, logs
|
||||||
|
|
||||||
|
class CodeInterpreterFunction(AIFunction):
|
||||||
|
def __init__(self):
|
||||||
|
self.func_id = "system.code_interpreter"
|
||||||
|
self.description = "execute python code"
|
||||||
|
self.parameters = ParameterDefine.create_parameters({
|
||||||
|
"code": {"type": "string", "description": "python code"}
|
||||||
|
})
|
||||||
|
|
||||||
|
def get_name(self) -> str:
|
||||||
|
return self.func_id
|
||||||
|
|
||||||
|
def get_description(self) -> str:
|
||||||
|
return self.description
|
||||||
|
|
||||||
|
def get_parameters(self) -> Dict:
|
||||||
|
return self.parameters
|
||||||
|
|
||||||
|
async def execute(self, **kwargs) -> str:
|
||||||
|
code = kwargs.get("code")
|
||||||
|
ret_code, result = execute_code(code=code)
|
||||||
|
if ret_code == 0:
|
||||||
|
return result.strip()
|
||||||
|
else:
|
||||||
|
return result.strip()
|
||||||
|
|
||||||
|
def is_local(self) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
def is_in_zone(self) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
def is_ready_only(self) -> bool:
|
||||||
|
return False
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
# pylint:disable=E0402
|
||||||
|
import json
|
||||||
|
from typing import Dict
|
||||||
|
|
||||||
|
from ..proto.ai_function import *
|
||||||
|
from ..agent.llm_context import GlobaToolsLibrary
|
||||||
|
from duckduckgo_search import AsyncDDGS
|
||||||
|
|
||||||
|
|
||||||
|
class DuckDuckGoTextSearchFunction(AIFunction):
|
||||||
|
def __init__(self):
|
||||||
|
self.name = "web.search.duckduckgo"
|
||||||
|
self.description = "Search web by text (powered by DuckDuckGo)"
|
||||||
|
self.region = "wt-wt"
|
||||||
|
self.safesearch = "moderate"
|
||||||
|
self.time = "y"
|
||||||
|
self.max_results = 5
|
||||||
|
self.parameters = ParameterDefine.create_parameters({
|
||||||
|
"query": {"type": "string", "description": "The query to search for."}
|
||||||
|
})
|
||||||
|
|
||||||
|
def register_function(self):
|
||||||
|
GlobaToolsLibrary.get_instance().register_tool_function(self)
|
||||||
|
|
||||||
|
def get_id(self) -> str:
|
||||||
|
return self.name
|
||||||
|
|
||||||
|
def get_description(self) -> str:
|
||||||
|
return self.description
|
||||||
|
|
||||||
|
def get_parameters(self) -> Dict:
|
||||||
|
return self.parameters
|
||||||
|
|
||||||
|
async def execute(self, **kwargs) -> str:
|
||||||
|
query = kwargs.get("query")
|
||||||
|
|
||||||
|
async with AsyncDDGS() as ddgs:
|
||||||
|
results = [r async for r in ddgs.text(
|
||||||
|
query,
|
||||||
|
region=self.region,
|
||||||
|
safesearch=self.safesearch,
|
||||||
|
timelimit=self.time,
|
||||||
|
backend="api",
|
||||||
|
max_results=self.max_results
|
||||||
|
)]
|
||||||
|
|
||||||
|
return json.dumps(results,,ensure_ascii=False)
|
||||||
|
|
||||||
|
def is_local(self) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
def is_in_zone(self) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
def is_ready_only(self) -> bool:
|
||||||
|
return False
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
# pylint:disable=E0402
|
||||||
|
import logging
|
||||||
|
from typing import Dict
|
||||||
|
|
||||||
|
from ..frame.compute_kernel import ComputeKernel
|
||||||
|
from ..proto.ai_function import *
|
||||||
|
from ..agent.llm_context import GlobaToolsLibrary
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
class Image2TextFunction(AIFunction):
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self.func_id = "aigc.image_2_text"
|
||||||
|
self.description = "According to the input image file address, return the description of the image content"
|
||||||
|
self.parameters = ParameterDefine.create_parameters({
|
||||||
|
"image_path": {"type": "string", "description": "image file path"}
|
||||||
|
})
|
||||||
|
logger.info(f"init Image2TextFunction")
|
||||||
|
|
||||||
|
def register_function(self):
|
||||||
|
GlobaToolsLibrary.get_instance().register_tool_function(self)
|
||||||
|
|
||||||
|
def get_id(self) -> str:
|
||||||
|
return self.func_id
|
||||||
|
|
||||||
|
def get_description(self) -> str:
|
||||||
|
return self.description
|
||||||
|
|
||||||
|
def get_parameters(self) -> Dict:
|
||||||
|
return self.parameters
|
||||||
|
|
||||||
|
async def execute(self, **kwargs) -> str:
|
||||||
|
logger.info(f"execute image_2_text function: {kwargs}")
|
||||||
|
image_path = kwargs.get("image_path")
|
||||||
|
data = await ComputeKernel.get_instance().do_image_2_text(image_path, '')
|
||||||
|
try:
|
||||||
|
result = data['message']['choices'][0]['message']['content']
|
||||||
|
except (KeyError, TypeError, IndexError):
|
||||||
|
logger.error(f"image_2_text error: {data}")
|
||||||
|
result = ""
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def is_local(self) -> bool:
|
||||||
|
return False
|
||||||
|
|
||||||
|
def is_in_zone(self) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
def is_ready_only(self) -> bool:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
+45
-36
@@ -1,57 +1,66 @@
|
|||||||
|
# pylint:disable=E0402
|
||||||
|
|
||||||
import io
|
import io
|
||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
import random
|
import random
|
||||||
|
from pathlib import Path
|
||||||
from typing import Dict
|
from typing import Dict
|
||||||
|
|
||||||
from aios_kernel import ComputeKernel
|
from ..proto.ai_function import *
|
||||||
from aios_kernel.ai_function import AIFunction
|
from ..frame.compute_kernel import ComputeKernel
|
||||||
|
from ..storage.storage import AIStorage
|
||||||
|
|
||||||
|
|
||||||
from pydub import AudioSegment
|
from pydub import AudioSegment
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
class TextToSpeechFunction(AIFunction):
|
|
||||||
def __init__(self):
|
|
||||||
self.func_id = "text_to_speech"
|
|
||||||
self.description = "根据输入的剧本生成音频文件，成功时会返回音频文件路径"
|
|
||||||
|
|
||||||
def get_name(self) -> str:
|
class ScriptToSpeechFunction(AIFunction):
|
||||||
|
def __init__(self):
|
||||||
|
self.func_id = "aigc.script_to_speech"
|
||||||
|
self.description = "Generate audio files according to the input script, and the audio file path will be returned when successful"
|
||||||
|
self.speech_path = os.path.join(AIStorage.get_instance().get_myai_dir(), "tts")
|
||||||
|
self.parameters = ParameterDefine.create_parameters({
|
||||||
|
"language": {"type": "string", "description": "Actual language", "enum": ["zh", "en"]},
|
||||||
|
"model": {"type": "string", "description": "Studio", "enum": ["tts-1", "tts-1-hd"]},
|
||||||
|
"roles": {"type": "array", "items": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"name": {"type": "string", "description": "Character name"},
|
||||||
|
"gender": {"type": "string", "description": "Gender", "enum": ["man", "female"]},
|
||||||
|
"age": {"type": "string", "description": "age", "enum": ["child", "adult"]},
|
||||||
|
}}},
|
||||||
|
"lines": {"type": "array", "items": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"name": {"type": "string", "description": "Character name"},
|
||||||
|
"tone": {"type": "string", "description": "Sovereign emotions",
|
||||||
|
"enum": ["happy", "sad", "angry", "fear", "disgust", "surprise", "neutral"]},
|
||||||
|
"text": {"type": "string", "description": "Line"},
|
||||||
|
}
|
||||||
|
}}
|
||||||
|
})
|
||||||
|
|
||||||
|
Path(self.speech_path).mkdir(exist_ok=True)
|
||||||
|
|
||||||
|
def get_id(self) -> str:
|
||||||
return self.func_id
|
return self.func_id
|
||||||
|
|
||||||
def get_description(self) -> str:
|
def get_description(self) -> str:
|
||||||
return self.description
|
return self.description
|
||||||
|
|
||||||
def get_parameters(self) -> Dict:
|
def get_parameters(self):
|
||||||
return {
|
return self.parameters
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"language": {"type": "string", "description": "演播语言", "enum": ["zh", "en"]},
|
|
||||||
"roles": {"type": "array", "items": {
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"name": {"type": "string", "description": "角色名字"},
|
|
||||||
"gender": {"type": "string", "description": "角色性别", "enum": ["man", "female"]},
|
|
||||||
"age": {"type": "string", "description": "年龄", "enum": ["child", "adult"]},
|
|
||||||
}}},
|
|
||||||
"lines": {"type": "array", "items": {
|
|
||||||
"type": "object",
|
|
||||||
"properties": {
|
|
||||||
"name": {"type": "string", "description": "角色名字"},
|
|
||||||
"tone": {"type": "string", "description": "演播情感",
|
|
||||||
"enum": ["happy", "sad", "angry", "fear", "disgust", "surprise", "neutral"]},
|
|
||||||
"text": {"type": "string", "description": "台词"},
|
|
||||||
}
|
|
||||||
}}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
async def execute(self, **kwargs) -> str:
|
async def execute(self, **kwargs) -> str:
|
||||||
logger.info(f"execute text_to_speech function: {kwargs}")
|
logger.info(f"execute aigc.script_to_speech function: {kwargs}")
|
||||||
|
|
||||||
language = kwargs.get("language")
|
language = kwargs.get("language")
|
||||||
if language is None:
|
if language is None:
|
||||||
language = "zh"
|
language = "zh"
|
||||||
|
model = kwargs.get("model")
|
||||||
roles = kwargs.get("roles")
|
roles = kwargs.get("roles")
|
||||||
lines = kwargs.get("lines")
|
lines = kwargs.get("lines")
|
||||||
|
|
||||||
@@ -71,23 +80,23 @@ class TextToSpeechFunction(AIFunction):
|
|||||||
i = 0
|
i = 0
|
||||||
while i < 3:
|
while i < 3:
|
||||||
try:
|
try:
|
||||||
data = await ComputeKernel.get_instance().do_text_to_speech(text, language, gender, age, name, tone)
|
data = await ComputeKernel.get_instance().do_text_to_speech(text, language, gender, age, name, tone, model_name=model)
|
||||||
if audio is None:
|
if audio is None:
|
||||||
audio = AudioSegment.from_mp3(io.BytesIO(data))
|
audio = AudioSegment.from_mp3(io.BytesIO(data))
|
||||||
else:
|
else:
|
||||||
audio = audio + AudioSegment.from_mp3(io.BytesIO(data))
|
audio = audio + AudioSegment.from_mp3(io.BytesIO(data))
|
||||||
break
|
break
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"do_text_to_speech failed: {e}")
|
logger.error(f"script_to_speech failed: {e}")
|
||||||
i += 1
|
i += 1
|
||||||
continue
|
continue
|
||||||
|
|
||||||
if audio is not None:
|
if audio is not None:
|
||||||
path = os.path.join(os.path.realpath(os.curdir), "{}.mp3".format(''.join(random.sample('zyxwvutsrqponmlkjihgfedcba', 10))))
|
path = os.path.join(self.speech_path, "{}.mp3".format(''.join(random.sample('zyxwvutsrqponmlkjihgfedcba', 10))))
|
||||||
audio.export(path, format="mp3")
|
audio.export(path, format="mp3")
|
||||||
return "exec text_to_speech OK，speech file store at {}".format(path)
|
return "exec script_to_speech OK，speech file store at ```{}```".format(path)
|
||||||
else:
|
else:
|
||||||
return "exec text_to_speech failed"
|
return "exec script_to_speech failed"
|
||||||
|
|
||||||
def is_local(self) -> bool:
|
def is_local(self) -> bool:
|
||||||
return True
|
return True
|
||||||
@@ -0,0 +1,113 @@
|
|||||||
|
# pylint:disable=E0402
|
||||||
|
from datetime import timedelta, datetime
|
||||||
|
from typing import Dict
|
||||||
|
|
||||||
|
from cachetools import TLRUCache, cached
|
||||||
|
|
||||||
|
from ..proto.ai_function import *
|
||||||
|
from ..environment.sql_database import SQLDatabase, get_from_env
|
||||||
|
|
||||||
|
|
||||||
|
def _my_ttu(_key, _value, now):
|
||||||
|
return now + timedelta(seconds=600)
|
||||||
|
|
||||||
|
|
||||||
|
database_cache = TLRUCache(ttu=_my_ttu, maxsize=10000, timer=datetime.now)
|
||||||
|
|
||||||
|
|
||||||
|
@cached(cache=database_cache)
|
||||||
|
def get_database(uri: str) -> SQLDatabase:
|
||||||
|
return SQLDatabase.from_uri(uri)
|
||||||
|
|
||||||
|
|
||||||
|
class GetTableInfosFunction(AIFunction):
|
||||||
|
def __init__(self):
|
||||||
|
super().__init__()
|
||||||
|
self.name = "get_table_infos"
|
||||||
|
self.description = "Get table informations in the database"
|
||||||
|
|
||||||
|
def get_id(self) -> str:
|
||||||
|
return self.name
|
||||||
|
|
||||||
|
def get_description(self) -> str:
|
||||||
|
return self.description
|
||||||
|
|
||||||
|
def get_parameters(self) -> Dict:
|
||||||
|
return {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"database_url": {"type": "string", "description": "Database URL,Can be set to None"},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async def execute(self, **kwargs) -> str:
|
||||||
|
database_url: str = kwargs.get("database_url")
|
||||||
|
if (database_url is None
|
||||||
|
or database_url.strip() == ""
|
||||||
|
or database_url.strip().lower() == "none"
|
||||||
|
or database_url.strip().lower() == "null"):
|
||||||
|
database_url = get_from_env(key="database url", env_key="DATABASE_URL")
|
||||||
|
if database_url is None:
|
||||||
|
return "error: database_url is None"
|
||||||
|
database = get_database(database_url)
|
||||||
|
tables = database.get_usable_table_names()
|
||||||
|
table_infos = database.get_table_info(tables)
|
||||||
|
return table_infos
|
||||||
|
|
||||||
|
def is_local(self) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
def is_in_zone(self) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
def is_ready_only(self) -> bool:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
class ExecuteSqlFunction(AIFunction):
|
||||||
|
def __init__(self):
|
||||||
|
super().__init__()
|
||||||
|
self.name = "execute_sql"
|
||||||
|
self.description = """
|
||||||
|
Input to this function is a detailed and correct SQL query, output is a result from the database.
|
||||||
|
If the query is not correct, an error message will be returned.
|
||||||
|
If an error is returned, rewrite the query, check the query, and try again.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def get_id(self) -> str:
|
||||||
|
return self.name
|
||||||
|
|
||||||
|
def get_description(self) -> str:
|
||||||
|
return self.description
|
||||||
|
|
||||||
|
def get_parameters(self) -> Dict:
|
||||||
|
return {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"database_url": {"type": "string", "description": "Database URL,Can be set to None"},
|
||||||
|
"sql": {"type": "string", "description": "SQL to execute"}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async def execute(self, **kwargs) -> str:
|
||||||
|
database_url = kwargs.get("database_url")
|
||||||
|
if (database_url is None
|
||||||
|
or database_url.strip() == ""
|
||||||
|
or database_url.strip().lower() == "none"
|
||||||
|
or database_url.strip().lower() == "null"):
|
||||||
|
database_url = get_from_env(key="database url", env_key="DATABASE_URL")
|
||||||
|
if database_url is None:
|
||||||
|
return "error: database_url is None"
|
||||||
|
sql = kwargs.get("sql")
|
||||||
|
|
||||||
|
database = get_database(database_url)
|
||||||
|
return database.run_no_throw(sql)
|
||||||
|
|
||||||
|
def is_local(self) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
def is_in_zone(self) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
def is_ready_only(self) -> bool:
|
||||||
|
return False
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
import io
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import random
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Dict
|
||||||
|
|
||||||
|
from ..proto.ai_function import *
|
||||||
|
from ..frame.compute_kernel import ComputeKernel
|
||||||
|
from ..storage.storage import AIStorage
|
||||||
|
|
||||||
|
|
||||||
|
from pydub import AudioSegment
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class TextToSpeechFunction(AIFunction):
|
||||||
|
def __init__(self):
|
||||||
|
self.func_id = "aigc.text_to_speech"
|
||||||
|
self.description = "To generate audio files according to the input text, the audio file path will be returned when successful"
|
||||||
|
self.speech_path = os.path.join(AIStorage.get_instance().get_myai_dir(), "tts")
|
||||||
|
self.parameters = ParameterDefine.create_parameters({
|
||||||
|
"language": {"type": "string", "description": "Actual language", "enum": ["zh", "en"]},
|
||||||
|
"model": {"type": "string", "description": "Studio", "enum": ["tts-1", "tts-1-hd"]},
|
||||||
|
"text": {"type": "string", "description": "text"}
|
||||||
|
})
|
||||||
|
Path(self.speech_path).mkdir(exist_ok=True)
|
||||||
|
|
||||||
|
def get_id(self) -> str:
|
||||||
|
return self.func_id
|
||||||
|
|
||||||
|
def get_description(self) -> str:
|
||||||
|
return self.description
|
||||||
|
|
||||||
|
def get_parameters(self) -> Dict:
|
||||||
|
return self.parameters
|
||||||
|
|
||||||
|
async def execute(self, **kwargs) -> str:
|
||||||
|
logger.info(f"execute aigc.text_to_speech function: {kwargs}")
|
||||||
|
|
||||||
|
language = kwargs.get("language")
|
||||||
|
if language is None:
|
||||||
|
language = "en"
|
||||||
|
model = kwargs.get("model")
|
||||||
|
text = kwargs.get("text")
|
||||||
|
|
||||||
|
i = 0
|
||||||
|
while i < 3:
|
||||||
|
try:
|
||||||
|
data = await ComputeKernel.get_instance().do_text_to_speech(text, language, None, None, None, None,
|
||||||
|
model_name=model)
|
||||||
|
if data is not None:
|
||||||
|
audio = AudioSegment.from_mp3(io.BytesIO(data))
|
||||||
|
break
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"do_text_to_speech failed: {e}")
|
||||||
|
i += 1
|
||||||
|
continue
|
||||||
|
|
||||||
|
if audio is not None:
|
||||||
|
path = os.path.join(self.speech_path, "{}.mp3".format(''.join(random.sample('zyxwvutsrqponmlkjihgfedcba', 10))))
|
||||||
|
audio.export(path, format="mp3")
|
||||||
|
return "exec text_to_speech OK，speech file store at ```{}```".format(path)
|
||||||
|
else:
|
||||||
|
return "exec text_to_speech failed"
|
||||||
|
|
||||||
|
def is_local(self) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
def is_in_zone(self) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
def is_ready_only(self) -> bool:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,123 @@
|
|||||||
|
# basic environment class
|
||||||
|
# we have some built-in environment: Calender(include timer),Home(connect to IoT device in your home), ,KnwoledgeBase,FileSystem,
|
||||||
|
# pylint:disable=E0402
|
||||||
|
from abc import ABC, abstractmethod
|
||||||
|
from typing import Any, Callable, Optional,Dict,Awaitable,List
|
||||||
|
import logging
|
||||||
|
from ..proto.ai_function import *
|
||||||
|
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class BaseEnvironment:
|
||||||
|
@classmethod
|
||||||
|
def get_env_by_id(cls,env_id:str)->'BaseEnvironment':
|
||||||
|
if cls.all_env is None:
|
||||||
|
cls.all_env = {}
|
||||||
|
return cls.all_env.get(env_id)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def register_env(cls,env_id:str,env:'BaseEnvironment')->None:
|
||||||
|
if cls.all_env is None:
|
||||||
|
cls.all_env = {}
|
||||||
|
cls.all_env[env_id] = env
|
||||||
|
|
||||||
|
|
||||||
|
def __init__(self, workspace: str) -> None:
|
||||||
|
pass
|
||||||
|
|
||||||
|
# @abstractmethod
|
||||||
|
# #TODO: how to use env? different env has different prompt
|
||||||
|
# def get_env_prompt(self) -> str:
|
||||||
|
# pass
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def get_ai_function(self,func_name:str) -> AIFunction:
|
||||||
|
pass
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def get_all_ai_functions(self) -> List[AIFunction]:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def get_ai_operation(self,op_name:str) -> AIAction:
|
||||||
|
pass
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def get_all_ai_operations(self) -> List[AIAction]:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def __getitem__(self, key):
|
||||||
|
return self.get_value(key)
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def get_value(self,key:str) -> Optional[str]:
|
||||||
|
pass
|
||||||
|
|
||||||
|
# _all_env = {}
|
||||||
|
# @classmethod
|
||||||
|
# def get_env_by_id(cls,env_id:str):
|
||||||
|
# return cls._all_env.get(env_id)
|
||||||
|
|
||||||
|
# @classmethod
|
||||||
|
# def set_env_by_id(cls,id,env):
|
||||||
|
# assert id == env.get_id()
|
||||||
|
# cls._all_env[env.get_id()] = env
|
||||||
|
|
||||||
|
class SimpleEnvironment(BaseEnvironment):
|
||||||
|
def __init__(self, workspace: str) -> None:
|
||||||
|
super().__init__(workspace)
|
||||||
|
self.functions: Dict[str,AIFunction] = {}
|
||||||
|
self.operations: Dict[str,AIAction] = {}
|
||||||
|
|
||||||
|
def add_ai_function(self,func:AIFunction) -> None:
|
||||||
|
self.functions[func.get_id()] = func
|
||||||
|
|
||||||
|
def get_ai_function(self,func_name:str) -> AIFunction:
|
||||||
|
func = self.functions.get(func_name)
|
||||||
|
if func is not None:
|
||||||
|
return func
|
||||||
|
return None
|
||||||
|
|
||||||
|
def get_all_ai_functions(self) -> List[AIFunction]:
|
||||||
|
func_list = []
|
||||||
|
func_list.extend(self.functions.values())
|
||||||
|
return func_list
|
||||||
|
|
||||||
|
def add_ai_operation(self,op:AIAction) -> None:
|
||||||
|
self.operations[op.get_id()] = op
|
||||||
|
|
||||||
|
def get_ai_operation(self,op_name:str) -> AIAction:
|
||||||
|
op = self.operations.get(op_name)
|
||||||
|
if op is not None:
|
||||||
|
return op
|
||||||
|
return None
|
||||||
|
|
||||||
|
def get_all_ai_operations(self) -> List[AIAction]:
|
||||||
|
op_list = []
|
||||||
|
op_list.extend(self.operations.values())
|
||||||
|
return op_list
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
class CompositeEnvironment(SimpleEnvironment):
|
||||||
|
def __init__(self, workspace: str) -> None:
|
||||||
|
super().__init__(workspace)
|
||||||
|
self.envs: List[BaseEnvironment] = []
|
||||||
|
|
||||||
|
def add_env(self, env: BaseEnvironment) -> None:
|
||||||
|
self.envs.append(env)
|
||||||
|
functions = env.get_all_ai_functions()
|
||||||
|
for func in functions:
|
||||||
|
self.functions[func.get_id()] = func
|
||||||
|
operations = env.get_all_ai_operations()
|
||||||
|
for op in operations:
|
||||||
|
self.operations[op.get_id()] = op
|
||||||
|
|
||||||
|
def get_value(self,key:str) -> Optional[str]:
|
||||||
|
for env in self.envs:
|
||||||
|
val = env.get_value(key)
|
||||||
|
if val is not None:
|
||||||
|
return val
|
||||||
@@ -0,0 +1,228 @@
|
|||||||
|
# pylint:disable=E0402
|
||||||
|
import sqlite3
|
||||||
|
import json
|
||||||
|
import threading
|
||||||
|
import logging
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
from typing import Optional, List
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
class SimpleKnowledgeDB:
|
||||||
|
def __init__(self,db_path:str):
|
||||||
|
self.db_path = db_path
|
||||||
|
self._get_conn()
|
||||||
|
|
||||||
|
def _get_conn(self):
|
||||||
|
""" get db connection """
|
||||||
|
local = threading.local()
|
||||||
|
if not hasattr(local, 'conn'):
|
||||||
|
local.conn = self._create_connection(self.db_path)
|
||||||
|
return local.conn
|
||||||
|
|
||||||
|
|
||||||
|
def _create_connection(self, db_file):
|
||||||
|
""" create a database connection to a SQLite database """
|
||||||
|
conn = None
|
||||||
|
try:
|
||||||
|
conn = sqlite3.connect(db_file)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("Error occurred while connecting to database: %s", e)
|
||||||
|
return None
|
||||||
|
|
||||||
|
if conn:
|
||||||
|
self._create_tables(conn)
|
||||||
|
|
||||||
|
return conn
|
||||||
|
|
||||||
|
def _create_tables(self,conn):
|
||||||
|
cursor = conn.cursor()
|
||||||
|
cursor.execute('''
|
||||||
|
CREATE TABLE IF NOT EXISTS documents (
|
||||||
|
doc_path TEXT PRIMARY KEY,
|
||||||
|
length INTEGER,
|
||||||
|
last_modify TEXT,
|
||||||
|
doc_hash TEXT,
|
||||||
|
create_time TEXT
|
||||||
|
)
|
||||||
|
''')
|
||||||
|
cursor.execute('''
|
||||||
|
CREATE TABLE IF NOT EXISTS knowledge (
|
||||||
|
doc_hash TEXT PRIMARY KEY,
|
||||||
|
title TEXT,
|
||||||
|
summary TEXT,
|
||||||
|
content TEXT,
|
||||||
|
catalogs TEXT,
|
||||||
|
tags TEXT,
|
||||||
|
llm_title TEXT,
|
||||||
|
llm_summary TEXT,
|
||||||
|
create_time TEXT
|
||||||
|
)
|
||||||
|
''')
|
||||||
|
|
||||||
|
cursor.execute('''
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_documents_doc_hash
|
||||||
|
ON documents (doc_hash)
|
||||||
|
''')
|
||||||
|
|
||||||
|
cursor.execute('''
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_knowledge_tags
|
||||||
|
ON knowledge (tags)
|
||||||
|
''')
|
||||||
|
|
||||||
|
conn.commit()
|
||||||
|
|
||||||
|
def add_doc(self, doc_path: str, length: int, last_modify: str, doc_hash: Optional[str] = None):
|
||||||
|
conn = self._get_conn()
|
||||||
|
cursor = conn.cursor()
|
||||||
|
create_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||||||
|
cursor.execute('''
|
||||||
|
INSERT INTO documents (doc_path, length, last_modify, doc_hash,create_time)
|
||||||
|
VALUES (?, ?, ?, ?,?)
|
||||||
|
''', (doc_path, length, last_modify, doc_hash,create_time))
|
||||||
|
conn.commit()
|
||||||
|
|
||||||
|
def is_doc_exist(self, doc_path: str) -> bool:
|
||||||
|
conn = self._get_conn()
|
||||||
|
cursor = conn.cursor()
|
||||||
|
cursor.execute('''
|
||||||
|
SELECT doc_path
|
||||||
|
FROM documents
|
||||||
|
WHERE doc_path = ?
|
||||||
|
''', (doc_path,))
|
||||||
|
return len(cursor.fetchall()) > 0
|
||||||
|
|
||||||
|
def set_doc_hash(self, doc_path: str, doc_hash: str):
|
||||||
|
conn = self._get_conn()
|
||||||
|
cursor = conn.cursor()
|
||||||
|
cursor.execute('''
|
||||||
|
UPDATE documents
|
||||||
|
SET doc_hash = ?
|
||||||
|
WHERE doc_path = ?
|
||||||
|
''', (doc_hash, doc_path))
|
||||||
|
conn.commit()
|
||||||
|
|
||||||
|
def get_docs_without_hash(self,limit:int=1024) -> List[str]:
|
||||||
|
conn = self._get_conn()
|
||||||
|
cursor = conn.cursor()
|
||||||
|
cursor.execute('''
|
||||||
|
SELECT doc_path
|
||||||
|
FROM documents
|
||||||
|
WHERE doc_hash IS NULL OR doc_hash = ''
|
||||||
|
ORDER BY create_time DESC
|
||||||
|
LIMIT ?
|
||||||
|
''',(limit,))
|
||||||
|
return [row[0] for row in cursor.fetchall()]
|
||||||
|
|
||||||
|
#metadata["summary"]
|
||||||
|
#metadata["catelogs"]
|
||||||
|
#metadata["tags"]
|
||||||
|
def add_knowledge(self, doc_hash: str, title: str, metadata: dict,content:str = None,):
|
||||||
|
conn = self._get_conn()
|
||||||
|
cursor = conn.cursor()
|
||||||
|
|
||||||
|
create_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||||||
|
summary = metadata.get("summary", "")
|
||||||
|
catalogs = metadata.get("catalogs","")
|
||||||
|
tags = ','.join(metadata.get("tags", []))
|
||||||
|
|
||||||
|
cursor.execute('''
|
||||||
|
INSERT INTO knowledge (doc_hash, title , summary , catalogs , tags,create_time)
|
||||||
|
VALUES (?, ?, ?, ?, ?,?)
|
||||||
|
''', (doc_hash, title, summary, catalogs, tags,create_time))
|
||||||
|
conn.commit()
|
||||||
|
|
||||||
|
#llm_result["summary"]
|
||||||
|
#llm_result["tags"]
|
||||||
|
#llm_result["catelog"]
|
||||||
|
def set_knowledge_llm_result(self, doc_hash: str, llm_result: dict):
|
||||||
|
conn = self._get_conn()
|
||||||
|
cursor = conn.cursor()
|
||||||
|
|
||||||
|
title = llm_result.get("title", "")
|
||||||
|
summary = llm_result.get("summary", "")
|
||||||
|
catalogs = json.dumps(llm_result.get("catalogs", {}))
|
||||||
|
tags = ','.join(llm_result.get("tags", []))
|
||||||
|
|
||||||
|
cursor.execute('''
|
||||||
|
UPDATE knowledge
|
||||||
|
SET llm_title = ?,llm_summary = ?, catalogs = ?, tags = ?
|
||||||
|
WHERE doc_hash = ?
|
||||||
|
''', (title,summary, catalogs, tags, doc_hash))
|
||||||
|
conn.commit()
|
||||||
|
|
||||||
|
def get_hash_by_doc_path(self, doc_path: str) -> Optional[str]:
|
||||||
|
conn = self._get_conn()
|
||||||
|
cursor = conn.cursor()
|
||||||
|
cursor.execute('''
|
||||||
|
SELECT doc_hash
|
||||||
|
FROM documents
|
||||||
|
WHERE doc_path = ?
|
||||||
|
''', (doc_path,))
|
||||||
|
row = cursor.fetchone()
|
||||||
|
if row is None:
|
||||||
|
return None
|
||||||
|
return row[0]
|
||||||
|
|
||||||
|
def get_knowledge(self, doc_hash: str) -> Optional[dict]:
|
||||||
|
conn = self._get_conn()
|
||||||
|
cursor = conn.cursor()
|
||||||
|
cursor.execute('''
|
||||||
|
SELECT title, summary, catalogs, tags, llm_title, llm_summary
|
||||||
|
FROM knowledge
|
||||||
|
WHERE doc_hash = ?
|
||||||
|
''', (doc_hash,))
|
||||||
|
row = cursor.fetchone()
|
||||||
|
if row is None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
# get doc path
|
||||||
|
cursor.execute('''
|
||||||
|
SELECT doc_path
|
||||||
|
FROM documents
|
||||||
|
WHERE doc_hash = ?
|
||||||
|
''', (doc_hash,))
|
||||||
|
row2 = cursor.fetchone()
|
||||||
|
if row2 is None:
|
||||||
|
return None
|
||||||
|
doc_path = row2[0]
|
||||||
|
|
||||||
|
|
||||||
|
return {
|
||||||
|
"full_path": doc_path,
|
||||||
|
"title": row[0],
|
||||||
|
"summary": row[1],
|
||||||
|
"catalogs": row[2],
|
||||||
|
"tags": row[3],
|
||||||
|
"llm_title" : row[4],
|
||||||
|
"llm_summary" : row[5],
|
||||||
|
}
|
||||||
|
|
||||||
|
def get_knowledge_without_llm_title(self,limit:int=16) -> List[str]:
|
||||||
|
conn = self._get_conn()
|
||||||
|
cursor = conn.cursor()
|
||||||
|
cursor.execute('''
|
||||||
|
SELECT doc_hash
|
||||||
|
FROM knowledge
|
||||||
|
WHERE llm_title IS NULL OR llm_title = ''
|
||||||
|
ORDER BY create_time DESC
|
||||||
|
LIMIT ?
|
||||||
|
''',(limit,))
|
||||||
|
return [row[0] for row in cursor.fetchall()]
|
||||||
|
|
||||||
|
def query_docs_by_tag(self, tag: str) -> List[str]:
|
||||||
|
conn = self._get_conn()
|
||||||
|
cursor = conn.cursor()
|
||||||
|
tag_json = json.dumps(tag,ensure_ascii=False) # 将标签转换为 JSON 字符串
|
||||||
|
cursor.execute('''
|
||||||
|
SELECT documents.doc_path
|
||||||
|
FROM documents
|
||||||
|
JOIN knowledge ON documents.doc_hash = knowledge.doc_hash
|
||||||
|
WHERE json_extract(knowledge.tags, '$') LIKE ?
|
||||||
|
''', (tag))
|
||||||
|
return [row[0] for row in cursor.fetchall()]
|
||||||
|
|
||||||
|
def query(self,sql:str):
|
||||||
|
pass
|
||||||
|
#cursor = self.conn.cursor()
|
||||||
@@ -0,0 +1,495 @@
|
|||||||
|
"""
|
||||||
|
Taken from: langchain
|
||||||
|
SQLAlchemy wrapper around a database.
|
||||||
|
"""
|
||||||
|
# pylint:disable=E0402
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
import os
|
||||||
|
|
||||||
|
import warnings
|
||||||
|
from typing import Any, Dict, Iterable, List, Literal, Optional, Sequence, Union
|
||||||
|
|
||||||
|
import sqlalchemy
|
||||||
|
from sqlalchemy import MetaData, Table, create_engine, inspect, select, text
|
||||||
|
from sqlalchemy.engine import Engine
|
||||||
|
from sqlalchemy.exc import ProgrammingError, SQLAlchemyError
|
||||||
|
from sqlalchemy.schema import CreateTable
|
||||||
|
from sqlalchemy.types import NullType
|
||||||
|
|
||||||
|
|
||||||
|
def get_from_env(key: str, env_key: str, default: Optional[str] = None) -> str:
|
||||||
|
"""Get a value from a dictionary or an environment variable."""
|
||||||
|
if env_key in os.environ and os.environ[env_key]:
|
||||||
|
return os.environ[env_key]
|
||||||
|
elif default is not None:
|
||||||
|
return default
|
||||||
|
else:
|
||||||
|
raise ValueError(
|
||||||
|
f"Did not find {key}, please add an environment variable"
|
||||||
|
f" `{env_key}` which contains it, or pass"
|
||||||
|
f" `{key}` as a named parameter."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _format_index(index: sqlalchemy.engine.interfaces.ReflectedIndex) -> str:
|
||||||
|
return (
|
||||||
|
f'Name: {index["name"]}, Unique: {index["unique"]},'
|
||||||
|
f' Columns: {str(index["column_names"])}'
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def truncate_word(content: Any, *, length: int, suffix: str = "...") -> str:
|
||||||
|
"""
|
||||||
|
Truncate a string to a certain number of words, based on the max string
|
||||||
|
length.
|
||||||
|
"""
|
||||||
|
|
||||||
|
if not isinstance(content, str) or length <= 0:
|
||||||
|
return content
|
||||||
|
|
||||||
|
if len(content) <= length:
|
||||||
|
return content
|
||||||
|
|
||||||
|
return content[: length - len(suffix)].rsplit(" ", 1)[0] + suffix
|
||||||
|
|
||||||
|
|
||||||
|
class SQLDatabase:
|
||||||
|
"""SQLAlchemy wrapper around a database."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
engine: Engine,
|
||||||
|
schema: Optional[str] = None,
|
||||||
|
metadata: Optional[MetaData] = None,
|
||||||
|
ignore_tables: Optional[List[str]] = None,
|
||||||
|
include_tables: Optional[List[str]] = None,
|
||||||
|
sample_rows_in_table_info: int = 3,
|
||||||
|
indexes_in_table_info: bool = False,
|
||||||
|
custom_table_info: Optional[dict] = None,
|
||||||
|
view_support: bool = False,
|
||||||
|
max_string_length: int = 300,
|
||||||
|
):
|
||||||
|
"""Create engine from database URI."""
|
||||||
|
self._engine = engine
|
||||||
|
self._schema = schema
|
||||||
|
if include_tables and ignore_tables:
|
||||||
|
raise ValueError("Cannot specify both include_tables and ignore_tables")
|
||||||
|
|
||||||
|
self._inspector = inspect(self._engine)
|
||||||
|
|
||||||
|
# including view support by adding the views as well as tables to the all
|
||||||
|
# tables list if view_support is True
|
||||||
|
self._all_tables = set(
|
||||||
|
self._inspector.get_table_names(schema=schema)
|
||||||
|
+ (self._inspector.get_view_names(schema=schema) if view_support else [])
|
||||||
|
)
|
||||||
|
|
||||||
|
self._include_tables = set(include_tables) if include_tables else set()
|
||||||
|
if self._include_tables:
|
||||||
|
missing_tables = self._include_tables - self._all_tables
|
||||||
|
if missing_tables:
|
||||||
|
raise ValueError(
|
||||||
|
f"include_tables {missing_tables} not found in database"
|
||||||
|
)
|
||||||
|
self._ignore_tables = set(ignore_tables) if ignore_tables else set()
|
||||||
|
if self._ignore_tables:
|
||||||
|
missing_tables = self._ignore_tables - self._all_tables
|
||||||
|
if missing_tables:
|
||||||
|
raise ValueError(
|
||||||
|
f"ignore_tables {missing_tables} not found in database"
|
||||||
|
)
|
||||||
|
usable_tables = self.get_usable_table_names()
|
||||||
|
self._usable_tables = set(usable_tables) if usable_tables else self._all_tables
|
||||||
|
|
||||||
|
if not isinstance(sample_rows_in_table_info, int):
|
||||||
|
raise TypeError("sample_rows_in_table_info must be an integer")
|
||||||
|
|
||||||
|
self._sample_rows_in_table_info = sample_rows_in_table_info
|
||||||
|
self._indexes_in_table_info = indexes_in_table_info
|
||||||
|
|
||||||
|
self._custom_table_info = custom_table_info
|
||||||
|
if self._custom_table_info:
|
||||||
|
if not isinstance(self._custom_table_info, dict):
|
||||||
|
raise TypeError(
|
||||||
|
"table_info must be a dictionary with table names as keys and the "
|
||||||
|
"desired table info as values"
|
||||||
|
)
|
||||||
|
# only keep the tables that are also present in the database
|
||||||
|
intersection = set(self._custom_table_info).intersection(self._all_tables)
|
||||||
|
self._custom_table_info = dict(
|
||||||
|
(table, self._custom_table_info[table])
|
||||||
|
for table in self._custom_table_info
|
||||||
|
if table in intersection
|
||||||
|
)
|
||||||
|
|
||||||
|
self._max_string_length = max_string_length
|
||||||
|
|
||||||
|
self._metadata = metadata or MetaData()
|
||||||
|
# including view support if view_support = true
|
||||||
|
self._metadata.reflect(
|
||||||
|
views=view_support,
|
||||||
|
bind=self._engine,
|
||||||
|
only=list(self._usable_tables),
|
||||||
|
schema=self._schema,
|
||||||
|
)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_uri(
|
||||||
|
cls, database_uri: str, engine_args: Optional[dict] = None, **kwargs: Any
|
||||||
|
) -> SQLDatabase:
|
||||||
|
"""Construct a SQLAlchemy engine from URI."""
|
||||||
|
_engine_args = engine_args or {}
|
||||||
|
return cls(create_engine(database_uri, **_engine_args), **kwargs)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_databricks(
|
||||||
|
cls,
|
||||||
|
catalog: str,
|
||||||
|
schema: str,
|
||||||
|
host: Optional[str] = None,
|
||||||
|
api_token: Optional[str] = None,
|
||||||
|
warehouse_id: Optional[str] = None,
|
||||||
|
cluster_id: Optional[str] = None,
|
||||||
|
engine_args: Optional[dict] = None,
|
||||||
|
**kwargs: Any,
|
||||||
|
) -> SQLDatabase:
|
||||||
|
"""
|
||||||
|
Class method to create an SQLDatabase instance from a Databricks connection.
|
||||||
|
This method requires the 'databricks-sql-connector' package. If not installed,
|
||||||
|
it can be added using `pip install databricks-sql-connector`.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
catalog (str): The catalog name in the Databricks database.
|
||||||
|
schema (str): The schema name in the catalog.
|
||||||
|
host (Optional[str]): The Databricks workspace hostname, excluding
|
||||||
|
'https://' part. If not provided, it attempts to fetch from the
|
||||||
|
environment variable 'DATABRICKS_HOST'. If still unavailable and if
|
||||||
|
running in a Databricks notebook, it defaults to the current workspace
|
||||||
|
hostname. Defaults to None.
|
||||||
|
api_token (Optional[str]): The Databricks personal access token for
|
||||||
|
accessing the Databricks SQL warehouse or the cluster. If not provided,
|
||||||
|
it attempts to fetch from 'DATABRICKS_TOKEN'. If still unavailable
|
||||||
|
and running in a Databricks notebook, a temporary token for the current
|
||||||
|
user is generated. Defaults to None.
|
||||||
|
warehouse_id (Optional[str]): The warehouse ID in the Databricks SQL. If
|
||||||
|
provided, the method configures the connection to use this warehouse.
|
||||||
|
Cannot be used with 'cluster_id'. Defaults to None.
|
||||||
|
cluster_id (Optional[str]): The cluster ID in the Databricks Runtime. If
|
||||||
|
provided, the method configures the connection to use this cluster.
|
||||||
|
Cannot be used with 'warehouse_id'. If running in a Databricks notebook
|
||||||
|
and both 'warehouse_id' and 'cluster_id' are None, it uses the ID of the
|
||||||
|
cluster the notebook is attached to. Defaults to None.
|
||||||
|
engine_args (Optional[dict]): The arguments to be used when connecting
|
||||||
|
Databricks. Defaults to None.
|
||||||
|
**kwargs (Any): Additional keyword arguments for the `from_uri` method.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
SQLDatabase: An instance of SQLDatabase configured with the provided
|
||||||
|
Databricks connection details.
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
ValueError: If 'databricks-sql-connector' is not found, or if both
|
||||||
|
'warehouse_id' and 'cluster_id' are provided, or if neither
|
||||||
|
'warehouse_id' nor 'cluster_id' are provided and it's not executing
|
||||||
|
inside a Databricks notebook.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
from databricks import sql # noqa: F401
|
||||||
|
except ImportError:
|
||||||
|
raise ValueError(
|
||||||
|
"databricks-sql-connector package not found, please install with"
|
||||||
|
" `pip install databricks-sql-connector`"
|
||||||
|
)
|
||||||
|
context = None
|
||||||
|
try:
|
||||||
|
from dbruntime.databricks_repl_context import get_context
|
||||||
|
|
||||||
|
context = get_context()
|
||||||
|
except ImportError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
default_host = context.browserHostName if context else None
|
||||||
|
if host is None:
|
||||||
|
host = get_from_env("host", "DATABRICKS_HOST", default_host)
|
||||||
|
|
||||||
|
default_api_token = context.apiToken if context else None
|
||||||
|
if api_token is None:
|
||||||
|
api_token = get_from_env("api_token", "DATABRICKS_TOKEN", default_api_token)
|
||||||
|
|
||||||
|
if warehouse_id is None and cluster_id is None:
|
||||||
|
if context:
|
||||||
|
cluster_id = context.clusterId
|
||||||
|
else:
|
||||||
|
raise ValueError(
|
||||||
|
"Need to provide either 'warehouse_id' or 'cluster_id'."
|
||||||
|
)
|
||||||
|
|
||||||
|
if warehouse_id and cluster_id:
|
||||||
|
raise ValueError("Can't have both 'warehouse_id' or 'cluster_id'.")
|
||||||
|
|
||||||
|
if warehouse_id:
|
||||||
|
http_path = f"/sql/1.0/warehouses/{warehouse_id}"
|
||||||
|
else:
|
||||||
|
http_path = f"/sql/protocolv1/o/0/{cluster_id}"
|
||||||
|
|
||||||
|
uri = (
|
||||||
|
f"databricks://token:{api_token}@{host}?"
|
||||||
|
f"http_path={http_path}&catalog={catalog}&schema={schema}"
|
||||||
|
)
|
||||||
|
return cls.from_uri(database_uri=uri, engine_args=engine_args, **kwargs)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_cnosdb(
|
||||||
|
cls,
|
||||||
|
url: str = "127.0.0.1:8902",
|
||||||
|
user: str = "root",
|
||||||
|
password: str = "",
|
||||||
|
tenant: str = "cnosdb",
|
||||||
|
database: str = "public",
|
||||||
|
) -> SQLDatabase:
|
||||||
|
"""
|
||||||
|
Class method to create an SQLDatabase instance from a CnosDB connection.
|
||||||
|
This method requires the 'cnos-connector' package. If not installed, it
|
||||||
|
can be added using `pip install cnos-connector`.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
url (str): The HTTP connection host name and port number of the CnosDB
|
||||||
|
service, excluding "http://" or "https://", with a default value
|
||||||
|
of "127.0.0.1:8902".
|
||||||
|
user (str): The username used to connect to the CnosDB service, with a
|
||||||
|
default value of "root".
|
||||||
|
password (str): The password of the user connecting to the CnosDB service,
|
||||||
|
with a default value of "".
|
||||||
|
tenant (str): The name of the tenant used to connect to the CnosDB service,
|
||||||
|
with a default value of "cnosdb".
|
||||||
|
database (str): The name of the database in the CnosDB tenant.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
SQLDatabase: An instance of SQLDatabase configured with the provided
|
||||||
|
CnosDB connection details.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
from cnosdb_connector import make_cnosdb_langchain_uri
|
||||||
|
|
||||||
|
uri = make_cnosdb_langchain_uri(url, user, password, tenant, database)
|
||||||
|
return cls.from_uri(database_uri=uri)
|
||||||
|
except ImportError:
|
||||||
|
raise ValueError(
|
||||||
|
"cnos-connector package not found, please install with"
|
||||||
|
" `pip install cnos-connector`"
|
||||||
|
)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def dialect(self) -> str:
|
||||||
|
"""Return string representation of dialect to use."""
|
||||||
|
return self._engine.dialect.name
|
||||||
|
|
||||||
|
def get_usable_table_names(self) -> Iterable[str]:
|
||||||
|
"""Get names of tables available."""
|
||||||
|
if self._include_tables:
|
||||||
|
return sorted(self._include_tables)
|
||||||
|
return sorted(self._all_tables - self._ignore_tables)
|
||||||
|
|
||||||
|
def get_table_names(self) -> Iterable[str]:
|
||||||
|
"""Get names of tables available."""
|
||||||
|
warnings.warn(
|
||||||
|
"This method is deprecated - please use `get_usable_table_names`."
|
||||||
|
)
|
||||||
|
return self.get_usable_table_names()
|
||||||
|
|
||||||
|
@property
|
||||||
|
def table_info(self) -> str:
|
||||||
|
"""Information about all tables in the database."""
|
||||||
|
return self.get_table_info()
|
||||||
|
|
||||||
|
def get_table_info(self, table_names: Optional[List[str]] = None) -> str:
|
||||||
|
"""Get information about specified tables.
|
||||||
|
|
||||||
|
Follows best practices as specified in: Rajkumar et al, 2022
|
||||||
|
(https://arxiv.org/abs/2204.00498)
|
||||||
|
|
||||||
|
If `sample_rows_in_table_info`, the specified number of sample rows will be
|
||||||
|
appended to each table description. This can increase performance as
|
||||||
|
demonstrated in the paper.
|
||||||
|
"""
|
||||||
|
all_table_names = self.get_usable_table_names()
|
||||||
|
if table_names is not None:
|
||||||
|
missing_tables = set(table_names).difference(all_table_names)
|
||||||
|
if missing_tables:
|
||||||
|
raise ValueError(f"table_names {missing_tables} not found in database")
|
||||||
|
all_table_names = table_names
|
||||||
|
|
||||||
|
meta_tables = [
|
||||||
|
tbl
|
||||||
|
for tbl in self._metadata.sorted_tables
|
||||||
|
if tbl.name in set(all_table_names)
|
||||||
|
and not (self.dialect == "sqlite" and tbl.name.startswith("sqlite_"))
|
||||||
|
]
|
||||||
|
|
||||||
|
tables = []
|
||||||
|
for table in meta_tables:
|
||||||
|
if self._custom_table_info and table.name in self._custom_table_info:
|
||||||
|
tables.append(self._custom_table_info[table.name])
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Ignore JSON datatyped columns
|
||||||
|
for k, v in table.columns.items():
|
||||||
|
if type(v.type) is NullType:
|
||||||
|
table._columns.remove(v)
|
||||||
|
|
||||||
|
# add create table command
|
||||||
|
create_table = str(CreateTable(table).compile(self._engine))
|
||||||
|
table_info = f"{create_table.rstrip()}"
|
||||||
|
has_extra_info = (
|
||||||
|
self._indexes_in_table_info or self._sample_rows_in_table_info
|
||||||
|
)
|
||||||
|
if has_extra_info:
|
||||||
|
table_info += "\n\n/*"
|
||||||
|
if self._indexes_in_table_info:
|
||||||
|
table_info += f"\n{self._get_table_indexes(table)}\n"
|
||||||
|
if self._sample_rows_in_table_info:
|
||||||
|
table_info += f"\n{self._get_sample_rows(table)}\n"
|
||||||
|
if has_extra_info:
|
||||||
|
table_info += "*/"
|
||||||
|
tables.append(table_info)
|
||||||
|
tables.sort()
|
||||||
|
final_str = "\n\n".join(tables)
|
||||||
|
return final_str
|
||||||
|
|
||||||
|
def _get_table_indexes(self, table: Table) -> str:
|
||||||
|
indexes = self._inspector.get_indexes(table.name)
|
||||||
|
indexes_formatted = "\n".join(map(_format_index, indexes))
|
||||||
|
return f"Table Indexes:\n{indexes_formatted}"
|
||||||
|
|
||||||
|
def _get_sample_rows(self, table: Table) -> str:
|
||||||
|
# build the select command
|
||||||
|
command = select(table).limit(self._sample_rows_in_table_info)
|
||||||
|
|
||||||
|
# save the columns in string format
|
||||||
|
columns_str = "\t".join([col.name for col in table.columns])
|
||||||
|
|
||||||
|
try:
|
||||||
|
# get the sample rows
|
||||||
|
with self._engine.connect() as connection:
|
||||||
|
sample_rows_result = connection.execute(command) # type: ignore
|
||||||
|
# shorten values in the sample rows
|
||||||
|
sample_rows = list(
|
||||||
|
map(lambda ls: [str(i)[:100] for i in ls], sample_rows_result)
|
||||||
|
)
|
||||||
|
|
||||||
|
# save the sample rows in string format
|
||||||
|
sample_rows_str = "\n".join(["\t".join(row) for row in sample_rows])
|
||||||
|
|
||||||
|
# in some dialects when there are no rows in the table a
|
||||||
|
# 'ProgrammingError' is returned
|
||||||
|
except ProgrammingError:
|
||||||
|
sample_rows_str = ""
|
||||||
|
|
||||||
|
return (
|
||||||
|
f"{self._sample_rows_in_table_info} rows from {table.name} table:\n"
|
||||||
|
f"{columns_str}\n"
|
||||||
|
f"{sample_rows_str}"
|
||||||
|
)
|
||||||
|
|
||||||
|
def _execute(
|
||||||
|
self,
|
||||||
|
command: str,
|
||||||
|
fetch: Union[Literal["all"], Literal["one"]] = "all",
|
||||||
|
) -> Sequence[Dict[str, Any]]:
|
||||||
|
"""
|
||||||
|
Executes SQL command through underlying engine.
|
||||||
|
|
||||||
|
If the statement returns no rows, an empty list is returned.
|
||||||
|
"""
|
||||||
|
with self._engine.begin() as connection:
|
||||||
|
if self._schema is not None:
|
||||||
|
if self.dialect == "snowflake":
|
||||||
|
connection.exec_driver_sql(
|
||||||
|
"ALTER SESSION SET search_path = %s", (self._schema,)
|
||||||
|
)
|
||||||
|
elif self.dialect == "bigquery":
|
||||||
|
connection.exec_driver_sql("SET @@dataset_id=?", (self._schema,))
|
||||||
|
elif self.dialect == "mssql":
|
||||||
|
pass
|
||||||
|
elif self.dialect == "trino":
|
||||||
|
connection.exec_driver_sql("USE ?", (self._schema,))
|
||||||
|
elif self.dialect == "duckdb":
|
||||||
|
# Unclear which parameterized argument syntax duckdb supports.
|
||||||
|
# The docs for the duckdb client say they support multiple,
|
||||||
|
# but `duckdb_engine` seemed to struggle with all of them:
|
||||||
|
# https://github.com/Mause/duckdb_engine/issues/796
|
||||||
|
connection.exec_driver_sql(f"SET search_path TO {self._schema}")
|
||||||
|
elif self.dialect == "oracle":
|
||||||
|
connection.exec_driver_sql(
|
||||||
|
f"ALTER SESSION SET CURRENT_SCHEMA = {self._schema}"
|
||||||
|
)
|
||||||
|
else: # postgresql and other compatible dialects
|
||||||
|
connection.exec_driver_sql("SET search_path TO %s", (self._schema,))
|
||||||
|
cursor = connection.execute(text(command))
|
||||||
|
if cursor.returns_rows:
|
||||||
|
if fetch == "all":
|
||||||
|
result = [x._asdict() for x in cursor.fetchall()]
|
||||||
|
elif fetch == "one":
|
||||||
|
first_result = cursor.fetchone()
|
||||||
|
result = [] if first_result is None else [first_result._asdict()]
|
||||||
|
else:
|
||||||
|
raise ValueError("Fetch parameter must be either 'one' or 'all'")
|
||||||
|
return result
|
||||||
|
return []
|
||||||
|
|
||||||
|
def run(
|
||||||
|
self,
|
||||||
|
command: str,
|
||||||
|
fetch: Union[Literal["all"], Literal["one"]] = "all",
|
||||||
|
) -> str:
|
||||||
|
"""Execute a SQL command and return a string representing the results.
|
||||||
|
|
||||||
|
If the statement returns rows, a string of the results is returned.
|
||||||
|
If the statement returns no rows, an empty string is returned.
|
||||||
|
"""
|
||||||
|
result = self._execute(command, fetch)
|
||||||
|
# Convert columns values to string to avoid issues with sqlalchemy
|
||||||
|
# truncating text
|
||||||
|
res = [
|
||||||
|
tuple(truncate_word(c, length=self._max_string_length) for c in r.values())
|
||||||
|
for r in result
|
||||||
|
]
|
||||||
|
if not res:
|
||||||
|
return ""
|
||||||
|
else:
|
||||||
|
return str(res)
|
||||||
|
|
||||||
|
def get_table_info_no_throw(self, table_names: Optional[List[str]] = None) -> str:
|
||||||
|
"""Get information about specified tables.
|
||||||
|
|
||||||
|
Follows best practices as specified in: Rajkumar et al, 2022
|
||||||
|
(https://arxiv.org/abs/2204.00498)
|
||||||
|
|
||||||
|
If `sample_rows_in_table_info`, the specified number of sample rows will be
|
||||||
|
appended to each table description. This can increase performance as
|
||||||
|
demonstrated in the paper.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
return self.get_table_info(table_names)
|
||||||
|
except ValueError as e:
|
||||||
|
"""Format the error message"""
|
||||||
|
return f"Error: {e}"
|
||||||
|
|
||||||
|
def run_no_throw(
|
||||||
|
self,
|
||||||
|
command: str,
|
||||||
|
fetch: Union[Literal["all"], Literal["one"]] = "all",
|
||||||
|
) -> str:
|
||||||
|
"""Execute a SQL command and return a string representing the results.
|
||||||
|
|
||||||
|
If the statement returns rows, a string of the results is returned.
|
||||||
|
If the statement returns no rows, an empty string is returned.
|
||||||
|
|
||||||
|
If the statement throws an error, the error message is returned.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
return self.run(command, fetch)
|
||||||
|
except SQLAlchemyError as e:
|
||||||
|
"""Format the error message"""
|
||||||
|
return f"Error: {e}"
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
|
# pylint:disable=E0402
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
import asyncio
|
import asyncio
|
||||||
import json
|
import json
|
||||||
@@ -7,20 +7,22 @@ from sqlite3 import Error
|
|||||||
import threading
|
import threading
|
||||||
import logging
|
import logging
|
||||||
from typing import Optional
|
from typing import Optional
|
||||||
|
|
||||||
from .text_to_speech_function import TextToSpeechFunction
|
|
||||||
from .compute_kernel import ComputeKernel, ComputeTaskResultCode
|
|
||||||
from .environment import Environment,EnvironmentEvent
|
|
||||||
from .ai_function import SimpleAIFunction
|
|
||||||
from .storage import AIStorage
|
|
||||||
from .contact_manager import ContactManager,Contact,FamilyMember
|
|
||||||
|
|
||||||
import aiosqlite
|
import aiosqlite
|
||||||
|
|
||||||
|
from ..agent.llm_context import GlobaToolsLibrary
|
||||||
|
from ..proto.compute_task import *
|
||||||
|
from ..proto.ai_function import *
|
||||||
|
from ..frame.compute_kernel import ComputeKernel
|
||||||
|
from ..frame.contact_manager import ContactManager,Contact
|
||||||
|
from ..storage.storage import AIStorage
|
||||||
|
|
||||||
|
from .environment import SimpleEnvironment, CompositeEnvironment
|
||||||
|
from ..ai_functions.script_to_speech_function import ScriptToSpeechFunction
|
||||||
|
from ..ai_functions.image_2_text_function import Image2TextFunction
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
class CalenderEvent(SimpleEnvironment):
|
||||||
class CalenderEvent(EnvironmentEvent):
|
|
||||||
def __init__(self,data) -> None:
|
def __init__(self,data) -> None:
|
||||||
super().__init__()
|
super().__init__()
|
||||||
self.event_name = "timer"
|
self.event_name = "timer"
|
||||||
@@ -30,48 +32,45 @@ class CalenderEvent(EnvironmentEvent):
|
|||||||
return f"#event timer:{self.data}"
|
return f"#event timer:{self.data}"
|
||||||
|
|
||||||
# AI Calender GOAL: Let user use "create notify after 2 days" to create a timer event
|
# AI Calender GOAL: Let user use "create notify after 2 days" to create a timer event
|
||||||
class CalenderEnvironment(Environment):
|
class CalenderEnvironment(SimpleEnvironment):
|
||||||
def __init__(self, env_id: str) -> None:
|
def __init__(self, env_id: str) -> None:
|
||||||
super().__init__(env_id)
|
super().__init__(env_id)
|
||||||
self.db_file = AIStorage.get_instance().get_myai_dir() / "calender.db"
|
self.db_file = AIStorage.get_instance().get_myai_dir() / "calender.db"
|
||||||
self.is_run = False
|
self.is_run = False
|
||||||
|
gl = GlobaToolsLibrary.get_instance()
|
||||||
|
|
||||||
self.add_ai_function(SimpleAIFunction("get_time",
|
gl.register_tool_function(SimpleAIFunction("system.now",
|
||||||
"get current time",
|
"get current time",
|
||||||
self._get_now))
|
self._get_now))
|
||||||
|
|
||||||
#self.add_ai_function(SimpleAIFunction("serach_events",
|
get_param = ParameterDefine.create_parameters({
|
||||||
# "search events in calender",
|
|
||||||
# self._search_events))
|
|
||||||
|
|
||||||
get_param = {
|
|
||||||
"start_time": "start time (UTC) of event",
|
"start_time": "start time (UTC) of event",
|
||||||
"end_time": "end time (UTC) of event"
|
"end_time": "end time (UTC) of event"
|
||||||
}
|
})
|
||||||
self.add_ai_function(SimpleAIFunction("get_events",
|
gl.register_tool_function(SimpleAIFunction("system.calender.get_events",
|
||||||
"get events in calender by time range",
|
"get events in calender by time range",
|
||||||
self._get_events_by_time_range,get_param))
|
self._get_events_by_time_range,get_param))
|
||||||
|
|
||||||
add_param = {
|
add_param = ParameterDefine.create_parameters({
|
||||||
"title": "title of event",
|
"title": "title of event",
|
||||||
"start_time": "start time (UTC) of event",
|
"start_time": "start time (UTC) of event",
|
||||||
"end_time": "end time (UTC) of event",
|
"end_time": "end time (UTC) of event",
|
||||||
"participants": "participants of event",
|
"participants": "participants of event",
|
||||||
"location": "location of event",
|
"location": "location of event",
|
||||||
"details": "details of event"
|
"details": "details of event"
|
||||||
}
|
})
|
||||||
self.add_ai_function(SimpleAIFunction("add_event",
|
gl.register_tool_function(SimpleAIFunction("system.calender.add_event",
|
||||||
"add event to calender",
|
"add event to calender",
|
||||||
self._add_event,add_param))
|
self._add_event,add_param))
|
||||||
|
|
||||||
delete_param = {
|
delete_param = ParameterDefine.create_parameters({
|
||||||
"event_id": "id of event"
|
"event_id": "id of event"
|
||||||
}
|
})
|
||||||
self.add_ai_function(SimpleAIFunction("delete_event",
|
gl.register_tool_function(SimpleAIFunction("system.calender.delete_event",
|
||||||
"delete event from calender",
|
"delete event from calender",
|
||||||
self._delete_event,delete_param))
|
self._delete_event,delete_param))
|
||||||
|
|
||||||
update_param = {
|
update_param = ParameterDefine.create_parameters({
|
||||||
"event_id": "id of event",
|
"event_id": "id of event",
|
||||||
"new_title": "new title of event",
|
"new_title": "new title of event",
|
||||||
"new_participants": "new participants of event",
|
"new_participants": "new participants of event",
|
||||||
@@ -79,36 +78,12 @@ class CalenderEnvironment(Environment):
|
|||||||
"new_details": "new details of event",
|
"new_details": "new details of event",
|
||||||
"start_time": "new start time (UTC) of event",
|
"start_time": "new start time (UTC) of event",
|
||||||
"end_time": "new end time (UTC) of event"
|
"end_time": "new end time (UTC) of event"
|
||||||
}
|
})
|
||||||
self.add_ai_function(SimpleAIFunction("update_event",
|
gl.register_tool_function(SimpleAIFunction("system.calender.update_event",
|
||||||
"update event in calender",
|
"update event in calender",
|
||||||
self._update_event,update_param))
|
self._update_event,update_param))
|
||||||
|
|
||||||
|
|
||||||
#maybe this function should be in other env?
|
|
||||||
paint_param = {
|
|
||||||
"prompt": "A description of the content of the painting",
|
|
||||||
"model_name": "Which model to use to draw the picture, can be None"
|
|
||||||
}
|
|
||||||
self.add_ai_function(SimpleAIFunction("paint",
|
|
||||||
"Draw a picture according to the description",
|
|
||||||
self._paint,paint_param))
|
|
||||||
|
|
||||||
self.add_ai_function(SimpleAIFunction("get_contact",
|
|
||||||
"get contact info",
|
|
||||||
self._get_contact,{"name":"name of contact"}))
|
|
||||||
|
|
||||||
self.add_ai_function(SimpleAIFunction("set_contact",
|
|
||||||
"set contact info",
|
|
||||||
self._set_contact,{"name":"name of contact","contact_info":"A json to descrpit contact"}))
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
#self.add_ai_function(SimpleAIFunction("user_confirm",
|
|
||||||
# "user confirm",
|
|
||||||
# self._user_confirm))
|
|
||||||
|
|
||||||
async def init_db(self):
|
async def init_db(self):
|
||||||
async with aiosqlite.connect(self.db_file) as db:
|
async with aiosqlite.connect(self.db_file) as db:
|
||||||
await db.execute("""
|
await db.execute("""
|
||||||
@@ -151,7 +126,7 @@ class CalenderEnvironment(Environment):
|
|||||||
_event["location"] = row[5]
|
_event["location"] = row[5]
|
||||||
_event["details"] = row[6]
|
_event["details"] = row[6]
|
||||||
result[row[0]] = _event
|
result[row[0]] = _event
|
||||||
return json.dumps(result, indent=4, sort_keys=True)
|
return json.dumps(result, indent=4, sort_keys=True,ensure_ascii=False)
|
||||||
|
|
||||||
async def _get_events_by_time_range(self,start_time, end_time):
|
async def _get_events_by_time_range(self,start_time, end_time):
|
||||||
async with aiosqlite.connect(self.db_file) as db:
|
async with aiosqlite.connect(self.db_file) as db:
|
||||||
@@ -177,7 +152,7 @@ class CalenderEnvironment(Environment):
|
|||||||
if not have_result:
|
if not have_result:
|
||||||
return "No event."
|
return "No event."
|
||||||
|
|
||||||
return json.dumps(result, indent=4, sort_keys=True)
|
return json.dumps(result, indent=4, sort_keys=True,ensure_ascii=False)
|
||||||
|
|
||||||
async def _update_event(self,event_id, new_title=None, new_participants=None, new_location=None, new_details=None ,start_time=None, end_time=None):
|
async def _update_event(self,event_id, new_title=None, new_participants=None, new_location=None, new_details=None ,start_time=None, end_time=None):
|
||||||
fields_to_update = []
|
fields_to_update = []
|
||||||
@@ -239,7 +214,7 @@ class CalenderEnvironment(Environment):
|
|||||||
cm = ContactManager.get_instance()
|
cm = ContactManager.get_instance()
|
||||||
contact : Contact = cm.find_contact_by_name(name)
|
contact : Contact = cm.find_contact_by_name(name)
|
||||||
if contact:
|
if contact:
|
||||||
s = json.dumps(contact.to_dict())
|
s = json.dumps(contact.to_dict(),ensure_ascii=False)
|
||||||
return f"Execute get_contact OK , contact {name} is {s}"
|
return f"Execute get_contact OK , contact {name} is {s}"
|
||||||
else:
|
else:
|
||||||
return f"Execute get_contact OK , contact {name} not found!"
|
return f"Execute get_contact OK , contact {name} not found!"
|
||||||
@@ -271,9 +246,6 @@ class CalenderEnvironment(Environment):
|
|||||||
|
|
||||||
return f"Execute set_contact OK , contact {name} updated!"
|
return f"Execute set_contact OK , contact {name} updated!"
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
async def start(self) -> None:
|
async def start(self) -> None:
|
||||||
if self.is_run:
|
if self.is_run:
|
||||||
return
|
return
|
||||||
@@ -318,18 +290,19 @@ class CalenderEnvironment(Environment):
|
|||||||
return f'exec paint OK, saved as a local file, path is: {result.result["file"]}'
|
return f'exec paint OK, saved as a local file, path is: {result.result["file"]}'
|
||||||
|
|
||||||
|
|
||||||
class PaintEnvironment(Environment):
|
class PaintEnvironment(SimpleEnvironment):
|
||||||
def __init__(self, env_id: str) -> None:
|
def __init__(self, env_id: str) -> None:
|
||||||
super().__init__(env_id)
|
super().__init__(env_id)
|
||||||
self.is_run = False
|
|
||||||
|
|
||||||
paint_param = {
|
|
||||||
"prompt": "Keywords of the content of the painting",
|
|
||||||
"model_name": "Which model to use to draw the picture, can be None",
|
|
||||||
"negative_prompt": "Keywords that describe what is not to be drawn, can be None"
|
def register_functions(self):
|
||||||
}
|
paint_param = ParameterDefine.create_parameters({
|
||||||
self.add_ai_function(SimpleAIFunction("paint",
|
"prompt": "Description of the content of the painting",
|
||||||
"Draw a picture according to the keywords",
|
})
|
||||||
|
GlobaToolsLibrary.get_instance().register_tool_function(SimpleAIFunction("aigc.text_2_image",
|
||||||
|
"Draw a picture according to the description",
|
||||||
self._paint,paint_param))
|
self._paint,paint_param))
|
||||||
|
|
||||||
def _do_get_value(self,key:str) -> Optional[str]:
|
def _do_get_value(self,key:str) -> Optional[str]:
|
||||||
@@ -337,21 +310,22 @@ class PaintEnvironment(Environment):
|
|||||||
|
|
||||||
|
|
||||||
async def _paint(self, prompt, model_name = None, negative_prompt = None) -> str:
|
async def _paint(self, prompt, model_name = None, negative_prompt = None) -> str:
|
||||||
err, result = await ComputeKernel.get_instance().do_text_2_image(prompt, model_name, negative_prompt)
|
result = await ComputeKernel.get_instance().do_text_2_image(prompt, model_name, negative_prompt)
|
||||||
if err is not None:
|
if result.result_code == ComputeTaskResultCode.ERROR:
|
||||||
return f"exec paint failed. err:{err}"
|
return f"exec paint failed. err:{result.error_str}"
|
||||||
else:
|
else:
|
||||||
return f'exec paint OK, saved as a local file, path is: {result.result["file"]}'
|
return f'exec paint OK, saved as a local file, path is: {result.result["file"]}'
|
||||||
|
|
||||||
|
|
||||||
# Default Workflow Environment(Context)
|
# Default Workflow Environment(Context)
|
||||||
class WorkflowEnvironment(Environment):
|
class WorkflowEnvironment(CompositeEnvironment):
|
||||||
def __init__(self, env_id: str,db_file:str) -> None:
|
def __init__(self, env_id: str,db_file:str) -> None:
|
||||||
super().__init__(env_id)
|
super().__init__(env_id)
|
||||||
self.db_file = db_file
|
self.db_file = db_file
|
||||||
self.local = threading.local()
|
self.local = threading.local()
|
||||||
self.table_name = "WorkflowEnv_" + env_id
|
self.table_name = "WorkflowEnv_" + env_id
|
||||||
self.add_ai_function(TextToSpeechFunction())
|
# self.add_ai_function(ScriptToSpeechFunction())
|
||||||
|
# self.add_ai_function(Image2TextFunction())
|
||||||
|
|
||||||
|
|
||||||
def _get_conn(self):
|
def _get_conn(self):
|
||||||
@@ -0,0 +1,328 @@
|
|||||||
|
# pylint:disable=E0402
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import aiofiles
|
||||||
|
import sqlite3
|
||||||
|
import asyncio
|
||||||
|
from typing import Any,List,Dict
|
||||||
|
import chardet
|
||||||
|
|
||||||
|
from ..proto.agent_task import *
|
||||||
|
from ..proto.ai_function import *
|
||||||
|
from ..proto.compute_task import *
|
||||||
|
from ..agent.agent_base import *
|
||||||
|
|
||||||
|
from ..storage.storage import AIStorage,ResourceLocation
|
||||||
|
from .environment import SimpleEnvironment, CompositeEnvironment
|
||||||
|
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
class TodoListType:
|
||||||
|
TO_WORK = "work"
|
||||||
|
TO_LEARN = "learn"
|
||||||
|
|
||||||
|
class TodoListEnvironment(SimpleEnvironment):
|
||||||
|
def __init__(self, workspace, list_type) -> None:
|
||||||
|
super().__init__(workspace)
|
||||||
|
self.root_path = os.path.join(workspace, list_type)
|
||||||
|
if not os.path.exists(self.root_path):
|
||||||
|
os.makedirs(self.root_path)
|
||||||
|
|
||||||
|
self.db_path = os.path.join(self.root_path, "todo.db")
|
||||||
|
self.conn = None
|
||||||
|
try:
|
||||||
|
self.conn = sqlite3.connect(self.db_path)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("Error occurred while connecting to database: %s", e)
|
||||||
|
return None
|
||||||
|
|
||||||
|
cursor = self.conn.cursor()
|
||||||
|
cursor.execute('''
|
||||||
|
CREATE TABLE IF NOT EXISTS todo_list (
|
||||||
|
id TEXT,
|
||||||
|
path TEXT
|
||||||
|
)
|
||||||
|
''')
|
||||||
|
self.conn.commit()
|
||||||
|
|
||||||
|
async def create_todo(params):
|
||||||
|
todoObj = AgentTodo.from_dict(params["todo"])
|
||||||
|
parent_id = params.get("parent")
|
||||||
|
return await self.create_todo(parent_id,todoObj)
|
||||||
|
|
||||||
|
self.add_ai_operation(SimpleAIAction(
|
||||||
|
op="create_todo",
|
||||||
|
description="create todo",
|
||||||
|
func_handler=create_todo,
|
||||||
|
))
|
||||||
|
|
||||||
|
|
||||||
|
async def update_todo(params):
|
||||||
|
todo_id = params["id"]
|
||||||
|
new_stat = params["state"]
|
||||||
|
return await self.update_todo(todo_id,new_stat)
|
||||||
|
|
||||||
|
self.add_ai_operation(SimpleAIAction(
|
||||||
|
op="update_todo",
|
||||||
|
description="update todo",
|
||||||
|
func_handler=update_todo,
|
||||||
|
))
|
||||||
|
|
||||||
|
def _get_todo_path(self,todo_id:str) -> str:
|
||||||
|
cursor = self.conn.cursor()
|
||||||
|
cursor.execute('''
|
||||||
|
SELECT path FROM todo_list WHERE id = ?
|
||||||
|
''',(todo_id,))
|
||||||
|
row = cursor.fetchone()
|
||||||
|
if row:
|
||||||
|
return row[0]
|
||||||
|
else:
|
||||||
|
return None
|
||||||
|
|
||||||
|
def _save_todo_path(self,todo_id:str,path:str):
|
||||||
|
cursor = self.conn.cursor()
|
||||||
|
cursor.execute('''
|
||||||
|
INSERT INTO todo_list (id,path) VALUES (?,?)
|
||||||
|
''',(todo_id,path))
|
||||||
|
self.conn.commit()
|
||||||
|
|
||||||
|
# Task/todo system , create,update,delete,query
|
||||||
|
async def get_todo_tree(self,path:str = None,deep:int = 4):
|
||||||
|
if path:
|
||||||
|
directory_path = os.path.join(self.root_path, path)
|
||||||
|
else:
|
||||||
|
directory_path = self.root_path
|
||||||
|
|
||||||
|
|
||||||
|
str_result:str = "/todos\n"
|
||||||
|
todo_count:int = 0
|
||||||
|
|
||||||
|
async def scan_dir(directory_path:str,deep:int):
|
||||||
|
nonlocal str_result
|
||||||
|
nonlocal todo_count
|
||||||
|
if deep <= 0:
|
||||||
|
return
|
||||||
|
|
||||||
|
if os.path.exists(directory_path) is False:
|
||||||
|
return
|
||||||
|
|
||||||
|
for entry in os.scandir(directory_path):
|
||||||
|
is_dir = entry.is_dir()
|
||||||
|
if not is_dir:
|
||||||
|
continue
|
||||||
|
|
||||||
|
if entry.name.startswith("."):
|
||||||
|
continue
|
||||||
|
|
||||||
|
todo_count = todo_count + 1
|
||||||
|
str_result = str_result + f"{' '*(4-deep)}{entry.name}\n"
|
||||||
|
await scan_dir(entry.path,deep-1)
|
||||||
|
|
||||||
|
await scan_dir(directory_path,deep)
|
||||||
|
return str_result,todo_count
|
||||||
|
|
||||||
|
async def get_todo_list(self,agent_id:str,path:str = None)->List[AgentTodo]:
|
||||||
|
logger.info("get_todo_list:%s,%s",agent_id,path)
|
||||||
|
if path:
|
||||||
|
directory_path = os.path.join(self.root_path, path)
|
||||||
|
else:
|
||||||
|
directory_path = self.root_path
|
||||||
|
|
||||||
|
result_list:List[AgentTodo] = []
|
||||||
|
|
||||||
|
async def scan_dir(directory_path:str,deep:int,parent:AgentTodo=None):
|
||||||
|
nonlocal result_list
|
||||||
|
if os.path.exists(directory_path) is False:
|
||||||
|
return
|
||||||
|
|
||||||
|
for entry in os.scandir(directory_path):
|
||||||
|
is_dir = entry.is_dir()
|
||||||
|
if not is_dir:
|
||||||
|
continue
|
||||||
|
|
||||||
|
if entry.name.startswith("."):
|
||||||
|
continue
|
||||||
|
|
||||||
|
todo = await self.get_todo_by_fullpath(entry.path)
|
||||||
|
if todo:
|
||||||
|
if todo.worker:
|
||||||
|
if todo.worker != agent_id:
|
||||||
|
continue
|
||||||
|
|
||||||
|
if parent:
|
||||||
|
parent.sub_todos[todo.todo_id] = todo
|
||||||
|
|
||||||
|
result_list.append(todo)
|
||||||
|
todo.rank = int(todo.create_time)>>deep
|
||||||
|
await scan_dir(entry.path,deep + 1,todo)
|
||||||
|
|
||||||
|
return
|
||||||
|
|
||||||
|
await scan_dir(directory_path,0)
|
||||||
|
#sort by rank
|
||||||
|
result_list.sort(key=lambda x:(x.rank,x.title))
|
||||||
|
logger.info("get_todo_list return,todolist.length() is %d",len(result_list))
|
||||||
|
return result_list
|
||||||
|
|
||||||
|
async def get_todo_by_fullpath(self,path:str) -> AgentTodo:
|
||||||
|
logger.info("get_todo_by_fullpath:%s",path)
|
||||||
|
|
||||||
|
detail_path = path + "/detail"
|
||||||
|
try:
|
||||||
|
with open(detail_path, mode='r', encoding="utf-8") as f:
|
||||||
|
todo_dict = json.load(f)
|
||||||
|
result_todo = AgentTodo.from_dict(todo_dict)
|
||||||
|
if result_todo:
|
||||||
|
relative_path = os.path.relpath(path, self.root_path)
|
||||||
|
if not relative_path.startswith('/'):
|
||||||
|
relative_path = '/' + relative_path
|
||||||
|
result_todo.todo_path = relative_path
|
||||||
|
else:
|
||||||
|
logger.error("get_todo_by_path:%s,parse failed!",path)
|
||||||
|
|
||||||
|
return result_todo
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("get_todo_by_path:%s,failed:%s",path,e)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
async def create_todo(self,parent_id:str,todo:AgentTodo) -> str:
|
||||||
|
try:
|
||||||
|
if parent_id:
|
||||||
|
parent_path = self._get_todo_path(parent_id)
|
||||||
|
todo_path = f"{parent_path}/{todo.todo_id}-{todo.title}"
|
||||||
|
else:
|
||||||
|
todo_path = f"{todo.todo_id}-{todo.title}"
|
||||||
|
|
||||||
|
dir_path = f"{self.root_path}/{todo_path}"
|
||||||
|
|
||||||
|
os.makedirs(dir_path)
|
||||||
|
detail_path = f"{dir_path}/detail"
|
||||||
|
if todo.todo_path is None:
|
||||||
|
todo.todo_path = todo_path
|
||||||
|
self._save_todo_path(todo.todo_id,todo_path)
|
||||||
|
logger.info("create_todo %s",detail_path)
|
||||||
|
async with aiofiles.open(detail_path, mode='w', encoding="utf-8") as f:
|
||||||
|
await f.write(json.dumps(todo.to_dict(),ensure_ascii=False))
|
||||||
|
except Exception as e:
|
||||||
|
logger.error("create_todo failed:%s",e)
|
||||||
|
return str(e)
|
||||||
|
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def update_todo(self,todo_id:str,new_stat:str)->str:
|
||||||
|
try:
|
||||||
|
todo_path = self._get_todo_path(todo_id)
|
||||||
|
full_path = f"{self.root_path}/{todo_path}"
|
||||||
|
todo : AgentTodo = await self.get_todo_by_fullpath(full_path)
|
||||||
|
if todo:
|
||||||
|
todo.state = new_stat
|
||||||
|
detail_path = f"{full_path}/detail"
|
||||||
|
async with aiofiles.open(detail_path, mode='w', encoding="utf-8") as f:
|
||||||
|
await f.write(json.dumps(todo.to_dict()),ensure_ascii=False)
|
||||||
|
return None
|
||||||
|
else:
|
||||||
|
return "todo not found."
|
||||||
|
except Exception as e:
|
||||||
|
return str(e)
|
||||||
|
|
||||||
|
async def wait_todo_done(self,todo_id:str,state=AgentTodoState.TODO_STATE_EXEC_OK) -> AgentTodo:
|
||||||
|
todo_path = self._get_todo_path(todo_id)
|
||||||
|
full_path = f"{self.root_path}/{todo_path}"
|
||||||
|
async def check_done():
|
||||||
|
while True:
|
||||||
|
todo : AgentTodo = await self.get_todo_by_fullpath(full_path)
|
||||||
|
if todo is None:
|
||||||
|
continue
|
||||||
|
if todo.state == AgentTodo.TODO_STATE_CANCEL:
|
||||||
|
break
|
||||||
|
elif todo.state == AgentTodo.TODO_STATE_EXPIRED:
|
||||||
|
break
|
||||||
|
elif todo.state == AgentTodo.TODO_STATE_WAITING_CHECK:
|
||||||
|
if state == AgentTodo.TODO_STATE_WAITING_CHECK:
|
||||||
|
break
|
||||||
|
elif todo.state == AgentTodo.TODO_STATE_DONE:
|
||||||
|
if state == AgentTodo.TODO_STATE_WAITING_CHECK:
|
||||||
|
break
|
||||||
|
elif todo.state == AgentTodo.TODO_STATE_DONE:
|
||||||
|
break
|
||||||
|
elif todo.state == AgentTodo.TODO_STATE_REVIEWED:
|
||||||
|
break
|
||||||
|
await asyncio.sleep(1)
|
||||||
|
|
||||||
|
await check_done()
|
||||||
|
return await self.get_todo_by_fullpath(full_path)
|
||||||
|
|
||||||
|
|
||||||
|
# async def append_worklog(self, todo:AgentTodo, result:AgentTodoResult):
|
||||||
|
# worklog = f"{self.root_path}/{todo.todo_path}/.worklog"
|
||||||
|
|
||||||
|
# async with aiofiles.open(worklog, mode='w+', encoding="utf-8") as f:
|
||||||
|
# content = await f.read()
|
||||||
|
# if len(content) > 0:
|
||||||
|
# json_obj = json.loads(content)
|
||||||
|
# else:
|
||||||
|
# json_obj = {}
|
||||||
|
# logs = json_obj.get("logs")
|
||||||
|
# if logs is None:
|
||||||
|
# logs = []
|
||||||
|
# logs.append(result.to_dict())
|
||||||
|
# json_obj["logs"] = logs
|
||||||
|
# await f.write(json.dumps(json_obj),ensure_ascii=False)
|
||||||
|
|
||||||
|
|
||||||
|
class WorkspaceEnvironment(CompositeEnvironment):
|
||||||
|
def __init__(self, env_id: str) -> None:
|
||||||
|
myai_path = AIStorage.get_instance().get_myai_dir()
|
||||||
|
root_path = f"{myai_path}/workspace/{env_id}"
|
||||||
|
super().__init__(root_path)
|
||||||
|
|
||||||
|
self.root_path = root_path
|
||||||
|
if not os.path.exists(self.root_path):
|
||||||
|
os.makedirs()
|
||||||
|
|
||||||
|
self.todo_list: Dict[str, TodoListEnvironment] = {}
|
||||||
|
self.todo_list[TodoListType.TO_WORK] = TodoListEnvironment(self.root_path,TodoListType.TO_WORK)
|
||||||
|
self.todo_list[TodoListType.TO_LEARN] = TodoListEnvironment(self.root_path,TodoListType.TO_LEARN)
|
||||||
|
|
||||||
|
# default environments in workspace
|
||||||
|
self.add_env(self.todo_list[TodoListType.TO_WORK])
|
||||||
|
|
||||||
|
def set_root_path(self,path:str):
|
||||||
|
self.root_path = path
|
||||||
|
|
||||||
|
def get_prompt(self) -> AgentMsg:
|
||||||
|
return None
|
||||||
|
|
||||||
|
def get_role_prompt(self,role_id:str) -> LLMPrompt:
|
||||||
|
return None
|
||||||
|
|
||||||
|
def get_do_prompt(self,todo:AgentTodo=None)->LLMPrompt:
|
||||||
|
return None
|
||||||
|
|
||||||
|
# result mean: list[op_error_str],have_error
|
||||||
|
async def exec_op_list(self,oplist:List,agent_id:str)->tuple[List[str],bool]:
|
||||||
|
result_str = "op list is none"
|
||||||
|
if oplist is None or len(oplist) == 0:
|
||||||
|
return None,False
|
||||||
|
|
||||||
|
result_str = []
|
||||||
|
have_error = False
|
||||||
|
for op in oplist:
|
||||||
|
operation = self.get_ai_operation(op["op"])
|
||||||
|
if operation:
|
||||||
|
error_str = await operation.execute(op)
|
||||||
|
else:
|
||||||
|
logger.error(f"execute op list failed: unknown op:{op['op']}")
|
||||||
|
error_str = f"execute op list failed: unknown op:{op['op']}"
|
||||||
|
|
||||||
|
if error_str:
|
||||||
|
have_error = True
|
||||||
|
result_str.append(error_str)
|
||||||
|
else:
|
||||||
|
result_str.append(f"execute success!")
|
||||||
|
|
||||||
|
return result_str,have_error
|
||||||
|
|
||||||
@@ -1,10 +1,11 @@
|
|||||||
from typing import Coroutine,Dict,Any
|
from typing import Coroutine,Dict,Any
|
||||||
from .agent_message import AgentMsg,AgentMsgStatus,AgentMsgType
|
|
||||||
import asyncio
|
import asyncio
|
||||||
from asyncio import Queue
|
from asyncio import Queue
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
|
|
||||||
|
from ..proto.agent_msg import *
|
||||||
|
from ..agent.agent_base import *
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
class AIBusHandler:
|
class AIBusHandler:
|
||||||
@@ -109,6 +110,9 @@ class AIBus:
|
|||||||
# means sub
|
# means sub
|
||||||
def register_message_handler(self,handler_name:str,handler:Any) -> Queue:
|
def register_message_handler(self,handler_name:str,handler:Any) -> Queue:
|
||||||
handler_node = AIBusHandler(handler,self)
|
handler_node = AIBusHandler(handler,self)
|
||||||
|
if self.handlers.get(handler_name) is not None:
|
||||||
|
logger.warn(f"handler {handler_name} already register on AI_BUS!")
|
||||||
|
|
||||||
self.handlers[handler_name] = handler_node
|
self.handlers[handler_name] = handler_node
|
||||||
return handler_node.queue
|
return handler_node.queue
|
||||||
|
|
||||||
@@ -3,12 +3,13 @@ import random
|
|||||||
from typing import Optional
|
from typing import Optional
|
||||||
import logging
|
import logging
|
||||||
import asyncio
|
import asyncio
|
||||||
|
import tiktoken
|
||||||
from asyncio import Queue
|
from asyncio import Queue
|
||||||
|
|
||||||
from knowledge import ObjectID
|
from ..proto.compute_task import *
|
||||||
from .agent import AgentPrompt
|
from ..knowledge import ObjectID
|
||||||
|
|
||||||
from .compute_node import ComputeNode
|
from .compute_node import ComputeNode
|
||||||
from .compute_task import ComputeTask, ComputeTaskState, ComputeTaskResult, ComputeTaskType,ComputeTaskResultCode
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -16,7 +17,6 @@ logger = logging.getLogger(__name__)
|
|||||||
# to suitable computing nodes, achieving a balance of speed, cost, and power consumption,
|
# to suitable computing nodes, achieving a balance of speed, cost, and power consumption,
|
||||||
# is the CORE GOAL of the entire computing task schedule system (aios_kernel).
|
# is the CORE GOAL of the entire computing task schedule system (aios_kernel).
|
||||||
|
|
||||||
|
|
||||||
class ComputeKernel:
|
class ComputeKernel:
|
||||||
_instance = None
|
_instance = None
|
||||||
@classmethod
|
@classmethod
|
||||||
@@ -104,16 +104,34 @@ class ComputeKernel:
|
|||||||
def is_task_support(self, task: ComputeTask) -> bool:
|
def is_task_support(self, task: ComputeTask) -> bool:
|
||||||
return True
|
return True
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def llm_num_tokens_from_text(text:str,model:str) -> int:
|
||||||
|
if model is None:
|
||||||
|
model = "gpt-4-turbo-preview"
|
||||||
|
|
||||||
|
try:
|
||||||
|
encoding = tiktoken.encoding_for_model(model)
|
||||||
|
except KeyError:
|
||||||
|
logger.debug("Warning: model not found. Using cl100k_base encoding.")
|
||||||
|
encoding = tiktoken.get_encoding("cl100k_base")
|
||||||
|
|
||||||
|
token_count = len(encoding.encode(text))
|
||||||
|
return token_count
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def llm_num_tokens(prompt: LLMPrompt, model_name: str = None) -> int:
|
||||||
|
return ComputeKernel.llm_num_tokens_from_text(prompt.as_str(), model_name)
|
||||||
|
|
||||||
# friendly interface for use:
|
# friendly interface for use:
|
||||||
def llm_completion(self, prompt: AgentPrompt, mode_name: Optional[str] = None, max_token: int = 0,inner_functions = None):
|
def llm_completion(self, prompt: LLMPrompt, resp_mode:str="text",mode_name: Optional[str] = None, max_token: int = 0,inner_functions = None):
|
||||||
# craete a llm_work_task ,push on queue's end
|
# craete a llm_work_task ,push on queue's end
|
||||||
# then task_schedule would run this task.(might schedule some work_task to another host)
|
# then task_schedule would run this task.(might schedule some work_task to another host)
|
||||||
task_req = ComputeTask()
|
task_req = ComputeTask()
|
||||||
task_req.set_llm_params(prompt, mode_name, max_token,inner_functions)
|
task_req.set_llm_params(prompt,resp_mode,mode_name, max_token,inner_functions)
|
||||||
self.run(task_req)
|
self.run(task_req)
|
||||||
return task_req
|
return task_req
|
||||||
|
|
||||||
async def _wait_task(self,task_req:ComputeTask)->ComputeTaskResult:
|
async def _wait_task(self,task_req:ComputeTask, timeout=60)->ComputeTaskResult:
|
||||||
async def check_timer():
|
async def check_timer():
|
||||||
check_times = 0
|
check_times = 0
|
||||||
while True:
|
while True:
|
||||||
@@ -123,7 +141,7 @@ class ComputeKernel:
|
|||||||
if task_req.state == ComputeTaskState.ERROR:
|
if task_req.state == ComputeTaskState.ERROR:
|
||||||
break
|
break
|
||||||
|
|
||||||
if check_times >= 120:
|
if timeout is not None and check_times >= timeout*2:
|
||||||
task_req.state = ComputeTaskState.ERROR
|
task_req.state = ComputeTaskState.ERROR
|
||||||
break
|
break
|
||||||
|
|
||||||
@@ -137,11 +155,13 @@ class ComputeKernel:
|
|||||||
time_out_result = ComputeTaskResult()
|
time_out_result = ComputeTaskResult()
|
||||||
time_out_result.result_code = ComputeTaskResultCode.TIMEOUT
|
time_out_result.result_code = ComputeTaskResultCode.TIMEOUT
|
||||||
time_out_result.set_from_task(task_req)
|
time_out_result.set_from_task(task_req)
|
||||||
## craete timeout task_result
|
task_req.result = time_out_result
|
||||||
|
return time_out_result
|
||||||
|
|
||||||
async def do_llm_completion(self, prompt: AgentPrompt, mode_name: Optional[str] = None, max_token: int = 0, inner_functions = None) -> str:
|
|
||||||
task_req = self.llm_completion(prompt, mode_name, max_token,inner_functions)
|
async def do_llm_completion(self, prompt: LLMPrompt,resp_mode:str="text", mode_name: Optional[str]=None, max_token:int=0, inner_functions=None, timeout=60) -> str:
|
||||||
return await self._wait_task(task_req)
|
task_req = self.llm_completion(prompt, resp_mode,mode_name, max_token,inner_functions)
|
||||||
|
return await self._wait_task(task_req, timeout)
|
||||||
|
|
||||||
|
|
||||||
def text_embedding(self,input:str,model_name:Optional[str] = None):
|
def text_embedding(self,input:str,model_name:Optional[str] = None):
|
||||||
@@ -181,7 +201,8 @@ class ComputeKernel:
|
|||||||
gender: Optional[str] = None,
|
gender: Optional[str] = None,
|
||||||
age: Optional[str] = None,
|
age: Optional[str] = None,
|
||||||
voice_name: Optional[str] = None,
|
voice_name: Optional[str] = None,
|
||||||
tone: Optional[str] = None):
|
tone: Optional[str] = None,
|
||||||
|
model_name: Optional[str] = None):
|
||||||
task_req = ComputeTask()
|
task_req = ComputeTask()
|
||||||
task_req.params["text"] = input
|
task_req.params["text"] = input
|
||||||
task_req.params["language_code"] = language_code
|
task_req.params["language_code"] = language_code
|
||||||
@@ -189,6 +210,7 @@ class ComputeKernel:
|
|||||||
task_req.params["age"] = age
|
task_req.params["age"] = age
|
||||||
task_req.params["voice_name"] = voice_name
|
task_req.params["voice_name"] = voice_name
|
||||||
task_req.params["tone"] = tone
|
task_req.params["tone"] = tone
|
||||||
|
task_req.params["model_name"] = model_name
|
||||||
task_req.task_type = ComputeTaskType.TEXT_2_VOICE
|
task_req.task_type = ComputeTaskType.TEXT_2_VOICE
|
||||||
self.run(task_req)
|
self.run(task_req)
|
||||||
|
|
||||||
@@ -197,6 +219,24 @@ class ComputeKernel:
|
|||||||
if task_req.state == ComputeTaskState.DONE:
|
if task_req.state == ComputeTaskState.DONE:
|
||||||
return task_result.result
|
return task_result.result
|
||||||
|
|
||||||
|
async def do_speech_to_text(self,
|
||||||
|
audio: str,
|
||||||
|
model: str,
|
||||||
|
prompt: Optional[str],
|
||||||
|
response_format: Optional[str]):
|
||||||
|
task_req = ComputeTask()
|
||||||
|
task_req.params["file"] = audio
|
||||||
|
task_req.params["model_name"] = model
|
||||||
|
task_req.params["prompt"] = prompt
|
||||||
|
task_req.params["response_format"] = response_format
|
||||||
|
task_req.task_type = ComputeTaskType.VOICE_2_TEXT
|
||||||
|
|
||||||
|
self.run(task_req)
|
||||||
|
|
||||||
|
task_result = await self._wait_task(task_req)
|
||||||
|
|
||||||
|
if task_req.state == ComputeTaskState.DONE:
|
||||||
|
return task_result
|
||||||
|
|
||||||
def text_2_image(self, prompt:str, model_name:Optional[str] = None, negative_prompt = None):
|
def text_2_image(self, prompt:str, model_name:Optional[str] = None, negative_prompt = None):
|
||||||
task = ComputeTask()
|
task = ComputeTask()
|
||||||
@@ -206,11 +246,19 @@ class ComputeKernel:
|
|||||||
|
|
||||||
async def do_text_2_image(self, prompt:str, model_name:Optional[str] = None, negative_prompt = None) -> ComputeTaskResult:
|
async def do_text_2_image(self, prompt:str, model_name:Optional[str] = None, negative_prompt = None) -> ComputeTaskResult:
|
||||||
task = self.text_2_image(prompt,model_name, negative_prompt)
|
task = self.text_2_image(prompt,model_name, negative_prompt)
|
||||||
task = await self._wait_task(task)
|
task_result = await self._wait_task(task)
|
||||||
|
|
||||||
return task.result
|
return task_result
|
||||||
# if task_req.state == ComputeTaskState.DONE:
|
# if task_req.state == ComputeTaskState.DONE:
|
||||||
# return None, task_result
|
# return None, task_result
|
||||||
|
|
||||||
# return task_req.error_str, None
|
def image_2_text(self, image_path: str, prompt:str, model_name:Optional[str] = None, negative_prompt = None):
|
||||||
|
task = ComputeTask()
|
||||||
|
task.set_image_2_text_params(image_path,prompt,model_name, negative_prompt)
|
||||||
|
self.run(task)
|
||||||
|
return task
|
||||||
|
async def do_image_2_text(self, image_path: str, prompt:str, model_name:Optional[str] = None, negative_prompt = None) -> ComputeTaskResult:
|
||||||
|
task = self.image_2_text(image_path,prompt, model_name, negative_prompt)
|
||||||
|
task = await self._wait_task(task)
|
||||||
|
return task.result
|
||||||
|
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
from abc import ABC, abstractmethod
|
from abc import ABC, abstractmethod
|
||||||
from .compute_task import ComputeTask, ComputeTaskType
|
|
||||||
|
|
||||||
|
from ..proto.compute_task import ComputeTask, ComputeTaskType
|
||||||
|
|
||||||
class ComputeNode(ABC):
|
class ComputeNode(ABC):
|
||||||
def __init__(self) -> None:
|
def __init__(self) -> None:
|
||||||
@@ -0,0 +1,76 @@
|
|||||||
|
from typing import List
|
||||||
|
import logging
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
from ..proto.agent_msg import AgentMsg
|
||||||
|
from .tunnel import AgentTunnel
|
||||||
|
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
class Contact:
|
||||||
|
def __init__(self, name, phone=None, email=None, telegram=None,added_by=None, tags=[], notes=""):
|
||||||
|
self.name = name
|
||||||
|
self.phone = phone
|
||||||
|
self.email = email
|
||||||
|
self.telegram = telegram
|
||||||
|
self.added_by = added_by
|
||||||
|
self.tags = tags
|
||||||
|
self.notes = notes
|
||||||
|
self.is_family_member = False
|
||||||
|
self.active_tunnels = {}
|
||||||
|
self.relationship = "friends"
|
||||||
|
|
||||||
|
def to_dict(self):
|
||||||
|
return {
|
||||||
|
"name": self.name,
|
||||||
|
"phone": self.phone,
|
||||||
|
"email": self.email,
|
||||||
|
"telegram" : self.telegram,
|
||||||
|
"is_family_member": self.is_family_member,
|
||||||
|
|
||||||
|
"added_by": self.added_by,
|
||||||
|
"tags": self.tags,
|
||||||
|
"notes": self.notes,
|
||||||
|
"now" : datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
|
||||||
|
"relationship" : self.relationship
|
||||||
|
}
|
||||||
|
|
||||||
|
async def _process_msg(self,msg:AgentMsg):
|
||||||
|
tunnel : AgentTunnel = self.get_active_tunnel(msg.sender)
|
||||||
|
if tunnel is not None:
|
||||||
|
await tunnel.post_message(msg)
|
||||||
|
return None
|
||||||
|
else:
|
||||||
|
tunnel = await self.create_default_tunnel(msg.sender)
|
||||||
|
if tunnel is not None:
|
||||||
|
self.active_tunnels[msg.sender] = tunnel
|
||||||
|
await tunnel.post_message(msg)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
logger.warn(f"contact {self.name} cann't get tunnel,post message failed!")
|
||||||
|
|
||||||
|
def get_active_tunnel(self,agent_id) -> AgentTunnel:
|
||||||
|
tunnel = self.active_tunnels.get(agent_id)
|
||||||
|
return tunnel
|
||||||
|
|
||||||
|
def set_active_tunnel(self,agent_id,tunnel:AgentTunnel):
|
||||||
|
self.active_tunnels[agent_id] = tunnel
|
||||||
|
|
||||||
|
async def create_default_tunnel(self,agent_id:str) -> AgentTunnel:
|
||||||
|
#TODO:fix this
|
||||||
|
from .email_tunnel import EmailTunnel
|
||||||
|
|
||||||
|
result_tunnels = AgentTunnel.get_tunnel_by_agentid(agent_id)
|
||||||
|
for tunnel in result_tunnels:
|
||||||
|
if isinstance(tunnel,EmailTunnel):
|
||||||
|
return tunnel
|
||||||
|
|
||||||
|
return None
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_dict(cls, data):
|
||||||
|
result = Contact(data.get("name"), data.get("phone"), data.get("email"), data.get("telegram"),data.get("added_by"), data.get("tags"), data.get("notes"))
|
||||||
|
if data.get("relationship") is not None:
|
||||||
|
result.relationship = data.get("relationship")
|
||||||
|
return result
|
||||||
@@ -1,48 +1,18 @@
|
|||||||
from typing import List
|
from typing import List
|
||||||
import toml
|
import toml
|
||||||
|
import time
|
||||||
|
import logging
|
||||||
|
|
||||||
class Contact:
|
from datetime import datetime
|
||||||
def __init__(self, name, phone=None, email=None, telegram=None,added_by=None, tags=[], notes=""):
|
from ..proto.agent_msg import AgentMsg
|
||||||
self.name = name
|
from ..proto.ai_function import ParameterDefine, SimpleAIFunction
|
||||||
self.phone = phone
|
from ..agent.llm_context import GlobaToolsLibrary
|
||||||
self.email = email
|
from .tunnel import AgentTunnel
|
||||||
self.telegram = telegram
|
from .contact import Contact
|
||||||
self.added_by = added_by
|
|
||||||
self.tags = tags
|
|
||||||
self.notes = notes
|
|
||||||
self.is_family_member = False
|
|
||||||
|
|
||||||
def to_dict(self):
|
|
||||||
return {
|
|
||||||
"name": self.name,
|
|
||||||
"phone": self.phone,
|
|
||||||
"email": self.email,
|
|
||||||
"telegram" : self.telegram,
|
|
||||||
|
|
||||||
"added_by": self.added_by,
|
logger = logging.getLogger(__name__)
|
||||||
"tags": self.tags,
|
|
||||||
"notes": self.notes
|
|
||||||
}
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def from_dict(cls, data):
|
|
||||||
return Contact(data.get("name"), data.get("phone"), data.get("email"), data.get("telegram"),data.get("added_by"), data.get("tags"), data.get("notes"))
|
|
||||||
|
|
||||||
class FamilyMember(Contact):
|
|
||||||
def __init__(self, name, relationship,phone=None, email=None,telegram=None):
|
|
||||||
super().__init__(name, phone, email, telegram)
|
|
||||||
self.name = name
|
|
||||||
self.relationship = relationship
|
|
||||||
self.is_family_member = True
|
|
||||||
|
|
||||||
def to_dict(self):
|
|
||||||
result = super().to_dict()
|
|
||||||
result["relationship"] = self.relationship
|
|
||||||
return result
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def from_dict(cls, data):
|
|
||||||
return FamilyMember(data.get("name"),data.get("relationship"),data.get("phone"), data.get("email"),data.get("telegram"))
|
|
||||||
|
|
||||||
class ContactManager:
|
class ContactManager:
|
||||||
_instance = None
|
_instance = None
|
||||||
@@ -52,10 +22,27 @@ class ContactManager:
|
|||||||
cls._instance = ContactManager(str(filename))
|
cls._instance = ContactManager(str(filename))
|
||||||
return cls._instance
|
return cls._instance
|
||||||
|
|
||||||
|
def register_global_functions(self):
|
||||||
|
gl = GlobaToolsLibrary.get_instance()
|
||||||
|
|
||||||
|
get_parameters = ParameterDefine.create_parameters({"name":"contact name name"})
|
||||||
|
gl.register_tool_function(SimpleAIFunction("system.contacts.get",
|
||||||
|
"get contact info",
|
||||||
|
self._get_contact,get_parameters))
|
||||||
|
|
||||||
|
# todo: use json to save contact info
|
||||||
|
update_parameters = ParameterDefine.create_parameters({"name":"name","contact_info":"A json to descrpit contact"})
|
||||||
|
gl.register_tool_function(SimpleAIFunction("system.contacts.set",
|
||||||
|
"set contact info",
|
||||||
|
self._set_contact,update_parameters))
|
||||||
|
|
||||||
|
return
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
def __init__(self, filename="contacts.toml"):
|
def __init__(self, filename="contacts.toml"):
|
||||||
self.filename = filename
|
self.filename = filename
|
||||||
self.contacts = []
|
self.contacts = []
|
||||||
self.family_members = []
|
|
||||||
|
|
||||||
self.is_auto_create_contact_from_telegram = True
|
self.is_auto_create_contact_from_telegram = True
|
||||||
|
|
||||||
@@ -69,12 +56,10 @@ class ContactManager:
|
|||||||
|
|
||||||
def load_from_config(self,config_data:dict):
|
def load_from_config(self,config_data:dict):
|
||||||
self.contacts = [Contact.from_dict(item) for item in config_data.get("contacts", [])]
|
self.contacts = [Contact.from_dict(item) for item in config_data.get("contacts", [])]
|
||||||
self.family_members = [FamilyMember.from_dict(item) for item in config_data.get("family_members", [])]
|
|
||||||
|
|
||||||
def save_data(self):
|
def save_data(self):
|
||||||
data = {
|
data = {
|
||||||
"contacts": [contact.to_dict() for contact in self.contacts],
|
"contacts": [contact.to_dict() for contact in self.contacts],
|
||||||
"family_members": [member.to_dict() for member in self.family_members]
|
|
||||||
}
|
}
|
||||||
with open(self.filename, "w") as f:
|
with open(self.filename, "w") as f:
|
||||||
toml.dump(data, f)
|
toml.dump(data, f)
|
||||||
@@ -108,46 +93,28 @@ class ContactManager:
|
|||||||
if contact.name == name:
|
if contact.name == name:
|
||||||
return contact
|
return contact
|
||||||
|
|
||||||
for member in self.family_members:
|
|
||||||
if member.name == name:
|
|
||||||
return member
|
|
||||||
return None
|
return None
|
||||||
|
|
||||||
def find_contact_by_telegram(self, telegram:str):
|
def find_contact_by_telegram(self, telegram:str):
|
||||||
for contact in self.contacts:
|
for contact in self.contacts:
|
||||||
if contact.telegram == telegram:
|
if contact.telegram == telegram:
|
||||||
return contact
|
return contact
|
||||||
for member in self.family_members:
|
|
||||||
if member.telegram == telegram:
|
|
||||||
return member
|
|
||||||
return None
|
return None
|
||||||
|
|
||||||
def find_contact_by_email(self, email:str):
|
def find_contact_by_email(self, email:str):
|
||||||
for contact in self.contacts:
|
for contact in self.contacts:
|
||||||
if contact.email == email:
|
if contact.email == email:
|
||||||
return contact
|
return contact
|
||||||
for member in self.family_members:
|
|
||||||
if member.email == email:
|
|
||||||
return member
|
|
||||||
return None
|
return None
|
||||||
|
|
||||||
def find_contact_by_phone(self, phone:str):
|
def find_contact_by_phone(self, phone:str):
|
||||||
for contact in self.contacts:
|
for contact in self.contacts:
|
||||||
if contact.phone == phone:
|
if contact.phone == phone:
|
||||||
return contact
|
return contact
|
||||||
for member in self.family_members:
|
|
||||||
if member.phone == phone:
|
|
||||||
return member
|
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
def add_family_member(self, name, new_member:FamilyMember):
|
|
||||||
assert name == new_member.name
|
|
||||||
self.family_members.append(new_member)
|
|
||||||
self.save_data()
|
|
||||||
|
|
||||||
def list_contacts(self):
|
def list_contacts(self):
|
||||||
return self.contacts
|
return self.contacts
|
||||||
|
|
||||||
def list_family_members(self):
|
|
||||||
return self.family_members
|
|
||||||
@@ -4,8 +4,7 @@ from asyncio import Queue
|
|||||||
import logging
|
import logging
|
||||||
from abc import abstractmethod
|
from abc import abstractmethod
|
||||||
|
|
||||||
from .compute_task import ComputeTask, ComputeTaskResult, ComputeTaskResultCode, ComputeTaskState, ComputeTaskType
|
from aios import ComputeTask, ComputeNode,ComputeTaskResult, ComputeTaskResultCode, ComputeTaskState, ComputeTaskType
|
||||||
from .compute_node import ComputeNode
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -1,7 +1,8 @@
|
|||||||
from abc import ABC, abstractmethod
|
from abc import ABC, abstractmethod
|
||||||
import logging
|
import logging
|
||||||
from typing import Coroutine
|
from typing import Coroutine
|
||||||
from .agent_message import AgentMsg
|
|
||||||
|
from ..proto.agent_msg import AgentMsg
|
||||||
from .bus import AIBus
|
from .bus import AIBus
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
@@ -54,13 +55,21 @@ class AgentTunnel(ABC):
|
|||||||
|
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
async def get_tunnel_by_agentid(cls,agent_id:str):
|
||||||
|
result = []
|
||||||
|
for tunnel in cls._all_tunnels.values():
|
||||||
|
if tunnel.target_id == agent_id:
|
||||||
|
result.append(tunnel)
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
def __init__(self) -> None:
|
def __init__(self) -> None:
|
||||||
super().__init__()
|
super().__init__()
|
||||||
self.tunnel_id = None
|
self.tunnel_id = None
|
||||||
self.target_id = None
|
self.target_id = None
|
||||||
self.target_type = None
|
self.target_type = None
|
||||||
self.ai_bus = None
|
self.ai_bus: AIBus = None
|
||||||
self.is_connected = False
|
self.is_connected = False
|
||||||
|
|
||||||
def connect_to(self, ai_bus:AIBus,target_id: str) -> None:
|
def connect_to(self, ai_bus:AIBus,target_id: str) -> None:
|
||||||
@@ -75,6 +84,9 @@ class AgentTunnel(ABC):
|
|||||||
self.ai_bus = ai_bus
|
self.ai_bus = ai_bus
|
||||||
self.is_connected = True
|
self.is_connected = True
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def post_message(self, msg: AgentMsg) -> None:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
@@ -3,3 +3,4 @@ from .vector import *
|
|||||||
from .data import *
|
from .data import *
|
||||||
from .store import KnowledgeStore
|
from .store import KnowledgeStore
|
||||||
from .core_object import *
|
from .core_object import *
|
||||||
|
from .pipeline import *
|
||||||
+3
-4
@@ -1,7 +1,6 @@
|
|||||||
from ..object import KnowledgeObject, ObjectRelationStore
|
from ..object import KnowledgeObject, ObjectRelationStore
|
||||||
from ..data import ChunkList, ChunkListWriter
|
from ..data import ChunkList, ChunkListWriter
|
||||||
from ..object import ObjectType
|
from ..object import ObjectType
|
||||||
from .. import KnowledgeStore
|
|
||||||
|
|
||||||
# desc
|
# desc
|
||||||
# meta
|
# meta
|
||||||
@@ -49,13 +48,13 @@ class DocumentObjectBuilder:
|
|||||||
self.text = text
|
self.text = text
|
||||||
return self
|
return self
|
||||||
|
|
||||||
def build(self) -> DocumentObject:
|
def build(self, store) -> DocumentObject:
|
||||||
chunk_list = KnowledgeStore().get_chunk_list_writer().create_chunk_list_from_text(self.text)
|
chunk_list = store.get_chunk_list_writer().create_chunk_list_from_text(self.text)
|
||||||
doc = DocumentObject(self.meta, self.tags, chunk_list)
|
doc = DocumentObject(self.meta, self.tags, chunk_list)
|
||||||
doc_id = doc.calculate_id()
|
doc_id = doc.calculate_id()
|
||||||
|
|
||||||
# Add relation to store
|
# Add relation to store
|
||||||
for chunk_id in chunk_list.chunk_list:
|
for chunk_id in chunk_list.chunk_list:
|
||||||
KnowledgeStore().get_relation_store().add_relation(chunk_id, doc_id)
|
store.get_relation_store().add_relation(chunk_id, doc_id)
|
||||||
|
|
||||||
return doc
|
return doc
|
||||||
+3
-4
@@ -1,4 +1,3 @@
|
|||||||
from .. import KnowledgeStore
|
|
||||||
from .rich_text_object import RichTextObject, RichTextObjectBuilder
|
from .rich_text_object import RichTextObject, RichTextObjectBuilder
|
||||||
from ..object import ObjectID, ObjectType, KnowledgeObject
|
from ..object import ObjectID, ObjectType, KnowledgeObject
|
||||||
from .document_object import DocumentObjectBuilder
|
from .document_object import DocumentObjectBuilder
|
||||||
@@ -68,11 +67,11 @@ class EmailObjectBuilder:
|
|||||||
self.folder = folder
|
self.folder = folder
|
||||||
return self
|
return self
|
||||||
|
|
||||||
def build(self) -> EmailObject:
|
def build(self, store) -> EmailObject:
|
||||||
|
|
||||||
# Just get the object store and relation store from global KnowledgeStore
|
# Just get the object store and relation store from global KnowledgeStore
|
||||||
store = KnowledgeStore().get_object_store()
|
store = store.get_object_store()
|
||||||
relation = KnowledgeStore().get_relation_store()
|
relation = store.get_relation_store()
|
||||||
|
|
||||||
# Read meta.json
|
# Read meta.json
|
||||||
meta = {}
|
meta = {}
|
||||||
+2
-3
@@ -1,7 +1,6 @@
|
|||||||
from ..object import KnowledgeObject
|
from ..object import KnowledgeObject
|
||||||
from ..data import ChunkList, ChunkListWriter
|
from ..data import ChunkList, ChunkListWriter
|
||||||
from ..object import ObjectType
|
from ..object import ObjectType
|
||||||
from .. import KnowledgeStore
|
|
||||||
import os
|
import os
|
||||||
|
|
||||||
# desc
|
# desc
|
||||||
@@ -86,10 +85,10 @@ class ImageObjectBuilder:
|
|||||||
self.restore_file = restore_file
|
self.restore_file = restore_file
|
||||||
return self
|
return self
|
||||||
|
|
||||||
def build(self) -> ImageObject:
|
def build(self, store) -> ImageObject:
|
||||||
|
|
||||||
file_size = os.path.getsize(self.image_file)
|
file_size = os.path.getsize(self.image_file)
|
||||||
chunk_list = KnowledgeStore().get_chunk_list_writer().create_chunk_list_from_file(
|
chunk_list = store.get_chunk_list_writer().create_chunk_list_from_file(
|
||||||
self.image_file, 1024 * 1024 * 4, self.restore_file
|
self.image_file, 1024 * 1024 * 4, self.restore_file
|
||||||
)
|
)
|
||||||
exif = get_exif_data(self.image_file)
|
exif = get_exif_data(self.image_file)
|
||||||
+1
-1
@@ -1,4 +1,4 @@
|
|||||||
from knowledge.object.object_id import ObjectType
|
from ..object.object_id import ObjectType
|
||||||
from ..object import KnowledgeObject
|
from ..object import KnowledgeObject
|
||||||
from ..data import ChunkList, ChunkListWriter
|
from ..data import ChunkList, ChunkListWriter
|
||||||
from ..object import ObjectType
|
from ..object import ObjectType
|
||||||
+2
-3
@@ -1,7 +1,6 @@
|
|||||||
from ..object import KnowledgeObject
|
from ..object import KnowledgeObject
|
||||||
from ..data import ChunkList, ChunkListWriter
|
from ..data import ChunkList, ChunkListWriter
|
||||||
from ..object import ObjectType
|
from ..object import ObjectType
|
||||||
from .. import KnowledgeStore
|
|
||||||
|
|
||||||
# desc
|
# desc
|
||||||
# meta
|
# meta
|
||||||
@@ -76,8 +75,8 @@ class VideoObjectBuilder:
|
|||||||
self.restore_file = restore_file
|
self.restore_file = restore_file
|
||||||
return self
|
return self
|
||||||
|
|
||||||
def build(self) -> VideoObject:
|
def build(self, store) -> VideoObject:
|
||||||
chunk_list = KnowledgeStore().get_chunk_list_writer().create_chunk_list_from_file(
|
chunk_list = store.get_chunk_list_writer().create_chunk_list_from_file(
|
||||||
self.video_file, 1024 * 1024 * 4, self.restore_file
|
self.video_file, 1024 * 1024 * 4, self.restore_file
|
||||||
)
|
)
|
||||||
info = get_video_info(self.video_file)
|
info = get_video_info(self.video_file)
|
||||||
@@ -86,7 +86,7 @@ def _split_text_with_regex(
|
|||||||
return [s for s in splits if s != ""]
|
return [s for s in splits if s != ""]
|
||||||
|
|
||||||
|
|
||||||
def _split_text(
|
def split_text(
|
||||||
text: str,
|
text: str,
|
||||||
separators: List[str],
|
separators: List[str],
|
||||||
chunk_size: int,
|
chunk_size: int,
|
||||||
@@ -127,7 +127,7 @@ def _split_text(
|
|||||||
if not new_separators:
|
if not new_separators:
|
||||||
final_chunks.append(s)
|
final_chunks.append(s)
|
||||||
else:
|
else:
|
||||||
other_info = _split_text(s, new_separators, chunk_size, chunk_overlap, length_function)
|
other_info = split_text(s, new_separators, chunk_size, chunk_overlap, length_function)
|
||||||
final_chunks.extend(other_info)
|
final_chunks.extend(other_info)
|
||||||
if _good_splits:
|
if _good_splits:
|
||||||
merged_text = _merge_splits(_good_splits, _separator, chunk_size, chunk_overlap, length_function)
|
merged_text = _merge_splits(_good_splits, _separator, chunk_size, chunk_overlap, length_function)
|
||||||
@@ -197,7 +197,7 @@ class ChunkListWriter:
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
text_list = _split_text(text, separators, chunk_size, chunk_overlap, length_function)
|
text_list = split_text(text, separators, chunk_size, chunk_overlap, length_function)
|
||||||
chunk_list = []
|
chunk_list = []
|
||||||
hash_obj = hashlib.sha256()
|
hash_obj = hashlib.sha256()
|
||||||
|
|
||||||
@@ -48,10 +48,22 @@ class KnowledgeObject(ABC):
|
|||||||
def get_body(self) -> dict:
|
def get_body(self) -> dict:
|
||||||
return self.body
|
return self.body
|
||||||
|
|
||||||
|
def get_summary(self) -> str:
|
||||||
|
return self.desc.get("summary")
|
||||||
|
|
||||||
|
# def get_articl_catelog(self) -> str:
|
||||||
|
# assert self.object_type == ObjectType.Document
|
||||||
|
# return self.desc.get("catelog")
|
||||||
|
|
||||||
|
# def get_article_full_content(self) -> str:
|
||||||
|
# assert self.object_type == ObjectType.Document
|
||||||
|
# return self.body
|
||||||
|
|
||||||
def calculate_id(self):
|
def calculate_id(self):
|
||||||
# Convert the object_type and desc to string and compute the SHA256 hash
|
# Convert the object_type and desc to string and compute the SHA256 hash
|
||||||
data = json.dumps(
|
data = json.dumps(
|
||||||
{"object_type": self.object_type, "desc": self.desc},
|
{"object_type": self.object_type, "desc": self.desc},
|
||||||
|
ensure_ascii=False,
|
||||||
cls=ObjectEnhancedJSONEncoder,
|
cls=ObjectEnhancedJSONEncoder,
|
||||||
)
|
)
|
||||||
sha256 = hashlib.sha256()
|
sha256 = hashlib.sha256()
|
||||||
@@ -63,5 +75,5 @@ class KnowledgeObject(ABC):
|
|||||||
return pickle.dumps(self)
|
return pickle.dumps(self)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def decode(data: bytes) -> "ImageObject":
|
def decode(data: bytes) -> "KnowledgeObject":
|
||||||
return pickle.loads(data)
|
return pickle.loads(data)
|
||||||
@@ -13,6 +13,17 @@ class ObjectType(IntEnum):
|
|||||||
Document = 103
|
Document = 103
|
||||||
RichText = 104
|
RichText = 104
|
||||||
Email = 105
|
Email = 105
|
||||||
|
UserDef = 200
|
||||||
|
|
||||||
|
def is_user_def(self) -> bool:
|
||||||
|
return self.value >= 200
|
||||||
|
|
||||||
|
def get_user_def_type_code(self):
|
||||||
|
return (self.value - 200) if self.is_user_def() else None
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_user_def_type_code(cls, value):
|
||||||
|
return value + 200
|
||||||
|
|
||||||
|
|
||||||
# define a object ID class to identify a object
|
# define a object ID class to identify a object
|
||||||
@@ -56,3 +67,6 @@ class ObjectID: # pylint: disable=too-few-public-methods
|
|||||||
|
|
||||||
def __eq__(self, other) -> bool:
|
def __eq__(self, other) -> bool:
|
||||||
return self.value == other.value
|
return self.value == other.value
|
||||||
|
|
||||||
|
def __hash__(self):
|
||||||
|
return hash(self.value)
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user