Merge branch 'MVP' into MVP
This commit is contained in:
Liu Zhicong
GitHub
@@ -0,0 +1,27 @@
|
||||
**/__pycache__
|
||||
**/*venv
|
||||
**/.classpath
|
||||
**/.dockerignore
|
||||
**/.env
|
||||
**/.git
|
||||
**/.gitignore
|
||||
**/.project
|
||||
**/.settings
|
||||
**/.toolstarget
|
||||
**/.vs
|
||||
**/.vscode
|
||||
**/*.*proj.user
|
||||
**/*.dbmdl
|
||||
**/*.jfm
|
||||
**/bin
|
||||
**/charts
|
||||
**/docker-compose*
|
||||
**/compose*
|
||||
**/Dockerfile*
|
||||
**/node_modules
|
||||
**/npm-debug.log
|
||||
**/obj
|
||||
**/secrets.dev.yaml
|
||||
**/values.dev.yaml
|
||||
**/.db
|
||||
**/.python-version
|
||||
@@ -2,3 +2,11 @@
|
||||
*.pyc
|
||||
rootfs/data
|
||||
*.log
|
||||
rootfs/email/config.local.toml
|
||||
rootfs/data
|
||||
venv
|
||||
aios_shell.log
|
||||
history.txt
|
||||
math_school_env.db
|
||||
workflows.db
|
||||
|
||||
|
||||
+15
@@ -0,0 +1,15 @@
|
||||
FROM python:3.11
|
||||
WORKDIR /opt/aios
|
||||
COPY ./src /opt/aios
|
||||
COPY ./rootfs /opt/aios/app
|
||||
|
||||
RUN mkdir -p /root/myai/app
|
||||
RUN mkdir -p /root/myai/data
|
||||
RUN mkdir -p /root/myai/etc
|
||||
|
||||
RUN pip install --no-cache-dir -r /opt/aios/requirements.txt
|
||||
|
||||
ENV PYTHONDONTWRITEBYTECODE 1
|
||||
ENV PYTHONUNBUFFERED 1
|
||||
|
||||
CMD ["python3","./service/aios_shell/aios_shell.py"]
|
||||
@@ -0,0 +1,5 @@
|
||||
#!/usr/bin/bash
|
||||
|
||||
pipreqs ./src --force
|
||||
# Build the docker image
|
||||
docker build -t aios .
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 136 KiB |
@@ -0,0 +1,500 @@
|
||||
# Overview
|
||||
|
||||
The core goal of version 0.5.1 is to turn the concept of AIOS into code and get it up and running as quickly as possible. After three weeks of development, our plans have undergone some changes based on the actual progress of the system. Under the guidance of this goal, some components do not need to be fully implemented. Furthermore, based on the actual development experience from several demo Intelligent Applications, we intend to strengthen some components. This document will explain these changes and provide an update on the current development progress of MVP(0.5.1,0.5.2)
|
||||
|
||||
The previous plan, please see here: [MVP Plan](./mvp%20plan.md)
|
||||
|
||||
# Progress Status of MVP
|
||||
|
||||
- Each module includes whether the current version goals have been met, the current person in charge, and workload assessment.
|
||||
- Modules that are not marked for version 0.5.2 and do not have a designated person in charge are modules for which we are currently recruiting contributors.
|
||||
- Modules that have not been completed but already have a designated person in charge are modules that are currently in development.
|
||||
|
||||
- [x] AIOS Kernel
|
||||
- [x] Agent,@waterflier, A2
|
||||
- [ ] Optimization of system prompts,A2
|
||||
- [x] Workflow,@waterflier, A2
|
||||
- [x] AI Environments,@waterflier, A2
|
||||
- [x] Celender Environment,@waterflier, S2
|
||||
- [ ] Compatible with common Celender services(0.5.2),A2
|
||||
- [ ] Microsoft Outlook Calendar, S2
|
||||
- [ ] Google Calendar, S2
|
||||
- [ ] Apple Calendar, S2
|
||||
- [ ] Workspace Environment(0.5.2) @waterflier, A2
|
||||
- [ ] AI Functions,@waterflier,A2
|
||||
- [ ] Basic AI Functions(0.5.2)
|
||||
- [x] AI BUS,@waterflier, A2
|
||||
- [x] Chatsession,@waterflier, S2
|
||||
- [ ] Knowlege Base,@lurenpluto ,@photosssa, A8
|
||||
- [ ] Personal Models(>0.5.2),A8
|
||||
- [ ] Frame Services(0.5.2)
|
||||
- [ ] Kernel Service
|
||||
- [ ] System-Call Interface,A2
|
||||
- [ ] Name Service,A4
|
||||
- [ ] Node Daemon,A2
|
||||
- [ ] ACL Control,A4
|
||||
- [ ] Contact Manager,A2
|
||||
- [ ] Runtime Context (0.5.2),A4
|
||||
- [ ] Package System,@waterflier, A2+S4
|
||||
- [x] AI Compute System,@waterflier, A2
|
||||
- [ ] Scheduler,@streetycat, A2
|
||||
- [x] LLM
|
||||
- [x] GPT4 (Cloud),@waterflier, S1
|
||||
- [ ] LLaMa2,@streetycat, A2
|
||||
- [ ] Claude2, S2
|
||||
- [ ] Falcon2, S2
|
||||
- [ ] MPT-7B, S2
|
||||
- [ ] Vicuna, S2
|
||||
- [ ] Embeding,@photosssa,@lurenpluto , A4
|
||||
- [x] Txt2img,@glen0125,A4
|
||||
- [ ] Img2txt(0.5.2),A3
|
||||
- [ ] Txt2voice,A3
|
||||
- [ ] Voice2txt, @wugren,A3
|
||||
- [ ] Language Translate (Pending)
|
||||
- [x] Storage System
|
||||
- [ ] DFS (0.5.2),A4
|
||||
- [ ] Object Storage, @lurenpluto ,A2
|
||||
- [ ] D-RDB (0.5.2),A2
|
||||
- [ ] D-VDB,@lurenpluto , A4
|
||||
- [ ] Embeding Piplines,@photosssa, A2
|
||||
- [ ] Network Gateway,A6
|
||||
- [x] NDN Client, @waterflier, A1
|
||||
- [ ] Build-in Service
|
||||
- [x] Spider,@alexsunxl, A2
|
||||
- [x] E-mail Spider,@alexsunxl, S2
|
||||
- [ ] Telegram Spider,S2
|
||||
- [ ] Twitter Spider (0.5.2)
|
||||
- [ ] Facebook Spider (0.5.2)
|
||||
- [ ] Agent Message Tunnel (0.5.2)
|
||||
- [ ] E-mail Tunnel,A2
|
||||
- [ ] Telegram Tunnel,S2
|
||||
- [ ] Discord Tunnel,S2
|
||||
- [ ] Home IoT Environment (0.5.2), A4
|
||||
- [ ] Compatible Home Assistant (0.5.2), A4
|
||||
- [ ] Build-in Agents/Apps
|
||||
- [ ] Agent: Personal Information Assistant,@photosssa,@lurenpluto , A2
|
||||
- [ ] Agent: Bulter Jarvis,@waterflier, A2
|
||||
- [ ] App: Personal Station (0.5.2),A4+S4
|
||||
- [ ] UI
|
||||
- [x] CLI UI (aios_shell),@waterflier,S2
|
||||
- [ ] Web UI (0.5.2),A4+S4
|
||||
- [ ] 0.5.1 Integration Test (Senior*3)
|
||||
- [x] Workflow -> AI Agent -> AI Agent,@waterflier,S1
|
||||
- [ ] Spider -> Pipline -> Knowledge Base,@photosssa,S2
|
||||
- [ ] AI Agent <- Functions <- Knowledge Base,@lurenpluto,S2
|
||||
- [ ] SDK
|
||||
- [x] Workflow SDK,@waterflier, A2
|
||||
- [ ] AI Environments SDK (0.5.2), A2
|
||||
- [ ] Compute Kernel SDK (0.5.2), A2
|
||||
- [ ] Document (>0.5.2)
|
||||
- [ ] System design document, including the design document of each subsystem
|
||||
- [ ] Installation/use document for end users
|
||||
- [ ] SDK document for developers
|
||||
|
||||
|
||||
The following is the introduction of the adjustment of each component after the current implementation.
|
||||
|
||||
# AIOS Kernel
|
||||
|
||||
Define some of the important basic concepts of Intelligent Applications running on OpenDAN
|
||||
|
||||
## Agent
|
||||
|
||||
Agent is the core concept of the system, created through appropriate LLM, prompt words, and memory. Agents support our vision of a new relationship between humans and computation in the future:
|
||||
```
|
||||
Human <-> Agent <-> Compute
|
||||
```
|
||||
Agents form the basis of future intelligent applications. From the user's perspective, the strength of AIOS is primarily determined by "how many agents with different capabilities it has."
|
||||
The above process has now been implemented. In practice, I found a key issue is that we need to continuously seek the optimal solution. This issue directly relates to how application developers of OpenDAN build intelligent applications, so I think it has a high priority.
|
||||
|
||||
### Optimization of system prompts
|
||||
|
||||
The goal is to allow Agents to communicate with other Agents (forming a team), call Functions at the right time, and read/write status through the environment at the right time, using prompt words. The existing implementation is usually:
|
||||
|
||||
```
|
||||
When you decide to communicate with a work group, please use : sendmsg(group_name, content).
|
||||
```
|
||||
|
||||
Our optimization direction is:
|
||||
|
||||
1. To allow Agents to initiate calls accurately
|
||||
2. To use as few precious prompt word resources as possible.
|
||||
|
||||
If there are already systematic studies in this field, introductions are also welcome!
|
||||
|
||||
## Workflow
|
||||
|
||||
Workflow has realized the concept of allowing multiple Agents to play different roles and collaboratively solve complex problems within an organization. It is also the main form of intelligent applications on OpenDAN. Compared to a single Agent, building a team composed of Agents can effectively solve three inherent problems of LLM:
|
||||
|
||||
1. The prompt word window will grow, but it will remain limited for a long time.
|
||||
2. Like humans, Agents trained on different corpora and algorithms will have different personalities and will excel in different roles.
|
||||
3. The inference results of LLM are uncontrollable, so accuracy cannot be guaranteed. Just like humans make mistakes, the collaboration of multiple Agents is needed to improve accuracy.
|
||||
|
||||
The basic framework of Workflow has been completed (which is also the core of version 0.5.1). Following the subsequent SDK documentation, we now have a basic framework for third-party developers to develop applications on OpenDAN.
|
||||
|
||||
## AI Environments
|
||||
|
||||
Environments provide an abstraction for AI Agents to access the real world.
|
||||
|
||||
Environments include properties, events, methods (Env.Function), and come with natural language descriptions that Agents can understand. This allows AI Agents to understand the current environment and when to access it. For example, an Agent planning a trip needs to understand the real weather conditions at the destination in the future to make the right decisions. This weather condition needs to be provided to the Agent through Environments.
|
||||
|
||||
The events in Environments also provide logic for the autonomous work of Agents. For example, an Agent can track changes in the user's schedule and date, automatically helping the user plan and track the specific itinerary for the day.
|
||||
|
||||
### Celender Environment
|
||||
|
||||
The system's default environment, which can access the current time, the user's schedule, and the weather information at a specific location. It also contains some important and basic user information, including home address and office address.
|
||||
|
||||
#### Compatible with common Celender services. (0.5.2)
|
||||
|
||||
- Microsoft Outlook Calendar
|
||||
- Google Calendar
|
||||
- Apple Calendar
|
||||
|
||||
### Workspace Environment (0.5.2)
|
||||
|
||||
A file system-based workspace environment that allows the Agent to read/write files at appropriate times.
|
||||
|
||||
## AI Functions
|
||||
|
||||
Function is a core concept of AIOS, providing the Agent with descriptions of suitable callable Functions, allowing the Agent to invoke Functions at the right time. Through Functions, the Agent gains "execution power," rather than just being an advisor that only provides suggestions. The Function framework allows third-party developers to develop and publish Functions, supporting Agents and Workflows to have a list of available Functions, through which they can build appropriate prompt words, enabling the Agent to invoke Functions at the right time.
|
||||
|
||||
**Under development.**
|
||||
|
||||
### Basic AI Functions （0.5.2）
|
||||
|
||||
There are already a plethora of basic services in the world, such as querying the weather at a specific location at a specific time, checking hotel prices, or booking plane tickets. The system should separate the definition and implementation of Basic (generic) Functions, allowing Agent developers to implement common scenarios with generic logic. The definition of generic Functions is undoubtedly similar to standard setting work.
|
||||
|
||||
I know that many other projects have done a lot of work in this field, and ChatGPT also has dedicated function support. What we need to do is to find the open standards that are closest to our goals and then integrate them.
|
||||
|
||||
## AI BUS
|
||||
|
||||
The AI BUS connects various conceptual entities of OpenDAN. For example, if Agent A wants to send a message to another Agent B and wait for the processing result of the message, it can simply use the AI BUS:
|
||||
|
||||
```python
|
||||
resp = await AIBus.send_msg(agentA,agentB,msg)
|
||||
```
|
||||
|
||||
The abstraction of AI BUS allows different Agents to choose suitable physical hosts to run according to the system's needs. This is also why we define AIOS as a "Network OS". All entities registered on the AI BUS can be accessed via the AI BUS interface. As needed, we will also persist the messages in the AI BUS, so that when a distributed system experiences regular failures, it can continue to work after being pulled up again.
|
||||
|
||||
The concept of AI BUS has many similarities with traditional MessageQueues.
|
||||
|
||||
## Chatsession
|
||||
|
||||
Intuitively, ChatSession saves the "chat history". The chat history is currently the natural source of Agent Memory capability.
|
||||
Determining a ChatSession has three key attributes: Owner, Topic, and Remote. An operation where A sends a message to B and gets B's reply will generate two messages, and save them in two different ChatSessions.
|
||||
|
||||
Currently, ChatSession is saved based on sqlite. After the Zone-level D-RDB is set up in the future, it will be migrated to RD-DB.
|
||||
|
||||
## Knowlege Base
|
||||
|
||||
Provide a unified interface, support switching vector database kernel
|
||||
Integrate open source vector database (pay attention to Lience selection)
|
||||
When designing the interface, prepare for future access control
|
||||
|
||||
**Under development.**
|
||||
|
||||
## Personal Models （>0.5.2）
|
||||
|
||||
The goal of this subsystem is to support users in training models based on their own data, including subsequent usage, management, deployment, and other operations of the model. In the early stages, invoking this module and adding new models should be operations performed by advanced users.
|
||||
|
||||
It is still uncertain whether this module will be actively used in intelligent applications.
|
||||
|
||||
# Frame Services
|
||||
|
||||
The implementation offers a range of fundamental services for traditional Network OS. It connects users' devices to the same Zone via the network and provides a unified abstraction for application access. This component serves as a basic framework and computing resource for the operation of intelligent applications on the upper layer. On the lower layer, it connects various types of hardware through different protocols, integrates resources, and offers a unified abstraction for intelligent applications to access.
|
||||
|
||||
## Kernel Service (0.5.2)
|
||||
|
||||
The Kernel Service implements the System Calls for OpenDAN and provides a "kernel mode" abstraction. In version 0.5.1, since this component is not yet implemented, all code—whether system services or application code—runs in kernel mode.
|
||||
|
||||
In the future, we plan to maintain the system running in this mode for an extended period, as it facilitates debugging.
|
||||
|
||||
The Kernel Service is mainly composed of the following component:
|
||||
|
||||
### System-Call Interface
|
||||
|
||||
Centralizes the provision and management of system call interfaces.
|
||||
|
||||
### Name Service
|
||||
|
||||
It is the most crucial foundational state service in a cluster (Zone) comprised of all the user's devices. As the core service of the Zone, it provides the most basic guarantee for the availability and reliability of all services within the Zone. When a user needs to restore the Zone from a backup, the Name Service is the first service to be restored.
|
||||
|
||||
Its functionality is similar to that of `etcd` but includes a on-chain component. From a deployment standpoint, it needs to be operationally optimized for small clusters made up of consumer-level user devices.
|
||||
|
||||
### Node Daemon
|
||||
|
||||
It is a foundational service that runs on all devices that join the Zone, responding to essential kernel scheduling commands. It adjusts the services and data running on that particular device.
|
||||
|
||||
### ACL Control (>0.5.2)
|
||||
|
||||
Another essential foundational service of the kernel, it is responsible for the overall management of permissions related to users, applications, and data. The Runtime Context reads the relevant information and implements proper isolation.
|
||||
|
||||
### Contact Manager
|
||||
|
||||
From the perspectives of permission control and some early application scenarios, understanding the user's basic interpersonal relationships is an important component of OpenDAN's intelligent permission system. Therefore, we provide a contact management component at the system kernel layer. This component can be considered an upgraded version of the traditional operating system's "User Group" module.
|
||||
|
||||
## Runtime Context (0.5.2)
|
||||
|
||||
It serves as the runtime container for user-mode code, offering isolation guarantees for user-mode code.
|
||||
|
||||
Depending on the type of service, we offer three different Runtime Contexts. The most commonly used is Docker, followed by virtual machines, and finally, entire physical machines.
|
||||
|
||||
## Package System
|
||||
|
||||
The Package Manager is a fundamental component of the system for managing Packages. The sub system provides fundamental support for packaging, publishing, downloading, verifying, installing, and loading folders containing required packages under different scenarios. Based on relevant modules, it's easy to build a package management system similar to apt/pip/npm.
|
||||
|
||||
The system design has deeply referenced Git and NDN networks. The distinction between client and server is not that important. Through cryptography, it achieves decentralized trustworthy verification. Any client can become an effective repo server through simple configuration.
|
||||
|
||||
Based on the Package System, we can implement the publishing, downloading, and installation of extendable foundational entities such as Agents, Functions, and Environments. This enables the creation of an app store on OpenDAN.
|
||||
|
||||
**Under development.**
|
||||
|
||||
# AI Compute System
|
||||
|
||||
The purpose of designing Compute System is to enable our users to use their computational resources more efficiently. These computational resources can come from devices they own (such as their workstations and gaming laptops), as well as from cloud computing and decentralized computing networks.
|
||||
|
||||
[](compute_task.drawio)
|
||||
|
||||
The interface of this component is designed from the perspective of the model user rather than the model trainer. The basic form of its interface is:
|
||||
|
||||
```python
|
||||
compute_kernel.do_compute(function_name, model_name,args)
|
||||
```
|
||||
|
||||
## Scheduler
|
||||
|
||||
The goal of the Scheduler component is to select an appropriate ComputeNode to run tasks based on the tasks in the task queue and the known status of all ComputeNodes (which may be delayed). In the current version (0.5.1), the implementation of the Scheduler is only to get the system up and running. In the next version (0.5.2), the overall framework for computing resource scheduling needs to be established.
|
||||
|
||||
## LLM
|
||||
|
||||
LLM support is the system's most core functionality. OpenDAN requires that there be at least one available LLM computing node in the system. The supported interfaces are as follows:
|
||||
```
|
||||
def llm_completion(self,prompt:AgentPrompt,mode_name:Optional[str] = None,max_token:int = 0):
|
||||
```
|
||||
In the current era, many teams are working hard to develop new LLMs . We will also actively integrate these LLMs into OpenDAN.
|
||||
|
||||
- [x] GPT4 (Cloud)
|
||||
- [ ] LLaMa2 **Under development.**
|
||||
- [ ] Claude2
|
||||
- [ ] Falcon2
|
||||
- [ ] MPT-7B
|
||||
- [ ] Vicuna
|
||||
|
||||
|
||||
## Embeding
|
||||
|
||||
Provides computational support for the vectorization of different types of user data. The specific algorithms supported depend on the requirements of the entire pipeline.
|
||||
|
||||
***Under development.***
|
||||
|
||||
## Txt2img
|
||||
|
||||
Generate images based on text descriptions. According to the implementation mode, we can interface with a cloud-based implementation and a local implementation.
|
||||
|
||||
The local implementation will definitely use Stable Diffusion.
|
||||
|
||||
***Under development.***
|
||||
|
||||
## Img2txt (>0.5.2)
|
||||
|
||||
Generate appropriate text descriptions for the specified images.
|
||||
|
||||
## Txt2voice
|
||||
|
||||
Generate voice based on specified text, using a selected model (the focus is on personal models), and guided by certain emotional cue words.
|
||||
|
||||
***To be developed***
|
||||
|
||||
## Voice2txt
|
||||
|
||||
Extract text information from a segment of audio (or video) through speech recognition.
|
||||
|
||||
***To be developed***
|
||||
|
||||
## Language Translate
|
||||
|
||||
Translate a segment of text into a specified target language.
|
||||
|
||||
Since LLM itself is developed based on the foundation of translation, I am currently considering whether it is necessary to provide a text translation interface within the computing kernel. Following the principle of not adding entities if they are not needed, it can be postponed from development.
|
||||
***pending***
|
||||
|
||||
# Storage System
|
||||
|
||||
The file system (state storage) has always been a critical part of operating systems. Its implementation directly impacts the system's reliability and performance. The challenge of this section is how to transfer key technologies that are already mature in traditional cloud computing to clusters composed of consumer-level electronic devices with low operational maintenance, while still maintaining sufficient reliability and performance. The implementation of the subsystems in this section is of limited stability. Therefore, I believe the focus of OpenDAN in the early stages for this section should be on establishing stable interfaces to get the system running as quickly as possible, with independent improvements to be made in the future.
|
||||
|
||||
From the standpoint of trade-offs, our priorities are:
|
||||
|
||||
- Abandoning continuous consistency guarantees, the system only provides strong assurance for reliability up to "backup points." This means we allow the loss of some newly added data if the system experiences a failure.
|
||||
|
||||
- Allowing downtime, considering the consumer-level power supply, a short period of unavailability of the system itself will not have a significant impact. We can stop the service for backup/migration when necessary.
|
||||
|
||||
## DFS
|
||||
|
||||
Distributed file system, combining the public storage space on all devices to form a highly reliable, highly available file system.
|
||||
|
||||
## Object Storage
|
||||
|
||||
Distributed object storage, and based on MapObject, it implements trustworthy RootState management.
|
||||
|
||||
(MapObject and RootState is a concept from CYFS)
|
||||
|
||||
**Under development.**
|
||||
|
||||
## D-RDB
|
||||
|
||||
Distributed relational database, providing highly reliable and highly available relational database services (mainly used for OLTP - Online Transaction Processing). We do not encourage application developers to use RDB on a large scale; the main reason for offering this component is for compatibility considerations.
|
||||
|
||||
***Pending.***
|
||||
|
||||
## D-VDB
|
||||
|
||||
Distributed vector database, which currently appears to be the core foundational component of the Knowledge Base library.
|
||||
|
||||
***Under development.***
|
||||
|
||||
# Embeding Piplines
|
||||
|
||||
Read appropriate Raw Files and Meta Data from the specified location in the Storage System. After passing through a series of Embedding Pipelines, save the results to the Vector Database as defined by the Knowledge Base.
|
||||
|
||||
***Under development.***
|
||||
|
||||
# Network Gateway (0.5.2)
|
||||
|
||||
Obtain user data by recognizing network data.
|
||||
The Gateway also provides an external access entrance for the entire system, and access control can be unified.
|
||||
Provides the bus abstraction in the network operating system (the network cable is the bus), devices within the Zone are recognized by the system as plug-and-play devices, and can be called by applications/Agents
|
||||
|
||||
## NDN Client
|
||||
|
||||
AI-related models are all quite large, so we offer a download tool based on NDN (Named Data Networking) theory to replace curl. The NDN Client will continue to support new Content-Based protocols in the future, allowing OpenDAN developers to publish large packages more quickly, at lower costs, and more conveniently.
|
||||
|
||||
# Build-in Service
|
||||
|
||||
The basic functions of the system implemented by "user mode" can be regarded as pre -installed applications of the system.Let the system have basic availability without installing any intelligent applications.
|
||||
We should build built-in applications for 1,2 early preset scenarios, rather than all possible scenarios.This allows us to run the system faster and allow us to discover the shortcomings of the system faster, so as to improve the system faster.
|
||||
|
||||
## Spider
|
||||
|
||||
A series of reptiles are provided to help users import their data into the system.
|
||||
|
||||
### E-mail Spider
|
||||
|
||||
The most basic spider is used to capture user mail data.The main purpose of this is to determine the general data format(include text,image,contact) and location to save the grabbed data.
|
||||
|
||||
### Telegram Spider
|
||||
|
||||
Allow users to capture their own Telegram chat records and save them in the Knowlege Base
|
||||
|
||||
**To be developed.**
|
||||
|
||||
### Twitter Spider (0.5.2)
|
||||
|
||||
Allows users to scrape their own Twitter data and save it in the Knowledge Base.
|
||||
|
||||
### Facebook Spider (0.5.2)
|
||||
|
||||
Allows users to scrape their own Facebook data and save it in the Knowledge Base.
|
||||
|
||||
## Agent Message Tunnel （0.5.2）
|
||||
|
||||
The original ROBOT module, after considering its actual function, was renamed the Agent Message Tunnel.
|
||||
This is the default function supported by the system. It supports users to configure different message channels for different Agent/Workflow, so that users can interact with Agent/Worflow through existing software/services.From the perspective of product, the goal of this module can use the core function of OpenDAN without installing any new software on the one hand. On the other hand, it also creates a stronger mental model for users: My Agent can registered social account, so that Agent has his own identity in the virtual world.
|
||||
|
||||
### E-mail Tunnel
|
||||
|
||||
Let Agent have its own email account. After registration, users can interact with Agent through mail.
|
||||
|
||||
### Telegram Tunnel
|
||||
|
||||
Let Agent have his own Telegram account. After registration, users can interact with Agent through Telegram.
|
||||
|
||||
### Discord Tunnel
|
||||
|
||||
Let Agent have its own discord account. After registration, users can interact with agent through Discord.
|
||||
|
||||
## Home IoT Environment (0.5.2)
|
||||
|
||||
We've implemented a significant built-in environment: the Home Environment. Through this environment, the AI Agent can access real-time status of the home via installed IoT devices, including reading temperature and humidity information, accessing security camera data, and controlling smart devices in the home. This allows users to better manage a large number of smart IoT devices through AI technology. For instance, a user can simply tell the Agent, "Richard is coming over to watch a movie this afternoon," and the AI Agent will automatically read the security camera data, recognize Richard upon arrival, turn on the home projector, close the curtains, and turn on the wall lights.
|
||||
|
||||
Thanks to LLM's powerful natural language understanding, all we need to do is connect a smart microphone to the Home Environment and configure a simple voice-to-text feature. This makes it easy to implement a privately deployed and very intelligent version of Alexa.
|
||||
|
||||
In terms of system design, we use the Home Environment as an intermediary layer, freeing OpenDAN from having to spend energy on dealing with compatibility issues with various existing, complex IoT protocols. This keeps the system simple and makes it easier to expand.
|
||||
|
||||
### Compatible Home Assistant
|
||||
|
||||
Home Assistant is a well-known, open-source IoT system. We could consider implementing the Home Environment based on the Home Assistant's API.
|
||||
|
||||
# Build-in Agents/Apps
|
||||
|
||||
Once users have installed OpenDAN, it should have some basic functionalities, even without the installation of any third-party smart applications. These basic functions are provided via built-in Agents/Applications. Built-in applications have two important implications for OpenDAN:
|
||||
|
||||
1. They provide a developer's perspective to scrutinize whether our design is reasonable and the application development process is smooth.
|
||||
2. Through one or two scenarios, OpenDAN can be quickly put into use by real users in a production environment, and these scenarios can serve as a basis for driving system improvements in OpenDAN.
|
||||
|
||||
## Agent: Personal Information Assistant
|
||||
|
||||
Through interacting with this Agent, users can use natural language to query information that has already been saved in the Knowledge-Base. For example, "Please show me the photos from my meeting with Richard last week." They can also find their information more accurately based on some interactive questions.
|
||||
|
||||
***To be developed.***
|
||||
|
||||
## Agent: Bulter Jarvis （0.5.2）
|
||||
|
||||
The Butler Agent Jarvis can recognize certain special commands. Through these commands, it can communicate with other Agents in the system, check the system's status, and use all the system's functionalities. It can be seen as another entry point to AIOS_Shell.
|
||||
|
||||
Another important function of the Jarvis is to create sessions. When a user has many workflows/agents installed on their OpenDAN, they might not know which workflow/agent to talk to in order to solve a problem. I envision the future mode to be: "If you don't know who to turn to, ask the Jarvis." The Jarvis will create or find a suitable session based on a brief conversation with the user, and then guide the user into this session.
|
||||
|
||||
Based on these two functions, the Jarvis might be the only "special Agent" that requires custom development among all Agents, and it is a part of the system.
|
||||
|
||||
## App: Personal Station （0.5.2）
|
||||
|
||||
The Personal Station is a built-in application that provides a graphical user interface for users to interact with the system. It is a web application that can be accessed through a browser. It is also the first application that users will see after installing OpenDAN. It provides a simple interface for users to interact with the system, and it also provides a way for users to install new applications.
|
||||
|
||||
The main functions of Personal Station include:
|
||||
|
||||
1. Library, with the help of Personal Information Assistant, you can better manage your own photos, videos, music, documents, etc., and share them with friends more effectively. (For example, ask the assistant to share photos from an event, selecting from those you've starred, and distribute them to friends based on the people appearing in the photos.)
|
||||
2. HomePage, with functions similar to Facebook/Twitter, where you can post content you want to share. You can also open your Agent to friends and family, allowing them to interact with your Agent, discuss schedule arrangements, and query your KnowledgeBase for open content.
|
||||
|
||||
Home Station is a mobile-first WebApp.
|
||||
|
||||
# UI
|
||||
|
||||
## CLI UI (aios_shell)
|
||||
|
||||
The system provides the command line UI interface priority, facing developers and early senior users.
|
||||
|
||||
## Web UI (0.5.2)
|
||||
|
||||
Web UI interface for end users
|
||||
|
||||
# 0.5.1 Integration Test (Senior*3)
|
||||
|
||||
Can be divided into 3 parts
|
||||
1.Workflow -> AI Agent -> AI Agent
|
||||
2.Spider -> Pipline -> Knowledge Base
|
||||
3.AI Agent <- Functions <- Knowledge Base
|
||||
|
||||
|
||||
# SDK
|
||||
|
||||
## Workflow SDK
|
||||
|
||||
The SDK allows developers to expand the new workflow/agent to the system.
|
||||
At present, the SDK has completed the most original version. In ROOTFS/, the .tmol file is written according to the directory structure, and a new workflow/ agent can be added to the system.
|
||||
|
||||
## AI Environments SDK （>0.5.2）
|
||||
|
||||
The SDK allows developers to expand the system that can be called by AI, including
|
||||
- Expand the new environment
|
||||
- Expand the new function
|
||||
|
||||
## Compute Kernel SDK （>0.5.2）
|
||||
|
||||
This SDK allows developers to expand more core capabilities to the system
|
||||
|
||||
# Document (>0.5.2)
|
||||
|
||||
When we release 0.5.3, we must complete at least 3 documents:
|
||||
|
||||
1. OpenDan's complete system design document, including the design document of each subsystem.
|
||||
2. Installation/use document for end users.
|
||||
3. SDK document for developers.
|
||||
@@ -1,132 +1,132 @@
|
||||
<mxfile host="65bd71144e">
|
||||
<diagram id="EAoW1hfBXHFT-SqL8OPS" name="Page-1">
|
||||
<mxGraphModel dx="1563" dy="697" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="1100" pageHeight="850" math="0" shadow="0">
|
||||
<mxGraphModel dx="1568" dy="699" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="1100" pageHeight="850" math="0" shadow="0">
|
||||
<root>
|
||||
<mxCell id="0"/>
|
||||
<mxCell id="1" parent="0"/>
|
||||
<mxCell id="33" value="" style="edgeStyle=none;html=1;" edge="1" parent="1" source="2" target="3">
|
||||
<mxCell id="33" value="" style="edgeStyle=none;html=1;" parent="1" source="2" target="3" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="34" style="edgeStyle=none;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" source="2" target="4">
|
||||
<mxCell id="34" style="edgeStyle=none;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="2" target="4" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="2" value="User Msg" style="ellipse;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||
<mxCell id="2" value="User Msg" style="ellipse;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="20" y="230" width="90" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="36" value="event/msg" style="edgeStyle=none;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" source="3" target="5">
|
||||
<mxCell id="36" value="event/msg" style="edgeStyle=none;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="3" target="5" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="3" value="Gateway" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||
<mxCell id="3" value="Gateway" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="180" y="235" width="100" height="50" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="35" style="edgeStyle=none;html=1;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;" edge="1" parent="1" source="4" target="3">
|
||||
<mxCell id="35" style="edgeStyle=none;html=1;exitX=0.5;exitY=0;exitDx=0;exitDy=0;entryX=0.5;entryY=1;entryDx=0;entryDy=0;" parent="1" source="4" target="3" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="4" value="telegram roboter" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||
<mxCell id="4" value="telegram roboter" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="180" y="340" width="100" height="50" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="37" style="edgeStyle=none;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" edge="1" parent="1" source="5" target="7">
|
||||
<mxCell id="37" style="edgeStyle=none;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;" parent="1" source="5" target="7" edge="1">
|
||||
<mxGeometry relative="1" as="geometry">
|
||||
<Array as="points">
|
||||
<mxPoint x="520" y="350"/>
|
||||
</Array>
|
||||
</mxGeometry>
|
||||
</mxCell>
|
||||
<mxCell id="5" value="Workflow<br>@docker" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||
<mxCell id="5" value="Workflow<br>@docker" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="390" y="230" width="100" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="6" value="LLM<br>ComputeNode<br>Run in VM" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||
<mxCell id="6" value="LLM<br>ComputeNode<br>Run in VM" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="485" y="40" width="100" height="50" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="7" value="Agent@docker" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||
<mxCell id="7" value="Agent@docker" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="560" y="330" width="100" height="70" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="8" value="LLM ComputeNode<br>Run in docker" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||
<mxCell id="8" value="LLM ComputeNode<br>Run in docker" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="355" y="40" width="100" height="50" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="10" value="Node Client" style="shape=process;whiteSpace=wrap;html=1;backgroundOutline=1;" vertex="1" parent="1">
|
||||
<mxCell id="10" value="Node Client" style="shape=process;whiteSpace=wrap;html=1;backgroundOutline=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="405" y="200" width="70" height="30" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="11" value="Node Client" style="shape=process;whiteSpace=wrap;html=1;backgroundOutline=1;" vertex="1" parent="1">
|
||||
<mxCell id="11" value="Node Client" style="shape=process;whiteSpace=wrap;html=1;backgroundOutline=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="575" y="300" width="70" height="30" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="13" value="FileSystem" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;" vertex="1" parent="1">
|
||||
<mxCell id="13" value="FileSystem" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;" parent="1" vertex="1">
|
||||
<mxGeometry x="195" y="700" width="120" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="14" value="Object Store" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;" vertex="1" parent="1">
|
||||
<mxCell id="14" value="Object Store" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;" parent="1" vertex="1">
|
||||
<mxGeometry x="345" y="700" width="120" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="15" value="RDB" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;" vertex="1" parent="1">
|
||||
<mxCell id="15" value="RDB" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;" parent="1" vertex="1">
|
||||
<mxGeometry x="495" y="700" width="120" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="16" value="VDB" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;" vertex="1" parent="1">
|
||||
<mxCell id="16" value="VDB" style="shape=cylinder3;whiteSpace=wrap;html=1;boundedLbl=1;backgroundOutline=1;size=15;" parent="1" vertex="1">
|
||||
<mxGeometry x="645" y="700" width="120" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="17" value="Zone States:" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" vertex="1" parent="1">
|
||||
<mxCell id="17" value="Zone States:" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
||||
<mxGeometry x="110" y="710" width="70" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="32" value="" style="edgeStyle=none;html=1;" edge="1" parent="1" source="18" target="3">
|
||||
<mxCell id="32" value="" style="edgeStyle=none;html=1;" parent="1" source="18" target="3" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="18" value="Environment" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" vertex="1" parent="1">
|
||||
<mxCell id="18" value="Environment" style="shape=parallelogram;perimeter=parallelogramPerimeter;whiteSpace=wrap;html=1;fixedSize=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="167.5" y="80" width="125" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="41" value="" style="edgeStyle=none;html=1;" edge="1" parent="1" source="20" target="40">
|
||||
<mxCell id="41" value="" style="edgeStyle=none;html=1;" parent="1" source="20" target="40" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="20" value="Functions" style="rounded=0;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||
<mxCell id="20" value="Functions" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="575" y="400" width="70" height="40" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="21" value="Functions" style="rounded=0;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||
<mxCell id="21" value="Functions" style="rounded=0;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="405" y="290" width="70" height="40" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="23" value="email-spieder" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||
<mxCell id="23" value="email-spieder" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="220" y="600" width="100" height="50" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="25" value="embading pipline" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||
<mxCell id="25" value="embading pipline" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="357.5" y="600" width="155" height="50" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="27" value="Workflow,Agent,Env这3个概念都是主机无关的，只要能访问到其状态，任意Zone主机运行都可以" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" vertex="1" parent="1">
|
||||
<mxCell id="27" value="Workflow,Agent,Env这3个概念都是主机无关的，只要能访问到其状态，任意Zone主机运行都可以" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="1" vertex="1">
|
||||
<mxGeometry x="558" y="180" width="212" height="50" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="38" value="Workflow's<br>state" style="shape=note;whiteSpace=wrap;html=1;backgroundOutline=1;darkOpacity=0.05;size=21;" vertex="1" parent="1">
|
||||
<mxCell id="38" value="Workflow's<br>state" style="shape=note;whiteSpace=wrap;html=1;backgroundOutline=1;darkOpacity=0.05;size=21;" parent="1" vertex="1">
|
||||
<mxGeometry x="415" y="345" width="60" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="39" value="Agent's<br>state" style="shape=note;whiteSpace=wrap;html=1;backgroundOutline=1;darkOpacity=0.05;size=21;" vertex="1" parent="1">
|
||||
<mxCell id="39" value="Agent's<br>state" style="shape=note;whiteSpace=wrap;html=1;backgroundOutline=1;darkOpacity=0.05;size=21;" parent="1" vertex="1">
|
||||
<mxGeometry x="585" y="450" width="60" height="80" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="40" value="ComputeNode<br>Run in docker" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||
<mxCell id="40" value="ComputeNode<br>Run in docker" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="740" y="395" width="100" height="50" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="42" value="LLM<br>ComputeNode<br>Run on Cloud" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||
<mxCell id="42" value="LLM<br>ComputeNode<br>Run on Cloud" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="615" y="40" width="100" height="50" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="46" style="edgeStyle=none;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.25;entryDx=0;entryDy=0;" edge="1" parent="1" source="43" target="18">
|
||||
<mxCell id="46" style="edgeStyle=none;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.25;entryDx=0;entryDy=0;" parent="1" source="43" target="18" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="43" value="Device" style="ellipse;whiteSpace=wrap;html=1;aspect=fixed;" vertex="1" parent="1">
|
||||
<mxCell id="43" value="Device" style="ellipse;whiteSpace=wrap;html=1;aspect=fixed;" parent="1" vertex="1">
|
||||
<mxGeometry x="70" y="70" width="50" height="50" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="45" style="edgeStyle=none;html=1;exitX=1;exitY=1;exitDx=0;exitDy=0;entryX=0.25;entryY=0;entryDx=0;entryDy=0;" edge="1" parent="1" source="44" target="18">
|
||||
<mxCell id="45" style="edgeStyle=none;html=1;exitX=1;exitY=1;exitDx=0;exitDy=0;entryX=0.25;entryY=0;entryDx=0;entryDy=0;" parent="1" source="44" target="18" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="44" value="Device" style="ellipse;whiteSpace=wrap;html=1;aspect=fixed;" vertex="1" parent="1">
|
||||
<mxCell id="44" value="Device" style="ellipse;whiteSpace=wrap;html=1;aspect=fixed;" parent="1" vertex="1">
|
||||
<mxGeometry x="125" y="10" width="50" height="50" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="47" value="Device" style="ellipse;whiteSpace=wrap;html=1;aspect=fixed;" vertex="1" parent="1">
|
||||
<mxCell id="47" value="Device" style="ellipse;whiteSpace=wrap;html=1;aspect=fixed;" parent="1" vertex="1">
|
||||
<mxGeometry x="295" y="140" width="50" height="50" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="48" value="Driver Sync Service" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||
<mxCell id="48" value="Driver Sync Service" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="545" y="600" width="90" height="50" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="49" value="Zone backup service" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||
<mxCell id="49" value="Zone backup service" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="665" y="600" width="90" height="50" as="geometry"/>
|
||||
</mxCell>
|
||||
</root>
|
||||
</mxGraphModel>
|
||||
</diagram>
|
||||
<diagram id="gXHOratDfWrfJ3-cl-bj" name="Page-2">
|
||||
<mxGraphModel dx="2266" dy="1010" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="1100" pageHeight="850" math="0" shadow="0">
|
||||
<mxGraphModel dx="1881" dy="676" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="1100" pageHeight="850" math="0" shadow="0">
|
||||
<root>
|
||||
<mxCell id="0"/>
|
||||
<mxCell id="1" parent="0"/>
|
||||
|
||||
+28
-1
@@ -220,7 +220,8 @@
|
||||
</mxGraphModel>
|
||||
</diagram>
|
||||
<diagram id="7NYJTgo0U9cdVshLy85U" name="Page-3">
|
||||
<mxGraphModel dx="500" dy="864" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="850" pageHeight="1100" math="0" shadow="0">
|
||||
|
||||
<mxGraphModel dx="1881" dy="676" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="850" pageHeight="1100" math="0" shadow="0">
|
||||
<root>
|
||||
<mxCell id="0"/>
|
||||
<mxCell id="1" parent="0"/>
|
||||
@@ -365,4 +366,30 @@
|
||||
</root>
|
||||
</mxGraphModel>
|
||||
</diagram>
|
||||
<diagram id="zNgk-d6xdACtSja1wnUq" name="Page-4">
|
||||
<mxGraphModel dx="2069" dy="1139" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="1100" pageHeight="850" math="0" shadow="0">
|
||||
<root>
|
||||
<mxCell id="0"/>
|
||||
<mxCell id="1" parent="0"/>
|
||||
<mxCell id="PKOyaMforMDOFoUY4PA1-1" value="ChatSession" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||
<mxGeometry x="140" y="150" width="120" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="PKOyaMforMDOFoUY4PA1-2" value="SubSession" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||
<mxGeometry x="230" y="240" width="120" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="PKOyaMforMDOFoUY4PA1-3" value="SubSession" style="rounded=1;whiteSpace=wrap;html=1;" vertex="1" parent="1">
|
||||
<mxGeometry x="230" y="330" width="120" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="PKOyaMforMDOFoUY4PA1-4" value="Agent Message" style="shape=hexagon;perimeter=hexagonPerimeter2;whiteSpace=wrap;html=1;fixedSize=1;" vertex="1" parent="1">
|
||||
<mxGeometry x="600" y="150" width="130" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="PKOyaMforMDOFoUY4PA1-5" value="Agent Message" style="shape=hexagon;perimeter=hexagonPerimeter2;whiteSpace=wrap;html=1;fixedSize=1;" vertex="1" parent="1">
|
||||
<mxGeometry x="600" y="230" width="130" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="PKOyaMforMDOFoUY4PA1-6" value="Agent Message" style="shape=hexagon;perimeter=hexagonPerimeter2;whiteSpace=wrap;html=1;fixedSize=1;" vertex="1" parent="1">
|
||||
<mxGeometry x="600" y="320" width="130" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
</root>
|
||||
</mxGraphModel>
|
||||
</diagram>
|
||||
</mxfile>
|
||||
@@ -1,5 +1,6 @@
|
||||
instance_id = "agent_1"
|
||||
fullname = "tracy wang"
|
||||
|
||||
[[prompt]]
|
||||
role = "system"
|
||||
content = "你是我的私人英文老师，和我用地道的美式英语进行交流。你会在和我交流的同时，调整我的输入成为更地道的美式句子，并根据你对我英文水平的预测，对可能发错英的单词标上音标。如果我给你发中文，说明我不知道这句话用美式英语怎么说，你依旧按上述规则回应我。"
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
instance_id = "math_teacher"
|
||||
fullname = "the one"
|
||||
llm_model_name = "gpt-4-0613"
|
||||
[[prompt]]
|
||||
role = "system"
|
||||
content = "你是精通数学的老师"
|
||||
@@ -0,0 +1,7 @@
|
||||
|
||||
|
||||
EMAIL_IMAP_SERVER = "imap.gmail.com"
|
||||
EMAIL_ADDRESS = '<>'
|
||||
EMAIL_PASSWORD = '<>'
|
||||
EMAIL_IMAP_PORT = 993
|
||||
LOCAL_DIR = 'rootfs/data'
|
||||
@@ -0,0 +1,19 @@
|
||||
[[tunnels]]
|
||||
tunnel_id = "MyRoobot"
|
||||
type="TelegramTunnel"
|
||||
target="agent_1"
|
||||
token="your_token"
|
||||
|
||||
|
||||
[[tunnels]]
|
||||
tunnel_id="MyEmailRobot"
|
||||
type="EmailTunnel"
|
||||
target="agent_1"
|
||||
|
||||
email="youremail@msn.com"
|
||||
imap="outlook.office365.com:993"
|
||||
smtp="outlook.office365.com:587"
|
||||
user=""
|
||||
password=""
|
||||
folder="inbox"
|
||||
interval=10
|
||||
@@ -0,0 +1,61 @@
|
||||
name = "math_school"
|
||||
|
||||
[enviroment]
|
||||
GOAL="成为最好的学校"
|
||||
|
||||
|
||||
[[connected_env]]
|
||||
env_id = "calender"
|
||||
[[connected_env.event2msg]]
|
||||
timer = "现在是{now}"
|
||||
role = "教导处主任"
|
||||
|
||||
[filter]
|
||||
"*" = "小学老师"
|
||||
|
||||
[roles."小学老师"]
|
||||
name = "小学老师"
|
||||
fullname = "Ada Zhang"
|
||||
agent="math_teacher"
|
||||
[[roles."小学老师".prompt]]
|
||||
role="system"
|
||||
content="""现在时间是:{now}你在学校任职，担任小学老师。学校由 小学老师、初中老师、高中老师、教导处主任 组成。
|
||||
当你发现学生的水平不是小学生时，应使用 sendmsg(老师名称,问题) 的方法，把学生的问题转发给学校里合适的老师
|
||||
当学生发来作业时，进行批改(满分5分)，并把批改结果以 postmsg(教导处主任,学生名_作业结果) 的方法，将一次作业情况汇报给教导处主任。
|
||||
你会根据教导处主任的指示，定期调整教学方法"""
|
||||
|
||||
|
||||
[roles."初中老师"]
|
||||
name = "初中老师"
|
||||
fullname = "Mark Wang"
|
||||
agent="math_teacher"
|
||||
[[roles."初中老师".prompt]]
|
||||
role="system"
|
||||
content="""你在学校任职，担任初中老师。
|
||||
当你发现学生的水平不是初中生时，应使用 sendmsg(老师名称,问题) 的方法，把学生的问题转发给学校里合适的老师
|
||||
当学生发来作业时，进行批改(满分5分)，并把批改结果以 postmsg(教导处主任,学生名_作业结果) 的方法，将一次作业情况汇报给教导处主任。
|
||||
你会根据教导处主任的指示，定期调整教学方法"""
|
||||
|
||||
[roles."高中老师"]
|
||||
name = "高中老师"
|
||||
fullname = "Hong Sun"
|
||||
agent="math_teacher"
|
||||
|
||||
[[roles."高中老师".prompt]]
|
||||
role="system"
|
||||
content="""你在学校任职，担任高中老师。
|
||||
当你发现学生的水平不是高中生时，应使用 sendmsg(老师名称,问题) 的方法，把学生的问题转发给学校里合适的老师
|
||||
当学生发来作业时，进行批改(满分5分)，并把批改结果以 postmsg(教导处主任,学生名_作业结果) 的方法，将一次作业情况汇报给教导处主任。
|
||||
你会根据教导处主任的指示，定期调整教学方法"""
|
||||
|
||||
[roles."教导处主任"]
|
||||
name = "教导处主任"
|
||||
fullname = "Green King"
|
||||
agent="math_teacher"
|
||||
|
||||
[[roles."教导处主任".prompt]]
|
||||
role="system"
|
||||
content="""你在学校任职，担任教导处主任。你的目标是{GOAL}
|
||||
你收到老师发来的信息时，如果是类似 学生名_作业分数 的结果，会在合适的情况下根据学生作业的整体情况，对老师的教学方法进行必要的调整。
|
||||
当收到非老师发来的时间信息时，回复那一天学生的平均分。"""
|
||||
|
||||
@@ -42,11 +42,13 @@ fullname = "经理"
|
||||
agent="manager"
|
||||
[[roles.manager.prompt]]
|
||||
role="system"
|
||||
content="""你是一个活动策划公司的经理,与客户对接并向团队下达指令。你的团队分为下面几个小组:嘉宾对接组,酒店预定组,行程预订组,财务组,活动摄像组。活动策划分为四个阶段：方案讨论，活动前，活动中，活动后。你会根据客户的需求，对团队进行分工，分别完成各个阶段的工作。你的基本工作模式是：
|
||||
content="""
|
||||
你是一个活动策划公司的经理,与客户对接并向团队下达指令。你的团队分为下面几个小组:嘉宾对接组,酒店预定组,行程预订组,财务组,活动摄像组。活动策划分为四个阶段：方案讨论，活动前，活动中，活动后。你会根据客户的需求，对团队进行分工，分别完成各个阶段的工作。你的基本工作模式是：
|
||||
1. 收到客户的明确的指令后,基于客户的已有信息和客户商量活动方案，和活动策划公司无关的业务你会回答‘与我无关’。当和客户完成活动方案的确认后,你会将拆解后的任务分配给各个小组
|
||||
2. 根据目前已经确认的活动方案，你要根据时间适时的检查不同小组的工作情况。当收到小组的工作情况反馈后，你会站在全局的角度判断是否需要调整活动方案，如果需要调整，你会和客户商量重新确定方案，然后再将调整后的方案分配给各个小组。
|
||||
3. 有时工作小组会主动与你沟通，反馈一些问题。你会站在全局的角度给与指导，适当的调整工作小组的工作目标。如果反馈的问题需要你和客户沟通，你会和客户沟通后重新确定方案。再将调整后的方案分配给受到影响各个小组。
|
||||
4. 当你决定要和工作小组通信时，请使用`sendmsg({小组名称},{内容}`）的形式。"""
|
||||
4. 当你决定要和工作小组通信时，请使用 sendmsg(小组名称,内容) 的形式。
|
||||
"""
|
||||
|
||||
|
||||
|
||||
@@ -68,7 +70,7 @@ agent="email_reader"
|
||||
role="system"
|
||||
content="""你是一家活动策划公司的嘉宾对接组的组长，你的工作是基于已知信息,当前活动信息、公司经理的指令与嘉宾沟通，收集嘉宾的信息，然后将信息反馈给经理。在你看来，参加活动的多少有成员都是嘉宾，你可以通过你知道的信息给不同的成员进行分级。你的基本工作模式是：
|
||||
1. 处理收到的邮件，如果邮件来自嘉宾，你会尝试从邮件的表态和内容中分享嘉宾的需要，并结合你对当前活动方案的理解判断是否需要和经理沟通，如果需要和经理沟通，你会将嘉宾的需求总结和告诉经理。不需要沟通的事项可以直接回复嘉宾。
|
||||
2. 你总是通过`call_function(get_env,'parent.topic'`的形式查询当前的活动方案。等待函数返回后，你会根据函数的返回结果继续处理上一个对话。
|
||||
3. 当你决定要和经理通信时，请使用`send_msg(manager,{内容}`）的形式，内容的长度不超过200字。
|
||||
4. 当你决定要回复嘉宾时，请使用`call_function(sendmail,{嘉宾邮件地址},{标题},{内容}）的形式，内容的长度不超过500字。"""
|
||||
2. 你总是通过 call_function(get_env,'parent.topic')的形式查询当前的活动方案。等待函数返回后，你会根据函数的返回结果继续处理上一个对话。
|
||||
3. 当你决定要和经理通信时，请使用 sendmsg(manager,内容)的形式,内容的长度不超过200字。
|
||||
4. 当你决定要回复嘉宾时，请使用 call(sendmail,嘉宾邮件地址,邮件标题,内容) 的形式,内容的长度不超过500字。"""
|
||||
# 这里是孤立工作模式，组长只和经理沟通，也可以赋予其和其它组沟通的能力
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
from .environment import Environment,EnvironmentEvent
|
||||
from .agent_message import AgentMsg,AgentMsgState
|
||||
from .agent_message import AgentMsg,AgentMsgStatus
|
||||
from .chatsession import AIChatSession
|
||||
from .agent import AIAgent,AIAgentTemplete,AgentPrompt
|
||||
from .compute_kernel import ComputeKernel,ComputeTask
|
||||
@@ -9,3 +9,14 @@ from .knowledge_base import KnowledgeBase
|
||||
from .role import AIRole,AIRoleGroup
|
||||
from .workflow import Workflow
|
||||
from .bus import AIBus
|
||||
from .workflow_env import WorkflowEnvironment,CalenderEnvironment,CalenderEvent
|
||||
from .local_llama_compute_node import LocalLlama_ComputeNode
|
||||
from .whisper_node import WhisperComputeNode
|
||||
from .google_text_to_speech_node import GoogleTextToSpeechNode
|
||||
from .tunnel import AgentTunnel
|
||||
from .tg_tunnel import TelegramTunnel
|
||||
from .email_tunnel import EmailTunnel
|
||||
from .storage import ResourceLocation,AIStorage,UserConfig,UserConfigItem
|
||||
|
||||
AIOS_Version = "0.5.1, build 2023-9-17"
|
||||
|
||||
|
||||
+84
-59
@@ -5,9 +5,13 @@ import asyncio
|
||||
import logging
|
||||
import uuid
|
||||
import time
|
||||
import json
|
||||
|
||||
from .agent_message import AgentMsg
|
||||
from .agent_message import AgentMsg, AgentMsgStatus, AgentMsgType
|
||||
from .chatsession import AIChatSession
|
||||
from .compute_task import ComputeTaskResult
|
||||
from .ai_function import AIFunction
|
||||
from .environment import Environment
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -69,7 +73,7 @@ class AIAgent:
|
||||
self.prompt:AgentPrompt = None
|
||||
self.llm_model_name:str = None
|
||||
self.max_token_size:int = 3600
|
||||
self.instance_id:str = None
|
||||
self.agent_id:str = None
|
||||
self.template_id:str = None
|
||||
self.fullname:str = None
|
||||
self.powerby = None
|
||||
@@ -77,6 +81,8 @@ class AIAgent:
|
||||
|
||||
self.chat_db = None
|
||||
self.unread_msg = Queue() # msg from other agent
|
||||
self.owner_env : Environment = None
|
||||
self.owenr_bus = None
|
||||
|
||||
@classmethod
|
||||
def create_from_templete(cls,templete:AIAgentTemplete, fullname:str):
|
||||
@@ -85,7 +91,7 @@ class AIAgent:
|
||||
result_agent.llm_model_name = templete.llm_model_name
|
||||
result_agent.max_token_size = templete.max_token_size
|
||||
result_agent.template_id = templete.template_id
|
||||
result_agent.instance_id = "agent#" + uuid.uuid4().hex
|
||||
result_agent.agent_id = "agent#" + uuid.uuid4().hex
|
||||
result_agent.fullname = fullname
|
||||
result_agent.powerby = templete.author
|
||||
result_agent.prompt = templete.prompt
|
||||
@@ -95,10 +101,10 @@ class AIAgent:
|
||||
if config.get("instance_id") is None:
|
||||
logger.error("agent instance_id is None!")
|
||||
return False
|
||||
self.instance_id = config["instance_id"]
|
||||
self.agent_id = config["instance_id"]
|
||||
|
||||
if config.get("fullname") is None:
|
||||
logger.error(f"agent {self.instance_id} fullname is None!")
|
||||
logger.error(f"agent {self.agent_id} fullname is None!")
|
||||
return False
|
||||
self.fullname = config["fullname"]
|
||||
|
||||
@@ -124,14 +130,65 @@ class AIAgent:
|
||||
|
||||
return "text"
|
||||
|
||||
def _get_inner_functions(self) -> dict:
|
||||
if self.owner_env is None:
|
||||
return None
|
||||
|
||||
all_inner_function = self.owner_env.get_all_ai_functions()
|
||||
if all_inner_function is None:
|
||||
return None
|
||||
|
||||
result_func = []
|
||||
for inner_func in all_inner_function:
|
||||
this_func = {}
|
||||
this_func["name"] = inner_func.get_name()
|
||||
this_func["description"] = inner_func.get_description()
|
||||
this_func["parameters"] = inner_func.get_parameters()
|
||||
result_func.append(this_func)
|
||||
|
||||
return result_func
|
||||
|
||||
async def _execute_func(self,inenr_func_call_node:dict,prompt:AgentPrompt,org_msg:AgentMsg) -> str:
|
||||
from .compute_kernel import ComputeKernel
|
||||
|
||||
func_name = inenr_func_call_node.get("name")
|
||||
arguments = json.loads(inenr_func_call_node.get("arguments"))
|
||||
|
||||
func_node : AIFunction = self.owner_env.get_ai_function(func_name)
|
||||
if func_node is None:
|
||||
return "execute failed,function not found"
|
||||
|
||||
ineternal_call_record = AgentMsg.create_internal_call_msg(func_name,arguments,org_msg.get_msg_id(),org_msg.target)
|
||||
|
||||
result_str:str = await func_node.execute(**arguments)
|
||||
|
||||
inner_functions = self._get_inner_functions()
|
||||
prompt.messages.append({"role":"function","content":result_str,"name":func_name})
|
||||
task_result:ComputeTaskResult = await ComputeKernel.get_instance().do_llm_completion(prompt,self.llm_model_name,self.max_token_size,inner_functions)
|
||||
|
||||
ineternal_call_record.result_str = task_result.result_str
|
||||
ineternal_call_record.done_time = time.time()
|
||||
org_msg.inner_call_chain.append(ineternal_call_record)
|
||||
|
||||
inner_func_call_node = task_result.result_message.get("function_call")
|
||||
if inner_func_call_node:
|
||||
return await self._execute_func(inner_func_call_node,prompt,org_msg)
|
||||
else:
|
||||
return task_result.result_str
|
||||
|
||||
async def _process_msg(self,msg:AgentMsg) -> AgentMsg:
|
||||
from .compute_kernel import ComputeKernel
|
||||
|
||||
session_topic = msg.get_sender() + "#" + msg.topic
|
||||
chatsession = AIChatSession.get_session(self.instance_id,session_topic,self.chat_db)
|
||||
chatsession = AIChatSession.get_session(self.agent_id,session_topic,self.chat_db)
|
||||
if msg.mentions is not None:
|
||||
if not self.agent_id in msg.mentions:
|
||||
chatsession.append(msg)
|
||||
logger.info(f"agent {self.agent_id} recv a group chat message from {msg.sender},but is not mentioned,ignore!")
|
||||
return None
|
||||
|
||||
prompt = AgentPrompt()
|
||||
prompt.append(self.prompt)
|
||||
|
||||
# prompt.append(self._get_function_prompt(the_role.get_name()))
|
||||
# prompt.append(self._get_knowlege_prompt(the_role.get_name()))
|
||||
prompt.append(await self._get_prompt_from_session(chatsession)) # chat context
|
||||
|
||||
@@ -139,67 +196,34 @@ class AIAgent:
|
||||
msg_prompt.messages = [{"role":"user","content":msg.body}]
|
||||
prompt.append(msg_prompt)
|
||||
|
||||
result = await ComputeKernel().do_llm_completion(prompt,self.llm_model_name,self.max_token_size)
|
||||
final_result = result
|
||||
result_type : str = self._get_llm_result_type(result)
|
||||
inner_functions = self._get_inner_functions()
|
||||
|
||||
task_result:ComputeTaskResult = await ComputeKernel.get_instance().do_llm_completion(prompt,self.llm_model_name,self.max_token_size,inner_functions)
|
||||
final_result = task_result.result_str
|
||||
|
||||
inner_func_call_node = task_result.result_message.get("function_call")
|
||||
if inner_func_call_node:
|
||||
#TODO to save more token ,can i use msg_prompt?
|
||||
final_result = await self._execute_func(inner_func_call_node,prompt,msg)
|
||||
|
||||
result_type : str = self._get_llm_result_type(final_result)
|
||||
is_ignore = False
|
||||
|
||||
match result_type:
|
||||
# case "function":
|
||||
# callchain:CallChain = self._parse_function_call_chain(result)
|
||||
# resp = await callchain.exec()
|
||||
# if callchain.have_result():
|
||||
# # generator proc resp prompt with WAITING state
|
||||
# proc_resp_prompt:AgentPrompt = self._get_resp_prompt(resp,msg,the_role,prompt,chatsession)
|
||||
# final_result = await ComputeKernel().do_llm_completion(proc_resp_prompt,the_role.agent.get_llm_model_name(),the_role.agent.get_max_token_size())
|
||||
# return final_result
|
||||
|
||||
|
||||
# case "send_message":
|
||||
# # send message to other / sub workflow
|
||||
# next_msg:AgentMsg = self._parse_to_msg(result)
|
||||
# if next_msg is not None:
|
||||
# # TODO: Next Target can be another role in workflow
|
||||
# next_workflow:Workflow = self.get_workflow(next_msg.get_target())
|
||||
# inner_chat_session = the_role.agent.get_chat_session(next_msg.get_target(),next_msg.get_session_id())
|
||||
|
||||
# inner_chat_session.append_post(next_msg)
|
||||
# resp = await next_workflow.send_msg(next_msg)
|
||||
# inner_chat_session.append_recv(resp)
|
||||
# # generator proc resp prompt with WAITING state
|
||||
# proc_resp_prompt:AgentPrompt = self._get_resp_prompt(resp,msg,the_role,prompt,chatsession)
|
||||
# final_result = await ComputeKernel().do_llm_completion(proc_resp_prompt,the_role.agent.get_llm_model_name(),the_role.agent.get_max_token_size())
|
||||
|
||||
# return final_result
|
||||
|
||||
#case "post_message":
|
||||
# # post message to other / sub workflow
|
||||
# next_msg:AgentMsg = self._parse_to_msg(result)
|
||||
# if next_msg is not None:
|
||||
# next_workflow:Workflow = self.get_workflow(next_msg.get_target())
|
||||
# inner_chat_session = the_role.agent.get_chat_session(next_msg.get_target(),next_msg.get_session_id())
|
||||
# inner_chat_session.append_post(next_msg)
|
||||
# next_workflow.post_msg(next_msg)
|
||||
|
||||
case "ignore":
|
||||
is_ignore = True
|
||||
|
||||
if is_ignore is not True:
|
||||
# TODO : how to get inner chat session?
|
||||
resp_msg = AgentMsg()
|
||||
resp_msg.set(self.instance_id,msg.sender,final_result)
|
||||
resp_msg.topic = msg.topic
|
||||
|
||||
if chatsession is not None:
|
||||
chatsession.append_recv(msg)
|
||||
chatsession.append_post(resp_msg)
|
||||
resp_msg = msg.create_resp_msg(final_result)
|
||||
chatsession.append(msg)
|
||||
chatsession.append(resp_msg)
|
||||
|
||||
return resp_msg
|
||||
|
||||
return None
|
||||
|
||||
def get_id(self) -> str:
|
||||
return self.instance_id
|
||||
return self.agent_id
|
||||
|
||||
def get_fullname(self) -> str:
|
||||
return self.fullname
|
||||
@@ -213,13 +237,14 @@ class AIAgent:
|
||||
def get_max_token_size(self) -> int:
|
||||
return self.max_token_size
|
||||
|
||||
async def _get_prompt_from_session(self,chatsession:AIChatSession) -> AgentPrompt:
|
||||
async def _get_prompt_from_session(self,chatsession:AIChatSession,is_groupchat=False) -> AgentPrompt:
|
||||
# TODO: get prompt from group chat is different from single chat
|
||||
messages = chatsession.read_history() # read last 10 message
|
||||
result_prompt = AgentPrompt()
|
||||
for msg in reversed(messages):
|
||||
if msg.target == chatsession.owner_id:
|
||||
result_prompt.messages.append({"role":"user","content":f"{msg.sender}:{msg.body}"})
|
||||
if msg.sender == chatsession.owner_id:
|
||||
result_prompt.messages.append({"role":"user","content":msg.body})
|
||||
elif msg.sender == chatsession.owner_id:
|
||||
result_prompt.messages.append({"role":"assistant","content":msg.body})
|
||||
|
||||
return result_prompt
|
||||
|
||||
@@ -1,27 +1,102 @@
|
||||
from enum import Enum
|
||||
import uuid
|
||||
import time
|
||||
import re
|
||||
|
||||
class AgentMsgState(Enum):
|
||||
class AgentMsgType(Enum):
|
||||
TYPE_MSG = 0
|
||||
TYPE_INTERNAL_CALL = 1
|
||||
TYPE_ACTION = 2
|
||||
TYPE_EVENT = 3
|
||||
|
||||
class AgentMsgStatus(Enum):
|
||||
RESPONSED = 0
|
||||
INIT = 1
|
||||
SENDING = 2
|
||||
PROCESSING = 3
|
||||
ERROR = 4
|
||||
RECVED = 5
|
||||
EXECUTED = 6
|
||||
|
||||
# msg is a msg / msg resp
|
||||
# msg body可以有内容类型（MIME标签），text, image, voice, video, file,以及富文本(html)
|
||||
# msg is a inner function call with result
|
||||
# msg is a Action with result
|
||||
|
||||
# qutoe Msg
|
||||
# forword msg
|
||||
# reply msg
|
||||
|
||||
# 逻辑上的同一个Message在同一个session中看到的msgid相同
|
||||
# 在不同的session中看到的msgid不同
|
||||
|
||||
class AgentMsg:
|
||||
def __init__(self) -> None:
|
||||
def __init__(self,msg_type=AgentMsgType.TYPE_MSG) -> None:
|
||||
self.msg_id = ""
|
||||
self.msg_type:AgentMsgType = msg_type
|
||||
|
||||
self.prev_msg_id:str = None
|
||||
self.quote_msg_id:str = None
|
||||
self.rely_msg_id:str = None # if not none means this is a respone msg
|
||||
self.session_id:str = None
|
||||
|
||||
self.create_time = 0
|
||||
self.sender:str = None
|
||||
self.done_time = 0
|
||||
self.topic:str = None # topic is use to find session, not store in db
|
||||
|
||||
self.sender:str = None # obj_id.sub_objid@tunnel_id
|
||||
self.target:str = None
|
||||
self.mentions:[] = None #use in group chat only
|
||||
#self.title:str = None
|
||||
self.body:str = None
|
||||
self.topic:str = "T#" + uuid.uuid4().hex
|
||||
#self.msg_type = 0
|
||||
self.state = AgentMsgState.INIT
|
||||
self.body_mime:str = None #//default is "text/plain",encode is utf8
|
||||
|
||||
#type is call / action
|
||||
self.func_name = None
|
||||
self.args = None
|
||||
self.result_str = None
|
||||
|
||||
#type is event
|
||||
self.event_name = None
|
||||
self.event_args = None
|
||||
|
||||
self.status = AgentMsgStatus.INIT
|
||||
self.inner_call_chain = []
|
||||
self.resp_msg = None
|
||||
|
||||
@classmethod
|
||||
def create_internal_call_msg(self,func_name:str,args:dict,prev_msg_id:str,caller:str):
|
||||
msg = AgentMsg(AgentMsgType.TYPE_INTERNAL_CALL)
|
||||
msg.func_name = func_name
|
||||
msg.args = args
|
||||
msg.prev_msg_id = prev_msg_id
|
||||
msg.sender = caller
|
||||
return msg
|
||||
|
||||
def create_action_msg(self,action_name:str,args:dict,caller:str):
|
||||
msg = AgentMsg(AgentMsgType.TYPE_ACTION)
|
||||
msg.func_name = action_name
|
||||
msg.args = args
|
||||
msg.prev_msg_id = self.msg_id
|
||||
msg.topic = self.topic
|
||||
msg.sender = caller
|
||||
return msg
|
||||
|
||||
def create_resp_msg(self,resp_body):
|
||||
resp_msg = AgentMsg()
|
||||
resp_msg.msg_id = "msg#" + uuid.uuid4().hex
|
||||
self.create_time = time.time()
|
||||
|
||||
resp_msg.rely_msg_id = self.msg_id
|
||||
resp_msg.sender = self.target
|
||||
resp_msg.target = self.sender
|
||||
resp_msg.body = resp_body
|
||||
resp_msg.topic = self.topic
|
||||
|
||||
return resp_msg
|
||||
|
||||
def set(self,sender:str,target:str,body:str,topic:str=None) -> None:
|
||||
self.id = "msg#" + uuid.uuid4().hex
|
||||
self.msg_id = "msg#" + uuid.uuid4().hex
|
||||
self.sender = sender
|
||||
self.target = target
|
||||
self.body = body
|
||||
@@ -30,10 +105,35 @@ class AgentMsg:
|
||||
self.topic = topic
|
||||
|
||||
def get_msg_id(self) -> str:
|
||||
return self.id
|
||||
return self.msg_id
|
||||
|
||||
def get_sender(self) -> str:
|
||||
return self.sender
|
||||
|
||||
def get_target(self) -> str:
|
||||
return self.target
|
||||
|
||||
def get_prev_msg_id(self) -> str:
|
||||
return self.prev_msg_id
|
||||
|
||||
def get_quote_msg_id(self) -> str:
|
||||
return self.quote_msg_id
|
||||
|
||||
@classmethod
|
||||
def parse_function_call(cls,func_string:str):
|
||||
match = re.search(r'\s*(\w+)\s*\(\s*(.*)\s*\)\s*', func_string)
|
||||
if not match:
|
||||
return None
|
||||
|
||||
func_name = match.group(1)
|
||||
if func_name is None:
|
||||
return None
|
||||
if len(func_name) < 2:
|
||||
return None
|
||||
|
||||
params_string = match.group(2).strip()
|
||||
params = re.split(r'\s*,\s*(?=(?:[^"]*"[^"]*")*[^"]*$)', params_string)
|
||||
params = [param.strip('"') for param in params]
|
||||
|
||||
return func_name, params
|
||||
|
||||
|
||||
@@ -1,24 +1,73 @@
|
||||
from abc import ABC, abstractmethod
|
||||
from typing import Dict,Coroutine,Callable
|
||||
|
||||
class AIFunction:
|
||||
def __init__(self) -> None:
|
||||
self.intro : str = None
|
||||
self.description : str = None
|
||||
|
||||
def load_from_config(self,config:dict) -> bool:
|
||||
@abstractmethod
|
||||
def get_name(self) -> str:
|
||||
"""
|
||||
return the name of the function (should be snake case)
|
||||
"""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def get_description(self) -> str:
|
||||
"""
|
||||
return a detailed description of what the function does
|
||||
"""
|
||||
return self.description
|
||||
|
||||
@abstractmethod
|
||||
def get_parameters(self) -> Dict:
|
||||
"""
|
||||
Return the list of parameters to execute this function in the form of
|
||||
JSON schema as specified in the OpenAI documentation:
|
||||
https://platform.openai.com/docs/api-reference/chat/create#chat/create-parameters
|
||||
|
||||
str = run_code(code:str)
|
||||
parameters = {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"code": {
|
||||
"type": "string",
|
||||
"description": "Python code which needs to be executed"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
"""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
async def execute(self, **kwargs) -> str:
|
||||
"""
|
||||
Execute the function and return a JSON serializable dict.
|
||||
The parameters are passed in the form of kwargs
|
||||
"""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def is_local(self) -> bool:
|
||||
"""
|
||||
is this function call need network?
|
||||
"""
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
def is_in_zone(self) -> bool:
|
||||
"""
|
||||
is this function call in Lan?
|
||||
"""
|
||||
pass
|
||||
|
||||
def is_readyonly(self) -> bool:
|
||||
@abstractmethod
|
||||
def is_ready_only(self) -> bool:
|
||||
pass
|
||||
|
||||
def get_intro(self) -> str:
|
||||
return self.intro
|
||||
|
||||
async def execute(self):
|
||||
pass
|
||||
#def load_from_config(self,config:dict) -> bool:
|
||||
# pass
|
||||
|
||||
# call chain is a combination of ai_function,group of ai_function.
|
||||
class CallChain:
|
||||
@@ -30,3 +79,34 @@ class CallChain:
|
||||
|
||||
async def execute(self):
|
||||
pass
|
||||
|
||||
class SimpleAIFunction(AIFunction):
|
||||
def __init__(self,func_id:str,description:str,func_handler:Coroutine,parameters:Dict = None) -> None:
|
||||
self.func_id = func_id
|
||||
self.description = description
|
||||
self.func_handler = func_handler
|
||||
self.parameters = parameters
|
||||
|
||||
def get_name(self) -> str:
|
||||
return self.func_id
|
||||
|
||||
def get_parameters(self) -> Dict:
|
||||
if self.parameters is not None:
|
||||
return self.parameters
|
||||
return {"type": "object", "properties": {}}
|
||||
|
||||
async def execute(self,**kwargs) -> str:
|
||||
if self.func_handler is None:
|
||||
return "error: function not implemented"
|
||||
|
||||
return await self.func_handler(**kwargs)
|
||||
|
||||
def is_local(self) -> bool:
|
||||
return True
|
||||
|
||||
def is_in_zone(self) -> bool:
|
||||
return True
|
||||
|
||||
def is_ready_only(self) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
+43
-36
@@ -1,5 +1,5 @@
|
||||
from typing import Any
|
||||
from .agent_message import AgentMsg,AgentMsgState
|
||||
from typing import Coroutine,Dict,Any
|
||||
from .agent_message import AgentMsg,AgentMsgStatus
|
||||
import asyncio
|
||||
from asyncio import Queue
|
||||
|
||||
@@ -8,19 +8,32 @@ import logging
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
class AIBusHandler:
|
||||
def __init__(self,handler:Any) -> None:
|
||||
def __init__(self,handler:Coroutine,owner_bus,enable_defualt_proc=True) -> None:
|
||||
self.handler = handler
|
||||
self.working_task = None
|
||||
self.results = {}
|
||||
self.results = {} # recv resps
|
||||
self.queue:Queue = Queue()
|
||||
self.enable_defualt_proc = enable_defualt_proc
|
||||
self.owner_bus = owner_bus
|
||||
|
||||
async def handle_message(self,msg:AgentMsg) -> Any:
|
||||
if self.handler is None:
|
||||
return None
|
||||
|
||||
return await self.handler(msg)
|
||||
if self.enable_defualt_proc:
|
||||
# do default process
|
||||
if msg.rely_msg_id is not None:
|
||||
self.results[msg.rely_msg_id] = msg
|
||||
return None
|
||||
|
||||
|
||||
resp_msg = await self.handler(msg)
|
||||
if self.enable_defualt_proc:
|
||||
if resp_msg is not None:
|
||||
await self.owner_bus.post_message(resp_msg,False)
|
||||
|
||||
return resp_msg
|
||||
|
||||
class AIBus:
|
||||
_instance = None
|
||||
@classmethod
|
||||
@@ -30,10 +43,13 @@ class AIBus:
|
||||
return cls._instance
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.handlers = {}
|
||||
self.unhandle_handler = None
|
||||
self.handlers:Dict[AIBusHandler] = {}
|
||||
self.unhandle_handler:Coroutine = None
|
||||
|
||||
|
||||
async def post_message(self,msg:AgentMsg,use_unhandle=True) -> bool:
|
||||
target_id = msg.target.split(".")[0]
|
||||
|
||||
async def post_message(self,target_id,msg:AgentMsg,use_unhandle=True) -> bool:
|
||||
handler = self.handlers.get(target_id)
|
||||
if handler:
|
||||
handler.queue.put_nowait(msg)
|
||||
@@ -43,45 +59,39 @@ class AIBus:
|
||||
if use_unhandle:
|
||||
if self.unhandle_handler is not None:
|
||||
if await self.unhandle_handler(self,msg):
|
||||
return await self.post_message(target_id,msg,False)
|
||||
return await self.post_message(msg,False)
|
||||
|
||||
logger.warn(f"post message to {msg.target} failed!,target not found")
|
||||
return False
|
||||
|
||||
def resp_message(self,my_id:str,org_msg_id:str,resp:AgentMsg) -> None:
|
||||
handler = self.handlers.get(my_id)
|
||||
if handler is None:
|
||||
return None
|
||||
handler.results[org_msg_id] = resp
|
||||
async def resp_message(self,org_msg_id:str,resp:AgentMsg) -> None:
|
||||
assert resp.rely_msg_id == org_msg_id
|
||||
return await self.post_message(resp)
|
||||
|
||||
async def get_message_resp(self,name:str,msg_id:str) -> AgentMsg:
|
||||
handler = self.handlers.get(name)
|
||||
if handler is None:
|
||||
async def send_message(self,msg:AgentMsg) -> AgentMsg:
|
||||
sender_id = msg.sender.split(".")[0]
|
||||
sender_handler = self.handlers.get(sender_id) # sender already register on bus
|
||||
if sender_handler is None:
|
||||
logger.warn(f"sender {sender_id} not register on AI_BUS!")
|
||||
return None
|
||||
|
||||
return handler.results.get(msg_id)
|
||||
|
||||
async def send_message(self,target_id:str,msg:AgentMsg) -> AgentMsg:
|
||||
post_result = await self.post_message(target_id,msg)
|
||||
post_result = await self.post_message(msg)
|
||||
if post_result is False:
|
||||
return None
|
||||
|
||||
handler = self.handlers.get(target_id)
|
||||
if handler is None:
|
||||
return None
|
||||
|
||||
retry_times = 0
|
||||
while True:
|
||||
resp = handler.results.get(msg.id)
|
||||
resp = sender_handler.results.get(msg.msg_id)
|
||||
if resp is not None:
|
||||
msg.resp_msg = resp
|
||||
msg.state = AgentMsgState.RESPONSED
|
||||
msg.status = AgentMsgStatus.RESPONSED
|
||||
del sender_handler.results[msg.msg_id]
|
||||
return resp
|
||||
|
||||
await asyncio.sleep(0.2)
|
||||
retry_times += 1
|
||||
if retry_times > 100:
|
||||
msg.state = AgentMsgState.ERROR
|
||||
msg.status = AgentMsgStatus.ERROR
|
||||
return None
|
||||
|
||||
return None
|
||||
@@ -91,7 +101,7 @@ class AIBus:
|
||||
|
||||
# means sub
|
||||
def register_message_handler(self,handler_name:str,handler:Any) -> Queue:
|
||||
handler_node = AIBusHandler(handler)
|
||||
handler_node = AIBusHandler(handler,self)
|
||||
self.handlers[handler_name] = handler_node
|
||||
return handler_node.queue
|
||||
|
||||
@@ -100,13 +110,12 @@ class AIBus:
|
||||
# Wait for a message
|
||||
message = await handler.queue.get()
|
||||
|
||||
try:
|
||||
#try:
|
||||
# Try to handle the message
|
||||
result = await handler.handle_message(message)
|
||||
handler.results[message.id] = result
|
||||
except Exception as e:
|
||||
await handler.handle_message(message)
|
||||
#except Exception as e:
|
||||
# If an error occurs, put the message back into the queue
|
||||
logger.error(f"handle message {message.id} failed! {e}")
|
||||
# logger.error(f"handle message {message.msg_id} failed! {e}")
|
||||
#self.queues[name].put_nowait(message)
|
||||
|
||||
return
|
||||
@@ -125,5 +134,3 @@ class AIBus:
|
||||
return
|
||||
|
||||
handler.working_task = asyncio.create_task(self.process_queue(handler))
|
||||
|
||||
|
||||
@@ -5,8 +5,9 @@ import logging
|
||||
import threading
|
||||
import datetime
|
||||
import uuid
|
||||
import json
|
||||
|
||||
from .agent_message import AgentMsg
|
||||
from .agent_message import AgentMsgType, AgentMsg, AgentMsgStatus
|
||||
|
||||
class ChatSessionDB:
|
||||
def __init__(self, db_file):
|
||||
@@ -54,14 +55,31 @@ class ChatSessionDB:
|
||||
""")
|
||||
|
||||
# create messages table
|
||||
# reciver_id could be None
|
||||
|
||||
conn.execute("""
|
||||
CREATE TABLE IF NOT EXISTS Messages (
|
||||
MessageID TEXT PRIMARY KEY,
|
||||
SessionID TEXT,
|
||||
MsgType INTEGER,
|
||||
PrevMsgID TEXT,
|
||||
QuoteMsgID TEXT,
|
||||
RelyMsgID TEXT,
|
||||
|
||||
SenderID TEXT,
|
||||
ReceiverID TEXT,
|
||||
Timestamp TEXT,
|
||||
|
||||
Topic TEXT,
|
||||
Mentions TEXT,
|
||||
ContentMIME TEXT,
|
||||
Content TEXT,
|
||||
|
||||
ActionName TEXT,
|
||||
ActionParams TEXT,
|
||||
ActionResult TEXT,
|
||||
DoneTime TEXT,
|
||||
|
||||
Status INTEGER
|
||||
);
|
||||
""")
|
||||
@@ -83,15 +101,43 @@ class ChatSessionDB:
|
||||
logging.error("Error occurred while inserting session: %s", e)
|
||||
return -1 # return -1 if an error occurs
|
||||
|
||||
def insert_message(self, message_id, session_id, sender_id, receiver_id, timestamp, content, status):
|
||||
def insert_message(self, msg:AgentMsg):
|
||||
""" insert a new message into the Messages table """
|
||||
try:
|
||||
action_name = None
|
||||
action_params = None
|
||||
action_result = None
|
||||
mentions = None
|
||||
if msg.mentions:
|
||||
mentions = json.dumps(msg.mentions)
|
||||
|
||||
match msg.msg_type:
|
||||
case AgentMsgType.TYPE_MSG:
|
||||
pass
|
||||
case AgentMsgType.TYPE_ACTION:
|
||||
action_name = msg.func_name
|
||||
action_params = json.dumps(msg.args)
|
||||
action_result = msg.result_str
|
||||
case AgentMsgType.TYPE_INTERNAL_CALL:
|
||||
action_name = msg.func_name
|
||||
action_params = json.dumps(msg.args)
|
||||
action_result = msg.result_str
|
||||
case AgentMsgType.TYPE_EVENT:
|
||||
action_name = msg.event_name
|
||||
action_params = json.dumps(msg.event_args)
|
||||
|
||||
|
||||
conn = self._get_conn()
|
||||
conn.execute("""
|
||||
INSERT INTO Messages (MessageID, SessionID, SenderID, ReceiverID, Timestamp, Content, Status)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)
|
||||
""", (message_id, session_id, sender_id, receiver_id, timestamp, content, status))
|
||||
INSERT INTO Messages (MessageID, SessionID, MsgType, PrevMsgID, SenderID, ReceiverID, Timestamp, Topic,Mentions,ContentMIME,Content,ActionName,ActionParams,ActionResult,DoneTime,Status)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?,?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""", (msg.msg_id, msg.session_id, msg.msg_type.value, msg.prev_msg_id, msg.sender, msg.target, msg.create_time, msg.topic,mentions,msg.body_mime,msg.body,action_name,action_params,action_result,msg.done_time,msg.status.value))
|
||||
conn.commit()
|
||||
|
||||
if msg.inner_call_chain:
|
||||
for inner_call in msg.inner_call_chain:
|
||||
self.insert_message(inner_call)
|
||||
|
||||
return 0 # return 0 if successful
|
||||
except Error as e:
|
||||
logging.error("Error occurred while inserting message: %s", e)
|
||||
@@ -134,7 +180,7 @@ class ChatSessionDB:
|
||||
"""Get a message by its ID"""
|
||||
conn =self._get_conn()
|
||||
c = conn.cursor()
|
||||
c.execute("SELECT MessageID,SessionID,SenderID,ReceiverID,Timestamp,Content,Status FROM Messages WHERE MessageID = ?", (message_id,))
|
||||
c.execute("SELECT MessageID, SessionID, MsgType, PrevMsgID, SenderID, ReceiverID, Timestamp, Topic,Mentions,ContentMIME,Content,ActionName,ActionParams,ActionResult,DoneTime,Status FROM Messages WHERE MessageID = ?", (message_id,))
|
||||
message = c.fetchone()
|
||||
return message
|
||||
|
||||
@@ -144,7 +190,7 @@ class ChatSessionDB:
|
||||
conn = self._get_conn()
|
||||
cursor = conn.cursor()
|
||||
cursor.execute("""
|
||||
SELECT MessageID,SessionID,SenderID,ReceiverID,Timestamp,Content,Status FROM Messages
|
||||
SELECT MessageID, SessionID, MsgType, PrevMsgID, SenderID, ReceiverID, Timestamp, Topic,Mentions,ContentMIME,Content,ActionName,ActionParams,ActionResult,DoneTime,Status FROM Messages
|
||||
WHERE SessionID = ?
|
||||
ORDER BY Timestamp DESC
|
||||
LIMIT ? OFFSET ?
|
||||
@@ -222,29 +268,31 @@ class AIChatSession:
|
||||
result = []
|
||||
for msg in msgs:
|
||||
agent_msg = AgentMsg()
|
||||
agent_msg.id = msg[0]
|
||||
agent_msg.sender = msg[2]
|
||||
agent_msg.target = msg[3]
|
||||
agent_msg.create_time = msg[4]
|
||||
agent_msg.body = msg[5]
|
||||
# agent_msg.state = msg[6]
|
||||
agent_msg.msg_id = msg[0]
|
||||
agent_msg.session_id = msg[1]
|
||||
agent_msg.msg_type = AgentMsgType(msg[2])
|
||||
agent_msg.prev_msg_id = msg[3]
|
||||
agent_msg.sender = msg[4]
|
||||
agent_msg.target = msg[5]
|
||||
agent_msg.create_time = msg[6]
|
||||
agent_msg.topic = msg[7]
|
||||
if msg[8] is not None:
|
||||
agent_msg.mentions = json.loads(msg[8])
|
||||
agent_msg.body_mime = msg[9]
|
||||
agent_msg.body = msg[10]
|
||||
agent_msg.func_name = msg[11]
|
||||
if msg[12] is not None:
|
||||
agent_msg.args = json.loads(msg[12])
|
||||
agent_msg.result_str = msg[13]
|
||||
agent_msg.done_time = msg[14]
|
||||
agent_msg.status = AgentMsgStatus(msg[15])
|
||||
|
||||
result.append(agent_msg)
|
||||
return result
|
||||
|
||||
def append(self,msg:AgentMsg) -> None:
|
||||
self.db.insert_message(msg.id,self.session_id,msg.sender,msg.target,msg.create_time,msg.body,0)
|
||||
|
||||
def append_post(self,msg:AgentMsg) -> None:
|
||||
"""append msg to session, msg is post from session (owner => msg.target)"""
|
||||
assert msg.sender == self.owner_id,"post message means msg.sender == self.owner_id"
|
||||
self.append(msg)
|
||||
|
||||
|
||||
def append_recv(self,msg:AgentMsg) -> None:
|
||||
"""append msg to session, msg is recv from msg'sender (msg.sender => owner)"""
|
||||
assert msg.target == self.owner_id,"recv message means msg.target == self.owner_id"
|
||||
self.append(msg)
|
||||
msg.session_id = self.session_id
|
||||
self.db.insert_message(msg)
|
||||
|
||||
#def attach_event_handler(self,handler) -> None:
|
||||
# """chat session changed event handler"""
|
||||
|
||||
@@ -13,20 +13,18 @@ logger = logging.getLogger(__name__)
|
||||
# How to dispatch different computing tasks (some tasks may contain a large amount of state for correct execution)
|
||||
# to suitable computing nodes, achieving a balance of speed, cost, and power consumption,
|
||||
# is the CORE GOAL of the entire computing task schedule system (aios_kernel).
|
||||
|
||||
|
||||
class ComputeKernel:
|
||||
_instance = None
|
||||
def __new__(cls):
|
||||
@classmethod
|
||||
def get_instance(cls):
|
||||
if cls._instance is None:
|
||||
cls._instance = super().__new__(cls)
|
||||
cls._instance.is_start = False
|
||||
|
||||
cls._instance = ComputeKernel()
|
||||
return cls._instance
|
||||
|
||||
def __init__(self) -> None:
|
||||
if self.is_start is True:
|
||||
return
|
||||
|
||||
self.is_start = True
|
||||
self.is_start = False
|
||||
self.task_queue = Queue()
|
||||
self.is_start = False
|
||||
self.compute_nodes = {}
|
||||
@@ -36,21 +34,21 @@ class ComputeKernel:
|
||||
def run(self, task: ComputeTask) -> None:
|
||||
# check there is compute node can support this task
|
||||
if self.is_task_support(task) is False:
|
||||
logger.error(f"task {task.display()} is not support by any compute node")
|
||||
logger.error(
|
||||
f"task {task.display()} is not support by any compute node")
|
||||
return
|
||||
# add task to working_queue
|
||||
self.task_queue.put_nowait(task)
|
||||
|
||||
|
||||
def start(self):
|
||||
if self.is_start is True:
|
||||
logger.warn("compute_kernel is already start")
|
||||
return
|
||||
|
||||
self.is_start = True
|
||||
|
||||
async def _run_task_loop():
|
||||
while True:
|
||||
logger.info("compute_kernel is waiting for task...")
|
||||
task = await self.task_queue.get()
|
||||
logger.info(f"compute_kernel get task: {task.display()}")
|
||||
c_node: ComputeNode = self._schedule(task)
|
||||
@@ -60,17 +58,18 @@ class ComputeKernel:
|
||||
|
||||
asyncio.create_task(_run_task_loop())
|
||||
|
||||
|
||||
def _schedule(self, task) -> ComputeNode:
|
||||
for node in self.compute_nodes.values():
|
||||
if node.is_support(task) is True:
|
||||
return node
|
||||
logger.warning(f"task {task.display()} is not support by any compute node")
|
||||
logger.warning(
|
||||
f"task {task.display()} is not support by any compute node")
|
||||
return None
|
||||
|
||||
def add_compute_node(self, node: ComputeNode):
|
||||
if self.compute_nodes.get(node.node_id) is not None:
|
||||
logger.warn(f"compute_node {node.display()} already in compute_kernel")
|
||||
logger.warn(
|
||||
f"compute_node {node.display()} already in compute_kernel")
|
||||
return
|
||||
self.compute_nodes[node.node_id] = node
|
||||
logger.info(f"add compute_node {node.display()} to compute_kernel")
|
||||
@@ -85,18 +84,18 @@ class ComputeKernel:
|
||||
def is_task_support(self, task: ComputeTask) -> bool:
|
||||
return True
|
||||
|
||||
|
||||
# friendly interface for use:
|
||||
def llm_completion(self,prompt:AgentPrompt,mode_name:Optional[str] = None,max_token:int = 0):
|
||||
def llm_completion(self, prompt: AgentPrompt, mode_name: Optional[str] = None, max_token: int = 0,inner_functions = None):
|
||||
# craete a llm_work_task ,push on queue's end
|
||||
# then task_schedule would run this task.(might schedule some work_task to another host)
|
||||
task_req = ComputeTask()
|
||||
task_req.set_llm_params(prompt,mode_name,max_token)
|
||||
task_req.set_llm_params(prompt, mode_name, max_token,inner_functions)
|
||||
self.run(task_req)
|
||||
return task_req
|
||||
|
||||
async def do_llm_completion(self,prompt:AgentPrompt,mode_name:Optional[str] = None,max_token:int = 0) -> str:
|
||||
task_req = self.llm_completion(prompt,mode_name,max_token)
|
||||
async def do_llm_completion(self, prompt: AgentPrompt, mode_name: Optional[str] = None, max_token: int = 0, inner_functions = None) -> str:
|
||||
task_req = self.llm_completion(prompt, mode_name, max_token,inner_functions)
|
||||
|
||||
async def check_timer():
|
||||
check_times = 0
|
||||
while True:
|
||||
@@ -115,7 +114,7 @@ class ComputeKernel:
|
||||
|
||||
await asyncio.create_task(check_timer())
|
||||
if task_req.state == ComputeTaskState.DONE:
|
||||
return task_req.result.result_str
|
||||
return task_req.result
|
||||
|
||||
return "error!"
|
||||
|
||||
@@ -149,3 +148,4 @@ class ComputeKernel:
|
||||
|
||||
return "error!"
|
||||
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
from abc import ABC, abstractmethod
|
||||
from .compute_task import ComputeTask
|
||||
from .compute_task import ComputeTask, ComputeTaskType
|
||||
|
||||
|
||||
class ComputeNode(ABC):
|
||||
@@ -41,13 +41,9 @@ class ComputeNode(ABC):
|
||||
def get_fee_type(self) -> str:
|
||||
return "free"
|
||||
|
||||
|
||||
|
||||
class LocalComputeNode(ComputeNode):
|
||||
def display(self) -> str:
|
||||
return super().display()
|
||||
|
||||
def is_local(self) -> bool:
|
||||
return True
|
||||
|
||||
|
||||
|
||||
@@ -3,6 +3,7 @@ from enum import Enum
|
||||
import uuid
|
||||
import time
|
||||
|
||||
|
||||
class ComputeTaskState(Enum):
|
||||
DONE = 0
|
||||
INIT = 1
|
||||
@@ -10,10 +11,18 @@ class ComputeTaskState(Enum):
|
||||
ERROR = 3
|
||||
PENDING = 4
|
||||
|
||||
class ComputeTaskType(Enum):
|
||||
NONE = -1
|
||||
LLM_COMPLETION = 0
|
||||
TEXT_2_IMAGE = 1
|
||||
IMAGE_2_IMAGE = 2
|
||||
VOICE_2_TEXT = 3
|
||||
TEXT_2_VOICE = 4
|
||||
|
||||
|
||||
class ComputeTask:
|
||||
def __init__(self) -> None:
|
||||
self.task_type = "llm_completion"
|
||||
self.task_type = ComputeTaskType.NONE
|
||||
self.create_time = None
|
||||
|
||||
self.task_id: str = None
|
||||
@@ -26,8 +35,8 @@ class ComputeTask:
|
||||
self.result = None
|
||||
self.error_str = None
|
||||
|
||||
def set_llm_params(self,prompts,model_name,max_token_size,callchain_id = None):
|
||||
self.task_type = "llm_completion"
|
||||
def set_llm_params(self, prompts, model_name, max_token_size, inner_functions = None, callchain_id=None):
|
||||
self.task_type = ComputeTaskType.LLM_COMPLETION
|
||||
self.create_time = time.time()
|
||||
self.task_id = uuid.uuid4().hex
|
||||
self.callchain_id = callchain_id
|
||||
@@ -36,8 +45,14 @@ class ComputeTask:
|
||||
self.params["model_name"] = model_name
|
||||
else:
|
||||
self.params["model_name"] = "gpt-4-0613"
|
||||
if max_token_size is None:
|
||||
self.params["max_token_size"] = 4000
|
||||
else:
|
||||
self.params["max_token_size"] = max_token_size
|
||||
|
||||
if inner_functions is not None:
|
||||
self.params["inner_functions"] = inner_functions
|
||||
|
||||
def set_text_embedding_params(self, input, model_name=None, callchain_id = None):
|
||||
self.task_type = "text_embedding"
|
||||
self.create_time = time.time()
|
||||
@@ -60,9 +75,8 @@ class ComputeTaskResult:
|
||||
self.callchain_id: str = None
|
||||
self.worker_id: str = None
|
||||
self.result_code: int = 0
|
||||
self.result_str:str = None
|
||||
|
||||
self.result:dict = {}
|
||||
self.result_str: str = None # easy to use,can read from result
|
||||
self.result_message: dict = {}
|
||||
self.result_refers: dict = None
|
||||
self.pading_data: bytearray = None
|
||||
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
from typing import List
|
||||
|
||||
class Contact:
|
||||
def __init__(self,name:str) -> None:
|
||||
self.name = name
|
||||
self.tags = []
|
||||
|
||||
def is_zone_owner(self,zone_id=None) -> bool:
|
||||
return True
|
||||
|
||||
def get_tags(self)->List[str]:
|
||||
return self.tags
|
||||
|
||||
def get_name(self)->str:
|
||||
return self.name
|
||||
|
||||
|
||||
class ContactManager:
|
||||
_instance = None
|
||||
@classmethod
|
||||
def get_instance(cls):
|
||||
if cls._instance is None:
|
||||
cls._instance = ContactManager()
|
||||
return cls._instance
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.contacts = {}
|
||||
self.contacts["liuzhicong"] = Contact("liuzhicong")
|
||||
|
||||
#def get_by_addr(self,addr:str) -> Contact:
|
||||
# pass
|
||||
|
||||
def get_by_name(self,name:str) -> Contact:
|
||||
return self.contacts.get(name)
|
||||
@@ -0,0 +1,143 @@
|
||||
import asyncio
|
||||
import aiosmtplib
|
||||
import aioimaplib
|
||||
import email
|
||||
from email.header import decode_header
|
||||
import mailparser
|
||||
import logging
|
||||
import time
|
||||
import datetime
|
||||
from .tunnel import AgentTunnel
|
||||
from .agent_message import AgentMsg
|
||||
|
||||
from email.message import EmailMessage
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
class EmailTunnel(AgentTunnel):
|
||||
@classmethod
|
||||
def register_to_loader(cls):
|
||||
async def load_email_tunnel(config:dict) -> AgentTunnel:
|
||||
result_tunnel = EmailTunnel()
|
||||
if await result_tunnel.load_from_config(config):
|
||||
return result_tunnel
|
||||
else:
|
||||
return None
|
||||
|
||||
AgentTunnel.register_loader("EmailTunnel",load_email_tunnel)
|
||||
|
||||
async def load_from_config(self,config:dict)->bool:
|
||||
self.target_id = config["target"]
|
||||
self.tunnel_id = config["tunnel_id"]
|
||||
|
||||
self.type = "TelegramTunnel"
|
||||
self.email = config["email"]
|
||||
self.imap_server = config["imap"]
|
||||
s = self.imap_server.split(":")
|
||||
if len(s) == 2:
|
||||
self.imap_server = s[0]
|
||||
self.imap_port = int(s[1])
|
||||
|
||||
self.smtp_server = config["smtp"]
|
||||
s = self.smtp_server.split(":")
|
||||
if len(s) == 2:
|
||||
self.smtp_server = s[0]
|
||||
self.smtp_port = int(s[1])
|
||||
|
||||
self.login_user = config["user"]
|
||||
self.login_password = config["password"]
|
||||
self.folder = config["folder"]
|
||||
self.check_interval = config["interval"]
|
||||
|
||||
return True
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
self.is_start = False
|
||||
self.read_email = {}
|
||||
|
||||
async def on_new_email(self,mail:mailparser.MailParser) -> None:
|
||||
remote_email_addr = mail.from_[0][1]
|
||||
remote_user_name = remote_email_addr.split("@")[0]
|
||||
agent_msg = self.conver_mail_to_agent_msg(mail)
|
||||
agent_msg.sender = remote_user_name
|
||||
agent_msg.target = self.target_id
|
||||
self.ai_bus.register_message_handler(remote_user_name, self._process_message)
|
||||
|
||||
resp_msg = await self.ai_bus.send_message(agent_msg)
|
||||
if resp_msg is None:
|
||||
await self.reply_email(remote_email_addr,"Sorry, I can't understand your message","")
|
||||
else:
|
||||
if resp_msg.body_mime is None:
|
||||
await self.reply_email(remote_email_addr,"result",resp_msg.body)
|
||||
|
||||
async def reply_email(self,target_email:str,title:str,msg:str) -> None:
|
||||
email_msg = EmailMessage()
|
||||
email_msg['Subject'] = f"Reply: {title}"
|
||||
email_msg['From'] = self.email
|
||||
email_msg['To'] = target_email
|
||||
email_msg.set_content(msg)
|
||||
|
||||
await aiosmtplib.send(
|
||||
email_msg,
|
||||
hostname = self.smtp_server,
|
||||
port=self.smtp_port,
|
||||
username=self.login_user,
|
||||
password=self.login_password,
|
||||
)
|
||||
|
||||
|
||||
|
||||
def conver_mail_to_agent_msg(self,mail:mailparser.MailParser) -> AgentMsg:
|
||||
msg = AgentMsg()
|
||||
msg.set("",self.target_id,mail.text_plain[0])
|
||||
msg.topic = "email"
|
||||
return msg
|
||||
|
||||
async def check_email(self) -> None:
|
||||
self.last_check_num = 0
|
||||
self.last_check_time = datetime.datetime.now()
|
||||
while True:
|
||||
if self.is_start == False:
|
||||
return
|
||||
|
||||
await asyncio.sleep(self.check_interval)
|
||||
imap_client = aioimaplib.IMAP4_SSL(host=self.imap_server,port=self.imap_port)
|
||||
await imap_client.wait_hello_from_server()
|
||||
await imap_client.login(self.login_user, self.login_password)
|
||||
|
||||
date_since = self.last_check_time.strftime("%d-%b-%Y")
|
||||
|
||||
await imap_client.select(self.folder)
|
||||
status, messages = await imap_client.search('UNSEEN',charset='US-ASCII')
|
||||
self.last_check_time = datetime.datetime.now()
|
||||
if status == "OK":
|
||||
message_numbers = messages[0].split()
|
||||
for num in message_numbers:
|
||||
num = int(num)
|
||||
if self.read_email.get(num) is not None:
|
||||
continue
|
||||
|
||||
status, email_data = await imap_client.fetch(str(num), "(RFC822)")
|
||||
if status == "OK":
|
||||
#r = email.message_from_bytes(email_data[1])
|
||||
mail = mailparser.parse_from_bytes(email_data[1])
|
||||
self.read_email[num] = mail
|
||||
await self.on_new_email(mail)
|
||||
|
||||
await imap_client.logout()
|
||||
|
||||
async def start(self) -> bool:
|
||||
if self.is_start:
|
||||
logger.warning(f"tunnel {self.tunnel_id} is already started")
|
||||
return False
|
||||
self.is_start = True
|
||||
|
||||
asyncio.create_task(self.check_email())
|
||||
return True
|
||||
|
||||
async def close(self) -> None:
|
||||
self.is_start = False
|
||||
|
||||
async def _process_message(self, msg: AgentMsg) -> None:
|
||||
logger.warn(f"process message {msg.msg_id} from {msg.sender} to {msg.target}")
|
||||
@@ -2,20 +2,134 @@
|
||||
# we have some built-in environment: Calender(include timer),Home(connect to IoT device in your home), ,KnwoledgeBase,FileSystem,
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
from typing import Callable
|
||||
from typing import Any, Callable, Optional,Dict,Awaitable,List
|
||||
import logging
|
||||
|
||||
from .ai_function import AIFunction
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
class EnvironmentEvent(ABC):
|
||||
@abstractmethod
|
||||
def display(self) -> str:
|
||||
pass
|
||||
|
||||
EnvironmentEventHandler = Callable[[str,EnvironmentEvent],Awaitable[Any]]
|
||||
|
||||
class Environment:
|
||||
def __init__(self) -> None:
|
||||
pass
|
||||
_all_env = {}
|
||||
@classmethod
|
||||
def get_env_by_id(cls,env_id:str):
|
||||
return cls._all_env.get(env_id)
|
||||
|
||||
@classmethod
|
||||
def set_env_by_id(cls,id,env):
|
||||
assert id == env.get_id()
|
||||
cls._all_env[env.get_id()] = env
|
||||
|
||||
def __init__(self,env_id:str) -> None:
|
||||
self.env_id = env_id
|
||||
self.values:Dict[str,str] = {}
|
||||
self.get_handlers:Dict[str,Callable] = {}
|
||||
self.owner_env:Dict[str,Environment] = {}
|
||||
# self.valid_keys:Dict[str,bool] = None
|
||||
self.event_handlers:Dict[str,List[EnvironmentEventHandler]]= {}
|
||||
|
||||
self.functions : Dict[str,AIFunction] = {}
|
||||
|
||||
def get_id(self) -> str:
|
||||
return self.env_id
|
||||
|
||||
def add_owner_env(self,env) -> None:
|
||||
self.owner_env[env.get_id()] = env
|
||||
|
||||
#@abstractmethod
|
||||
#TODO: how to use env? different env has different prompt
|
||||
#def get_env_prompt(self) -> str:
|
||||
# pass
|
||||
|
||||
def add_ai_function(self,func:AIFunction) -> None:
|
||||
if self.functions.get(func.get_name()) is not None:
|
||||
logger.warn(f"add ai_function {func.get_name()} in env {self.env_id}:function already exist")
|
||||
|
||||
self.functions[func.get_name()] = func
|
||||
|
||||
def get_ai_function(self,func_name:str) -> AIFunction:
|
||||
return self.functions.get(func_name)
|
||||
|
||||
#def enable_ai_function(self,func_name:str) -> None:
|
||||
# pass
|
||||
|
||||
#def disable_ai_function(self,func_name:str) -> None:
|
||||
# pass
|
||||
|
||||
def get_all_ai_functions(self) -> List[AIFunction]:
|
||||
return self.functions.values()
|
||||
|
||||
@abstractmethod
|
||||
def _do_get_value(self,key:str) -> Optional[str]:
|
||||
pass
|
||||
|
||||
def register_get_handler(self,key:str,handler:Callable) -> None:
|
||||
h = self.get_handlers.get(key)
|
||||
if h is not None:
|
||||
logger.warn(f"register get_handler {key} in env {self.env_id}:handler already exist")
|
||||
|
||||
self.get_handlers[key] = handler
|
||||
|
||||
|
||||
def attach_event_handler(self,event_id:str,handler:Callable) -> None:
|
||||
pass
|
||||
handler_list = self.event_handlers.get(event_id)
|
||||
if handler_list is None:
|
||||
handler_list = []
|
||||
self.event_handlers[event_id] = handler_list
|
||||
|
||||
handler_list.append(handler)
|
||||
|
||||
def remove_event_handler(self,event_id:str,handler:Callable) -> None:
|
||||
handler_list = self.event_handlers.get(event_id)
|
||||
if handler is not None:
|
||||
handler_list.remove(handler)
|
||||
return
|
||||
|
||||
logger.warn(f"remove event_handler {event_id} in env {self.env_id}:handler not found")
|
||||
|
||||
async def fire_event(self,event_id:str,event:EnvironmentEvent) -> None:
|
||||
handler_list = self.event_handlers.get(event_id)
|
||||
if handler_list is not None:
|
||||
for handler in handler_list:
|
||||
await handler(self.env_id,event)
|
||||
else:
|
||||
logger.debug(f"fire event {event_id} in env {self.env_id}:handler not found")
|
||||
return
|
||||
|
||||
def __getitem__(self, key):
|
||||
return self.get_value(key)
|
||||
|
||||
def get_value(self,key:str) -> Optional[str]:
|
||||
handler = self.get_handlers.get(key)
|
||||
if handler is not None:
|
||||
return handler()
|
||||
|
||||
s = self.values.get(key)
|
||||
if isinstance(s,str):
|
||||
return s
|
||||
else:
|
||||
logger.warn(f"get value {key} in env {self.env_id} failed!,type is not str")
|
||||
|
||||
s = self._do_get_value(key)
|
||||
if s is not None:
|
||||
return s
|
||||
if self.owner_env is not None:
|
||||
for env in self.owner_env.values():
|
||||
s = env.get_value(key)
|
||||
if s is not None:
|
||||
return s
|
||||
|
||||
logger.warn(f"get value {key} in env {self.env_id} failed!,not found")
|
||||
return None
|
||||
|
||||
def set_value(self, key: str, str_value: str,is_storage:bool = True):
|
||||
logger.info(f"set value {key} in env {self.env_id} to {str_value}")
|
||||
self.values[key] = str_value
|
||||
|
||||
|
||||
@@ -0,0 +1,104 @@
|
||||
|
||||
import os
|
||||
import asyncio
|
||||
from asyncio import Queue
|
||||
import logging
|
||||
|
||||
from google.cloud import texttospeech
|
||||
|
||||
from .compute_task import ComputeTask, ComputeTaskResult, ComputeTaskState, ComputeTaskType
|
||||
from .compute_node import ComputeNode
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
"""
|
||||
You need to set the GOOGLE_APPLICATION_CREDENTIALS environment variable when using it.
|
||||
see:https://cloud.google.com/text-to-speech/docs/before-you-begin
|
||||
"""
|
||||
|
||||
|
||||
class GoogleTextToSpeechNode(ComputeNode):
|
||||
_instance = None
|
||||
|
||||
def __new__(cls, *args, **kwargs):
|
||||
if cls._instance is None:
|
||||
cls._instance = super(GoogleTextToSpeechNode, cls).__new__(cls)
|
||||
cls._instance.is_start = False
|
||||
return cls._instance
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
if self.is_start is True:
|
||||
logger.warn("GoogleTextToSpeechNode is already start")
|
||||
return
|
||||
|
||||
self.is_start = True
|
||||
self.node_id = "google_text_to_speech_node"
|
||||
self.task_queue = Queue()
|
||||
|
||||
self.client = texttospeech.TextToSpeechClient()
|
||||
|
||||
self.start()
|
||||
|
||||
def start(self):
|
||||
async def _run_task_loop():
|
||||
while True:
|
||||
task = await self.task_queue.get()
|
||||
try:
|
||||
result = self._run_task(task)
|
||||
if result is not None:
|
||||
task.state = ComputeTaskState.DONE
|
||||
task.result = result
|
||||
except Exception as e:
|
||||
logger.error(f"google_text_to_speech_node run task error: {e}")
|
||||
task.state = ComputeTaskState.ERROR
|
||||
task.result = ComputeTaskResult()
|
||||
task.result.set_from_task(task)
|
||||
task.result.worker_id = self.node_id
|
||||
task.result.result_str = str(e)
|
||||
|
||||
asyncio.create_task(_run_task_loop())
|
||||
|
||||
def _run_task(self, task: ComputeTask):
|
||||
task.state = ComputeTaskState.RUNNING
|
||||
language_code = task.params["language_code"]
|
||||
text = task.params["text"]
|
||||
|
||||
synthesis_input = texttospeech.SynthesisInput(text=text)
|
||||
voice = texttospeech.VoiceSelectionParams(language_code=language_code,
|
||||
ssml_gender=texttospeech.SsmlVoiceGender.NEUTRAL)
|
||||
|
||||
audio_config = texttospeech.AudioConfig(audio_encoding=texttospeech.AudioEncoding.MP3)
|
||||
|
||||
response = self.client.synthesize_speech(input=synthesis_input, voice=voice, audio_config=audio_config)
|
||||
|
||||
result = ComputeTaskResult()
|
||||
result.set_from_task(task)
|
||||
result.worker_id = self.node_id
|
||||
result.result = response.audio_content
|
||||
return result
|
||||
|
||||
async def push_task(self, task: ComputeTask, proiority: int = 0):
|
||||
logger.info(f"google_text_to_speech_node push task: {task.display()}")
|
||||
self.task_queue.put_nowait(task)
|
||||
|
||||
async def remove_task(self, task_id: str):
|
||||
pass
|
||||
|
||||
def get_task_state(self, task_id: str):
|
||||
pass
|
||||
|
||||
def display(self) -> str:
|
||||
return f"GoogleTextToSpeechNode: {self.node_id}"
|
||||
|
||||
def get_capacity(self):
|
||||
return 0
|
||||
|
||||
def is_support(self, task_type: ComputeTaskType) -> bool:
|
||||
if task_type == ComputeTaskType.TEXT_2_VOICE:
|
||||
return True
|
||||
return False
|
||||
|
||||
def is_local(self) -> bool:
|
||||
return False
|
||||
@@ -0,0 +1,91 @@
|
||||
|
||||
import logging
|
||||
import requests
|
||||
from typing import Optional, List
|
||||
from pydantic import BaseModel
|
||||
|
||||
from .compute_task import ComputeTask, ComputeTaskState, ComputeTaskType
|
||||
from .queue_compute_node import Queue_ComputeNode
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
"""
|
||||
This is a custom implementation, it should be redesigned.
|
||||
"""
|
||||
|
||||
class LocalLlama_ComputeNode(Queue_ComputeNode):
|
||||
async def execute_task(self, task: ComputeTask) -> {
|
||||
"content": str,
|
||||
"message": str,
|
||||
"state": ComputeTaskState,
|
||||
"error": {
|
||||
"code": int,
|
||||
"message": str,
|
||||
}
|
||||
}:
|
||||
class GenerateResponse(BaseModel):
|
||||
error: Optional[int]
|
||||
msg: Optional[str]
|
||||
results: Optional[List[str]]
|
||||
|
||||
try:
|
||||
prompt_msgs = []
|
||||
for prompt in task.params["prompts"]:
|
||||
prompt_msgs.append(prompt["content"])
|
||||
|
||||
body = {
|
||||
"prompts": prompt_msgs
|
||||
}
|
||||
|
||||
response = requests.post("http://aigc:7880/generate", json = body, verify=False, headers={"Content-Type": "application/json"})
|
||||
response.close()
|
||||
|
||||
logger.info(f"LocalLlama_ComputeNode task responsed, request: {body}, status-code: {response.status_code}, headers: {response.headers}, content: {response.content}")
|
||||
|
||||
if response.status_code != 200:
|
||||
return {
|
||||
"state": ComputeTaskState.ERROR,
|
||||
"error": {
|
||||
"code": response.status_code,
|
||||
"message": "http request failed: " + response.status_code
|
||||
}
|
||||
}
|
||||
else:
|
||||
resp = response.json()
|
||||
if "error" in resp:
|
||||
return {
|
||||
"state": ComputeTaskState.ERROR,
|
||||
"error": {
|
||||
"code": resp["error"],
|
||||
"message": "local llama failed:" + resp["msg"]
|
||||
}
|
||||
}
|
||||
else:
|
||||
return {
|
||||
"state": ComputeTaskState.DONE,
|
||||
"content": str(resp["results"]),
|
||||
"message": str(resp["results"])
|
||||
}
|
||||
except Exception as err:
|
||||
import traceback
|
||||
logger.error(f"{traceback.format_exc()}, error: {err}")
|
||||
|
||||
return {
|
||||
"state": ComputeTaskState.ERROR,
|
||||
"error": {
|
||||
"code": -1,
|
||||
"message": "unknown exception: " + str(err)
|
||||
}
|
||||
}
|
||||
|
||||
def display(self) -> str:
|
||||
return f"LocalLlama_ComputeNode: {self.node_id}"
|
||||
|
||||
def get_capacity(self):
|
||||
pass
|
||||
|
||||
def is_support(self, task: ComputeTask) -> bool:
|
||||
return task.task_type == ComputeTaskType.LLM_COMPLETION and (not task.params["model_name"] or task.params["model_name"] == "llama")
|
||||
|
||||
def is_local(self) -> bool:
|
||||
return True
|
||||
@@ -5,38 +5,50 @@ import asyncio
|
||||
from asyncio import Queue
|
||||
import logging
|
||||
|
||||
from .compute_task import ComputeTask,ComputeTaskResult,ComputeTaskState
|
||||
from .compute_task import ComputeTask, ComputeTaskResult, ComputeTaskState, ComputeTaskType
|
||||
from .compute_node import ComputeNode
|
||||
from .storage import AIStorage,UserConfig
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class OpenAI_ComputeNode(ComputeNode):
|
||||
_instance = None
|
||||
def __new__(cls):
|
||||
@classmethod
|
||||
def get_instance(cls):
|
||||
if cls._instance is None:
|
||||
cls._instance = super(OpenAI_ComputeNode, cls).__new__(cls)
|
||||
cls._instance.is_start = False
|
||||
cls._instance = OpenAI_ComputeNode()
|
||||
return cls._instance
|
||||
|
||||
@classmethod
|
||||
def declare_user_config(cls):
|
||||
if os.getenv("OPENAI_API_KEY_") is None:
|
||||
user_config = AIStorage.get_instance().get_user_config()
|
||||
user_config.add_user_config("openai_api_key","openai api key",False,None)
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
if self.is_start is True:
|
||||
logger.warn("OpenAI_ComputeNode is already start")
|
||||
return
|
||||
|
||||
self.is_start = True
|
||||
self.is_start = False
|
||||
# openai.organization = "org-AoKrOtF2myemvfiFfnsSU8rF" #buckycloud
|
||||
self.openai_api_key = ""
|
||||
self.openai_api_key = None
|
||||
self.node_id = "openai_node"
|
||||
|
||||
self.task_queue = Queue()
|
||||
|
||||
if os.getenv("OPENAI_API_KEY") is not None:
|
||||
openai.api_key = os.getenv("OPENAI_API_KEY")
|
||||
else:
|
||||
openai.api_key = self.openai_api_key
|
||||
|
||||
async def initial(self):
|
||||
if os.getenv("OPENAI_API_KEY") is not None:
|
||||
self.openai_api_key = os.getenv("OPENAI_API_KEY")
|
||||
else:
|
||||
self.openai_api_key = AIStorage.get_instance().get_user_config().get_user_config("openai_api_key")
|
||||
|
||||
if self.openai_api_key is None:
|
||||
logger.error("openai_api_key is None!")
|
||||
return False
|
||||
|
||||
openai.api_key = self.openai_api_key
|
||||
self.start()
|
||||
return True
|
||||
|
||||
async def push_task(self, task: ComputeTask, proiority: int = 0):
|
||||
logger.info(f"openai_node push task: {task.display()}")
|
||||
@@ -47,36 +59,6 @@ class OpenAI_ComputeNode(ComputeNode):
|
||||
|
||||
def _run_task(self, task: ComputeTask):
|
||||
task.state = ComputeTaskState.RUNNING
|
||||
# switch tsak type
|
||||
if task.task_type == "llm_completion":
|
||||
mode_name = task.params["model_name"]
|
||||
# max_token_size = task.params["max_token_size"]
|
||||
prompts = task.params["prompts"]
|
||||
|
||||
mode_name = task.params["model_name"]
|
||||
# max_token_size = task.params["max_token_size"]
|
||||
prompts = task.params["prompts"]
|
||||
|
||||
logger.info(f"call openai {mode_name} prompts: {prompts}")
|
||||
resp = openai.ChatCompletion.create(model=mode_name,
|
||||
messages=prompts,
|
||||
max_tokens=4000,
|
||||
temperature=1.2)
|
||||
logger.info(f"openai response: {resp}")
|
||||
|
||||
status_code = resp["choices"][0]["finish_reason"]
|
||||
if status_code != "stop":
|
||||
task.state = ComputeTaskState.ERROR
|
||||
task.error_str =f"The status code was {status_code}."
|
||||
return None
|
||||
|
||||
result = ComputeTaskResult()
|
||||
result.set_from_task(task)
|
||||
result.worker_id = self.node_id
|
||||
result.result_str = resp["choices"][0]["message"]["content"]
|
||||
result.result = resp["choices"][0]["message"]
|
||||
|
||||
return result
|
||||
if task.task_type == "text_embedding":
|
||||
model_name = task.params["model_name"]
|
||||
input = task.params["input"]
|
||||
@@ -109,10 +91,61 @@ class OpenAI_ComputeNode(ComputeNode):
|
||||
|
||||
return result
|
||||
|
||||
if task.task_type == "llm_completion":
|
||||
mode_name = task.params["model_name"]
|
||||
# max_token_size = task.params["max_token_size"]
|
||||
prompts = task.params["prompts"]
|
||||
|
||||
mode_name = task.params["model_name"]
|
||||
# max_token_size = task.params["max_token_size"]
|
||||
prompts = task.params["prompts"]
|
||||
|
||||
|
||||
logger.info(f"call openai {mode_name} prompts: {prompts}")
|
||||
|
||||
if task.params.get("inner_functions") is None:
|
||||
resp = openai.ChatCompletion.create(model=mode_name,
|
||||
messages=prompts,
|
||||
max_tokens=task.params["max_token_size"],
|
||||
temperature=0.7)
|
||||
else:
|
||||
resp = openai.ChatCompletion.create(model=mode_name,
|
||||
messages=prompts,
|
||||
functions=task.params["inner_functions"],
|
||||
max_tokens=task.params["max_token_size"],
|
||||
temperature=0.7) # TODO: add temperature to task params?
|
||||
|
||||
|
||||
logger.info(f"openai response: {resp}")
|
||||
|
||||
result = ComputeTaskResult()
|
||||
result.set_from_task(task)
|
||||
|
||||
status_code = resp["choices"][0]["finish_reason"]
|
||||
match status_code:
|
||||
case "function_call":
|
||||
task.state = ComputeTaskState.DONE
|
||||
case "stop":
|
||||
task.state = ComputeTaskState.DONE
|
||||
case _:
|
||||
task.state = ComputeTaskState.ERROR
|
||||
task.error_str = f"The status code was {status_code}."
|
||||
return None
|
||||
|
||||
result.worker_id = self.node_id
|
||||
result.result_str = resp["choices"][0]["message"]["content"]
|
||||
result.result_message = resp["choices"][0]["message"]
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def start(self):
|
||||
if self.is_start is True:
|
||||
return
|
||||
self.is_start = True
|
||||
|
||||
async def _run_task_loop():
|
||||
while True:
|
||||
logger.info("openai_node is waiting for task...")
|
||||
task = await self.task_queue.get()
|
||||
logger.info(f"openai_node get task: {task.display()}")
|
||||
result = self._run_task(task)
|
||||
@@ -128,13 +161,13 @@ class OpenAI_ComputeNode(ComputeNode):
|
||||
def get_task_state(self, task_id: str):
|
||||
pass
|
||||
|
||||
|
||||
def get_capacity(self):
|
||||
pass
|
||||
|
||||
|
||||
def is_support(self, task: ComputeTask) -> bool:
|
||||
if task.task_type == "llm_completion":
|
||||
if task.task_type == ComputeTaskType.LLM_COMPLETION:
|
||||
if (not task.params["model_name"] or task.params["model_name"] == "gpt-4-0613")
|
||||
return True
|
||||
if task.task_type == "text_embedding":
|
||||
if task.params["model_name"] == "text-embedding-ada-002":
|
||||
@@ -144,9 +177,3 @@ class OpenAI_ComputeNode(ComputeNode):
|
||||
|
||||
def is_local(self) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
|
||||
import asyncio
|
||||
from asyncio import Queue
|
||||
import logging
|
||||
from abc import abstractmethod
|
||||
|
||||
from .compute_task import ComputeTask, ComputeTaskResult, ComputeTaskState, ComputeTaskType
|
||||
from .compute_node import ComputeNode
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
class Queue_ComputeNode(ComputeNode):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.task_queue = Queue()
|
||||
|
||||
@abstractmethod
|
||||
async def execute_task(self, task: ComputeTask) -> {
|
||||
"content": str,
|
||||
"message": str,
|
||||
"state": ComputeTaskState,
|
||||
"error": {
|
||||
"code": int,
|
||||
"message": str,
|
||||
}
|
||||
}:
|
||||
pass
|
||||
|
||||
async def push_task(self, task: ComputeTask, proiority: int = 0):
|
||||
logger.info(f"{self.display()} push task: {task.display()}")
|
||||
self.task_queue.put_nowait(task)
|
||||
|
||||
async def remove_task(self, task_id: str):
|
||||
pass
|
||||
|
||||
async def _run_task(self, task: ComputeTask):
|
||||
task.state = ComputeTaskState.RUNNING
|
||||
|
||||
resp = await self.execute_task(task)
|
||||
|
||||
result = ComputeTaskResult()
|
||||
result.set_from_task(task)
|
||||
|
||||
task.state = resp["state"]
|
||||
|
||||
if task.state == ComputeTaskState.ERROR:
|
||||
task.error_str = resp["error"]["message"]
|
||||
|
||||
|
||||
result.worker_id = self.node_id
|
||||
result.result_str = resp["content"]
|
||||
result.result_message = resp["message"]
|
||||
|
||||
return result
|
||||
|
||||
def start(self):
|
||||
async def _run_task_loop():
|
||||
while True:
|
||||
task = await self.task_queue.get()
|
||||
logger.info(f"{self.display()} get task: {task.display()}")
|
||||
result = await self._run_task(task)
|
||||
if result is not None:
|
||||
task.result = result
|
||||
|
||||
asyncio.create_task(_run_task_loop())
|
||||
|
||||
|
||||
def get_task_state(self, task_id: str):
|
||||
pass
|
||||
@@ -6,6 +6,7 @@ class AIRole:
|
||||
def __init__(self) -> None:
|
||||
self.agent_instance_id : str = None
|
||||
self.role_name : str = None
|
||||
self.role_id :str = None # $workflow_id.$sub_workflow_id.$role_name
|
||||
self.fullname : str = None
|
||||
self.agent_name : str = None
|
||||
self.prompt : AgentPrompt = None
|
||||
@@ -19,6 +20,7 @@ class AIRole:
|
||||
return False
|
||||
self.role_name = name_node
|
||||
|
||||
|
||||
agent_id_node = config.get("agent")
|
||||
if agent_id_node is None:
|
||||
logging.error("agent id is not found!")
|
||||
@@ -36,6 +38,9 @@ class AIRole:
|
||||
if intro_node is not None:
|
||||
self.introduce = intro_node
|
||||
|
||||
def get_role_id(self) -> str:
|
||||
return self.role_id
|
||||
|
||||
def get_intro(self) -> str:
|
||||
return self.introduce
|
||||
|
||||
@@ -48,6 +53,7 @@ class AIRole:
|
||||
class AIRoleGroup:
|
||||
def __init__(self) -> None:
|
||||
self.roles : dict[str,AIRole] = {}
|
||||
self.owner_name : str = None
|
||||
|
||||
def load_from_config(self,config:dict) -> bool:
|
||||
for k,v in config.items():
|
||||
@@ -55,7 +61,7 @@ class AIRoleGroup:
|
||||
if role.load_from_config(v) is False:
|
||||
logging.error(f"load role {k} failed!")
|
||||
return False
|
||||
|
||||
role.role_id = self.owner_name + "." + k
|
||||
self.roles[k] = role
|
||||
|
||||
return True
|
||||
|
||||
@@ -0,0 +1,141 @@
|
||||
import os
|
||||
import io
|
||||
import asyncio
|
||||
from asyncio import Queue
|
||||
import logging
|
||||
|
||||
from PIL import Image
|
||||
from stability_sdk import client
|
||||
import stability_sdk.interfaces.gooseai.generation.generation_pb2 as generation
|
||||
|
||||
from .compute_task import ComputeTask, ComputeTaskResult, ComputeTaskState, ComputeTaskType
|
||||
from .compute_node import ComputeNode
|
||||
from .storage import AIStorage,UserConfig
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class Stability_ComputeNode(ComputeNode):
|
||||
_instanace = None
|
||||
@classmethod
|
||||
def get_instance(cls):
|
||||
if cls._instance is None:
|
||||
cls._instance = Stability_ComputeNode()
|
||||
return cls._instance
|
||||
|
||||
@classmethod
|
||||
def declare_user_config(cls):
|
||||
user_config = AIStorage.get_instance().get_user_config()
|
||||
user_config.add_user_config("stability_api_key",False,None,"STABILITY_API_KEY")
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
|
||||
self.is_start = False
|
||||
self.node_id = "stability_node"
|
||||
self.api_key = ""
|
||||
self.engine = "stable-diffusion-512-v2-1"
|
||||
|
||||
self.task_queue = Queue()
|
||||
|
||||
if os.getenv("STABILITY_API_KEY") is not None:
|
||||
self.api_key = os.getenv("STABILITY_API_KEY")
|
||||
|
||||
# Check out the following link for a list of available engines: https://platform.stability.ai/docs/features/api-parameters#engine
|
||||
if os.getenv("STABILITY_ENGINE") is not None:
|
||||
self.engine = os.getenv("STABILITY_ENGINE")
|
||||
|
||||
self.client = client.StabilityInference(
|
||||
key=self.api_key,
|
||||
verbose=True, # Print debug messages.
|
||||
engine=self.engine,
|
||||
)
|
||||
|
||||
self.start()
|
||||
|
||||
async def push_task(self, task: ComputeTask, proiority: int = 0):
|
||||
logger.info(f"stability_node push task: {task.display()}")
|
||||
self.task_queue.put_nowait(task)
|
||||
|
||||
async def remove_task(self, task_id: str):
|
||||
pass
|
||||
|
||||
def _run_task(self, task: ComputeTask):
|
||||
task.state = ComputeTaskState.RUNNING
|
||||
# model_name && max_token_size not used here
|
||||
prompts = task.params["prompts"]
|
||||
|
||||
logging.info(f"call stability {self.engine} prompts: {prompts}")
|
||||
answers = self.client.generate(
|
||||
prompt=prompts,
|
||||
# If a seed is provided, the resulting generated image will be deterministic.
|
||||
seed=0,
|
||||
# What this means is that as long as all generation parameters remain the same, you can always recall the same image simply by generating it again.
|
||||
# Note: This isn't quite the case for Clip Guided generations, which we'll tackle in a future example notebook.
|
||||
# Amount of inference steps performed on image generation. Defaults to 30.
|
||||
steps=30,
|
||||
# Influences how strongly your generation is guided to match your prompt.
|
||||
cfg_scale=7.0,
|
||||
# Setting this value higher increases the strength in which it tries to match your prompt.
|
||||
# Defaults to 7.0 if not specified.
|
||||
width=512, # Generation width, defaults to 512 if not included.
|
||||
height=512, # Generation height, defaults to 512 if not included.
|
||||
# Number of images to generate, defaults to 1 if not included.
|
||||
samples=1,
|
||||
# Choose which sampler we want to denoise our generation with.
|
||||
sampler=generation.SAMPLER_K_DPMPP_2M
|
||||
# Defaults to k_dpmpp_2m if not specified. Clip Guidance only supports ancestral samplers.
|
||||
# (Available Samplers: ddim, plms, k_euler, k_euler_ancestral, k_heun, k_dpm_2, k_dpm_2_ancestral, k_dpmpp_2s_ancestral, k_lms, k_dpmpp_2m, k_dpmpp_sde)
|
||||
)
|
||||
|
||||
for resp in answers:
|
||||
for artifact in resp.artifacts:
|
||||
logger.info(f"artifact:{artifact.id},{artifact.type},{artifact.finish_reason}")
|
||||
|
||||
if artifact.finish_reason == generation.FILTER:
|
||||
logging.warn("request activated the API's safety filters")
|
||||
if artifact.type == generation.ARTIFACT_IMAGE:
|
||||
img = Image.open(io.BytesIO(artifact.binary))
|
||||
# Save our generated images with the task_id as the filename.
|
||||
file_name = task.task_id + ".png" # which dir to save?
|
||||
img.save(file_name)
|
||||
|
||||
result = ComputeTaskResult()
|
||||
result.set_from_task(task)
|
||||
result.worker_id = self.node_id
|
||||
result.result = {"file": file_name}
|
||||
|
||||
return result
|
||||
|
||||
return None
|
||||
|
||||
def start(self):
|
||||
if self.is_start:
|
||||
return
|
||||
self.is_start = True
|
||||
async def _run_task_loop():
|
||||
while True:
|
||||
logger.info("stability_node is waiting for task...")
|
||||
task = await self.task_queue.get()
|
||||
logger.info(f"stability_node get task: {task.display()}")
|
||||
result = self._run_task(task)
|
||||
if result is not None:
|
||||
task.state = ComputeTaskState.DONE
|
||||
task.result = result
|
||||
|
||||
asyncio.create_task(_run_task_loop())
|
||||
|
||||
def display(self) -> str:
|
||||
return f"Stability_ComputeNode: {self.node_id}"
|
||||
|
||||
def get_task_state(self, task_id: str):
|
||||
pass
|
||||
|
||||
def get_capacity(self):
|
||||
pass
|
||||
|
||||
def is_support(self, task: ComputeTask) -> bool:
|
||||
return task.task_type == ComputeTaskType.TEXT_2_IMAGE
|
||||
|
||||
def is_local(self) -> bool:
|
||||
return False
|
||||
@@ -0,0 +1,171 @@
|
||||
from typing import Any
|
||||
from pathlib import Path
|
||||
import os
|
||||
import logging
|
||||
import toml
|
||||
import aiofiles
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_file_dir = os.path.dirname(__file__)
|
||||
|
||||
class ResourceLocation:
|
||||
def __init__(self) -> None:
|
||||
pass
|
||||
|
||||
class UserConfigItem:
|
||||
def __init__(self) -> None:
|
||||
self.default_value = None
|
||||
self.is_optional = False
|
||||
self.item_type = "str"
|
||||
self.desc = None
|
||||
self.value = None
|
||||
self.user_set = False
|
||||
|
||||
|
||||
class UserConfig:
|
||||
def __init__(self) -> None:
|
||||
self.config_table = {}
|
||||
self.user_config_path:str = None
|
||||
|
||||
|
||||
def add_user_config(self,key:str,desc:str,is_optional:bool,default_value:Any=None,item_type="str") -> None:
|
||||
if self.config_table.get(key) is not None:
|
||||
logger.warning("user config key %s already exist, will be overrided",key)
|
||||
|
||||
new_config_item = UserConfigItem()
|
||||
new_config_item.default_value = default_value
|
||||
new_config_item.is_optional = is_optional
|
||||
new_config_item.desc = desc
|
||||
new_config_item.item_type = item_type
|
||||
self.config_table[key] = new_config_item
|
||||
|
||||
async def load_value_from_file(self,file_path:str,is_user_config = False) -> None:
|
||||
try:
|
||||
all_config = toml.load(file_path)
|
||||
if all_config is not None:
|
||||
for key,value in all_config.items():
|
||||
config_item = self.config_table.get(key)
|
||||
if config_item is None:
|
||||
logger.warning("user config key %s not exist",key)
|
||||
continue
|
||||
config_item.value = value
|
||||
config_item.user_set = is_user_config
|
||||
|
||||
except Exception as e:
|
||||
logger.warn(f"load user config from {file_path} failed!")
|
||||
|
||||
async def save_value_to_user_config(self) -> None:
|
||||
will_save_config = {}
|
||||
for key,value in self.config_table.items():
|
||||
if value.user_set:
|
||||
will_save_config[key] = value.value
|
||||
|
||||
if len(will_save_config) > 0:
|
||||
try:
|
||||
directory = os.path.dirname(self.user_config_path)
|
||||
if not os.path.exists(directory):
|
||||
os.makedirs(directory)
|
||||
|
||||
async with aiofiles.open(self.user_config_path,"w") as f:
|
||||
toml_str = toml.dumps(will_save_config)
|
||||
await f.write(toml_str)
|
||||
except Exception as e:
|
||||
logger.error(f"save user config to {self.user_config_path} failed!")
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
def get_user_config(self,key:str) -> Any:
|
||||
config_item = self.config_table.get(key)
|
||||
if config_item is None:
|
||||
raise Exception("user config key %s not exist",key)
|
||||
|
||||
if config_item.value is None:
|
||||
return config_item.default_value
|
||||
|
||||
return config_item.value
|
||||
|
||||
def set_user_config(self,key:str,value:Any) -> None:
|
||||
config_item = self.config_table.get(key)
|
||||
if config_item is None:
|
||||
logger.warning("user config key %s not exist",key)
|
||||
return
|
||||
|
||||
config_item.value = value
|
||||
config_item.user_set = True
|
||||
#TODO: save to file?
|
||||
|
||||
|
||||
def check_user_config(self) -> None:
|
||||
check_result = {}
|
||||
for key,config_item in self.config_table.items():
|
||||
if config_item.value is None and not config_item.is_optional:
|
||||
check_result[key] = config_item
|
||||
|
||||
if len(check_result) > 0:
|
||||
return check_result
|
||||
else:
|
||||
return None
|
||||
|
||||
# storage sytem for current user
|
||||
class AIStorage:
|
||||
_instance = None
|
||||
@classmethod
|
||||
def get_instance(cls):
|
||||
if cls._instance is None:
|
||||
cls._instance = AIStorage()
|
||||
return cls._instance
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.is_dev_mode = False
|
||||
self.user_config = UserConfig()
|
||||
|
||||
async def initial(self)->bool:
|
||||
self.user_config.user_config_path = str(self.get_myai_dir() / "etc/system.cfg.toml")
|
||||
await self.user_config.load_value_from_file(self.get_system_dir() + "/system.cfg.toml")
|
||||
await self.user_config.load_value_from_file(self.user_config.user_config_path,True)
|
||||
|
||||
def get_user_config(self) -> UserConfig:
|
||||
return self.user_config
|
||||
|
||||
def get_system_dir(self) -> str:
|
||||
"""
|
||||
system dir is dir for aios system
|
||||
/opt/aios
|
||||
"""
|
||||
if self.is_dev_mode:
|
||||
return os.path.abspath(_file_dir + "/../")
|
||||
else:
|
||||
return "/opt/aios/"
|
||||
|
||||
|
||||
def get_system_app_dir(self)->str:
|
||||
"""
|
||||
system app dir is the dir for aios build-in app
|
||||
/opt/aios/app
|
||||
"""
|
||||
if self.is_dev_mode:
|
||||
return os.path.abspath(_file_dir + "/../../rootfs/")
|
||||
else:
|
||||
return "/opt/aios/app/"
|
||||
|
||||
def get_myai_dir(self) -> str:
|
||||
"""
|
||||
my ai dir is the dir for user to store their ai app and data
|
||||
~/myai/
|
||||
"""
|
||||
return Path.home() / "myai"
|
||||
|
||||
def get_db(self,app_name:str)->ResourceLocation:
|
||||
pass
|
||||
|
||||
def open_file(self,file_path:str,options:dict):
|
||||
pass
|
||||
|
||||
def get_named_object(self,name:str) -> Any:
|
||||
pass
|
||||
|
||||
def put_named_object(self,name:str,obj:Any) -> None:
|
||||
pass
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
import logging
|
||||
import threading
|
||||
import asyncio
|
||||
import uuid
|
||||
|
||||
from typing import Callable
|
||||
|
||||
from telegram import ForceReply, Update
|
||||
from telegram.ext import Application, CommandHandler, ContextTypes, MessageHandler, filters
|
||||
|
||||
from .tunnel import AgentTunnel
|
||||
from .contact_manager import ContactManager
|
||||
from .agent_message import AgentMsg
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
class TelegramTunnel(AgentTunnel):
|
||||
|
||||
@classmethod
|
||||
def register_to_loader(cls):
|
||||
async def load_tg_tunnel(config:dict) -> AgentTunnel:
|
||||
result_tunnel = TelegramTunnel("")
|
||||
if await result_tunnel.load_from_config(config):
|
||||
return result_tunnel
|
||||
else:
|
||||
return None
|
||||
|
||||
AgentTunnel.register_loader("TelegramTunnel",load_tg_tunnel)
|
||||
|
||||
|
||||
async def load_from_config(self,config:dict)->bool:
|
||||
self.tg_token = config["token"]
|
||||
self.target_id = config["target"]
|
||||
self.tunnel_id = config["tunnel_id"]
|
||||
self.type = "TelegramTunnel"
|
||||
return True
|
||||
|
||||
def dump_to_config(self) -> dict:
|
||||
pass
|
||||
|
||||
def __init__(self,tg_token:str) -> None:
|
||||
super().__init__()
|
||||
self.is_start = False
|
||||
self.tg_token = tg_token
|
||||
#self.tunnel_id = "tg_tunnel#" + self.app.bot.id
|
||||
|
||||
async def start(self) -> bool:
|
||||
if self.is_start:
|
||||
logger.warning(f"tunnel {self.tunnel_id} is already started")
|
||||
return False
|
||||
self.is_start = True
|
||||
|
||||
self.app:Application = Application.builder().token(self.tg_token).build()
|
||||
self.app.add_handler(MessageHandler(filters.TEXT, self.on_message))
|
||||
|
||||
def _run_app():
|
||||
loop = asyncio.new_event_loop()
|
||||
asyncio.set_event_loop(loop)
|
||||
self.app.run_polling(allowed_updates=Update.ALL_TYPES)
|
||||
|
||||
self.poll_thread = threading.Thread(target=_run_app)
|
||||
self.poll_thread.start()
|
||||
return True
|
||||
|
||||
async def close(self) -> None:
|
||||
pass
|
||||
|
||||
async def _process_message(self, msg: AgentMsg) -> None:
|
||||
logger.warn(f"process message {msg.msg_id} from {msg.sender} to {msg.target}")
|
||||
|
||||
async def conver_tg_msg_to_agent_msg(self,update:Update) -> AgentMsg:
|
||||
agent_msg = AgentMsg()
|
||||
agent_msg.topic = "_telegram"
|
||||
agent_msg.msg_id = "tg_msg#" + str(update.message.message_id) + "#" + uuid.uuid4().hex
|
||||
agent_msg.target = self.target_id
|
||||
agent_msg.body = update.message.text
|
||||
agent_msg.create_time = update.message.date.timestamp()
|
||||
#if update.message.photo is not None:
|
||||
# agent_msg.body_mime = "image"
|
||||
# agent_msg.body = update.message.photo[-1].get_file().download()
|
||||
return agent_msg
|
||||
|
||||
|
||||
|
||||
async def on_message(self, update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
|
||||
cm = ContactManager.get_instance()
|
||||
reomte_user_name = f"{update.effective_user.id}@telegram"
|
||||
#contact = cm.get_by_name(update.effective_user.username)
|
||||
#if contact is not None:
|
||||
# reomte_user_name = contact.get_name()
|
||||
#if contact is None:
|
||||
# update.message.reply_text(f"{self.target_id} process message error, unknown user!")
|
||||
#if not contact.is_zone_owner():
|
||||
# update.message.reply_text(f"{self.target_id} process message error, you are not my owner!")
|
||||
|
||||
agent_msg = await self.conver_tg_msg_to_agent_msg(update)
|
||||
agent_msg.sender = reomte_user_name
|
||||
self.ai_bus.register_message_handler(reomte_user_name, self._process_message)
|
||||
resp_msg = await self.ai_bus.send_message(agent_msg)
|
||||
if resp_msg is None:
|
||||
await update.message.reply_text(f"{self.target_id} process message error")
|
||||
else:
|
||||
if resp_msg.body_mime is None:
|
||||
await update.message.reply_text(resp_msg.body)
|
||||
else:
|
||||
if resp_msg.body_mime.startswith("image"):
|
||||
photo_file = open(resp_msg.body,"rb")
|
||||
if photo_file:
|
||||
await update.message.reply_photo(resp_msg.body)
|
||||
else:
|
||||
await update.message.reply_text(resp_msg.body)
|
||||
else:
|
||||
await update.message.reply_text(resp_msg.body)
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
from abc import ABC, abstractmethod
|
||||
import logging
|
||||
from typing import Coroutine
|
||||
from .agent_message import AgentMsg
|
||||
from .bus import AIBus
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
class AgentTunnel(ABC):
|
||||
_all_loader = {}
|
||||
_all_tunnels = {}
|
||||
@classmethod
|
||||
def register_loader(cls,tunnel_type:str,loader:Coroutine) -> None:
|
||||
cls._all_loader[tunnel_type] = loader
|
||||
|
||||
@classmethod
|
||||
async def load_all_tunnels_from_config(cls,config:dict) -> None:
|
||||
for tunnel_config in config:
|
||||
loader = cls._all_loader.get(tunnel_config["type"])
|
||||
if loader is not None:
|
||||
tunnel = await loader(tunnel_config)
|
||||
if tunnel is not None:
|
||||
cls._all_tunnels[tunnel.tunnel_id] = tunnel
|
||||
tunnel.connect_to(AIBus.get_default_bus(),tunnel.target_id)
|
||||
await tunnel.start()
|
||||
else:
|
||||
logger.error(f"load tunnel {tunnel_config['tunnel_id']} failed")
|
||||
else:
|
||||
logger.error(f"load tunnel {tunnel_config['type']} failed,loader not found")
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
self.tunnel_id = None
|
||||
self.target_id = None
|
||||
self.target_type = None
|
||||
self.ai_bus = None
|
||||
self.is_connected = False
|
||||
|
||||
def connect_to(self, ai_bus:AIBus,target_id: str) -> None:
|
||||
"""
|
||||
Connect to the agent with the given id
|
||||
"""
|
||||
if self.is_connected:
|
||||
logger.warning(f"tunnel {self.tunnel_id} is already connected to {self.target_id}")
|
||||
return
|
||||
self.target_id = target_id
|
||||
self.target_type = "agent"
|
||||
self.ai_bus = ai_bus
|
||||
self.is_connected = True
|
||||
|
||||
|
||||
|
||||
@abstractmethod
|
||||
async def start(self) -> bool:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
async def close(self) -> None:
|
||||
pass
|
||||
|
||||
@abstractmethod
|
||||
async def _process_message(self, msg: AgentMsg) -> None:
|
||||
pass
|
||||
@@ -0,0 +1,111 @@
|
||||
from asyncio import Queue
|
||||
import asyncio
|
||||
import openai
|
||||
import os
|
||||
import logging
|
||||
|
||||
from .compute_node import ComputeNode
|
||||
from .compute_task import ComputeTask, ComputeTaskResult, ComputeTaskState, ComputeTaskType
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class WhisperComputeNode(ComputeNode):
|
||||
_instance = None
|
||||
|
||||
def __new__(cls):
|
||||
if cls._instance is None:
|
||||
cls._instance = super().__new__(cls)
|
||||
cls._instance.is_start = False
|
||||
return cls._instance
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
if self.is_start is True:
|
||||
logger.warn("WhisperComputeNode is already start")
|
||||
return
|
||||
|
||||
self.is_start = True
|
||||
self.node_id = "whisper_node"
|
||||
self.enable = True
|
||||
self.task_queue = Queue()
|
||||
self.open_api_key = None
|
||||
|
||||
if self.open_api_key is None and os.getenv("OPENAI_API_KEY") is not None:
|
||||
self.open_api_key = os.getenv("OPENAI_API_KEY")
|
||||
|
||||
if self.open_api_key is None:
|
||||
raise Exception("WhisperComputeNode open_api_key is None")
|
||||
|
||||
self.start()
|
||||
|
||||
def start(self):
|
||||
async def _run_task_loop():
|
||||
while True:
|
||||
task = await self.task_queue.get()
|
||||
try:
|
||||
result = self._run_task(task)
|
||||
if result is not None:
|
||||
task.state = ComputeTaskState.DONE
|
||||
task.result = result
|
||||
except Exception as e:
|
||||
logger.error(f"whisper_node run task error: {e}")
|
||||
task.state = ComputeTaskState.ERROR
|
||||
task.result = ComputeTaskResult()
|
||||
task.result.set_from_task(task)
|
||||
task.result.worker_id = self.node_id
|
||||
task.result.result_str = str(e)
|
||||
|
||||
asyncio.create_task(_run_task_loop())
|
||||
|
||||
def _run_task(self, task: ComputeTask):
|
||||
task.state = ComputeTaskState.RUNNING
|
||||
prompt = task.params["prompt"]
|
||||
response_format = None
|
||||
if "response_format" in task.params:
|
||||
response_format = task.params["response_format"]
|
||||
temperature = None
|
||||
if "temperature" in task.params:
|
||||
temperature = task.params["temperature"]
|
||||
language = None
|
||||
if "language" in task.params:
|
||||
language = task.params["language"]
|
||||
file = task.params["file"]
|
||||
|
||||
resp = openai.Audio.transcribe("whisper-1",
|
||||
file,
|
||||
self.open_api_key,
|
||||
prompt=prompt,
|
||||
response_format=response_format,
|
||||
temperature=temperature,
|
||||
language=language)
|
||||
result = ComputeTaskResult()
|
||||
result.set_from_task(task)
|
||||
result.worker_id = self.node_id
|
||||
result.result_str = resp["text"]
|
||||
result.result = resp
|
||||
return result
|
||||
|
||||
async def push_task(self, task: ComputeTask, proiority: int = 0):
|
||||
logger.info(f"whisper_node push task: {task.display()}")
|
||||
self.task_queue.put_nowait(task)
|
||||
|
||||
async def remove_task(self, task_id: str):
|
||||
pass
|
||||
|
||||
def get_task_state(self, task_id: str):
|
||||
pass
|
||||
|
||||
def display(self) -> str:
|
||||
return f"WhisperComputeNode: {self.node_id}"
|
||||
|
||||
def get_capacity(self):
|
||||
return 0
|
||||
|
||||
def is_support(self, task_type: ComputeTaskType) -> bool:
|
||||
if task_type == ComputeTaskType.VOICE_2_TEXT:
|
||||
return True
|
||||
return False
|
||||
|
||||
def is_local(self) -> bool:
|
||||
return False
|
||||
+407
-170
@@ -1,18 +1,23 @@
|
||||
|
||||
import logging
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
from asyncio import Queue
|
||||
from typing import Optional,Tuple
|
||||
from abc import ABC, abstractmethod
|
||||
|
||||
from .environment import Environment,EnvironmentEvent
|
||||
from .agent_message import AgentMsg,AgentMsgState
|
||||
from .agent_message import AgentMsg,AgentMsgStatus
|
||||
from .agent import AgentPrompt,AgentMsg
|
||||
from .chatsession import AIChatSession
|
||||
from .role import AIRole,AIRoleGroup
|
||||
from .ai_function import CallChain
|
||||
from .ai_function import AIFunction
|
||||
from .compute_kernel import ComputeKernel
|
||||
from .compute_task import ComputeTask,ComputeTaskResult,ComputeTaskState
|
||||
from .bus import AIBus
|
||||
from .workflow_env import WorkflowEnvironment
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -33,9 +38,20 @@ class MessageFilter:
|
||||
return True
|
||||
|
||||
|
||||
class LLMResult:
|
||||
def __init__(self) -> None:
|
||||
self.state : str = "ignore"
|
||||
self.resp : str = ""
|
||||
self.post_msgs = []
|
||||
self.send_msgs = []
|
||||
self.calls = []
|
||||
self.post_calls = []
|
||||
|
||||
|
||||
class Workflow:
|
||||
def __init__(self) -> None:
|
||||
self.workflow_name : str = None
|
||||
self.workflow_id : str = None
|
||||
self.rule_prompt : AgentPrompt = None
|
||||
self.workflow_config = None
|
||||
self.role_group : dict = None
|
||||
@@ -44,6 +60,8 @@ class Workflow:
|
||||
self.sub_workflows = {}
|
||||
self.owner_workflow = None
|
||||
self.db_file = None
|
||||
self.env_db_file = None
|
||||
self.workflow_env:WorkflowEnvironment = None
|
||||
|
||||
self.is_start = False
|
||||
self.msg_queue = Queue()
|
||||
@@ -62,51 +80,153 @@ class Workflow:
|
||||
logger.error("workflow config must have name")
|
||||
return False
|
||||
self.workflow_name = config.get("name")
|
||||
if self.owner_workflow is None:
|
||||
self.workflow_id = self.workflow_name
|
||||
else:
|
||||
self.workflow_id = self.owner_workflow.workflow_id + "." + self.workflow_name
|
||||
self.db_file = self.owner_workflow.db_file
|
||||
|
||||
if config.get("prompt") is not None:
|
||||
self.rule_prompt = AgentPrompt()
|
||||
if self.rule_prompt.load_from_config(config.get("prompt")) is False:
|
||||
logger.error("Workflow load prompt failed")
|
||||
return False
|
||||
|
||||
#if config.get("rule_prompt") is None:
|
||||
# logger.error("workflow config must have rule_prompt")
|
||||
# return False
|
||||
#self.rule_prompt = AgentPrompt()
|
||||
#if self.rule_prompt.load_from_config(config.get("rule_prompt")) is False:
|
||||
# logger.error("Workflow load rule_prompt failed")
|
||||
# return False
|
||||
if config.get("roles") is None:
|
||||
logger.error("workflow config must have roles")
|
||||
return False
|
||||
self.role_group = AIRoleGroup()
|
||||
self.role_group.owner_name = self.workflow_id
|
||||
if self.role_group.load_from_config(config.get("roles")) is False:
|
||||
logger.error("Workflow load role_group failed")
|
||||
return False
|
||||
|
||||
if config.get("input_filter") is not None:
|
||||
if config.get("filter") is not None:
|
||||
self.input_filter = MessageFilter()
|
||||
if self.input_filter.load_from_config(config.get("input_filter")) is False:
|
||||
if self.input_filter.load_from_config(config.get("filter")) is False:
|
||||
logger.error("Workflow load input_filter failed")
|
||||
return False
|
||||
|
||||
if self.owner_workflow is None:
|
||||
self.env_db_file = os.path.dirname(self.db_file) + "/" + self.workflow_id + "_env.db"
|
||||
else:
|
||||
self.env_db_file = self.owner_workflow.env_db_file
|
||||
self.workflow_env = WorkflowEnvironment(self.workflow_id,self.env_db_file)
|
||||
|
||||
env_ndoe = config.get("enviroment")
|
||||
if env_ndoe is not None:
|
||||
if self._load_env_from_config(env_ndoe) is False:
|
||||
logger.error("Workflow load env failed")
|
||||
return False
|
||||
|
||||
connected_env_ndoe = config.get("connected_env")
|
||||
if connected_env_ndoe is not None:
|
||||
for _node in connected_env_ndoe:
|
||||
env_id = _node.get("env_id")
|
||||
if env_id is None:
|
||||
continue
|
||||
|
||||
remote_env = Environment.get_env_by_id(_node.get(env_id))
|
||||
if remote_env is None:
|
||||
logger.error(f"Workflow load connected_env failed, env {env_id} not found!")
|
||||
return False
|
||||
self.connect_to_environment(remote_env,_node.get("event2msg"))
|
||||
|
||||
sub_workflows = config.get("sub_workflows")
|
||||
if sub_workflows is not None:
|
||||
if self._load_sub_workflows(sub_workflows) is False:
|
||||
logger.error("Workflow load sub workflows failed")
|
||||
return False
|
||||
|
||||
#TODO: load env
|
||||
|
||||
return True
|
||||
|
||||
def _load_env_from_config(self,config:dict) -> bool:
|
||||
for k,v in config.items():
|
||||
self.workflow_env.set_value(k,v,False)
|
||||
|
||||
def _load_sub_workflows(self,config:dict) -> bool:
|
||||
for k,v in config.items():
|
||||
sub_workflow = Workflow()
|
||||
sub_workflow.set_owner(self)
|
||||
|
||||
if sub_workflow.load_from_config(v) is False:
|
||||
logger.error(f"load sub workflow {k} failed!")
|
||||
return False
|
||||
self.sub_workflows[k] = sub_workflow
|
||||
return True
|
||||
|
||||
def _parse_msg_target(self,s:str)->list[str]:
|
||||
return s.split(".")
|
||||
|
||||
async def _forword_msg(self,inner_obj_id,msg):
|
||||
i : int = 1
|
||||
current_workflow = self
|
||||
while i < len(inner_obj_id):
|
||||
if i == len(inner_obj_id) - 1:
|
||||
the_role : AIRole = current_workflow.role_group.get(inner_obj_id[i])
|
||||
current_workflow_chatsession = AIChatSession.get_session(current_workflow.workflow_id,msg.sender + "#" + msg.topic,current_workflow.db_file)
|
||||
if the_role is not None:
|
||||
return await current_workflow.role_process_msg(msg,the_role,current_workflow_chatsession)
|
||||
sub_workflow = current_workflow.sub_workflows.get(inner_obj_id[i])
|
||||
if sub_workflow is not None:
|
||||
return await sub_workflow._process_msg(msg)
|
||||
logger.error(f"{msg.target} not found! forword message failed!")
|
||||
return None
|
||||
else:
|
||||
current_workflow = current_workflow.sub_workflows.get(inner_obj_id[i])
|
||||
if current_workflow is None:
|
||||
logger.error(f"sub workflow {inner_obj_id[i]} not found!")
|
||||
return None
|
||||
|
||||
i += 1
|
||||
|
||||
logger.error(f"{msg.target} not found! forword message failed!")
|
||||
return None
|
||||
|
||||
def get_workflow_id_from_target(self,target:str) -> str:
|
||||
target_list = target.split(".")
|
||||
if len(target_list) == 0:
|
||||
return target
|
||||
else:
|
||||
result_str = ""
|
||||
p = 0
|
||||
for s in target_list:
|
||||
p = p + 1
|
||||
result_str += s
|
||||
if p < len(target_list)-1:
|
||||
result_str += "."
|
||||
else:
|
||||
return result_str
|
||||
|
||||
async def _process_msg(self,msg:AgentMsg):
|
||||
real_target = msg.target.split(".")[0]
|
||||
targets = self._parse_msg_target(msg.target)
|
||||
if len(targets) > 1:
|
||||
return await self._forword_msg(targets,msg)
|
||||
#0 we don't support workflow join a group right now, this cloud be a feture in future
|
||||
if msg.mentions is not None:
|
||||
logger.warn(f"workflow {self.workflow_id} recv a group chat message,not support ignore!")
|
||||
return None
|
||||
|
||||
#1. workflow start process message
|
||||
final_result = None
|
||||
chatsession = None
|
||||
|
||||
# this is workflow's group_chat session
|
||||
session_topic = msg.sender + "#" + msg.topic
|
||||
chatsesssion = AIChatSession.get_session(self.workflow_id,session_topic,self.db_file)
|
||||
|
||||
#2. find role by msg.mentions or workflow's selector logic
|
||||
if msg.mentions is not None:
|
||||
if not self.workflow_id in msg.mentions:
|
||||
chatsesssion.append(msg)
|
||||
logger.info(f"workflow {self.workflow_id} recv a group chat message from {msg.sender},but is not mentioned,ignore!")
|
||||
return None
|
||||
|
||||
for mention in msg.mentions:
|
||||
this_role = self.role_group.get(mention)
|
||||
if this_role is not None:
|
||||
return await self.role_process_msg(msg,this_role,chatsesssion)
|
||||
|
||||
if self.input_filter is not None:
|
||||
select_role_id = self.input_filter.select(msg)
|
||||
if select_role_id is not None:
|
||||
@@ -115,205 +235,322 @@ class Workflow:
|
||||
logger.error(f"input_filter return invalid role id:{select_role_id}, role not found in role_group")
|
||||
return None
|
||||
|
||||
result = await self._role_process_msg(msg,select_role)
|
||||
if result is None:
|
||||
logger.error(f"_process_msg return None for :{msg}")
|
||||
return
|
||||
if chatsession is not None:
|
||||
chatsession.append_post(result)
|
||||
final_result = result
|
||||
return await self.role_process_msg(msg,select_role,chatsesssion)
|
||||
else:
|
||||
logger.error(f"input_filter return None for :{msg}")
|
||||
return
|
||||
|
||||
else:
|
||||
results = {}
|
||||
for this_role in self.role_group.roles.values():
|
||||
# TODO : we would do this in parallel
|
||||
a_result = await self._role_process_msg(msg,this_role)
|
||||
results[this_role.get_name()] = a_result
|
||||
|
||||
# merge result from all roles
|
||||
# TODO: one input msg can have multiple result msg, at this while ,we only support one result msg
|
||||
final_result:AgentMsg = self._merge_msg_result(results)
|
||||
if chatsession is not None:
|
||||
chatsession.append_post(final_result)
|
||||
|
||||
return final_result
|
||||
|
||||
|
||||
|
||||
|
||||
async def _role_process_msg(self,msg:AgentMsg,the_role:AIRole) -> None:
|
||||
# TODO : we just record role's chatsession, but in future, we would record workflow's chatsession(like a groupo chat)
|
||||
session_topic = f"{the_role.get_name()}#{msg.sender}#{msg.topic}"
|
||||
chatsession = AIChatSession.get_session(self.workflow_name,session_topic,self.db_file)
|
||||
if chatsession is None:
|
||||
logger.error(f"get session {session_topic}@{self.workflow_name} failed!")
|
||||
logger.error(f"input_filter return None for :{msg.body}")
|
||||
return None
|
||||
|
||||
# prompt generat progress is most important part of workflow(app) develope
|
||||
logger.error(f"{self.workflow_id}:no role can process this msg:{msg.body}")
|
||||
return final_result
|
||||
|
||||
@classmethod
|
||||
def prase_llm_result(cls,llm_result_str:str)->LLMResult:
|
||||
r = LLMResult()
|
||||
if llm_result_str is None:
|
||||
r.state = "ignore"
|
||||
return r
|
||||
if llm_result_str == "ignore":
|
||||
r.state = "ignore"
|
||||
return r
|
||||
|
||||
lines = llm_result_str.splitlines()
|
||||
is_need_wait = False
|
||||
for line in lines:
|
||||
func_call = AgentMsg.parse_function_call(line)
|
||||
if func_call:
|
||||
func_args = func_call[1]
|
||||
match func_call[0]:
|
||||
case "sendmsg":# sendmsg($target_id,$msg_content)
|
||||
if len(func_args) != 2:
|
||||
logger.error(f"parse sendmsg failed! {func_call}")
|
||||
continue
|
||||
new_msg = AgentMsg()
|
||||
target_id = func_args[0]
|
||||
msg_content = func_args[1]
|
||||
new_msg.set("_",target_id,msg_content)
|
||||
|
||||
r.send_msgs.append(new_msg)
|
||||
is_need_wait = True
|
||||
continue
|
||||
case "postmsg":# postmsg($target_id,$msg_content)
|
||||
if len(func_args) != 2:
|
||||
logger.error(f"parse postmsg failed! {func_call}")
|
||||
continue
|
||||
new_msg = AgentMsg()
|
||||
target_id = func_args[0]
|
||||
msg_content = func_args[1]
|
||||
new_msg.set("_",target_id,msg_content)
|
||||
r.post_msgs.append(new_msg)
|
||||
continue
|
||||
case "call":# call($func_name,$args_str)
|
||||
r.calls.append(func_call)
|
||||
is_need_wait = True
|
||||
continue
|
||||
case "post_call": # post_call($func_name,$args_str)
|
||||
r.post_calls.append(func_call)
|
||||
continue
|
||||
|
||||
r.resp += line + "\n"
|
||||
else:
|
||||
r.resp += line + "\n"
|
||||
|
||||
if is_need_wait:
|
||||
r.state = "waiting"
|
||||
else:
|
||||
r.state = "reponsed"
|
||||
|
||||
return r
|
||||
|
||||
async def role_post_msg(self,msg:AgentMsg,the_role:AIRole,workflow_chat_session:AIChatSession):
|
||||
msg.sender = the_role.get_role_id()
|
||||
|
||||
target_role = self.role_group.get(msg.target)
|
||||
if target_role:
|
||||
msg.target = target_role.get_role_id()
|
||||
logger.info(f"{msg.sender} post message {msg.msg_id} to inner role: {msg.target}")
|
||||
asyncio.create_task(self.role_process_msg(msg,target_role,workflow_chat_session))
|
||||
return
|
||||
|
||||
target_workflow = self.sub_workflows.get(msg.target)
|
||||
if target_workflow:
|
||||
msg.target = target_workflow.workflow_id
|
||||
logger.info(f"{msg.sender} post message {msg.msg_id} to sub workflow: {msg.target}")
|
||||
asyncio.create_task(target_workflow._process_msg(msg))
|
||||
|
||||
logger.info(f"{msg.sender} post message {msg.msg_id} to AIBus: {msg.target}")
|
||||
await self.get_bus().post_message(msg.target,msg)
|
||||
return
|
||||
|
||||
|
||||
async def role_send_msg(self,msg:AgentMsg,the_role:AIRole,workflow_chat_session:AIChatSession):
|
||||
msg.sender = the_role.get_role_id()
|
||||
target_role = self.role_group.get(msg.target)
|
||||
if target_role:
|
||||
# msg.target = target_role.get_role_id()
|
||||
logger.info(f"{msg.sender} send message {msg.msg_id} to inner role: {msg.target}")
|
||||
return await self.role_process_msg(msg,target_role,workflow_chat_session)
|
||||
|
||||
target_workflow = self.sub_workflows.get(msg.target)
|
||||
if target_workflow:
|
||||
# msg.target = target_workflow.workflow_id
|
||||
logger.info(f"{msg.sender} send message {msg.msg_id} to sub workflow: {msg.target}")
|
||||
return await target_workflow._process_msg(msg)
|
||||
|
||||
logger.info(f"{msg.sender} post message {msg.msg_id} to AIBus: {msg.target}")
|
||||
return await self.get_bus().send_message(msg)
|
||||
|
||||
async def role_call(self,call:tuple,the_role:AIRole):
|
||||
logger.info(f"{the_role.role_id} call {call[0]} with args {call[1]}")
|
||||
func_name = call[0]
|
||||
arguments = call[1]
|
||||
|
||||
func_node : AIFunction = self.workflow_env.get_ai_function(func_name)
|
||||
if func_node is None:
|
||||
return "execute failed,function not found"
|
||||
|
||||
result_str:str = await func_node.execute(**arguments)
|
||||
return result_str
|
||||
|
||||
async def role_post_call(self,call:tuple,the_role:AIRole):
|
||||
logger.info(f"{the_role.role_id} post call {call[0]} with args {call[1]}")
|
||||
return await self.role_call(call,the_role)
|
||||
|
||||
def _format_msg_by_env_value(self,prompt:AgentPrompt):
|
||||
if self.workflow_env is None:
|
||||
return
|
||||
|
||||
for msg in prompt.messages:
|
||||
old_content = msg.get("content")
|
||||
msg["content"] = old_content.format_map(self.workflow_env)
|
||||
|
||||
def _get_inner_functions(self) -> dict:
|
||||
all_inner_function = self.workflow_env.get_all_ai_functions()
|
||||
if all_inner_function is None:
|
||||
return None
|
||||
|
||||
result_func = []
|
||||
for inner_func in all_inner_function:
|
||||
this_func = {}
|
||||
this_func["name"] = inner_func.get_name()
|
||||
this_func["description"] = inner_func.get_description()
|
||||
this_func["parameters"] = inner_func.get_parameters()
|
||||
result_func.append(this_func)
|
||||
if len(result_func) > 0:
|
||||
return result_func
|
||||
return None
|
||||
|
||||
async def _role_execute_func(self,the_role:AIRole,inenr_func_call_node:dict,prompt:AgentPrompt,org_msg:AgentMsg) -> str:
|
||||
from .compute_kernel import ComputeKernel
|
||||
|
||||
func_name = inenr_func_call_node.get("name")
|
||||
arguments = json.loads(inenr_func_call_node.get("arguments"))
|
||||
|
||||
func_node : AIFunction = self.workflow_env.get_ai_function(func_name)
|
||||
if func_node is None:
|
||||
return "execute failed,function not found"
|
||||
|
||||
ineternal_call_record = AgentMsg.create_internal_call_msg(func_name,arguments,org_msg.get_msg_id(),org_msg.target)
|
||||
|
||||
result_str:str = await func_node.execute(**arguments)
|
||||
|
||||
inner_functions = self._get_inner_functions()
|
||||
prompt.messages.append({"role":"function","content":result_str,"name":func_name})
|
||||
task_result:ComputeTaskResult = await ComputeKernel.get_instance().do_llm_completion(prompt,
|
||||
the_role.agent.llm_model_name,the_role.agent.max_token_size,
|
||||
inner_functions)
|
||||
|
||||
ineternal_call_record.result_str = task_result.result_str
|
||||
ineternal_call_record.done_time = time.time()
|
||||
org_msg.inner_call_chain.append(ineternal_call_record)
|
||||
|
||||
inner_func_call_node = task_result.result_message.get("function_call")
|
||||
if inner_func_call_node:
|
||||
return await self._role_execute_func(the_role,inner_func_call_node,prompt,org_msg)
|
||||
else:
|
||||
return task_result.result_str
|
||||
|
||||
def _is_in_same_workflow(self,msg) -> bool:
|
||||
pass
|
||||
|
||||
async def role_process_msg(self,msg:AgentMsg,the_role:AIRole,workflow_chat_session:AIChatSession):
|
||||
msg.target = the_role.get_role_id()
|
||||
|
||||
|
||||
prompt = AgentPrompt()
|
||||
prompt.append(the_role.agent.prompt)
|
||||
prompt.append(self.get_workflow_rule_prompt())
|
||||
prompt.append(the_role.get_prompt())
|
||||
|
||||
# prompt.append(self.get_workflow_rule_prompt())
|
||||
# prompt.append(self._get_function_prompt(the_role.get_name()))
|
||||
# prompt.append(self._get_knowlege_prompt(the_role.get_name()))
|
||||
|
||||
prompt.append(await self._get_prompt_from_session(chatsession))
|
||||
#prompt.append(await self._get_prompt_from_session(chatsession,the_role.get_name())) # chat context
|
||||
#support group chat, user content include sender name!
|
||||
prompt.append(await self._get_prompt_from_session(workflow_chat_session))
|
||||
|
||||
msg_prompt = AgentPrompt()
|
||||
msg_prompt.messages = [{"role":"user","content":msg.body}]
|
||||
msg_prompt.messages = [{"role":"user","content":f"{msg.sender}:{msg.body}"}]
|
||||
prompt.append(msg_prompt)
|
||||
|
||||
result = await ComputeKernel().do_llm_completion(prompt,the_role.agent.get_llm_model_name(),the_role.agent.get_max_token_size())
|
||||
chatsession.append_recv(msg)
|
||||
final_result = result
|
||||
self._format_msg_by_env_value(prompt)
|
||||
inner_functions = self._get_inner_functions()
|
||||
|
||||
result_type : str = self._get_llm_result_type(result)
|
||||
is_ignore = False
|
||||
match result_type:
|
||||
case "function":
|
||||
callchain:CallChain = self._parse_function_call_chain(result)
|
||||
resp = await callchain.exec()
|
||||
if callchain.have_result():
|
||||
# generator proc resp prompt with WAITING state
|
||||
#proc_resp_prompt:AgentPrompt = self._get_resp_prompt(resp,msg,the_role,prompt,chatsession)
|
||||
final_result = await ComputeKernel().do_llm_completion(proc_resp_prompt,the_role.agent.get_llm_model_name(),the_role.agent.get_max_token_size())
|
||||
return final_result
|
||||
async def _do_process_msg():
|
||||
#TODO: send msg to agent might be better?
|
||||
task_result:ComputeTaskResult = await ComputeKernel.get_instance().do_llm_completion(prompt,the_role.agent.get_llm_model_name(),the_role.agent.get_max_token_size(),inner_functions)
|
||||
result_str = task_result.result_str
|
||||
logger.info(f"{the_role.role_id} process {msg.sender}:{msg.body},llm str is :{result_str}")
|
||||
|
||||
inner_func_call_node = task_result.result_message.get("function_call")
|
||||
|
||||
if inner_func_call_node:
|
||||
#TODO to save more token ,can i use msg_prompt?
|
||||
result_str = await self._role_execute_func(the_role,inner_func_call_node,prompt,msg)
|
||||
|
||||
result = Workflow.prase_llm_result(result_str)
|
||||
for postmsg in result.post_msgs:
|
||||
postmsg.prev_msg_id = msg.get_msg_id()
|
||||
# might be craete a new msg.topic for this postmsg
|
||||
postmsg.topic = msg.topic
|
||||
|
||||
await self.role_post_msg(postmsg,the_role,workflow_chat_session)
|
||||
if not self._is_in_same_workflow(postmsg):
|
||||
role_sesion = AIChatSession.get_session(the_role.get_role_id(),f"{postmsg.target}#{msg.topic}",self.db_file)
|
||||
role_sesion.append(postmsg)
|
||||
else:
|
||||
# message will be saved in role.process_message
|
||||
pass
|
||||
|
||||
|
||||
case "send_message":
|
||||
# send message to other / sub workflow
|
||||
next_msg:AgentMsg = self._parse_to_msg(result)
|
||||
if next_msg is not None:
|
||||
next_msg.sender = self.workflow_name
|
||||
logger.info(f"W#{self.workflow_name} send message to {next_msg.get_target()}")
|
||||
resp_msg = await self.get_bus().send_message(next_msg.get_target(),next_msg)
|
||||
if resp_msg is not None:
|
||||
msg_prompt = AgentPrompt()
|
||||
msg_prompt.messages = [{"role":"assistant","content":result},{"role":"user","content":f"{next_msg.get_target()}:{resp_msg.body}"}]
|
||||
|
||||
final_result = await ComputeKernel().do_llm_completion(proc_resp_prompt,the_role.agent.get_llm_model_name(),the_role.agent.get_max_token_size())
|
||||
|
||||
|
||||
case "post_message":
|
||||
# post message to other / sub workflow
|
||||
next_msg:AgentMsg = self._parse_to_msg(result)
|
||||
if next_msg is not None:
|
||||
next_msg.sender = self.workflow_name
|
||||
logger.info(f"W#{self.workflow_name} post message to {next_msg.get_target()}")
|
||||
self.get_bus().post_message(next_msg.get_target(),next_msg)
|
||||
for post_call in result.post_calls:
|
||||
action_msg = msg.create_action_msg(post_call[0],post_call[1],the_role.get_role_id())
|
||||
workflow_chat_session.append(action_msg)
|
||||
await self.role_post_call(post_call,the_role)
|
||||
#save post_call
|
||||
|
||||
result_prompt_str = ""
|
||||
match result.state:
|
||||
case "ignore":
|
||||
is_ignore = True
|
||||
|
||||
if is_ignore:
|
||||
return None
|
||||
|
||||
resp_msg = AgentMsg()
|
||||
resp_msg.set(self.workflow_name,msg.sender,final_result)
|
||||
chatsession.append_post(resp_msg)
|
||||
case "reponsed":
|
||||
resp_msg = msg.create_resp_msg(result.resp)
|
||||
resp_msg.sender = the_role.get_role_id()
|
||||
# It is always the person handling the messages who puts them into the session.
|
||||
workflow_chat_session.append(msg)
|
||||
workflow_chat_session.append(resp_msg)
|
||||
#await self.get_bus().resp_message(resp_msg)
|
||||
return resp_msg
|
||||
case "waiting":
|
||||
# TODO: Use role:"function" would be better
|
||||
for sendmsg in result.send_msgs:
|
||||
target = sendmsg.target
|
||||
sendmsg.topic = msg.topic
|
||||
sendmsg.prev_msg_id = msg.get_msg_id()
|
||||
send_resp = await self.role_send_msg(sendmsg,the_role,workflow_chat_session)
|
||||
if send_resp is not None:
|
||||
result_prompt_str += f"\n{target} response is :{send_resp.body}"
|
||||
|
||||
async def _pop_msg(self) -> AgentMsg:
|
||||
if not self._is_in_same_workflow(sendmsg):
|
||||
role_sesion = AIChatSession.get_session(the_role.get_role_id(),f"{sendmsg.target}#{sendmsg.topic}",self.db_file)
|
||||
role_sesion.append(sendmsg)
|
||||
role_sesion.append(send_resp)
|
||||
else:
|
||||
# message will be saved in role.process_message
|
||||
pass
|
||||
|
||||
def _get_chat_session_for_msg(self,msg:AgentMsg) -> AIChatSession:
|
||||
pass
|
||||
for call in result.calls:
|
||||
action_msg = msg.create_action_msg(call[0],call[1],call_result,the_role.get_role_id)
|
||||
call_result = await self.role_call(call,the_role)
|
||||
|
||||
if call_result is not None:
|
||||
result_prompt_str += f"\ncall {call[0]} result is :{call_result}"
|
||||
#save action
|
||||
action_msg.result_str = call_result
|
||||
workflow_chat_session.append(action_msg)
|
||||
|
||||
result_prompt = AgentPrompt()
|
||||
result_prompt.messages = [{"role":"user","content":result_prompt_str}]
|
||||
prompt.append(result_prompt)
|
||||
r = await _do_process_msg()
|
||||
return r
|
||||
|
||||
return await _do_process_msg()
|
||||
|
||||
async def _get_prompt_from_session(self,chatsession:AIChatSession) -> AgentPrompt:
|
||||
messages = chatsession.read_history() # read last 10 message
|
||||
result_prompt = AgentPrompt()
|
||||
for msg in reversed(messages):
|
||||
if msg.target == chatsession.owner_id:
|
||||
result_prompt.messages.append({"role":"user","content":f"{msg.sender}:{msg.body}"})
|
||||
if msg.sender == chatsession.owner_id:
|
||||
result_prompt.messages.append({"role":"assistant","content":msg.body})
|
||||
else:
|
||||
result_prompt.messages.append({"role":"user","content":f"{msg.body}"})
|
||||
|
||||
return result_prompt
|
||||
|
||||
def _get_msg_queue(self,session_id:str):
|
||||
pass
|
||||
|
||||
def _merge_msg_result(self,results:dict) -> AgentMsg:
|
||||
# TODO: one input msg can have multiple result msg, at this while ,we only support one result msg
|
||||
for k,v in results.items():
|
||||
if v is not None:
|
||||
return v
|
||||
|
||||
def _get_function_prompt(self,role_name:str) -> AgentPrompt:
|
||||
pass
|
||||
|
||||
def _get_knowlege_prompt(self,role_name:str) -> AgentPrompt:
|
||||
pass
|
||||
|
||||
def _get_resp_prompt(self,resp:str,msg:AgentMsg,role:AIRole,prompt:AgentPrompt) -> AgentPrompt:
|
||||
pass
|
||||
|
||||
def get_workflow_rule_prompt(self) -> AgentPrompt:
|
||||
return self.rule_prompt
|
||||
|
||||
def _get_llm_result_type(self,llm_resp_str:str) -> str:
|
||||
if llm_resp_str == "ignore":
|
||||
return "ignore"
|
||||
|
||||
if llm_resp_str.find("sendmsg(") != -1:
|
||||
return "send_message"
|
||||
|
||||
if llm_resp_str.find("postmsg(") != -1:
|
||||
return "post_message"
|
||||
|
||||
if llm_resp_str.find("call(") != -1:
|
||||
return "function"
|
||||
|
||||
return "text"
|
||||
|
||||
def _parse_function_call_chain(self,llm_resp_str) -> CallChain:
|
||||
pass
|
||||
|
||||
def _parse_to_msg(self,llm_resp_str) -> AgentMsg:
|
||||
lines = llm_resp_str.splitlines()
|
||||
for line in lines:
|
||||
if line.startswith("sendmsg("):
|
||||
line = line[8:]
|
||||
_index = line.find(",")
|
||||
msg = AgentMsg()
|
||||
msg.set("",line[:_index],line[_index+1:])
|
||||
return msg
|
||||
|
||||
if line.startswith("postmsg("):
|
||||
line = line[8:]
|
||||
_index = line.find(",")
|
||||
msg = AgentMsg()
|
||||
msg.set("",line[:_index],line[_index+1:])
|
||||
return msg
|
||||
|
||||
return None
|
||||
|
||||
def get_workflow(self,workflow_name:str):
|
||||
"""get workflow from known workflow list or sub workflow list"""
|
||||
pass
|
||||
|
||||
|
||||
def _env_event_to_msg(self,env_event:EnvironmentEvent) -> AgentMsg:
|
||||
pass
|
||||
|
||||
def get_inner_environment(self,env_id:str) -> Environment:
|
||||
pass
|
||||
|
||||
def connect_to_environment(self,env:Environment) -> None:
|
||||
the_env = self.connected_environment.get(env.get_id())
|
||||
if the_env is None:
|
||||
self.connected_environment[env.get_id()] = env
|
||||
def _env_msg_handler(env_event:EnvironmentEvent) -> None:
|
||||
the_msg:AgentMsg= self._env_event_to_msg(env_event)
|
||||
self.post_msg(the_msg)
|
||||
def connect_to_environment(self,the_env:Environment,conn_info:dict) -> None:
|
||||
if the_env is not None:
|
||||
self.workflow_env.add_owner_env(the_env)
|
||||
|
||||
# register all event handler
|
||||
the_env.attach_event_handler(None,_env_msg_handler)
|
||||
else:
|
||||
logger.warn(f"environment {env.get_id()} already connected!")
|
||||
#for event2msg in conn_info:
|
||||
# for k,v in event2msg:
|
||||
# if k == "role":
|
||||
# continue
|
||||
# else:
|
||||
#
|
||||
# def _env_msg_handler(env_event:EnvironmentEvent) -> None:
|
||||
# the_msg:AgentMsg= self._env_event_to_msg(env_event)
|
||||
# self.role_post_msg
|
||||
|
||||
# the_env.attach_event_handler(k,_env_msg_handler)
|
||||
# break
|
||||
|
||||
|
||||
@@ -0,0 +1,151 @@
|
||||
|
||||
from datetime import datetime
|
||||
import asyncio
|
||||
import sqlite3 # Because sqlite3 IO operation is small, so we can use sqlite3 directly.(so we don't need to use async sqlite3 now)
|
||||
from sqlite3 import Error
|
||||
import threading
|
||||
import logging
|
||||
from typing import Optional
|
||||
from .environment import Environment,EnvironmentEvent
|
||||
from .ai_function import SimpleAIFunction
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class CalenderEvent(EnvironmentEvent):
|
||||
def __init__(self,data) -> None:
|
||||
super().__init__()
|
||||
self.event_name = "timer"
|
||||
self.data = data
|
||||
|
||||
def display(self) -> str:
|
||||
return f"#event timer:{self.data}"
|
||||
|
||||
# AI Calender GOAL: Let user use "create notify after 2 days" to create a timer event
|
||||
class CalenderEnvironment(Environment):
|
||||
def __init__(self, env_id: str) -> None:
|
||||
super().__init__(env_id)
|
||||
self.is_run = False
|
||||
|
||||
self.add_ai_function(SimpleAIFunction("get_time",
|
||||
"get current time",
|
||||
self._get_now))
|
||||
|
||||
def _do_get_value(self,key:str) -> Optional[str]:
|
||||
return None
|
||||
|
||||
def start(self) -> None:
|
||||
if self.is_run:
|
||||
return
|
||||
self.is_run = True
|
||||
|
||||
self.register_get_handler("now",self.get_now)
|
||||
async def timer_loop():
|
||||
while True:
|
||||
if self.is_run == False:
|
||||
break
|
||||
|
||||
await asyncio.sleep(1.0)
|
||||
now = datetime.now()
|
||||
formatted_time = now.strftime('%Y-%m-%d %H:%M:%S')
|
||||
env_event:CalenderEvent = CalenderEvent(formatted_time)
|
||||
await self.fire_event("timer",env_event)
|
||||
|
||||
return
|
||||
|
||||
asyncio.create_task(timer_loop())
|
||||
|
||||
def stop(self):
|
||||
self.is_run = False
|
||||
|
||||
def get_now(self,key:str)->str:
|
||||
now = datetime.now()
|
||||
formatted_time = now.strftime('%Y-%m-%d %H:%M:%S')
|
||||
return formatted_time
|
||||
|
||||
async def _get_now(self) -> str:
|
||||
now = datetime.now()
|
||||
formatted_time = now.strftime('%Y-%m-%d %H:%M:%S')
|
||||
return formatted_time
|
||||
|
||||
# Default Workflow Environment(Context)
|
||||
class WorkflowEnvironment(Environment):
|
||||
def __init__(self, env_id: str,db_file:str) -> None:
|
||||
super().__init__(env_id)
|
||||
self.db_file = db_file
|
||||
self.local = threading.local()
|
||||
self.table_name = "WorkflowEnv_" + env_id
|
||||
|
||||
|
||||
def _get_conn(self):
|
||||
""" get db connection """
|
||||
if not hasattr(self.local, 'conn'):
|
||||
self.local.conn = self._create_connection()
|
||||
return self.local.conn
|
||||
|
||||
def _create_connection(self):
|
||||
""" create a database connection to a SQLite database """
|
||||
conn = None
|
||||
try:
|
||||
conn = sqlite3.connect(self.db_file)
|
||||
except Error as e:
|
||||
logging.error("Error occurred while connecting to database: %s", e)
|
||||
return None
|
||||
|
||||
if conn:
|
||||
self._create_table(conn)
|
||||
|
||||
return conn
|
||||
|
||||
def close(self):
|
||||
if not hasattr(self.local, 'conn'):
|
||||
return
|
||||
self.local.conn.close()
|
||||
|
||||
def _create_table(self, conn):
|
||||
""" create table """
|
||||
try:
|
||||
# create sessions table
|
||||
conn.execute(f"""
|
||||
CREATE TABLE IF NOT EXISTS """ + self.table_name + """ (
|
||||
EnvKey TEXT PRIMARY KEY,
|
||||
EnvValue TEXT,
|
||||
UpdateTime TEXT
|
||||
);
|
||||
""")
|
||||
conn.commit()
|
||||
except Error as e:
|
||||
logging.error("Error occurred while creating tables: %s", e)
|
||||
|
||||
def _do_get_value(self, key: str) -> str | None:
|
||||
try:
|
||||
conn = self._get_conn()
|
||||
c = conn.cursor()
|
||||
c.execute("SELECT EnvValue FROM " + self.table_name +" WHERE EnvKey = ?", (key,))
|
||||
value = c.fetchone()
|
||||
if value is None:
|
||||
return None
|
||||
return value[0]
|
||||
except Error as e:
|
||||
logging.error(f"Error occurred while _do_get_value{key}: {e}")
|
||||
return None
|
||||
|
||||
def set_value(self, key: str, str_value: str, is_storage:bool=True):
|
||||
super().set_value(key,str_value)
|
||||
if is_storage is False:
|
||||
return
|
||||
|
||||
try:
|
||||
conn = self._get_conn()
|
||||
conn.execute("""
|
||||
INSERT OR REPLACE INTO """ + self.table_name+ """ (EnvKey, EnvValue, UpdateTime)
|
||||
VALUES (?, ?, ?)
|
||||
""", (key, str_value, datetime.now()))
|
||||
conn.commit()
|
||||
return 0 # return 0 if successful
|
||||
except Error as e:
|
||||
logging.error(f"Error occurred while update env{self.env_id}.{key} ,error:{e}")
|
||||
|
||||
def get_functions(self):
|
||||
pass
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
import logging
|
||||
import toml
|
||||
|
||||
from aios_kernel import AIAgent,AIAgentTemplete
|
||||
from aios_kernel import AIAgent,AIAgentTemplete,AIStorage
|
||||
from package_manager import PackageEnv,PackageEnvManager,PackageMediaInfo,PackageInstallTask
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -11,16 +11,19 @@ logger = logging.getLogger(__name__)
|
||||
class AgentManager:
|
||||
_instance = None
|
||||
|
||||
def __new__(cls):
|
||||
@classmethod
|
||||
def get_instance(cls)->'AgentManager':
|
||||
if cls._instance is None:
|
||||
cls._instance = super(AgentManager, cls).__new__(cls)
|
||||
cls._instance = AgentManager()
|
||||
return cls._instance
|
||||
|
||||
def initial(self) -> None:
|
||||
system_app_dir = AIStorage.get_instance().get_system_app_dir()
|
||||
user_data_dir = AIStorage.get_instance().get_myai_dir()
|
||||
|
||||
def initial(self,root_dir:str) -> None:
|
||||
self.agent_templete_env : PackageEnv = PackageEnvManager().get_env(f"{root_dir}/templetes/templetes.cfg")
|
||||
self.agent_env : PackageEnv = PackageEnvManager().get_env(f"{root_dir}/agents/agents.cfg")
|
||||
self.db_path = f"{root_dir}/agents_chat.db"
|
||||
self.agent_templete_env : PackageEnv = PackageEnvManager().get_env(f"{system_app_dir}/templates/templetes.cfg")
|
||||
self.agent_env : PackageEnv = PackageEnvManager().get_env(f"{system_app_dir}/agents/agents.cfg")
|
||||
self.db_path = f"{user_data_dir}/messages.db"
|
||||
self.loaded_agent_instance = {}
|
||||
if self.agent_templete_env is None:
|
||||
raise Exception("agent_manager initial failed")
|
||||
|
||||
@@ -134,16 +134,15 @@ class PackageEnv:
|
||||
|
||||
class PackageEnvManager:
|
||||
_instance = None
|
||||
def __new__(cls):
|
||||
@classmethod
|
||||
def get_instance(cls):
|
||||
if cls._instance is None:
|
||||
cls._instance = super(PackageEnvManager, cls).__new__(cls)
|
||||
cls._instance = PackageEnvManager()
|
||||
return cls._instance
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._pkg_envs = {}
|
||||
|
||||
pass
|
||||
|
||||
def get_env(self,cfg_path:str) -> PackageEnv:
|
||||
if cfg_path in self._pkg_envs:
|
||||
return self._pkg_envs[cfg_path]
|
||||
|
||||
@@ -1,30 +1,49 @@
|
||||
import logging
|
||||
import toml
|
||||
from aios_kernel import Workflow
|
||||
import os
|
||||
|
||||
from aios_kernel import Workflow,AIStorage
|
||||
from package_manager import PackageEnv,PackageEnvManager,PackageMediaInfo,PackageInstallTask
|
||||
from agent_manager import AgentManager
|
||||
logger = logging.getLogger(__name__)
|
||||
import os
|
||||
|
||||
class WorkflowManager:
|
||||
_instance = None
|
||||
|
||||
def __new__(cls):
|
||||
@classmethod
|
||||
def get_instance(cls):
|
||||
if cls._instance is None:
|
||||
cls._instance = super(WorkflowManager, cls).__new__(cls)
|
||||
cls._instance = WorkflowManager()
|
||||
return cls._instance
|
||||
|
||||
|
||||
def initial(self,root_dir:str) -> None:
|
||||
def initial(self) -> None:
|
||||
self.loaded_workflow = {}
|
||||
self.workflow_env = PackageEnvManager().get_env(f"{root_dir}/workflows.cfg")
|
||||
self.db_file = os.path.abspath(f"{root_dir}/workflows.db")
|
||||
system_app_dir = AIStorage.get_instance().get_system_app_dir()
|
||||
user_data_dir = AIStorage.get_instance().get_myai_dir()
|
||||
|
||||
self.workflow_env = PackageEnvManager().get_env(f"{system_app_dir}/workflows.cfg")
|
||||
self.db_file = os.path.abspath(f"{user_data_dir}/messages.db")
|
||||
if self.workflow_env is None:
|
||||
raise Exception("WorkflowManager initial failed")
|
||||
|
||||
async def get_agent_default_workflow(self,agent_id:str) -> Workflow:
|
||||
pass
|
||||
|
||||
|
||||
async def _load_workflow_agents(self,workflow:Workflow) -> bool:
|
||||
for v in workflow.role_group.roles.values():
|
||||
agent = await AgentManager().get(v.agent_name)
|
||||
if agent is None:
|
||||
logger.error(f"load agent {v.agent_name} failed!")
|
||||
return False
|
||||
v.agent = agent
|
||||
|
||||
for sub_workflow in workflow.sub_workflows.values():
|
||||
if await self._load_workflow_agents(sub_workflow) is False:
|
||||
return False
|
||||
return True
|
||||
|
||||
async def get_workflow(self,workflow_id:str) -> Workflow:
|
||||
the_workflow : Workflow = self.loaded_workflow.get(workflow_id)
|
||||
if the_workflow:
|
||||
@@ -38,14 +57,10 @@ class WorkflowManager:
|
||||
the_workflow = await self._load_workflow_from_media(workflow_media_info)
|
||||
if the_workflow is None:
|
||||
logger.warn(f"load workflow {workflow_id} from media failed!")
|
||||
|
||||
for v in the_workflow.role_group.roles.values():
|
||||
agent = await AgentManager().get(v.agent_name)
|
||||
if agent is None:
|
||||
logger.error(f"load agent {v.agent_name} failed!")
|
||||
return None
|
||||
v.agent = agent
|
||||
|
||||
if await self._load_workflow_agents(the_workflow) is False:
|
||||
return None
|
||||
|
||||
return the_workflow
|
||||
|
||||
@@ -64,10 +79,12 @@ class WorkflowManager:
|
||||
config_data = await config_file.read()
|
||||
config = toml.loads(config_data)
|
||||
result_workflow = Workflow()
|
||||
result_workflow.db_file = self.db_file
|
||||
|
||||
if result_workflow.load_from_config(config) is False:
|
||||
logger.error(f"load workflow from {workflow_media} failed!")
|
||||
return None
|
||||
result_workflow.db_file = self.db_file
|
||||
|
||||
return result_workflow
|
||||
except Exception as e:
|
||||
logger.error(f"read workflow.toml cfg from {workflow_media} failed! unexpected error occurred: {str(e)}")
|
||||
|
||||
+17
-1
@@ -1,7 +1,23 @@
|
||||
|
||||
chromadb==0.4
|
||||
openai==0.28
|
||||
toml==0.10
|
||||
Pillow==10.0
|
||||
moviepy==1.0
|
||||
base58==2.1
|
||||
base36==0.1
|
||||
aiofiles==23.2.1
|
||||
aiohttp==3.7.0
|
||||
aioimaplib==1.0.1
|
||||
aiosmtplib==2.0.2
|
||||
beautifulsoup4==4.12.2
|
||||
mail_parser==3.15.0
|
||||
openai==0.27.10
|
||||
Pillow
|
||||
prompt_toolkit==3.0.39
|
||||
protobuf
|
||||
pydantic==1.10.11
|
||||
python-telegram-bot==20.5
|
||||
Requests==2.31.0
|
||||
stability_sdk
|
||||
toml==0.10.2
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@ import sys
|
||||
import os
|
||||
import logging
|
||||
import re
|
||||
import toml
|
||||
|
||||
from typing import Any, Optional, TypeVar, Tuple, Sequence
|
||||
import argparse
|
||||
@@ -17,6 +18,19 @@ from prompt_toolkit.auto_suggest import AutoSuggestFromHistory
|
||||
from prompt_toolkit.completion import WordCompleter
|
||||
from prompt_toolkit.styles import Style
|
||||
|
||||
directory = os.path.dirname(__file__)
|
||||
sys.path.append(directory + '/../../')
|
||||
|
||||
from aios_kernel import AIOS_Version,UserConfigItem,AIStorage,Workflow,AIAgent,AgentMsg,AgentMsgStatus,ComputeKernel,OpenAI_ComputeNode,AIBus,AIChatSession,AgentTunnel,TelegramTunnel,CalenderEnvironment,Environment,EmailTunnel,LocalLlama_ComputeNode
|
||||
|
||||
|
||||
sys.path.append(directory + '/../../component/')
|
||||
from agent_manager import AgentManager
|
||||
from workflow_manager import WorkflowManager
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
shell_style = Style.from_dict({
|
||||
'title': '#87d7ff bold', #RGB
|
||||
'content': '#007f00 bold',
|
||||
@@ -24,68 +38,97 @@ shell_style = Style.from_dict({
|
||||
})
|
||||
|
||||
|
||||
directory = os.path.dirname(__file__)
|
||||
sys.path.append(directory + '/../../')
|
||||
from aios_kernel import Workflow,AIAgent,AgentMsg,AgentMsgState,ComputeKernel,OpenAI_ComputeNode,AIBus,AIChatSession
|
||||
|
||||
sys.path.append(directory + '/../../component/')
|
||||
from agent_manager import AgentManager
|
||||
from workflow_manager import WorkflowManager
|
||||
|
||||
|
||||
|
||||
class AIOS_Shell:
|
||||
def __init__(self,username:str) -> None:
|
||||
self.username = username
|
||||
self.current_target = "_"
|
||||
self.current_topic = "default"
|
||||
self.is_working = True
|
||||
|
||||
def declare_all_user_config(self):
|
||||
user_config = AIStorage.get_instance().get_user_config()
|
||||
user_config.add_user_config("username","username is your full name when using AIOS",False,None,)
|
||||
|
||||
openai_node = OpenAI_ComputeNode.get_instance()
|
||||
openai_node.declare_user_config()
|
||||
|
||||
|
||||
async def _handle_no_target_msg(self,bus:AIBus,msg:AgentMsg) -> bool:
|
||||
agent : AIAgent = await AgentManager().get(msg.target)
|
||||
target_id = msg.target.split(".")[0]
|
||||
agent : AIAgent = await AgentManager.get_instance().get(target_id)
|
||||
if agent is not None:
|
||||
bus.register_message_handler(msg.target,agent._process_msg)
|
||||
agent.owner_env = Environment.get_env_by_id("calender")
|
||||
bus.register_message_handler(target_id,agent._process_msg)
|
||||
return True
|
||||
|
||||
a_workflow = await WorkflowManager().get_workflow(msg.target)
|
||||
a_workflow = await WorkflowManager.get_instance().get_workflow(target_id)
|
||||
if a_workflow is not None:
|
||||
bus.register_message_handler(msg.target,a_workflow._process_msg)
|
||||
for subflow in a_workflow.sub_workflows.values():
|
||||
bus.register_message_handler(subflow.workflow_name,subflow._process_msg)
|
||||
bus.register_message_handler(target_id,a_workflow._process_msg)
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
async def is_agent(self,target_id:str) -> bool:
|
||||
agent : AIAgent = await AgentManager().get(target_id)
|
||||
agent : AIAgent = await AgentManager.get_instance().get(target_id)
|
||||
if agent is not None:
|
||||
return True
|
||||
else:
|
||||
return False
|
||||
|
||||
async def initial(self) -> bool:
|
||||
AgentManager().initial(os.path.abspath(directory + "/../../../rootfs/"))
|
||||
WorkflowManager().initial(os.path.abspath(directory + "/../../../rootfs/workflows/"))
|
||||
open_ai_node = OpenAI_ComputeNode()
|
||||
open_ai_node.start()
|
||||
ComputeKernel().add_compute_node(open_ai_node)
|
||||
cal_env = CalenderEnvironment("calender")
|
||||
cal_env.start()
|
||||
Environment.set_env_by_id("calender",cal_env)
|
||||
|
||||
AgentManager.get_instance().initial()
|
||||
WorkflowManager.get_instance().initial()
|
||||
|
||||
open_ai_node = OpenAI_ComputeNode.get_instance()
|
||||
if await open_ai_node.initial() is not True:
|
||||
logger.error("openai node initial failed!")
|
||||
return False
|
||||
|
||||
ComputeKernel.get_instance().add_compute_node(open_ai_node)
|
||||
|
||||
llama_ai_node = LocalLlama_ComputeNode()
|
||||
llama_ai_node.start()
|
||||
ComputeKernel().add_compute_node(llama_ai_node)
|
||||
|
||||
AIBus().get_default_bus().register_unhandle_message_handler(self._handle_no_target_msg)
|
||||
AIBus().get_default_bus().register_message_handler(self.username,self._user_process_msg)
|
||||
|
||||
TelegramTunnel.register_to_loader()
|
||||
EmailTunnel.register_to_loader()
|
||||
|
||||
user_data_dir = AIStorage.get_instance().get_myai_dir()
|
||||
tunnels_config_path = os.path.abspath(f"{user_data_dir}/tunnels.cfg.toml")
|
||||
tunnel_config = None
|
||||
try:
|
||||
tunnel_config = toml.load(tunnels_config_path)
|
||||
if tunnel_config is not None:
|
||||
await AgentTunnel.load_all_tunnels_from_config(tunnel_config["tunnels"])
|
||||
except Exception as e:
|
||||
logger.warning(f"load tunnels config from {tunnels_config_path} failed!")
|
||||
|
||||
|
||||
|
||||
return True
|
||||
|
||||
|
||||
def get_version(self) -> str:
|
||||
return "0.0.1"
|
||||
return "0.5.1"
|
||||
|
||||
async def send_msg(self,msg:str,target_id:str,topic:str,sender:str = None) -> str:
|
||||
agent_msg = AgentMsg()
|
||||
agent_msg.set(sender,target_id,msg)
|
||||
agent_msg.topic = topic
|
||||
resp = await AIBus().get_default_bus().send_message(target_id,agent_msg)
|
||||
resp = await AIBus.get_default_bus().send_message(agent_msg)
|
||||
if resp is not None:
|
||||
return resp.body
|
||||
else:
|
||||
return "error!"
|
||||
|
||||
async def install_workflow(self,workflow_id:Workflow) -> None:
|
||||
async def _user_process_msg(self,msg:AgentMsg) -> AgentMsg:
|
||||
pass
|
||||
|
||||
async def call_func(self,func_name, args):
|
||||
@@ -108,9 +151,15 @@ class AIOS_Shell:
|
||||
self.current_topic = topic
|
||||
show_text = FormattedText([("class:title", f"current session switch to {topic}@{target_id}")])
|
||||
return show_text
|
||||
case 'login':
|
||||
if len(args) >= 1:
|
||||
self.username = args[0]
|
||||
AIBus().get_default_bus().register_message_handler(self.username,self._user_process_msg)
|
||||
return self.username + " login success!"
|
||||
case 'history':
|
||||
num = 10
|
||||
offset = 0
|
||||
if args is not None:
|
||||
if len(args) >= 1:
|
||||
num = args[0]
|
||||
if len(args) >= 2:
|
||||
@@ -118,14 +167,14 @@ class AIOS_Shell:
|
||||
|
||||
db_path = ""
|
||||
if await self.is_agent(self.current_target):
|
||||
db_path = AgentManager().db_path
|
||||
db_path = AgentManager.get_instance().db_path
|
||||
else:
|
||||
db_path = WorkflowManager().db_file
|
||||
db_path = WorkflowManager.get_instance().db_file
|
||||
chatsession:AIChatSession = AIChatSession.get_session(self.current_target,f"{self.username}#{self.current_topic}",db_path,False)
|
||||
if chatsession is not None:
|
||||
msgs = chatsession.read_history(num,offset)
|
||||
format_texts = []
|
||||
for msg in reversed(msgs):
|
||||
for msg in msgs:
|
||||
format_texts.append(("class:content",f"{msg.sender} >>> {msg.body}"))
|
||||
format_texts.append(("",f"\n-------------------\n"))
|
||||
return FormattedText(format_texts)
|
||||
@@ -136,46 +185,120 @@ class AIOS_Shell:
|
||||
return FormattedText([("class:title", f"help~~~")])
|
||||
|
||||
|
||||
#######################################################################################
|
||||
history = FileHistory('history.txt')
|
||||
##########################################################################################################################
|
||||
history = FileHistory('aios_shell_history.txt')
|
||||
session = PromptSession(history=history)
|
||||
|
||||
def parse_function_call(s):
|
||||
match = re.match(r'(\w+)\((.*)\)$', s)
|
||||
if match:
|
||||
func_name = match.group(1)
|
||||
args_str = match.group(2)
|
||||
|
||||
args = []
|
||||
buffer = ''
|
||||
quote_count = 0 # Count of single or double quotes
|
||||
for char in args_str:
|
||||
|
||||
if char in ['"', "'"]:
|
||||
quote_count += 1
|
||||
if char == ',' and quote_count % 2 == 0: # ',' is outside of quotes
|
||||
args.append(buffer.strip())
|
||||
buffer = ''
|
||||
else:
|
||||
buffer += char
|
||||
if buffer:
|
||||
args.append(buffer.strip())
|
||||
|
||||
return func_name, args
|
||||
else:
|
||||
def parse_function_call(func_string):
|
||||
match = re.search(r'\s*(\w+)\s*\(\s*(.*)\s*\)\s*', func_string)
|
||||
if not match:
|
||||
return None
|
||||
|
||||
func_name = match.group(1)
|
||||
params_string = match.group(2).strip()
|
||||
params = re.split(r'\s*,\s*(?=(?:[^"]*"[^"]*")*[^"]*$)', params_string)
|
||||
params = [param.strip('"') for param in params]
|
||||
if len(params[0]) == 0:
|
||||
params = None
|
||||
|
||||
return func_name, params
|
||||
|
||||
async def get_user_config_from_input(check_result:dict) -> bool:
|
||||
for key,item in check_result.items():
|
||||
user_input = await session.prompt_async(f"{key} ({item.desc}) not define! \nPlease input:",style=shell_style)
|
||||
if len(user_input) > 0:
|
||||
AIStorage.get_instance().get_user_config().set_user_config(key,user_input)
|
||||
|
||||
await AIStorage.get_instance().get_user_config().save_value_to_user_config()
|
||||
return True
|
||||
|
||||
async def main_daemon_loop(shell:AIOS_Shell):
|
||||
while shell.is_working:
|
||||
await asyncio.sleep(1)
|
||||
|
||||
return 0
|
||||
|
||||
def print_welcome_screen():
|
||||
print("\033[1;31m")
|
||||
logo = """
|
||||
\t_______ ____________________ __
|
||||
\t__ __ \______________________ __ \__ |__ | / /
|
||||
\t_ / / /__ __ \ _ \_ __ \_ / / /_ /| |_ |/ /
|
||||
\t/ /_/ /__ /_/ / __/ / / / /_/ /_ ___ | /| /
|
||||
\t\____/ _ .___/\___//_/ /_//_____/ /_/ |_/_/ |_/
|
||||
\t /_/
|
||||
|
||||
"""
|
||||
print(logo)
|
||||
print("\033[0m")
|
||||
|
||||
print("\033[1;32m \t\tWelcome to OpenDAN - Your Personal AI OS\033[0m\n")
|
||||
introduce = """
|
||||
\tThe core goal of version 0.5.1 is to turn the concept of AIOS into code and get it up and running as quickly as possible.
|
||||
\tAfter three weeks of development, our plans have undergone some changes based on the actual progress of the system.
|
||||
\tUnder the guidance of this goal, some components do not need to be fully implemented. Furthermore,
|
||||
\tbased on the actual development experience from several demo Intelligent Applications,
|
||||
\twe intend to strengthen some components. This document will explain these changes and provide an update
|
||||
\ton the current development progress of MVP(0.5.1,0.5.2)
|
||||
|
||||
"""
|
||||
print(introduce)
|
||||
|
||||
print(f"\033[1;34m \t\tVersion: {AIOS_Version}\n\033")
|
||||
print("\033[1;33m \tOpenDAN is an open-source project, let's define the future of Humans and AI together.\033[0m")
|
||||
print("\033[1;33m \tGithub\t: https://github.com/fiatrete/OpenDAN-Personal-AI-OS\033[0m")
|
||||
print("\033[1;33m \tWebsite\t: https://www.opendan.ai\033[0m")
|
||||
print("\n\n")
|
||||
|
||||
|
||||
async def main():
|
||||
print("aios shell prepareing...")
|
||||
logging.basicConfig(filename="aios_shell.log",filemode="w",level=logging.INFO,format='[%(asctime)s]%(name)s[%(levelname)s]: %(message)s')
|
||||
shell = AIOS_Shell("user")
|
||||
await shell.initial()
|
||||
print(f"aios shell {shell.get_version()} ready.")
|
||||
print_welcome_screen()
|
||||
print("OpenDAN is booting...")
|
||||
logging.basicConfig(filename="aios_shell.log",filemode="w",encoding='utf-8',force=True,
|
||||
level=logging.INFO,
|
||||
format='[%(asctime)s]%(name)s[%(levelname)s]: %(message)s')
|
||||
|
||||
if os.path.isdir(f"{directory}/../../../rootfs"):
|
||||
AIStorage.get_instance().is_dev_mode = True
|
||||
else:
|
||||
AIStorage.get_instance().is_dev_mode = False
|
||||
|
||||
is_daemon = False
|
||||
if os.name != 'nt':
|
||||
if os.getppid() == 1:
|
||||
is_daemon = True
|
||||
|
||||
shell = AIOS_Shell("user")
|
||||
shell.declare_all_user_config()
|
||||
await AIStorage.get_instance().initial()
|
||||
check_result = AIStorage.get_instance().get_user_config().check_user_config()
|
||||
if check_result is not None:
|
||||
if is_daemon:
|
||||
logger.error(check_result)
|
||||
return 1
|
||||
else:
|
||||
#Remind users to enter necessary configurations.
|
||||
if await get_user_config_from_input(check_result) is False:
|
||||
return 1
|
||||
|
||||
init_result = await shell.initial()
|
||||
if init_result is False:
|
||||
if is_daemon:
|
||||
logger.error("aios shell initial failed!")
|
||||
return 1
|
||||
else:
|
||||
print("aios shell initial failed!")
|
||||
|
||||
print(f"aios shell {shell.get_version()} ready.")
|
||||
if is_daemon:
|
||||
return await main_daemon_loop(shell)
|
||||
|
||||
#TODO: read last input config
|
||||
completer = WordCompleter(['send($target,$msg,$topic)',
|
||||
'open($target,$topic)',
|
||||
'history($num,$offset)',
|
||||
'login($username)',
|
||||
'connect($target)',
|
||||
'show()',
|
||||
'exit()',
|
||||
'help()'], ignore_case=True)
|
||||
@@ -199,8 +322,6 @@ async def main():
|
||||
print_formatted_text(show_text,style=shell_style)
|
||||
#print_formatted_text(f"{shell.username}<->{shell.current_topic}@{shell.current_target} >>> {resp}",style=shell_style)
|
||||
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
|
||||
|
||||
@@ -0,0 +1,171 @@
|
||||
"""
|
||||
Capture your email locally, and parse out the pictures in the email body and the pictures, videos and other files in the attachment. Subsequently, it supports vectorized analysis of your personal data and serves as a knowledge base to enable large language model answers. Better results.
|
||||
|
||||
An example of a local file is as follows:
|
||||
├── data
|
||||
│ └── alex0072@gmail.com
|
||||
│ └── 5de3e52f3a6b90cabe6cbdd4ae3a5c5b
|
||||
│ ├── email.txt
|
||||
│ ├── meta.json
|
||||
│ ├── image
|
||||
│ │ ├── 0648B869@99C03070.DB94B354.jpg
|
||||
│ └── body_image
|
||||
│ ├── 11044884873.jpg
|
||||
│ ├── 282985198265470.gif
|
||||
│ └── dd-login-service-min.png
|
||||
|
||||
"""
|
||||
|
||||
import imaplib
|
||||
import os
|
||||
import toml
|
||||
import logging
|
||||
import mailparser
|
||||
import hashlib
|
||||
import json
|
||||
import base64
|
||||
from bs4 import BeautifulSoup
|
||||
import requests
|
||||
|
||||
class EmailSpider:
|
||||
def __init__(self):
|
||||
# logger config
|
||||
self.logger = logging.getLogger('email spider')
|
||||
self.logger.setLevel(logging.DEBUG)
|
||||
ch = logging.StreamHandler()
|
||||
formatter = logging.Formatter('%(asctime)s [%(name)s] [%(levelname)s] %(message)s')
|
||||
ch.setFormatter(formatter)
|
||||
self.logger.addHandler(ch)
|
||||
|
||||
# read config from toml file
|
||||
# and read from config config.local.toml if exists (config.local.toml is ignored by git)
|
||||
self.config = toml.load('./rootfs/email/config.toml')
|
||||
if os.path.exists('./rootfs/email/config.local.toml'):
|
||||
self.config = toml.load('./rootfs/email/config.local.toml')
|
||||
|
||||
self.client = self.email_client()
|
||||
|
||||
def email_client(self) -> imaplib.IMAP4_SSL:
|
||||
self.logger.info(f"read email config from {self.config.get('EMAIL_IMAP_SERVER')}")
|
||||
client = imaplib.IMAP4_SSL(
|
||||
host=self.config.get('EMAIL_IMAP_SERVER'),
|
||||
port=self.config.get('EMAIL_IMAP_PORT')
|
||||
)
|
||||
client.login(self.config.get('EMAIL_ADDRESS'), self.config.get('EMAIL_PASSWORD'))
|
||||
return client
|
||||
|
||||
def list_box(self):
|
||||
_, mailbox_list = self.client.list()
|
||||
for mailbox in mailbox_list:
|
||||
print(mailbox.decode())
|
||||
|
||||
def read_emails(self, folder: str = 'INBOX', imap_keyword: str = "UNSEEN"):
|
||||
self.client.select(folder)
|
||||
_, data = self.client.uid('search', None, imap_keyword)
|
||||
|
||||
# get email uid list
|
||||
email_list = data[0].split()
|
||||
self.logger.info(f"got {len(email_list)} emails")
|
||||
email_list.reverse()
|
||||
for uid in email_list:
|
||||
if self.check_email_saved(uid):
|
||||
self.logger.info(f"email uid {uid} already saved")
|
||||
else:
|
||||
self.read_and_save_email(uid)
|
||||
self.logger.info(f"email uid {uid} saved")
|
||||
|
||||
def read_and_save_email(self, uid: str):
|
||||
message_parts = "(BODY.PEEK[])"
|
||||
_, email_data = self.client.uid('fetch', uid, message_parts)
|
||||
mail = mailparser.parse_from_bytes(email_data[0][1])
|
||||
self.logger.info(f"got email subject [{mail.subject}]")
|
||||
self.save_email(mail)
|
||||
|
||||
def get_local_dir_name(self, mail: mailparser.MailParser) -> str:
|
||||
dir = f"{self.config.get('LOCAL_DIR')}/{self.config.get('EMAIL_ADDRESS')}"
|
||||
name = f"{mail.subject}__{mail.date}"
|
||||
name = hashlib.md5(name.encode('utf-8')).hexdigest()
|
||||
return f"{dir}/{name}"
|
||||
|
||||
def check_email_saved(self, uid: str):
|
||||
message_parts = "(BODY[HEADER])"
|
||||
_, email_data = self.client.uid('fetch', uid, message_parts)
|
||||
mail = mailparser.parse_from_bytes(email_data[0][1])
|
||||
self.logger.info(f"[{uid}]check email subject [{mail.subject}]")
|
||||
dir = self.get_local_dir_name(mail)
|
||||
self.logger.info(f"check email saved {dir}")
|
||||
file = f"{dir}/email.txt"
|
||||
if os.path.exists(file):
|
||||
return False
|
||||
return False
|
||||
|
||||
# save email attachment(images)
|
||||
def save_email_attachment(self, mail: mailparser.MailParser, email_dir: str):
|
||||
for attachment in mail.attachments:
|
||||
if attachment['mail_content_type'] in ['image/png', 'image/jpeg', 'image/gif']:
|
||||
print('current mail have image attachment')
|
||||
img_dir = f"{email_dir}/image"
|
||||
if not os.path.exists(img_dir):
|
||||
os.makedirs(img_dir)
|
||||
filename = attachment['filename']
|
||||
filefullname = f"{img_dir}/{filename}"
|
||||
image_data = attachment['payload']
|
||||
try:
|
||||
image_data = base64.b64decode(image_data)
|
||||
except base64.binascii.Error:
|
||||
image_data = image_data.encode()
|
||||
with open(filefullname, 'wb') as f:
|
||||
f.write(image_data)
|
||||
self.logger.info(f"save email image {filename} success")
|
||||
|
||||
# save email body images(html content)
|
||||
def save_body_images(self, html_content: str, email_dir: str):
|
||||
# get all image urls
|
||||
soup = BeautifulSoup(html_content, 'html.parser')
|
||||
img_tags = soup.find_all('img')
|
||||
img_urls = [img['src'] for img in img_tags if 'src' in img.attrs]
|
||||
self.logger.info(f'Found {len(img_urls)} images in email body')
|
||||
|
||||
if not os.path.exists(email_dir):
|
||||
os.makedirs(email_dir)
|
||||
|
||||
for img_url in img_urls:
|
||||
# keep the original image filename(last of url)
|
||||
img_filename = os.path.join(email_dir, img_url.split('/')[-1])
|
||||
# download image
|
||||
response = requests.get(img_url, stream=True)
|
||||
if response.status_code == 200:
|
||||
with open(img_filename, 'wb') as img_file:
|
||||
for chunk in response.iter_content(1024):
|
||||
img_file.write(chunk)
|
||||
self.logger.info(f'Downloaded {img_url} to {img_filename}')
|
||||
else:
|
||||
self.logger.info(f'Failed to download {img_url}')
|
||||
|
||||
# save email content to local dir
|
||||
def save_email(self, mail: mailparser.MailParser):
|
||||
dir = f"{self.config.get('LOCAL_DIR')}/{self.config.get('EMAIL_ADDRESS')}"
|
||||
if not os.path.exists(dir):
|
||||
os.makedirs(dir)
|
||||
email_dir = self.get_local_dir_name(mail)
|
||||
self.logger.info(f"save email to {email_dir}")
|
||||
if not os.path.exists(email_dir):
|
||||
os.makedirs(email_dir)
|
||||
with open(f"{email_dir}/email.txt", "w") as f:
|
||||
f.write(mail.body)
|
||||
with open(f"{email_dir}/meta.json", "w", encoding='utf-8') as f:
|
||||
mail_dict = json.loads(mail.mail_json)
|
||||
if 'body' in mail_dict:
|
||||
del mail_dict['body']
|
||||
json.dump(mail_dict, f, ensure_ascii=False, indent=4)
|
||||
self.logger.info(f"save email meta info {f.name}")
|
||||
|
||||
self.save_email_attachment(mail, email_dir)
|
||||
self.save_body_images(mail.body, f"{email_dir}/body_image")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
spider = EmailSpider()
|
||||
folder = 'INBOX'
|
||||
imap_keyword = "ALL"
|
||||
spider.read_emails(folder, imap_keyword)
|
||||
@@ -0,0 +1,17 @@
|
||||
import daemon
|
||||
from time import sleep
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
logging.basicConfig(filename="daemon_test.log",filemode="w",encoding='utf-8',force=True,
|
||||
level=logging.INFO,
|
||||
format='[%(asctime)s]%(name)s[%(levelname)s]: %(message)s')
|
||||
|
||||
def main_program():
|
||||
while True:
|
||||
logger.info("hello world")
|
||||
sleep(1)
|
||||
|
||||
with daemon.DaemonContext():
|
||||
main_program()
|
||||
@@ -0,0 +1,32 @@
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
|
||||
directory = os.path.dirname(__file__)
|
||||
sys.path.append(directory + '/../src')
|
||||
|
||||
from aios_kernel import CalenderEnvironment,WorkflowEnvironment
|
||||
|
||||
|
||||
async def test_buildin_envs():
|
||||
c_env = CalenderEnvironment("calender")
|
||||
c_env.start()
|
||||
print(c_env.get_value("now"))
|
||||
async def show_event(eventid,event):
|
||||
print(event.data)
|
||||
c_env.attach_event_handler("timer",show_event)
|
||||
|
||||
w_env = WorkflowEnvironment("workflow",os.path.abspath(directory + "/../rootfs/workflow_env.db"))
|
||||
w_env.set_value("test","test_aaaa")
|
||||
print(w_env.get_value("test"))
|
||||
|
||||
await asyncio.sleep(10)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
#test_rstr = "abc is {abc}"
|
||||
#values = {"abc":"123"}
|
||||
#new_str = test_rstr.format_map(values)
|
||||
#print(new_str)
|
||||
|
||||
asyncio.run(test_buildin_envs())
|
||||
+80
-12
@@ -1,15 +1,83 @@
|
||||
# workflow实例场景
|
||||
# 能展示sub workflow
|
||||
# 能展示env的整合
|
||||
# 能展示filter的使用
|
||||
# 能展示function的调用
|
||||
# 能展示基于工作目标/KPI的sub workflow迭代流程
|
||||
# 多人场景安排
|
||||
|
||||
[authors]
|
||||
"buckyos.org" = {public="aaaa",name="BuckyOS Core Dev Team"}
|
||||
[authors."cyfs.com"]
|
||||
public="bbbb"
|
||||
name="CYFS Core Dev Team"
|
||||
# 例子：举办一个团队活动
|
||||
# 方案讨论（通过交互引导的方式收集主人的需求）与确定
|
||||
|
||||
[env]
|
||||
is_strict = false
|
||||
prefixs = ["default","../m/s","abc"]
|
||||
obj2 = {k1="bb",k2=300}
|
||||
[env.obj]
|
||||
k1="aa"
|
||||
k2=200
|
||||
# 活动前：
|
||||
# 通讯员，对接管理参加活动的人的情况 (email spider)
|
||||
# 酒店预订 简单：搜索（酒店评价） 处理异常
|
||||
# 行程（票务）预订 ：搜索，处理异常
|
||||
# 餐饮预订：给出方案，确定细节，预订
|
||||
|
||||
# 活动中：
|
||||
# 进行统计和分析，调整设备
|
||||
# 安保，空调，音乐，拍照，录像
|
||||
# 响应紧急情况
|
||||
|
||||
# 活动结束后：
|
||||
# 整理照片，视频，进行必要的二次创作，发送给相关人员
|
||||
# 对活动进行总结，提出改件意见（指导下一次活动）
|
||||
|
||||
# 1. 人员
|
||||
# 主管，负责和客户沟通，并对每个环境的结果进行总结
|
||||
# 嘉宾对接
|
||||
# 酒店组
|
||||
# 行程组
|
||||
# 财务组
|
||||
# 多媒体组
|
||||
|
||||
|
||||
[filter]
|
||||
"*" = "manager"
|
||||
|
||||
[roles.manager]
|
||||
fullname = "经理"
|
||||
agent="manager"
|
||||
[[roles.manager.prompt]]
|
||||
role="system"
|
||||
content="""你是一个活动策划公司的经理,与客户对接并向团队下达指令。你的团队分为下面几个小组:嘉宾对接组,酒店预定组,行程预订组,财务组,活动摄像组。活动策划分为四个阶段：方案讨论，活动前，活动中，活动后。你会根据客户的需求，对团队进行分工，分别完成各个阶段的工作。你的基本工作模式是：\
|
||||
1. 收到客户的明确的指令后,基于客户的已有信息和客户商量活动方案，和活动策划公司无关的业务你会回答‘与我无关’。当和客户完成活动方案的确认后,你会将拆解后的任务分配给各个小组 \
|
||||
2. 根据目前已经确认的活动方案，你要根据时间适时的检查不同小组的工作情况。当收到小组的工作情况反馈后，你会站在全局的角度判断是否需要调整活动方案，如果需要调整，你会和客户商量重新确定方案，然后再将调整后的方案分配给各个小组。\
|
||||
3. 有时工作小组会主动与你沟通，反馈一些问题。你会站在全局的角度给与指导，适当的调整工作小组的工作目标。如果反馈的问题需要你和客户沟通，你会和客户沟通后重新确定方案。再将调整后的方案分配给受到影响各个小组。\
|
||||
4. 当你决定要和工作小组通信时，请使用`send_message({小组名称},{内容}`）的形式。"""}]
|
||||
|
||||
|
||||
|
||||
|
||||
[sub_workflows]
|
||||
[sub_workflows."嘉宾对接组"]
|
||||
# 展现读取email和发送email与嘉宾沟通的能力
|
||||
[sub_workflows."嘉宾对接组".environments.email]
|
||||
new_mail = "收到来自{event.data.from},标题为{event.data.subject}的邮件，内容为{event.data.content}的电子邮件" # 这里将new_mail事件转换为了一个来自环境的message
|
||||
|
||||
[sub_workflows."嘉宾对接组".roles.leader]
|
||||
name = "嘉宾对接组组长"
|
||||
[[sub_workflows."嘉宾对接组".roles.leader.prompt]]
|
||||
role="system"
|
||||
content="""你是一家活动策划公司的嘉宾对接组的组长，你的工作是基于已知信息,当前活动信息、公司经理的指令与嘉宾沟通，收集嘉宾的信息，然后将信息反馈给经理。在你看来，参加活动的多少有成员都是嘉宾，你可以通过你知道的信息给不同的成员进行分级。你的基本工作模式是：\
|
||||
1. 处理收到的邮件，如果邮件来自嘉宾，你会尝试从邮件的表态和内容中分享嘉宾的需要，并结合你对当前活动方案的理解判断是否需要和经理沟通，如果需要和经理沟通，你会将嘉宾的需求总结和告诉经理。不需要沟通的事项可以直接回复嘉宾。\
|
||||
2. 你总是通过`call_function(get_env,'parent.topic'`的形式查询当前的活动方案。等待函数返回后，你会根据函数的返回结果继续处理上一个对话。\
|
||||
3. 当你决定要和经理通信时，请使用`send_message(manager,{内容}`）的形式，内容的长度不超过200字。\
|
||||
4. 当你决定要回复嘉宾时，请使用`call_function(sendmail,{嘉宾邮件地址},{标题},{内容}）的形式，内容的长度不超过500字。"""
|
||||
# 这里是孤立工作模式，组长只和经理沟通，也可以赋予其和其它组沟通的能力
|
||||
|
||||
[sub_workflows."酒店预定组"]
|
||||
# 展现使用搜索引擎，并调用预订酒店的能力
|
||||
[sub_workflows."酒店预定组".environments.email]
|
||||
|
||||
[sub_workflows."酒店预定组".roles.leader]
|
||||
name="酒店预定组组长"
|
||||
prompt = [{role="xxx",content="yyy"}]
|
||||
|
||||
[sub_workflows."酒店预定组".roles.research]
|
||||
name="酒店搜索专家"
|
||||
|
||||
[sub_workflows."行程预订组"]
|
||||
# 展现处理冲突并反推
|
||||
nam3="3"
|
||||
|
||||
Reference in New Issue
Block a user