Merge pull request #74 from fiatrete/MVP
MVP 0.5.1 Meets Preset Version Goals
This commit is contained in:
fiatrete
GitHub
@@ -0,0 +1,27 @@
|
||||
**/__pycache__
|
||||
**/*venv
|
||||
**/.classpath
|
||||
**/.dockerignore
|
||||
**/.env
|
||||
**/.git
|
||||
**/.gitignore
|
||||
**/.project
|
||||
**/.settings
|
||||
**/.toolstarget
|
||||
**/.vs
|
||||
**/.vscode
|
||||
**/*.*proj.user
|
||||
**/*.dbmdl
|
||||
**/*.jfm
|
||||
**/bin
|
||||
**/charts
|
||||
**/docker-compose*
|
||||
**/compose*
|
||||
**/Dockerfile*
|
||||
**/node_modules
|
||||
**/npm-debug.log
|
||||
**/obj
|
||||
**/secrets.dev.yaml
|
||||
**/values.dev.yaml
|
||||
**/.db
|
||||
**/.python-version
|
||||
+13
@@ -0,0 +1,13 @@
|
||||
.vscode/
|
||||
*.pyc
|
||||
*.log
|
||||
rootfs/email/config.local.toml
|
||||
rootfs/data
|
||||
venv
|
||||
aios_shell.log
|
||||
history.txt
|
||||
aios_shell_history.txt
|
||||
math_school_env.db
|
||||
workflows.db
|
||||
|
||||
|
||||
+15
@@ -0,0 +1,15 @@
|
||||
FROM python:3.11
|
||||
WORKDIR /opt/aios
|
||||
COPY ./src /opt/aios
|
||||
COPY ./rootfs /opt/aios/app
|
||||
|
||||
RUN mkdir -p /root/myai/app
|
||||
RUN mkdir -p /root/myai/data
|
||||
RUN mkdir -p /root/myai/etc
|
||||
|
||||
RUN pip install --no-cache-dir -r /opt/aios/requirements.txt
|
||||
|
||||
ENV PYTHONDONTWRITEBYTECODE 1
|
||||
ENV PYTHONUNBUFFERED 1
|
||||
|
||||
CMD ["python3","./service/aios_shell/aios_shell.py"]
|
||||
@@ -0,0 +1,90 @@
|
||||
# **OpenDAN: Personal AI OS**
|
||||
[](https://opendan.ai)
|
||||
[](https://github.com/fiatrete/OpenDAN-Personal-AI-OS/stargazers)
|
||||
[](https://twitter.com/openDAN_AI)
|
||||
|
||||
OpenDAN is an open source Personal AI OS , which consolidates various AI modules in one place for your personal use.
|
||||
|
||||
## **Project Introduction**
|
||||
|
||||
The goal of OpenDAN (Open and Do Anything Now with AI) is to create a Personal AI OS , which provides a runtime environment for various Al modules as well as protocols for interoperability between them. With OpenDAN, users can securely collaborate with various AI modules using their private data to create powerful personal AI agents, such as butler, lawyer, doctor, teacher, assistant, girl or boy friends.
|
||||
|
||||
This project is still in its very early stages, and there may be significant changes in the future.
|
||||
|
||||
## **Updates**
|
||||
|
||||
### 1. Adding Knowledge Base Infrastructure
|
||||
We're currently working on implementing the infrastructure for the Knowledge Base, aiming to enhance AI agent's access to Personal Data stored on Personal Servers. Leveraging these new features, we want to empower everyone to build their own personal homepage in the AI era using OpenDAN.
|
||||
|
||||
### 2. Development Based on SourceDAO Contract
|
||||
We are in the process of developing the OpenDAN's DAO page, which is based on the SourceDAO contract. The future of OpenDAN should be determined by the community.
|
||||
|
||||
### 3. Welcoming waterflier as a Core Contributor
|
||||
We're excited to officially invite waterflier to join as a core contributor to our community. We recognize and appreciate his innovative ideas and deep insights in the realm of Personal AI OS. Waterflier will be spearheading the development of the MVP version. As a key contributor to the CYFS CoreDev Team, he brings extensive experience in the domain of Personal Servers.
|
||||
|
||||
## **Intro video - What is OpenDAN?**
|
||||
Click the image below for a demo:
|
||||
|
||||
[](https://www.youtube.com/watch?v=l2QmsIOXhdQ "Intro Video")
|
||||
|
||||
## **Demo video - What can OpenDAN do?**
|
||||
Click the image below for a demo:
|
||||
|
||||
[](https://youtu.be/13wdyoT0VHQ "Demo Video")
|
||||
|
||||
<center class="half">
|
||||
<img src="https://github.com/fiatrete/OpenDAN-Personal-AI-OS/assets/126534313/a0251e66-67e1-40e8-a05e-54ff8b4cafa2" width="400"/><img src="https://github.com/fiatrete/OpenDAN-Personal-AI-OS/assets/126534313/f36a8d92-c3e5-423f-906c-7bb692bae4d6" width="400"/>
|
||||
<img src="https://github.com/fiatrete/OpenDAN-Personal-AI-OS/assets/126534313/b26019c8-27da-49a8-ba8e-8dbae674a4a4" width="400"/><img src="https://github.com/fiatrete/OpenDAN-Personal-AI-OS/assets/126534313/23e2a296-de67-451b-8f4a-422c9844dd23" width="400"/>
|
||||
<img src="https://github.com/fiatrete/OpenDAN-Personal-AI-OS/assets/126534313/5661259a-d69a-4923-a864-b05521aef26b" width="400"/><img src="https://github.com/fiatrete/OpenDAN-Personal-AI-OS/assets/126534313/c0ecf73e-6db1-4c06-b948-f5bf75ba440f" width="400"/>
|
||||
</center>
|
||||
|
||||
|
||||
## **Subscribe to updates here**
|
||||
|
||||
https://twitter.com/openDAN_AI
|
||||
|
||||
## **Core Features of OpenDAN**
|
||||
|
||||
To achieve the goal of OpenDAN, we provide the following key features:
|
||||
|
||||
1. **Hardware-specific optimization**: Optimize for specific hardware to enable smooth local running of most open-source AI applications.
|
||||
2. **Open AI App Marketplace**: Offer a solution for one-click installation and use of various AI applications, helping users easily access and manage AI apps.
|
||||
3. **Open AI Model Solution**: Provide a unified entry point for model search, download, and access control, making it convenient for users to find and use models suitable for their needs.
|
||||
4. **Strict Privacy Protection and Management**: Strictly manage personal data, ranging from family albums to chat records and social media records, and provide a unified access control interface for AI applications.
|
||||
5. **Integrated Tools**: Offer tools for users to train their own voice models, Lora models, knowledge models, etc., using personal data.
|
||||
6. **AI Butler Assistant**: Driven by a large language model, the AI assistant completes tasks through natural language interaction.
|
||||
7. **Development Framework**: Provide a development framework for customizing AI assistants for specific purposes, making it easy for developers to create unique AI applications for users.
|
||||
|
||||
## **Roadmap**
|
||||
|
||||
- [x] Project Initialization
|
||||
- [x] Basic code for operating system image packaging script
|
||||
- [x] Opendan.ai website homepage
|
||||
- [x] OpenDAN Prototype Version
|
||||
- [x] AI butler assistant driven by GPT-3.5 or GPT-4.0
|
||||
- [x] Integration of Stable Diffusion
|
||||
- [x] Integration of TTS (Text-to-Speech)
|
||||
- [x] Integration of Telegram Chatbot as an interaction entrance
|
||||
- [ ] OpenDAN 1.0
|
||||
- [ ] AI butler assistant's large language model core switched to locally running open-source model
|
||||
- [ ] Offer more AI applications through AI App Marketplace
|
||||
- [ ] Provide an AI application development framework to support the community in integrating more AI applications
|
||||
- [ ] Provide a model management framework
|
||||
|
||||
## **Contributing**
|
||||
|
||||
We welcome community members to contribute to the project, including but not limited to submitting issues, improving documentation, fixing bugs, or providing new features. You can participate in the contribution through the following ways:
|
||||
|
||||
- Submit an Issue in the GitHub repository
|
||||
- Submit a Pull Request to the repository
|
||||
- Participate in discussions and development
|
||||
|
||||
## **⭐Star History**
|
||||
|
||||
|
||||
[](https://star-history.com/#fiatrete/OpenDAN-Personal-AI-Server-OS&Date)
|
||||
|
||||
|
||||
## **License**
|
||||
|
||||
MIT
|
||||
+1
-1
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 3.7 MiB After Width: | Height: | Size: 3.7 MiB |
|
Before Width: | Height: | Size: 92 KiB After Width: | Height: | Size: 92 KiB |
|
Before Width: | Height: | Size: 440 KiB After Width: | Height: | Size: 440 KiB |
@@ -1,4 +1,5 @@
|
||||
# **OpenDAN: Personal AI OS**
|
||||
# **OpenDAN : Your Personal AIOS**
|
||||
|
||||
[](https://opendan.ai)
|
||||
[](https://github.com/fiatrete/OpenDAN-Personal-AI-OS/stargazers)
|
||||
[](https://twitter.com/openDAN_AI)
|
||||
@@ -7,69 +8,207 @@ OpenDAN is an open source Personal AI OS , which consolidates various AI modules
|
||||
|
||||
## **Project Introduction**
|
||||
|
||||
The goal of OpenDAN (Open and Do Anything Now with AI) is to create a Personal AI OS , which provides a runtime environment for various Al modules as well as protocols for interoperability between them. With OpenDAN, users can securely collaborate with various AI modules using their private data to create powerful personal AI agents, such as butler, lawyer, doctor, teacher, assistant, girl or boy friends.
|
||||
OpenDAN (Open and Do Anything Now with AI) is revolutionizing the AI landscape with its Personal AI Operating System. Designed for seamless integration of diverse AI modules, it ensures unmatched interoperability. OpenDAN empowers users to craft powerful AI agents—from butlers and assistants to personal tutors and digital companions—all while retaining control. These agents can team up to tackle complex challenges, integrate with existing services, and command smart(IoT) devices.
|
||||
|
||||
With OpenDAN, we're putting AI in your hands, making life simpler and smarter.
|
||||
|
||||
This project is still in its very early stages, and there may be significant changes in the future.
|
||||
|
||||
## **Updates**
|
||||
|
||||
### 1. Adding Knowledge Base Infrastructure
|
||||
We're currently working on implementing the infrastructure for the Knowledge Base, aiming to enhance AI agent's access to Personal Data stored on Personal Servers. Leveraging these new features, we want to empower everyone to build their own personal homepage in the AI era using OpenDAN.
|
||||
After over three months of development, the code for the first version of OpenDAN MVP (0.5.1), driven by the new contributor `waterflier`, has been merged into the Master branch. This version has realized many concepts proposed in the PoC version of OpenDAN and completed the basic framework of the OS, especially defining the application form on AIOS. Currently, the 0.5.1 version operates in an "all-in-one" mode. For 0.5.2, we will advance the formal implementation of the OpenDAN OS kernel based on the partial framework code of the [CYFS Owner Online Device(OOD) OS](https://github.com/buckyos/CYFS) that has already been completed.
|
||||
|
||||
### 2. Development Based on SourceDAO Contract
|
||||
We are in the process of developing the OpenDAN's DAO page, which is based on the SourceDAO contract. The future of OpenDAN should be determined by the community.
|
||||
|
||||
|
||||
### 3. Welcoming waterflier as a Core Contributor
|
||||
We're excited to officially invite waterflier to join as a core contributor to our community. We recognize and appreciate his innovative ideas and deep insights in the realm of Personal AI OS. Waterflier will be spearheading the development of the MVP version. As a key contributor to the CYFS CoreDev Team, he brings extensive experience in the domain of Personal Servers.
|
||||
**The main new features of OpenDAN 0.5.1 (MVP) :**
|
||||
|
||||
- [x] Rapid installation and deployment of OpenDAN based on Docker, making OpenDAN compatible with a wide range of hardware environments (PC/Mac/RaspberryPI/NAS) through Docker.
|
||||
- [x] AI Agent's large language model can be switched, now supporting locally running the open-source model (LLaMa).
|
||||
- [x] Introduction of more built-in AI Agents:
|
||||
- [x] Personal Assistant Jarvis : Consultant.Assistant who anages your schedule and communication records. ChatGPT alternative.
|
||||
- [x] Information Assistant Mia : Manage your personal data and sort it into a knowledge base
|
||||
- [x] Private English Teacher Tracy : Your private English teacher
|
||||
- [x] ai_bash (for developers) :No longer need to memory complicated command line parameters! Bash is used by "Find FILES in ~/Documents that Contain OpenDAN".
|
||||
- [x] Connectivity to AI Agent/Workflow via Telegram/Email.
|
||||
- [x] Building a local private Knowledge Base based on existing file or email spiders, enabling AI Agent access to personal data.
|
||||
- [x] Supports text files and common image formats.
|
||||
- [ ] Supports other common formats.
|
||||
- [x] Implemented Workflow: Collaboration of Agents to solve more complex issues.
|
||||
- [x] Built-in Workflow story_maker, integrated the AIGC tool to create audio fairy tale books.
|
||||
- [x] Distributed AI computing core available for complex selections.
|
||||
- [x] Manual download and installation of new Agent/Workflow.
|
||||
- [ ] OpenDAN Store : Agent/Workflow/Models One-Stop installation (Delayed to 0.5.2).
|
||||
|
||||
[Try it NOW!](./doc/QuickStart.md)
|
||||
|
||||
Developers [click here](https://github.com/fiatrete/OpenDAN-Personal-AI-OS/issues/46) to learn about OpenDan's system development updates.
|
||||
|
||||
## **Intro video - What is OpenDAN?**
|
||||
|
||||
Click the image below for a demo:
|
||||
|
||||
[](https://www.youtube.com/watch?v=l2QmsIOXhdQ "Intro Video")
|
||||
|
||||
## **Demo video - What can OpenDAN do?**
|
||||
Click the image below for a demo:
|
||||
|
||||
[](https://youtu.be/13wdyoT0VHQ "Demo Video")
|
||||
|
||||
<center class="half">
|
||||
<img src="https://github.com/fiatrete/OpenDAN-Personal-AI-OS/assets/126534313/a0251e66-67e1-40e8-a05e-54ff8b4cafa2" width="400"/><img src="https://github.com/fiatrete/OpenDAN-Personal-AI-OS/assets/126534313/f36a8d92-c3e5-423f-906c-7bb692bae4d6" width="400"/>
|
||||
<img src="https://github.com/fiatrete/OpenDAN-Personal-AI-OS/assets/126534313/b26019c8-27da-49a8-ba8e-8dbae674a4a4" width="400"/><img src="https://github.com/fiatrete/OpenDAN-Personal-AI-OS/assets/126534313/23e2a296-de67-451b-8f4a-422c9844dd23" width="400"/>
|
||||
<img src="https://github.com/fiatrete/OpenDAN-Personal-AI-OS/assets/126534313/5661259a-d69a-4923-a864-b05521aef26b" width="400"/><img src="https://github.com/fiatrete/OpenDAN-Personal-AI-OS/assets/126534313/c0ecf73e-6db1-4c06-b948-f5bf75ba440f" width="400"/>
|
||||
</center>
|
||||
|
||||
|
||||
## **Subscribe to updates here**
|
||||
|
||||
https://twitter.com/openDAN_AI
|
||||
<https://twitter.com/openDAN_AI>
|
||||
|
||||
## **Core Features of OpenDAN**
|
||||
## **Installation**
|
||||
|
||||
To achieve the goal of OpenDAN, we provide the following key features:
|
||||
There are two ways to install the Internal Test Version of OpenDAN:
|
||||
|
||||
1. **Hardware-specific optimization**: Optimize for specific hardware to enable smooth local running of most open-source AI applications.
|
||||
2. **Open AI App Marketplace**: Offer a solution for one-click installation and use of various AI applications, helping users easily access and manage AI apps.
|
||||
3. **Open AI Model Solution**: Provide a unified entry point for model search, download, and access control, making it convenient for users to find and use models suitable for their needs.
|
||||
4. **Strict Privacy Protection and Management**: Strictly manage personal data, ranging from family albums to chat records and social media records, and provide a unified access control interface for AI applications.
|
||||
5. **Integrated Tools**: Offer tools for users to train their own voice models, Lora models, knowledge models, etc., using personal data.
|
||||
6. **AI Butler Assistant**: Driven by a large language model, the AI assistant completes tasks through natural language interaction.
|
||||
7. **Development Framework**: Provide a development framework for customizing AI assistants for specific purposes, making it easy for developers to create unique AI applications for users.
|
||||
1. Installation through docker, this is also the installation method we recommend now
|
||||
2. Installing through the source code, this method may encounter some traditional Pyhont dependence problems and requires you to have a certain ability to solve.But if you want to do secondary development of OpenDAN, this method is necessary.
|
||||
|
||||
## **Roadmap**
|
||||
### Preparation before installation
|
||||
|
||||
- [x] Project Initialization
|
||||
- [x] Basic code for operating system image packaging script
|
||||
- [x] Opendan.ai website homepage
|
||||
- [x] OpenDAN Prototype Version
|
||||
- [x] AI butler assistant driven by GPT-3.5 or GPT-4.0
|
||||
- [x] Integration of Stable Diffusion
|
||||
- [x] Integration of TTS (Text-to-Speech)
|
||||
- [x] Integration of Telegram Chatbot as an interaction entrance
|
||||
- [ ] OpenDAN 1.0
|
||||
- [ ] AI butler assistant's large language model core switched to locally running open-source model
|
||||
- [ ] Offer more AI applications through AI App Marketplace
|
||||
- [ ] Provide an AI application development framework to support the community in integrating more AI applications
|
||||
- [ ] Provide a model management framework
|
||||
1. Docker environment
|
||||
This article does not introduce how to install the docker, execute it under your console
|
||||
|
||||
```
|
||||
docker -version
|
||||
```
|
||||
|
||||
If you can see the docker version number (> 20.0), it means that you have installed Docker.
|
||||
If you don't know how to install docker, you can refer to [here](https://docs.docker.com/engine/install/)
|
||||
|
||||
2. OpenAI API Token
|
||||
If there is no api token, you can apply for [here](https://beta.openai.com/)
|
||||
|
||||
Applying for the API Token may have some thresholds for new players. You can find friends around you, and you can give you a temporary, or join our internal test experience group. We will also release some free experience API token from time to time.These token is limited to the maximum consumption and effective time
|
||||
|
||||
### Install
|
||||
|
||||
After executing the following command, you can install the Docker Image of OpenDAN
|
||||
|
||||
```
|
||||
docker pull paios/aios:latest
|
||||
```
|
||||
|
||||
## **Run OpenDAN**
|
||||
|
||||
The first Run of OpenDAN needs to be initialized. You need to enter some information in the process of initialization. Therefore, when starting the docker, remember to bring the -it parameter.
|
||||
|
||||
OpenDAN is your Personal AIOS, so it will generate some important personal data (such as chat history with agent, schedule data, etc.) during its operation. These data will be stored on your local disk. ThereforeWe recommend that you mount the local disk into the container of Docker so that the data can be guaranteed.
|
||||
|
||||
```
|
||||
docker run -v /your/local/myai/:/root/myai --name aios -it paios/aios:latest
|
||||
```
|
||||
|
||||
In the above command, we also set up a Docker instance for Docker Run named AIOS, which is convenient for subsequent operations.You can also use your favorite name instead.
|
||||
|
||||
After the first operation of the docker instance is created, it only needs to be executed again:
|
||||
|
||||
```
|
||||
docker start -ai aios
|
||||
```
|
||||
|
||||
If you plan to run in a service mode (NO UI), you don't need to bring the -AI parameter:
|
||||
|
||||
```
|
||||
docker start aios
|
||||
```
|
||||
|
||||
### Hello, Jarvis
|
||||
|
||||
After the configuration is completed, you will enter a AIOS Shell, which is similar to Linux Bash and similar. The meaning of this interface is:
|
||||
The current user "username" is communicating with the name "Agent/Workflow of Jarvis". The current topic is default.
|
||||
|
||||
Say Hello to your private AI assistant Jarvis !
|
||||
|
||||
**If everything is OK, you will get a reply from Jarvis after a moment .At this time, the OpenDAN system is running .**
|
||||
|
||||
|
||||
## **Core Concepts and Features of OpenDAN**
|
||||
1. **AI Agent**: Driven by a large language model, having own memory.The AI Agent completes tasks through natural language interaction.
|
||||
2. **AI Workflow**: Organize different AI Agents into an AI Agent Group to complete complex tasks.
|
||||
|
||||
3. **AI Envriment**: Supports AI Agents to access file systems, IoT devices, network services, smart contracts, and everything on today's internet once authorized.
|
||||
4. **AI Marketplace**: Offer a solution for one-click installation and use of various AI applications, helping users easily access and manage AI apps.
|
||||
5. **AI Model Solution**: Provide a unified entry point for model search, download, and access control, making it convenient for users to find and use models suitable for their needs.
|
||||
6. **Hardware-specific optimization**: Optimize for specific hardware to enable smooth local running of most open-source AI applications.
|
||||
7. **Strict Privacy Protection and Management**: Strictly manage personal data, ranging from family albums to chat records and social media records, and provide a unified access control interface for AI applications.
|
||||
8. **Personal Knowlege Base**:
|
||||
9. **Integrated AIGC Workflow**: Offer AIGC Agent/Workflow for users to train their own voice models, Lora models, knowledge models, etc., using personal data. Based on these private model data, integrate the most advanced AIGC algorithm to help people release creativity easily and build more COOL and more personalized content.
|
||||
10. **Development Framework**: Provide a development framework for customizing AI assistants for specific purposes, making it easy for developers to create unique AI applications / service for their customers.
|
||||
|
||||
## **Deeply Understanding OpenDAN**
|
||||
|
||||
### Build OpenDAN from source code
|
||||
1. Install the latest version of python (>= 3.11) and pip
|
||||
2. Clone the source code
|
||||
```
|
||||
git clone https://github.com/fiatrete/OpenDAN-Personal-AI-OS.git
|
||||
cd OpenDAN-Personal-AI-OS
|
||||
```
|
||||
3. Install the dependent python library
|
||||
```
|
||||
pip install -r ./src/requirements.txt
|
||||
```
|
||||
Waiting for installation.
|
||||
|
||||
4. Start OpenDAN through aios_shell
|
||||
```
|
||||
python ./src/srvice/aios_shell/aios_shell.py
|
||||
```
|
||||
Now OpenDAN runs in the development mode, and the directory is:
|
||||
- AIOS_ROOT: ./rootfs (/opt/aios in docker)
|
||||
- AIOS_MYAI: ~/myai (/root/myai in docer)
|
||||
|
||||
### OpenDAN Cookbook
|
||||
|
||||
#### Chapter 1: Hello, Jarvis!
|
||||
- 1.1 Installation of OpenDAN
|
||||
- 1.2 Initial Configuration of OpenDAN
|
||||
- 1.3 Introduction to Agent and Using Jarvis
|
||||
- 1.4 Communicating with Jarvis Anytime and Anywhere via Telegram and Email
|
||||
- 1.5 Using Jarvis in Daily Life
|
||||
- 1.6 Mia and the Knowledge Base
|
||||
- 1.7 Introduction to Other Built-in Agents
|
||||
|
||||
[Click to Read](./doc/QuickStart.md)
|
||||
|
||||
#### Chapter 2: AIGC Workflow （Coming Soon）
|
||||
Using Workflow to activate the AIGC feature and let the Agent team (director, artist, and narrator) collaborate to create a unique bedtime story for your child based on your instructions!
|
||||
|
||||
- 2.1 Using Workflow `story_maker`
|
||||
- 2.2 Enabling Your Own AIGC Computation Node
|
||||
- 2.3 Training and Using Your Own AIGC LoRA Model.
|
||||
|
||||
#### Chapter 3: Develop Agent/Workflow on OpenDAN (Writing)
|
||||
|
||||
What's the most crucial design aspect of an operating system? Defining new forms of applications!
|
||||
|
||||
This article will systematically introduce what future Intelligence Applications look like, how to develop and release Intelligence Applications, and how to connect new-age Intelligence Applications with traditional computing.
|
||||
|
||||
- 3.1 Developing Agents that Run on OpenDAN
|
||||
- 3.2 Developing Workflows that Run on OpenDAN
|
||||
- 3.3 Extending the Environments Accessible by Agents
|
||||
- 3.4 Releasing Various Models Trained by Yourself
|
||||
- 3.5 Expanding More Tunnels to Enhance the Accessibility of Agents/Workflow
|
||||
- 3.6 Developing Traditional dApps on the Personal Server.
|
||||
|
||||
#### Chapter 4: OpenDAN Kernel Development (Writing)
|
||||
This article will introduce the design and implementation of OpenDAN's architecture
|
||||
|
||||
|
||||
|
||||
|
||||
- 4.1 Integrate your own LLM core into OpenDAN.
|
||||
- 4.2 Knowledge Base: Expand more file types, allowing Agents to better understand your knowledge graph.
|
||||
- 4.3 AI computation engine, integrating more AIGC capabilities, and accessing more computational power.
|
||||
- 4.4 OpenDAN's state management: File system and vector database.
|
||||
- 4.5 Kernel services and permission isolation.
|
||||
- 4.6 Smart gateway.
|
||||
|
||||
|
||||
## **Upcoming Roadmap**
|
||||
|
||||
- [x] Release PoC of OpenDAN
|
||||
- [x] **0.5.1** Implement personal data embeding to Knownlege-Base(KB) via Spider, followed by access by AI Agent
|
||||
- [ ] 0.5.2 Separate user mode and kernel mode, Knowledge Base supports scene format and more Spiders, supports personal AIGC model training
|
||||
- [ ] 0.5.3 Release Home Environment, allowing Agents to access and control your home's IoT devices
|
||||
- [ ] 0.5.x Official version of OpenDAN Alpha. Release OpenDAN SDK 1.0.
|
||||
|
||||
## **Contributing**
|
||||
|
||||
@@ -79,12 +218,14 @@ We welcome community members to contribute to the project, including but not lim
|
||||
- Submit a Pull Request to the repository
|
||||
- Participate in discussions and development
|
||||
|
||||
## **⭐Star History**
|
||||
OpenDAN utilizes the SourceDAO smart contract to incentivize the community. Developers who contribute can receive rewards in the form of OpenDAN DAO Tokens. DAO Token holders can collaboratively determine the development direction of OpenDAN. You can learn more about the rules of SourceDAO by reading this article（ https://github.com/fiatrete/OpenDAN-Personal-AI-OS/issues/25 ）
|
||||
|
||||
The DAO governance page for OpenDAN is under development. Once officially launched, all contributors will receive DAO Tokens according to the rules.
|
||||
|
||||
## **⭐Star History**
|
||||
|
||||
[](https://star-history.com/#fiatrete/OpenDAN-Personal-AI-Server-OS&Date)
|
||||
|
||||
|
||||
## **License**
|
||||
|
||||
MIT
|
||||
The current license is MIT, but it will transition to SourceDAO in the future.
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
#!/bin/bash
|
||||
# Build the docker image
|
||||
docker build -t aios .
|
||||
docker tag aios:latest paios/aios:latest
|
||||
docker push paios/aios:latest
|
||||
@@ -0,0 +1,334 @@
|
||||
# OpenDAN Quick Start
|
||||
OpenDAN (Open and Do Anything Now with AI) is revolutionizing the AI landscape with its Personal AI Operating System. Designed for seamless integration of diverse AI modules, it ensures unmatched interoperability. OpenDAN empowers users to craft powerful AI agents—from butlers and assistants to personal tutors and digital companions—all while retaining control. These agents can team up to tackle complex challenges, integrate with existing services, and command smart(IoT) devices.
|
||||
|
||||
With OpenDAN, we're putting AI in your hands, making life simpler and smarter.
|
||||
|
||||
This project is still in its very early stages, and there may be significant changes in the future.
|
||||
|
||||
## Installation
|
||||
|
||||
OpenDAN的Internal test版本有两种安装方式：
|
||||
1.通过Docker安装，这也是我们现在推荐的安装方法
|
||||
2.通过源代码安装，这种方法可能会遇到一些传统的Python依赖问题，需要你有一定的解决能力。但是如果你想要对OpenDAN进行二次开发，这种方法是必须的。
|
||||
|
||||
### 安装前准备工作
|
||||
|
||||
1. Docker环境
|
||||
OpenDAN通过适配Docker实现了对多平台的适配。本文不介绍怎么安装Docker,在你的控制台下执行
|
||||
|
||||
```
|
||||
docker --version
|
||||
```
|
||||
|
||||
如果能够看到Docker的版本号（>20.0），说明你已经安装了Docker.
|
||||
不知道怎么安装Docker的话，可以参考[这里](https://docs.docker.com/engine/install/)
|
||||
|
||||
2. OpenAI的API Token
|
||||
如果你还没有API Token的话，可以通过[这里](https://beta.openai.com/)申请
|
||||
（申请API Token对新玩家可能有一些门槛，可以在身边找找朋友，可以让他们给你一个临时的，或则加入我们的内测体验群，我们也会不时放出一些免费体验的API Token,这些Token被限制了最大消费和有效时间）
|
||||
|
||||
#### 安装OpenDAN
|
||||
|
||||
执行下面的命令，就可以安装OpenDAN的Docker Image了
|
||||
|
||||
```
|
||||
docker pull paios/aios:latest
|
||||
```
|
||||
|
||||
## 运行OpenDAN
|
||||
|
||||
首次运行OpenDAN需要进行初始化，初始化过程中会下载一些用于本地Knowledge Base库的基础模型，并需要你输入一些个人信息，因此启动Docker的时候记住要带上 -it参数。
|
||||
OpenDAN是你的Personal AIOS,其运行过程中会产生一些重要的个人数据（比如和Agent的对话记录，日程数据等），这些数据会保存在你的本地磁盘上，因此在启动Docker的时候，要将本地磁盘挂载到Docker的容器中，这样才能保证数据的持久化。
|
||||
|
||||
```
|
||||
docker run -v /your/local/myai/:/root/myai --name aios -it paios/aios:latest
|
||||
```
|
||||
|
||||
在上述命令中，我们还为docker run创建的docker 实例起了一个名字叫aios,方便后续的操作。你也可以用自己喜欢的名字来代替
|
||||
|
||||
执行上述命令后，如果一切正常，你会看到如下界面
|
||||
|
||||
|
||||
首次运行完成Docker实例的创建后，再次运行只需要执行：
|
||||
|
||||
```
|
||||
docker start -ai aios
|
||||
```
|
||||
|
||||
如果打算以服务模式运行，则不用带 -ai参数：
|
||||
|
||||
```
|
||||
docker start aios
|
||||
```
|
||||
|
||||
## OpenDAN的首次运行配置
|
||||
|
||||
如果你过去没有用字符界面(CLI)的产品，可能会有一点点不习惯。但别紧张，即使在Internal Test版本中，你也只会在极少数的情况下需要使用CLI。
|
||||
|
||||
OpenDAN必须是所有人都能轻松使用的未来操作系统，因此我们希望OpenDAN的使用和配置都是非常友好和简单的。但在Internal Test版本中，我们还没有足够的资源来实现这一目标。经过思考，我们决定先支持以CLI的方式来使用OpenDAN。
|
||||
|
||||
OpenDAN以LLM为AIOS的内核，通过不同的Agent/Workflow整合了很多很Cool的AI功能，你能在OpenDAN里一站式的体验AI工业的一些最新的成功。激活全部的功能需要做比较多的配置，但首次运行我们只需要做两项配置就可以了
|
||||
|
||||
1. LLM内核。OpenDAN是围绕LLM构建的未来智能操作系统，因此系统必须有至少一个LLM内核。
|
||||
OpenDAN以Agent为单位对LLM进行配置，未指定LLM模型名的Agent将会默认使用GPT4（GPT4也是目前最聪明的LLM）。你可以修改该配置到llama或其它安装的Local LLM。今天使用Local LLM需要相当强的本地算力的支持，这需要一笔不小的一次性投入。
|
||||
但我们相信LLM领域也会遵循摩尔定律，未来的LLM模型会越来越强大，越来越小，越来越便宜。因此我们相信在未来，每个人都会有自己的Local LLM。
|
||||
2. 你的个人信息，这能让你的私人AI管家Jarvis更好的为你服务。注意这里一定要输入你自己正确的Telegram username ,否则由于权限控制，后续将无法通过Telegram访问OpenDAN上安装的Agent/Workflow。
|
||||
|
||||
好的，简单的了解了上述背景后，请按界面提示完成必要信息的输入。
|
||||
|
||||
P.S:
|
||||
上述配置会保存在`/your/local/myai/etc/system.cfg.toml`中，如果你想要修改配置，可以直接修改这个文件。如果你想要调整配置，可以直接编辑这个文件。
|
||||
|
||||
|
||||
## （实验性）安装本地LLM内核
|
||||
|
||||
首次快速体验OpenDAN,我们强烈的推荐你使用GPT4，虽然它很慢，也很贵，但它也是目前最强大和稳定的LLM内核。OpenDAN在架构设计上，允许不同的Agent选择不同的LLM内核（系统里至少要有一个可用的LLM内核），如果你因为各种原因无法使用GPT4，可以是用下面方法安装Local LLM让系统能跑起来。OpenDAN是面向未来设计的系统，我们相信今天GPT4的能力一定会是未来所有LLM的下限。但目前的现实情况，其它的LLM不管是效果还是功能和GPT4都还有比较明显的差距，所以要完整体验OpenDAN，在一定时间内，我们还是推荐使用GPT4.
|
||||
|
||||
目前我们只完成了基于Llama.cpp的Local LLM的适配，为OpenDAN适配新的LLM内核并不是复杂的工作，有需要的工程师朋友可以自行扩展（记得给我们PR~）。如果你有一定的动手能力，可以用下面的方法安装基于Llama.cpp的Compute Node:
|
||||
|
||||
### 安装Llama.cpp ComputeNode
|
||||
|
||||
OpenDAN支持分布式计算资源调度，因此你可以把LLaMa的计算节点安装在和OpenDAN不同的机器上。根据模型的大小需要相当的算力支持，请根据自己的机器配置量力而行。我们使用llama.cpp构建LLaMa LLM ComputeNode,llama.cpp也是一个正在高速演化的项目，正致力降低LLM的运行需要的设备门槛，提高运行速度。请阅读llamap.cpp的项目了解其支持的各个模型的最低系统要求。
|
||||
|
||||
|
||||
安装LLama.cpp 总共分两步：
|
||||
|
||||
Step1: 下载LLama.cpp的模型，有3个选择：7B-Chat,13B-Chat,70B-Chat. 我们的实践经验最少需要13B的才能工作。LLaMa2 目前官方的模型并不支持inner function call,而目前OpenDAN的很多Agent都高度依赖inner function call.所以我们推荐您下载通过Fine-Tune 的 13B模型：
|
||||
|
||||
```
|
||||
https://huggingface.co/Trelis/Llama-2-13b-chat-hf-function-calling
|
||||
```
|
||||
|
||||
Step2 运行llama-cpp-python镜像
|
||||
|
||||
```
|
||||
docker run --rm -it -p 8000:8000 -v /path/to/models:/models -e MODEL=/models/llama-2-13b-chat.gguf ghcr.io/abetlen/llama-cpp-python:latest
|
||||
```
|
||||
|
||||
完成上述步骤后，如果输出如下，说明LLaMa已经正确加载模型并正常运行了
|
||||
```
|
||||
....................................................................................................
|
||||
llama_new_context_with_model: kv self size = 640.00 MB
|
||||
llama_new_context_with_model: compute buffer total size = 305.47 MB
|
||||
AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 |
|
||||
INFO: Started server process [171]
|
||||
INFO: Waiting for application startup.
|
||||
INFO: Application startup complete.
|
||||
INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
|
||||
```
|
||||
|
||||
### 将LLama.cpp ComputeNode增加到OpenDAN中
|
||||
|
||||
ComputeNode是OpenDAN的底层组件，而且可能不会与OpenDAN运行在同一个机器上。因此从依赖关系的角度，OpenDAN并没有“主动检测”ComputeNode的能力，需要用户（或系统管理员）在OpenDAN的命令行中通过下面命令手工添加
|
||||
|
||||
```
|
||||
/node add llama Llama-2-13b-chat http://localhost:8000
|
||||
```
|
||||
|
||||
上面添加的是运行在本地的13b模型，如果你使用的是其它模型，或则跑在了不同的机器上。请修改上述命令中的模型名和端口号。
|
||||
|
||||
### 配置Agent使用LLaMa
|
||||
|
||||
OpenDAN的Agent可以选择最适合其职责的LLM-Model，我们内置了一个Agent叫Lachlan的私人西班牙语老师Agent，已经被配置成了使用LLaMa-2-13b-chat模型。你可以通过下面命令与其聊天：
|
||||
|
||||
```
|
||||
/open Lachlan
|
||||
```
|
||||
|
||||
|
||||
因此添加了一个新的LLM后，需要手工修改Agent的配置，才能让其使用新的LLM。比如我们的私人英文老师Tracy，其配置文件是`/opt/aios/agents/Tracy/Agent.toml`，修改配置如下：
|
||||
```
|
||||
llm_model_name="Llama-2-13b-chat"
|
||||
max_token_size = 4000
|
||||
```
|
||||
然后重新启动OpenDAN,你就可以让Tracy使用LLaMa了(你也可以通过该方法查看其它内置的Agent使用了哪些LLM模型)
|
||||
|
||||
|
||||
## Hello, Jarvis!
|
||||
|
||||
配置完成后，你会进入一个AIOS Shell,这和linux bash 和相似，这个界面的含义是：
|
||||
当前用户 "username" 正在 和 名“为Jarvis的Agent/Workflow” 进行交流，当前话题是default。
|
||||
和你的私人AI管家Jarvis Say Hello吧！
|
||||
|
||||
***如果一切正常，你将会在一小会后得到Jarvis的回复。此时OpenDAN系统已经正常运行了***
|
||||
|
||||
## 给Jarvis注册Telegram账号
|
||||
你已经完成了OpenDAN的安装和配置，并已经验证了其可以正常工作。下面让我们尽快回到熟悉的图形界面，回到移动互联网吧！
|
||||
我们将给Jarvis注册一个Telegram账号，通过Telegram，我们可以使用熟悉的方式和Jarvis进行交流了~
|
||||
在OpenDAN的aios_shell输入
|
||||
|
||||
```
|
||||
/connect Jarvis
|
||||
```
|
||||
|
||||
按照提示输入Telegram Bot Token就完成了Jarvis的账号注册. 你可以通过阅读下面文章来了解如何获取Telegram Bot Token
|
||||
https://core.telegram.org/bots#how-do-i-create-a-bot，
|
||||
|
||||
我们还支持给Agent注册email账号，用下面命令行
|
||||
|
||||
```
|
||||
/connect Jarvis email
|
||||
```
|
||||
|
||||
然后根据提示就可以给Jarvis绑定电子邮件账号了。但由于目前系统并未对email内容定制前置过滤，所以可能会带来潜在的大量LLM访问费用，因此Email的支持是实验性的。我们推荐给Agent创建全新的电子邮件账号。
|
||||
|
||||
## 以服务方式运行OpenDAN
|
||||
|
||||
上述的运行方式是以交互方式运行OpenDAN，这种方式适合在开发和调试的时候使用。在实际使用的时候，我们推荐以服务方式运行OpenDAN，这样可以让OpenDAN在后台默默的运行，不会影响你的正常使用。
|
||||
先输入
|
||||
|
||||
```
|
||||
/exit
|
||||
```
|
||||
|
||||
关闭并退出OpenDAN,随后我们再用服务的方式启动OpenDAN：
|
||||
|
||||
```
|
||||
docker start aios
|
||||
```
|
||||
|
||||
Jarvis是运行在OpenDAN上的Agent,当OpenDAN退出后，其活动也会被终止。因此如果想随时随地通过Telegram和Jarvis交流，请记住保持OpenDAN的运行（不要关闭你的电脑，并保持其网络连接）。
|
||||
|
||||
实际上,OpenDAN是一个典型的Personal OS，运行在Personal Server之上。关于Personal Servier的详细定义可以参考[CYFS Owner Online Device(OOD) ](https://github.com/buckyos/CYFS)。因此运行在PC或笔记本上并不是一个正式选择，但谁要我们正在Internal Test呢？
|
||||
|
||||
我们正在进行的很多研发工作，其中有很大一部分的目标，就是能让你轻松的拥有一个搭载AIOS的Personal Server.相对PC，我们将把这个新设备叫PI(Personal Intelligence)，OpenDAN是面向PI的首个OS。
|
||||
|
||||
## 你的私人管家 Jarvis 前来报道！
|
||||
|
||||
现在你已经可以随时随地通过Telegram和Jarvis交流了，但只是把他看成更易于访问的ChatGPT,未免有点小瞧他了。让我们来看一下运行在OpenDAN里的Jarvis有什么新本事吧！
|
||||
|
||||
## 让Jarvis给你安排日程
|
||||
|
||||
相信不少朋友有长期使用Outlook等传统Calender软件来管理自己日程的习惯。像我自己通常每周会花至少2个小时来是使用这类软件，当发生一些计划外的情况时，对计划进行手工调整是一个枯燥的工作。作为你的私人管家，Jarvis必须能够帮用自然语言的方式帮你管理日程！
|
||||
试试和Jarvis说：
|
||||
|
||||
```
|
||||
我周六和Alic上午去爬山，下午去看电影！
|
||||
```
|
||||
|
||||
如果一切正常，你会看到Jarvis的回复，并且已经记住了你的日程安排。
|
||||
|
||||
你可以通过自然语言的方式和Jarvis查询
|
||||
```
|
||||
我这周末有哪些安排？
|
||||
```
|
||||
|
||||
你会看到Jarvis的回复，其中包含了你的日程安排。
|
||||
由于Jarvis使用LLM作为思考内核，他能以非常自然的方式和你进行交流，并在合适的时候管理你的日程。比如你可以说
|
||||
|
||||
```
|
||||
我周六有朋友从LA过来，很久没见了，所有周六的约会都移动到周日吧！
|
||||
```
|
||||
|
||||
你会看到Jarvis会自动的帮你吧周六的日程移动到周日。
|
||||
实际上在整个交流的过程中，你不需要有明确的“使用日程管理语言的意识”，Jarvis作为你的管家，在理解你的个人数据的基础上，会在合适的时机和你进行交流，帮你管理日程。
|
||||
这是一个非常简单而又常用的例子，通过这个例子，我们可以看到未来人们不再需要学习一些今天非常重要的基础软件了。
|
||||
|
||||
欢迎来到新时代！
|
||||
|
||||
Agent安排的日程数据都保存在 ~/myai/calender.db 文件中，格式是sqlite DB. 我们后续计划授权让Jarvis可以操作你生产环境中的Calender(比如常用的Google Calender)。但我们还是希望未来，人们可以把重要的个人数据都保存在自己物理上拥有的Personal Server中。
|
||||
|
||||
## 介绍Jarvis给你的朋友
|
||||
|
||||
把Jarvis的telegram账号分享给你的朋友，可以做一些有趣的事情。比如你的朋友可以在联系不到你的时候，通过Jarvis，你的高级私人助理来处理一些事务性的工作，比如了解你最近的日程安排或计划。
|
||||
尝试后你会发现，Jarvis并不会按预期工作。是因为站在数据隐私的角度，Jarvis默认只会和“可信的人”进行交流。要实现上面目标，你需要让Jarvis能了解你的人际关系。
|
||||
|
||||
### 让Jarvis管理你的联系人
|
||||
|
||||
OpenDAN在 myai/contacts.toml 文件中保存了系统已知的所有人的信息。现在非常简单的分成了两组
|
||||
1. Family Member,现在该文件里保存里你自己的信息（在系统首次初始化时登陆的）添加
|
||||
2. Contact，通常是你的好友
|
||||
|
||||
任何不存在上述列表中的联系人，都会被系统划分到`Guest`。Jarvis默认不允许和`Guest`进行交流。因此如果你想要让Jarvis和你的朋友进行交流，你需要把他添加到`Contact`中。
|
||||
你可以手工修改 myai/contacts.toml 文件，也可以通过Jarvis来添加联系人。试试和Jarvis说
|
||||
|
||||
```
|
||||
Jarvis,请添加我的朋友Alic到我的联系人中，他的telegram username是xxxx,email是xxxx
|
||||
```
|
||||
|
||||
Jarvis能够理解你的意图，并完成添加联系人的工作。
|
||||
添加联系人后，你的朋友就可以和你的私人管家Jarvis进行交流了。
|
||||
|
||||
## 更新OpenDAN的镜像
|
||||
|
||||
现在OpenDAN还处在早期阶段，因此我们会定期发布OpenDAN的镜像来修正一些BUG。因此你可能需要定期更新你的OpenDAN镜像。更新OpenDAN的镜像非常简单，只需要执行下面的命令就可以了
|
||||
```
|
||||
docker stop aios
|
||||
docker rm aios
|
||||
docker pull paios/aios:latest
|
||||
docker run -v /your/local/myai/:/root/myai --name aios -it paios/aios:latest
|
||||
```
|
||||
|
||||
|
||||
## 让Agent进一步访问你的信息
|
||||
|
||||
你已经知道Jarvis可以帮你管理一些重要的信息。但这些信息都是“新增信息”。在上世纪80年代PC发明以后，我们的一切都在高速的数字化。每个人都已有了海量的数字信息，包括你通过智能手机拍摄的照片，视频，你工作中产生的邮件文档等等。过去我们通过文件系统来管理这些信息，在AI时代，我们将通过Knowledge Base来管理这些信息，保存在Knowledge Base中的信息能更好的被AI访问，让你的Agent更理解你，更好的为你服务。
|
||||
|
||||
Knowledge Base是OpenDAN里非常重要的一个基础概念，也是我们为什么需要Personal AIOS的一个关键原因。Knowledge Base相关的技术目前正在快速发展，因此OpenDAN的Knowledge Base的实现也在快速的进化。目前我们的实现更多的是让大家能体验Knowledge Base与Agent结合带来的新能力，其效果还远远未达我们的预期。站在系统设计的角度，我们尽快开放这个组件的另一个目的，是希望找到在产品上对用户更友好，更平滑的方法来把已经存在的个人信息导入进Knowledge Base。
|
||||
|
||||
Knowledge Base功能已经默认开启了，将自己的数据放入Knowledge Base有两种方法
|
||||
|
||||
1. 把要放入KnowledgeBase的数据复制到 `~myai/data`` 文件夹中。
|
||||
2. 通过输入`/Knowledge add dir` ，系统会要求你输入一个将要导入到Knowledge Base的本地目录。注意OpenDAN默认运行在容器中，因此$dir是相对于容器的路径，如果你想要加入本地磁盘的数据，需要先把本地数据挂载到容器中。
|
||||
|
||||
OpenDAN会在后台不断分析已加入Knowledge Base文件夹中的文件，分析结果保存在 ~/myai/knowledge 目录中。将该目录删除后，系统会重新分析已加入Knowledge Base的文件。由于目前OpenDAN的Knowledge Base还处在早期阶段，因此目前只支持分析识别文本文件，图片，短视频等。未来OpenDAN将会支持所有的主流文件格式，尽可能把所有的已有信息都能导入到Knowledge。可以aios_shel中通过下面命令来查询Knowledge Base 分析任务的运行状态。
|
||||
|
||||
```
|
||||
/Knowledge journal
|
||||
```
|
||||
|
||||
### Mia：个人信息助手
|
||||
|
||||
然后我们可以通过 Agent "Mia"来访问Knwolege Base,
|
||||
|
||||
```
|
||||
/open Mia
|
||||
```
|
||||
|
||||
试着与Mia交流一下吧！我想这会带来完全不同的体验！
|
||||
Mia找到的信息会用下面方式展示：
|
||||
|
||||
```
|
||||
{"id": "7otCtsrzbAH3Nq8KQGtWivMXV5p54qrv15xFZPtXWmxh", "type": "image"}
|
||||
```
|
||||
|
||||
可以用`/knowledge query 7otCtsrzbAH3Nq8KQGtWivMXV5p54qrv15xFZPtXWmxh` 命令来调用本地的文件查看器来查看结果。
|
||||
|
||||
我们更推荐把Mia接入到Telegram中,这样Mia会把查询结果直接用图片的方式展现，用起来更加方便~
|
||||
|
||||
### Embeding Pipeline
|
||||
|
||||
Knowledge Base读取并分析文件，产生Agent可以访问的信息的过程被称作Embeding.这个过程需要一定的计算资源。经过我们的测试，目前OpenDAN基于“Sentence Transformers”构建的Embeding Pipeline是可以在绝大多数类型的机器上运行起来的。不同能力的机器的区别主要在于Embeding的速度和质量。了解OpenDAN进度的朋友可能知道，我们在实现的过程中也曾支持过云端Embeding,用来彻底的减少OpenDAN的最小系统性能要求。不过考虑到Embeding过程中涉及到的大量的个人隐私数据，我们还是决定关闭云端Embeding这个特性。有需要的同学可以通过修改源代码来打开云端Embeding,让OpenDAN可以在非常低性能的设备上工作起来。
|
||||
|
||||
遗憾的是，现在并没有统一的Embeding标准，因此不同的Embeding Pipeline产生的结果不能互相兼容。这意味着一旦切换了Embeding Pipline,知识库的所有信息都要重新扫描。
|
||||
|
||||
## bash@aios
|
||||
|
||||
如果你有一定的工程背景，通过让Agent 执行bash命令，也可以非常简单快速的让OpenDAN具有你的私有数据的访问能力。
|
||||
使用命令
|
||||
|
||||
```
|
||||
/open ai_bash
|
||||
```
|
||||
|
||||
打开ai_bash,然后你就可以在aios_shell的命令行中执行传统的bash命令了。同时你还拥有了智能命令的能力，比如查找文件，你可以用
|
||||
|
||||
```
|
||||
帮我查找 ~/Documents 目录下所有包含OpenDAN的文件
|
||||
```
|
||||
|
||||
来代替输入find命令~ 非常酷吧！
|
||||
|
||||
OpenDAN目前默认运行在容器中，因此ai_bash也只能访问docker容器中的文件。这相对安全，但我们还是提醒你不要轻易的把ai_bash这个agent暴露出去，可能会带来潜在的安全风险。
|
||||
|
||||
## 我们为什么需要Personal AIOS?
|
||||
|
||||
很多人会第一个想到隐私，这是一个重要的原因，但我们不认为这是人们真正离开ChatGPT,选择Personal AIOS的真正原因。毕竟今天很多人并不对隐私敏感。而且今天的平台厂商一般都是默默的使用你的隐私赚钱，而很少会真正泄露你的隐私。
|
||||
|
||||
我们认为Personal AIOS的真正价值在于：
|
||||
|
||||
1. 成本是一个重要的决定因素。LLM是非常强大的，边界非常清楚的核心组件,是新时代的CPU。从产品和商业的角度，ChatGPT类产品只允许用有限的方法来使用它。让我想起了小型机刚刚出现时大家分时使用系统的时代：有用，但有限。要真正发挥LLM的价值，我们需要让每个人都能拥有自己的LLM，并能自由的使用LLM作为任何应用的底层组件，这就必须要有一个新的，以LLM为核心构建的操作系统来重新抽象应用（Agent/Workflow）和应用所使用的资源（算力，数据，环境）
|
||||
|
||||
2. 当拥有LLM后，当能把LLM放到每一个计算前面时，你会看到真正的宝藏！现在的ChatGPT通过Plugin对LLM能力的扩展，其能力和边界都是非常有限的，这里既有商业成本的原因，也有传统云服务的法律边界问题：平台要承担的责任太多了。而通过在Personal AIOS中使用LLM，你可以自由的把自然语言，LLM，已有服务，私人数据，智能设备连接在一起，并不用担心隐私泄露和责任问题（你自己承担了授权给LLM后产生后果的责任）！
|
||||
|
||||
OpenDAN is an open-source project, let's define the future of Humans and AI together!
|
||||
@@ -0,0 +1,348 @@
|
||||
# OpenDAN Quick Start
|
||||
|
||||
OpenDAN (Open and Do Anything Now with AI) is revolutionizing the AI landscape with its Personal AI Operating System. Designed for seamless integration of diverse AI modules, it ensures unmatched interoperability. OpenDAN empowers users to craft powerful AI agents—from butlers and assistants to personal tutors and digital companions—all while retaining control. These agents can team up to tackle complex challenges, integrate with existing services, and command smart(IoT) devices.
|
||||
|
||||
With OpenDAN, we're putting AI in your hands, making life simpler and smarter.
|
||||
|
||||
This project is still in its very early stages, and there may be significant changes in the future.
|
||||
|
||||
## Installation
|
||||
|
||||
There are two ways to install the Internal Test Version of OpenDAN:
|
||||
|
||||
1. Installation through docker, this is also the installation method we recommend now
|
||||
2. Installing through the source code, this method may encounter some traditional Pyhont dependence problems and requires you to have a certain ability to solve.But if you want to do secondary development of OpenDAN, this method is necessary.
|
||||
|
||||
### Preparation before installation
|
||||
|
||||
1. Docker environment
|
||||
This article does not introduce how to install the docker, execute it under your console
|
||||
|
||||
```
|
||||
docker -version
|
||||
```
|
||||
|
||||
If you can see the docker version number (> 20.0), it means that you have installed Docker.
|
||||
If you don't know how to install docker, you can refer to [here](https://docs.docker.com/engine/install/)
|
||||
|
||||
2. OpenAI API Token
|
||||
If there is no api token, you can apply for [here](https://beta.openai.com/)
|
||||
(Applying for the API Token may have some thresholds for new players. You can find friends around you, and you can give you a temporary, or join our internal test experience group. We will also release some free experience API token from time to time.These token is limited to the maximum consumption and effective time)
|
||||
|
||||
### Install OpenDAN
|
||||
|
||||
After executing the following command, you can install the Docker Image of OpenDAN
|
||||
```
|
||||
docker pull paios/aios:latest
|
||||
```
|
||||
|
||||
## Run
|
||||
|
||||
When you first run OpenDAN, you need to initialize it. During the initialization process, some basic models will be downloaded for the local Knowledge Base library, and you'll need to input some personal information. Therefore, remember to include the -it parameter when starting Docker.
|
||||
|
||||
OpenDAN is your Personal AIOS, so it will generate some important personal data (such as chat history with agent, schedule data, etc.) during its operation. These data will be stored on your local disk. ThereforeWe recommend that you mount the local disk into the container of Docker so that the data can be guaranteed.
|
||||
|
||||
```
|
||||
docker run -v /your/local/myai/:/root/myai --name aios -it paios/aios:latest
|
||||
```
|
||||
|
||||
In the above command, we also set up a Docker instance for Docker Run named AIOS, which is convenient for subsequent operations.You can also use your favorite name instead
|
||||
|
||||
After executing the above command, if everything is normal, you will see the following interface
|
||||
|
||||
|
||||
After the first operation of the docker instance is created, it only needs to be executed again:
|
||||
|
||||
```
|
||||
docker start -ai aios
|
||||
```
|
||||
|
||||
If you plan to run in a service mode (NO UI), you don't need to bring the -AI parameter:
|
||||
|
||||
```
|
||||
docker start aios
|
||||
```
|
||||
|
||||
## The first run configuration
|
||||
|
||||
If you have not used the character interface (CLI) in the past, you may not be used to it.But don't be nervous, even in the Internal Test version, you will only need to use CLI in a few cases.
|
||||
|
||||
OpenDAN must be a future operating system that everyone can easily use, so we hope that the use and configuration of OpenDAN are very friendly and simple.But in the Internal Test, we have not enough resources to achieve this goal.After thinking, we decided to support the use of OpenDAN by CLI.
|
||||
|
||||
OpenDAN uses LLM as the kernel of AIOS, and integrates many very COOL AI functions through different Agent/Workflow. You can experience some of the latest success of the AI industry in OpenDAN.To activate all the functions requires more configuration, but we only need to do two configurations for the first operation.
|
||||
|
||||
1. LLM Kernel
|
||||
|
||||
OpenDAN is a future AI Operating Yystem built around LLM, so the system must have `at least one LLM core`.
|
||||
|
||||
OpenDan configures LLM in the agent unit. Agent, which does not specify the LLM model name, will use GPT4 by default (GPT4 is also the smartest LLM).You can modify the local LLM that configures to LLaMa or other installed.Today, Local LLM requires the support of quite strong local computing resource, which requires a lot of one-time investment.
|
||||
|
||||
But we believe that the LLM field will also follow `Moore's law`. The future LLM model will become more and more powerful, smaller and cheaper.Therefore, we believe that in the future, everyone will have their own Local LLM.
|
||||
|
||||
2. Your personal information
|
||||
|
||||
This allows your personal AI assistant Jarvis to better serve you. Note that you must enter your own correct Telegram username, otherwise due to authority control, you will not be able to access Agent/Workflow installed on OpenDan through Telegram.
|
||||
|
||||
Okay, after a simple understanding of the above background, press the interface to prompt the input of the necessary information.
|
||||
|
||||
P.S:
|
||||
The above configuration will be saved in the `/your/local/myai/etc/system.cfg.toml`, if you want to modify the configuration, you can directly modify this file.If you want to adjust the configuration, you can edit this file directly.
|
||||
|
||||
|
||||
## (Experimental) Install the local LLM kernel
|
||||
|
||||
For a quick first experience with OpenDAN, we strongly recommend using GPT4. Although it's slow and expensive, it's currently the most powerful and stable LLM core. OpenDAN's architectural design allows different agents to choose different LLM cores (there must be at least one available LLM core in the system). If for any reason you can't use GPT4, you can install Local LLM using the method below to get the system running. OpenDAN is a system designed for the future, and we believe that the capabilities of GPT4 today will definitely be the minimum for all LLMs in the future. But the current reality is that other LLMs, whether in terms of effects or functions, still have a noticeable gap with GPT4, so to fully experience OpenDAN, we still recommend using GPT4 for a certain period of time.
|
||||
|
||||
At present, we have only completed the adaptation of Local LLM based on Llama.cpp. Adapting new LLM cores for OpenDAN is not a complicated task, and engineers who need it can expand it on their own (remember to PR us~). If you have certain hands-on abilities, you can install the Compute Node based on Llama.cpp using the method below:
|
||||
|
||||
### Install LLama.cpp ComputeNode
|
||||
|
||||
OpenDAN supports distributed computing resource scheduling, so you can install the LLaMa computing node on a machine different from OpenDAN. Depending on the size of the model, substantial computing power is needed, so please proceed according to your machine's configuration. We use llama.cpp to build the LLaMa LLM ComputeNode. Llama.cpp is a rapidly evolving project that is committed to reducing the device threshold required to run LLM and improving running speed. Please read the llama.cpp project to understand the minimum system requirements for each supported model.
|
||||
|
||||
Installing LLaMa.cpp involves two steps:
|
||||
|
||||
Step1: Download the LLaMa.cpp model. There are three choices: 7B-Chat, 13B-Chat, 70B-Chat. Our practical experience shows that at least the 13B model is required to work. LLaMa2's official models currently do not support inner function calls, and many of OpenDAN's Agents heavily depend on inner function calls. So we recommend you download the Fine-Tuned 13B model:
|
||||
|
||||
```
|
||||
https://huggingface.co/Trelis/Llama-2-13b-chat-hf-function-calling
|
||||
```
|
||||
|
||||
Step2: Run the llama-cpp-python image
|
||||
|
||||
```
|
||||
docker run --rm -it -p 8000:8000 -v /path/to/models:/models -e MODEL=/models/llama-2-13b-chat.gguf ghcr.io/abetlen/llama-cpp-python:latest
|
||||
```
|
||||
|
||||
After completing the above steps, if the output is as follows, it means that LLaMa has correctly loaded the model and is running normally.
|
||||
|
||||
```
|
||||
....................................................................................................
|
||||
llama_new_context_with_model: kv self size = 640.00 MB
|
||||
llama_new_context_with_model: compute buffer total size = 305.47 MB
|
||||
AVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 |
|
||||
INFO: Started server process [171]
|
||||
INFO: Waiting for application startup.
|
||||
INFO: Application startup complete.
|
||||
INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
|
||||
```
|
||||
|
||||
### Add LLaMa.cpp ComputeNode to OpenDAN
|
||||
|
||||
ComputeNode is a basic component of OpenDAN and may not run on the same machine as OpenDAN. Therefore, from a dependency perspective, OpenDAN does not have the ability to "actively detect" ComputeNode. Users (or system administrators) need to manually add it in the command line of OpenDAN with the following command:
|
||||
|
||||
```
|
||||
/node add llama Llama-2-13b-chat http://localhost:8000
|
||||
```
|
||||
|
||||
The above command adds a 13b model running locally. If you are using a different model or running it on a different machine, please modify the model name and port number in the command.
|
||||
|
||||
### Configure Agent to Use LLaMa
|
||||
|
||||
OpenDAN's Agents can choose the LLM-Model that best suits their responsibilities. We have built-in an Agent called Lachlan, a private Spanish teacher Agent, which has been configured to use the LLaMa-2-13b-chat model. You can chat with it using the following command:
|
||||
|
||||
```
|
||||
/open Lachlan
|
||||
```
|
||||
|
||||
Therefore, after adding a new LLM-Kernel, you need to manually modify the Agent's configuration to allow it to use the new LLM. For example, our private English teacher Tracy, whose configuration file is `/opt/aios/agents/Tracy/Agent.toml`, modify the configuration as follows:
|
||||
|
||||
```
|
||||
llm_model_name="Llama-2-13b-chat"
|
||||
max_token_size = 4000
|
||||
```
|
||||
|
||||
Then, after restarting OpenDAN, you can have Tracy use LLaMa (you can also use this method to see which LLM models other built-in Agents are using).
|
||||
|
||||
## Hello, Jarvis!
|
||||
|
||||
After the configuration is completed, you will enter a AIOS Shell, which is similar to Linux Bash and similar. The meaning of this interface is:
|
||||
The current user "username" is communicating with the name "Agent/Workflow of Jarvis". The current topic is default.
|
||||
|
||||
Say Hello to your private AI assistant Jarvis !
|
||||
|
||||
If everything is OK, you will get a reply from Jarvis after a moment .At this time, the OpenDAN system is running
|
||||
|
||||
## Give a Telegram account to Jarvis
|
||||
|
||||
You've successfully installed and configured OpenDAN, and verified that it's working properly. Now, let's quickly return to the familiar graphical interface, back to the mobile internet world!
|
||||
We'll be registering a Telegram account for Jarvis. Through Telegram, you can communicate with Jarvis in a way that feels familiar.
|
||||
In the OpenDAN's aios_shell, type:
|
||||
|
||||
```
|
||||
/connect Jarvis
|
||||
```
|
||||
|
||||
Follow the prompts to input the Telegram Bot Token, and you'll have Jarvis set up. To learn how to obtain a Telegram Bot Token, you can refer to the following article:
|
||||
https://core.telegram.org/bots#how-do-i-create-a-bot.
|
||||
|
||||
Additionally, we offer the option to register an email account for the Agent. Use the following command:
|
||||
|
||||
```
|
||||
/connect Jarvis email
|
||||
```
|
||||
|
||||
Then follow the prompts to link an email account to Jarvis. However, as the current system doesn't have a pre-filter customized for email contents, there's a potential for significant LLM access costs. Hence, email support is experimental. We recommend creating a brand-new email account for the Agent."
|
||||
|
||||
## Running OpenDAN as a Daemon
|
||||
|
||||
The method described above runs OpenDAN interactively, which is suitable for development and debugging purposes. For regular use, we recommend running OpenDAN as a service. This ensures OpenDAN operates silently in the background without disturbing your usual tasks.
|
||||
First, input:
|
||||
|
||||
```
|
||||
/exit
|
||||
```
|
||||
|
||||
to shut down and exit OpenDAN. Then, we'll start OpenDAN as a service using:
|
||||
|
||||
```
|
||||
docker start aios
|
||||
```
|
||||
|
||||
Jarvis, which is an Agent running on OpenDAN, will also be terminated once OpenDAN exits. So, if you wish to communicate with Jarvis via Telegram anytime, anywhere, remember to keep OpenDAN running (don't turn off your computer and maintain an active internet connection).
|
||||
|
||||
In fact, OpenDAN is a quintessential Personal OS, operating atop a Personal Server. For a detailed definition of a Personal Server, you can refer to the [CYFS Owner Online Device(OOD)](https://github.com/buckyos/CYFS). Running on a PC or laptop isn't the formal choice, but then again, aren't we in an Internal Test phase?
|
||||
|
||||
Much of our ongoing research and development work aims to provide an easy setup for a Personal Server equipped with AIOS. Compared to a PC, we're coining this new device as PI (Personal Intelligence), with OpenDAN being the premier OS tailored for the PI.
|
||||
|
||||
## Introducing Jarvis: Your Personal Butler!
|
||||
|
||||
Now you can talk with Jarvis anytime, anywhere via Telegram. However, merely seeing him as a more accessible ChatGPT doesn't do justice to his capabilities. Let's dive in and see what new tricks Jarvis, running on OpenDAN, brings to the table!
|
||||
|
||||
## Let Jarvis Plan Your Schedule
|
||||
|
||||
Many folks rely on traditional calendar software like Outlook to manage their schedules. I personally spend at least two hours each week using such applications. Manual adjustments to plans, especially unforeseen ones, can be tedious. As your personal butler, Jarvis should effortlessly manage your schedule through natural language!
|
||||
Try telling Jarvis:
|
||||
|
||||
```
|
||||
I'm going hiking with Alice on Saturday morning and seeing a movie in the afternoon!
|
||||
```
|
||||
|
||||
If everything's in order, you'll see Jarvis' response, and he'll remember your plans.
|
||||
You can inquire about your plans with Jarvis using natural language, like:
|
||||
|
||||
```
|
||||
What are my plans for this weekend?
|
||||
```
|
||||
|
||||
Jarvis will respond with a list of your scheduled activities.
|
||||
Since Jarvis uses LLM as its thinking core, he can communicate with you seamlessly, adjusting your schedule when needed. For instance, you can tell him:
|
||||
|
||||
```
|
||||
A friend is coming over from LA on Saturday, and it's been ages since we last met. Shift all of Saturday's appointments to Sunday, please!
|
||||
```
|
||||
|
||||
Jarvis will seamlessly reschedule your Saturday plans for Sunday.
|
||||
Throughout these interactions, there's no need to consciously use "schedule management language." As your butler, Jarvis understands your personal data and engages at the right moments, helping manage your schedule.
|
||||
This is a basic yet practical illustration. Through this example, it's clear that people might no longer need to familiarize themselves with foundational software of today.
|
||||
|
||||
Welcome to the new era!
|
||||
|
||||
All the schedules set by the Agent are stored in the ~/myai/calender.db file, formatted as sqlite DB. In future updates, we plan to authorize Jarvis to access your production environment calendars (like the commonly-used Google Calendar). Still, our hope for the future is that people store vital personal data on a physically-owned Personal Server.
|
||||
|
||||
## Introducing Jarvis to Your Friends
|
||||
|
||||
Sharing Jarvis's Telegram account with your friends can lead to some interesting interactions. For instance, if they can't get in touch with you directly, they can communicate with Jarvis, your advanced personal assistant, to handle transactional tasks like inquiring about your recent schedules or plans.
|
||||
|
||||
After trying, you'll realize that Jarvis doesn't operate as anticipated. From a data privacy standpoint, Jarvis, by default, interacts only with "trusted individuals". To achieve the above objectives, you need to let Jarvis understand your interpersonal relationships.
|
||||
|
||||
### Let Jarvis Manage Your Contacts
|
||||
|
||||
OpenDAN stores the information of all known individuals in the myai/contacts.toml file. Currently, it's simply divided into two groups:
|
||||
|
||||
1. Family Member, At present, this group stores your information (logged during the system's initial setup).
|
||||
2. Contact，These are typically your friends.
|
||||
|
||||
Anyone not listed in the aforementioned categories is classified as a Guest by the system. By default, Jarvis doesn't engage with Guests. Hence, if you want Jarvis to interact with your friends, you must add them to the Contact list.
|
||||
|
||||
You can manually edit the myai/contacts.toml file, or you can let Jarvis handle the contact addition. Try telling Jarvis:
|
||||
|
||||
```
|
||||
Please add my friend Alice to my contacts. Her Telegram username is xxxx, and her email is xxxx.
|
||||
```
|
||||
|
||||
Jarvis will comprehend your intent and carry out the task of adding the contact.
|
||||
Once the contact is added, your friend can interact with your personal butler, Jarvis.
|
||||
|
||||
## Update OpenDAN
|
||||
|
||||
As OpenDAN is still in its early stages, we regularly release images of OpenDAN to fix some bugs. Therefore, you may need to update your OpenDAN image regularly. Updating the OpenDAN image is very simple, just execute the following commands:
|
||||
|
||||
```
|
||||
docker stop aios
|
||||
docker rm aios
|
||||
docker pull paios/aios:latest
|
||||
docker run -v /your/local/myai/:/root/myai --name aios -it paios/aios:latest
|
||||
```
|
||||
|
||||
## Allowing Agents to Further Access Your Information
|
||||
|
||||
You already know that Jarvis can help you manage some important information. But these are all "new information". Since the invention of the PC in the 1980s, everything has been rapidly digitizing. Everyone already has a massive amount of digital information, including photos and videos you take with your smartphone, emails and documents generated at work, and so on. In the past, we managed this information through a file system, in the AI era, we will manage these pieces of information through a Knowledge Base. Information stored in the Knowledge Base can be better accessed by AI, allowing your Agent to understand you better and provide better service.
|
||||
|
||||
The Knowledge Base is a very important basic concept in OpenDAN, and it is a key reason why we need Personal AIOS. Technologies related to the Knowledge Base are currently developing rapidly, so the implementation of OpenDAN's Knowledge Base is also evolving rapidly. At present, our implementation is more about letting everyone experience the new capabilities brought by the combination of the Knowledge Base and Agent, and its effect is far from our expectations. From the perspective of system design, another purpose of our opening this component as soon as possible is to find a more user-friendly and smoother method on the product to import existing personal information into the Knowledge Base.
|
||||
|
||||
The Knowledge Base function is already turned on by default. There are two ways to put your own data into the Knowledge Base:
|
||||
|
||||
1. Copy the data to be put into the Knowledge Base into the `~myai/data` folder.
|
||||
2. By entering `/Knowledge add dir`, the system will ask you to input a local directory to be imported into the Knowledge Base. Note that OpenDAN runs in a container by default, so $dir is a path relative to the container. If you want to add data from the local disk, you need to mount the local data to the container first.
|
||||
|
||||
OpenDAN will continuously analyze the files in the Knowledge Base folder in the background, and the analysis results are saved in the ~/myai/knowledge directory. After deleting this directory, the system will re-analyze the files that have been added to the Knowledge Base. Since OpenDAN's Knowledge Base is still in its early stages, it currently only supports analyzing and recognizing text files, pictures, short videos, etc. In the future, OpenDAN will support all mainstream file formats and try to import all existing information into Knowledge. You can query the running status of the Knowledge Base analysis task in aios_shel with the following command:
|
||||
|
||||
```
|
||||
/Knowledge journal
|
||||
```
|
||||
|
||||
### Meet Your Personal Information Assistant, Mia
|
||||
|
||||
Then we can access the Knowledge Base through the Agent "Mia",
|
||||
|
||||
```
|
||||
/open Mia
|
||||
```
|
||||
|
||||
Try to communicate with Mia! I think this will bring a completely different experience! The information Mia finds will be displayed in the following way:
|
||||
|
||||
```
|
||||
{"id": "7otCtsrzbAH3Nq8KQGtWivMXV5p54qrv15xFZPtXWmxh", "type": "image"}
|
||||
```
|
||||
|
||||
You can use the `/knowledge query 7otCtsrzbAH3Nq8KQGtWivMXV5p54qrv15xFZPtXWmxh` command to call the local file viewer to view the results.
|
||||
|
||||
We recommend integrating Mia into Telegram, so Mia will directly display the query results in the form of images, which is more convenient to use~
|
||||
|
||||
### Embeding Pipeline
|
||||
|
||||
The process by which the Knowledge Base reads and analyzes files to generate information that the Agent can access is called Embedding. This process requires certain computational resources. According to our tests, the Embedding Pipeline built by OpenDAN based on "Sentence Transformers" can run on the vast majority of types of machines. The difference between machines of different capabilities mainly lies in the speed and quality of Embedding. Friends who understand the progress of OpenDAN may know that we have also supported cloud Embedding during the implementation process to completely reduce the minimum system performance requirements of OpenDAN. However, considering the large amount of personal privacy data involved in the Embedding process, we decided to turn off the cloud Embedding feature. Those in need can modify the source code to open cloud Embedding, allowing OpenDAN to work on very low-performance devices.
|
||||
|
||||
Unfortunately, there is no unified Embedding standard now, so the results generated by different Embedding Pipelines are not compatible with each other. This means that once the Embedding Pipeline is switched, all the information in the Knowledge Base needs to be rescanned.
|
||||
|
||||
## bash@ai
|
||||
|
||||
If you have a certain engineering background, by letting the Agent execute bash commands, you can also quickly and easily give OpenDAN access to your private data.
|
||||
|
||||
Use the command
|
||||
|
||||
```
|
||||
/open ai_bash
|
||||
```
|
||||
|
||||
to activate ai_bash. From there, you can execute traditional bash commands within the aios_shell command line. Plus, you'll have the ability to use smart commands. For example, to search for files, instead of using the 'find' command, you can simply say:
|
||||
|
||||
```
|
||||
Help me find all files in ~/Documents that contain OpenDAN.
|
||||
```
|
||||
|
||||
Pretty cool, right?
|
||||
|
||||
By default, OpenDAN operates inside a container, meaning ai_bash can only access files within that docker container. While this provides a relative level of security, we still advise caution. Do not expose the ai_bash agent recklessly, as it could pose potential security risks."
|
||||
|
||||
## Why Do We Need Personal AIOS?
|
||||
|
||||
Many people will first think of privacy, which is an important reason, but we don't think this is the real reason why people leave ChatGPT and choose Personal AIOS. After all, many people are not sensitive to privacy today. Moreover, today's platform manufacturers generally use your privacy to make money silently, and rarely really leak your privacy.
|
||||
|
||||
We believe the real value of Personal AIOS lies in:
|
||||
|
||||
1. Cost is an important determinant. LLM is a very powerful, clearly bounded core component, the CPU of the new era. From a product and business perspective, ChatGPT-like products only allow it to be used in limited ways. It reminds me of the era when minicomputers first appeared and everyone used the system in a time-sharing manner: useful, but limited. To truly realize the value of LLM, we need to allow everyone to have their own LLM and freely use LLM as the underlying component of any application. This requires a new operating system built around LLM to re-abstract applications (Agent/Workflow) and the resources used by applications (computing power, data, environment).
|
||||
|
||||
2. When you have LLM, when you can put LLM in front of every compute, you will see the real treasure! The current ChatGPT's extension of LLM capabilities through plugins is very limited in both capabilities and boundaries. This is due to both commercial cost reasons and the legal boundary issues of traditional cloud services: the platform bears too much responsibility. But by using LLM in Personal AIOS, you can freely connect natural language, LLM, existing services, private data, and smart devices without worrying about privacy leaks and liability issues (you bear the consequences of authorizing LLM yourself)!
|
||||
|
||||
OpenDAN is an open-source project, let's define the future of Humans and AI together!
|
||||
@@ -0,0 +1,9 @@
|
||||
|
||||
# aios kernel部分的核心介绍
|
||||
|
||||
## 核心理论
|
||||
|
||||
|
||||
## 以 workflow为核心组织ai agent
|
||||
定义了未来工程师使用LLM构造应用的方法
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 136 KiB |
@@ -0,0 +1,76 @@
|
||||
<mxfile host="65bd71144e">
|
||||
<diagram id="C5RBs43oDa-KdzZeNtuy" name="Page-1">
|
||||
<mxGraphModel dx="711" dy="944" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="827" pageHeight="1169" math="0" shadow="0">
|
||||
<root>
|
||||
<mxCell id="WIyWlLk6GJQsqaUBKTNV-0"/>
|
||||
<mxCell id="WIyWlLk6GJQsqaUBKTNV-1" parent="WIyWlLk6GJQsqaUBKTNV-0"/>
|
||||
<mxCell id="ia-fclrWNqMUroYMcw65-37" value="" style="rounded=0;whiteSpace=wrap;html=1;dashed=1;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||
<mxGeometry x="340" y="345" width="290" height="105" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="ia-fclrWNqMUroYMcw65-36" value="" style="rounded=0;whiteSpace=wrap;html=1;dashed=1;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||
<mxGeometry x="90" y="345" width="250" height="105" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="ia-fclrWNqMUroYMcw65-1" value="local compute <br>node" style="shape=step;perimeter=stepPerimeter;whiteSpace=wrap;html=1;fixedSize=1;fillColor=#6d8764;fontColor=#ffffff;strokeColor=#3A5431;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||
<mxGeometry x="90" y="380" width="120" height="50" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="ia-fclrWNqMUroYMcw65-31" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="ia-fclrWNqMUroYMcw65-2" target="ia-fclrWNqMUroYMcw65-28" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="ia-fclrWNqMUroYMcw65-2" value="my device<br>&nbsp;compute <br>node" style="shape=step;perimeter=stepPerimeter;whiteSpace=wrap;html=1;fixedSize=1;fillColor=#6d8764;fontColor=#ffffff;strokeColor=#3A5431;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||
<mxGeometry x="210" y="380" width="120" height="50" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="ia-fclrWNqMUroYMcw65-3" value="service compute<br>node" style="shape=step;perimeter=stepPerimeter;whiteSpace=wrap;html=1;fixedSize=1;fillColor=#e3c800;fontColor=#000000;strokeColor=#B09500;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||
<mxGeometry x="370" y="380" width="120" height="50" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="ia-fclrWNqMUroYMcw65-4" value="untruest compute<br>node" style="shape=step;perimeter=stepPerimeter;whiteSpace=wrap;html=1;fixedSize=1;fillColor=#fa6800;fontColor=#000000;strokeColor=#C73500;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||
<mxGeometry x="490" y="380" width="130" height="50" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="ia-fclrWNqMUroYMcw65-18" value="TaskParams:json" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#60a917;fontColor=#ffffff;strokeColor=#2D7600;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||
<mxGeometry x="280" y="50" width="120" height="40" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="ia-fclrWNqMUroYMcw65-19" value="padding input:buffers" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#0050ef;fontColor=#ffffff;strokeColor=#001DBC;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||
<mxGeometry x="280" y="130" width="120" height="70" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="ia-fclrWNqMUroYMcw65-22" value="Refer:Name Data" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#6a00ff;fontColor=#ffffff;strokeColor=#3700CC;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||
<mxGeometry x="280" y="90" width="120" height="40" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="ia-fclrWNqMUroYMcw65-23" value="aios_kernel<br>(scheduler)" style="shape=hexagon;perimeter=hexagonPerimeter2;whiteSpace=wrap;html=1;fixedSize=1;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||
<mxGeometry x="280" y="250" width="120" height="50" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="ia-fclrWNqMUroYMcw65-24" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="ia-fclrWNqMUroYMcw65-19" target="ia-fclrWNqMUroYMcw65-23" edge="1">
|
||||
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||
<mxPoint x="430" y="240" as="sourcePoint"/>
|
||||
<mxPoint x="480" y="190" as="targetPoint"/>
|
||||
</mxGeometry>
|
||||
</mxCell>
|
||||
<mxCell id="ia-fclrWNqMUroYMcw65-25" value="" style="endArrow=classic;html=1;rounded=0;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" source="ia-fclrWNqMUroYMcw65-23" target="ia-fclrWNqMUroYMcw65-2" edge="1">
|
||||
<mxGeometry width="50" height="50" relative="1" as="geometry">
|
||||
<mxPoint x="420" y="350" as="sourcePoint"/>
|
||||
<mxPoint x="470" y="300" as="targetPoint"/>
|
||||
</mxGeometry>
|
||||
</mxCell>
|
||||
<mxCell id="ia-fclrWNqMUroYMcw65-26" value="Owne Nodes" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||
<mxGeometry x="90" y="345" width="100" height="30" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="ia-fclrWNqMUroYMcw65-27" value="Rental Nodes" style="text;html=1;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;whiteSpace=wrap;rounded=0;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||
<mxGeometry x="505" y="345" width="100" height="30" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="ia-fclrWNqMUroYMcw65-28" value="ResultParams:json" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#60a917;fontColor=#ffffff;strokeColor=#2D7600;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||
<mxGeometry x="280" y="516" width="120" height="40" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="ia-fclrWNqMUroYMcw65-29" value="padding result:buffers" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#0050ef;fontColor=#ffffff;strokeColor=#001DBC;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||
<mxGeometry x="280" y="596" width="120" height="70" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="ia-fclrWNqMUroYMcw65-30" value="Refer:Name Data" style="rounded=0;whiteSpace=wrap;html=1;fillColor=#6a00ff;fontColor=#ffffff;strokeColor=#3700CC;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||
<mxGeometry x="280" y="556" width="120" height="40" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="ia-fclrWNqMUroYMcw65-34" value="<br><span style="color: rgb(209, 213, 219); font-family: Söhne, ui-sans-serif, system-ui, -apple-system, &quot;Segoe UI&quot;, Roboto, Ubuntu, Cantarell, &quot;Noto Sans&quot;, sans-serif, &quot;Helvetica Neue&quot;, Arial, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;, &quot;Segoe UI Symbol&quot;, &quot;Noto Color Emoji&quot;; font-size: 16px;">Some tasks can be divided, and this can be handled either at the higher layer for splitting and merging, or by the aios_kernel.</span>" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;labelBackgroundColor=default;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||
<mxGeometry x="450" y="80" width="170" height="150" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="ia-fclrWNqMUroYMcw65-35" value="<span style="color: rgb(209, 213, 219); font-family: Söhne, ui-sans-serif, system-ui, -apple-system, &quot;Segoe UI&quot;, Roboto, Ubuntu, Cantarell, &quot;Noto Sans&quot;, sans-serif, &quot;Helvetica Neue&quot;, Arial, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;, &quot;Segoe UI Symbol&quot;, &quot;Noto Color Emoji&quot;; font-size: 16px;">The core of the scheduler is to select a compute_node to complete the computing task based on a variety of factors, such as the known status of the compute_node, the distance to the Refer data, the cost of computation, and the speed of operation. It can also include some iterative retry logic.</span>" style="text;html=1;strokeColor=none;fillColor=none;align=left;verticalAlign=middle;whiteSpace=wrap;rounded=0;labelBackgroundColor=default;" parent="WIyWlLk6GJQsqaUBKTNV-1" vertex="1">
|
||||
<mxGeometry x="450" y="470" width="200" height="250" as="geometry"/>
|
||||
</mxCell>
|
||||
</root>
|
||||
</mxGraphModel>
|
||||
</diagram>
|
||||
</mxfile>
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 166 KiB |
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,20 @@
|
||||
# Objected knowleadge base, a specialized implemention for emails
|
||||
## Vectorized Knowledge
|
||||
Large language models are trained on general corpora and without fine-tuning on user-specific data, they struggle to utilize user-related context effectively.
|
||||
|
||||
Users accumulate a vast amount of content that reflects their personality during their regular internet usage. This includes personal photos, tweets, Facebook posts, emails, etc. While it's possible to include all this content in the prompt during each interaction with the large language model, this approach is costly and can easily reach the token limit.
|
||||
|
||||
A common solution is to generate feature vectors from this content using word embedding techniques and store them in a vector database. During an interaction, the vector that is most relevant to the prompt is retrieved from the database, merged with the prompt, and then passed to the large language model.
|
||||
|
||||
We refer to this vectorized content as "knowledge".
|
||||
|
||||
## Objected knownleadge base
|
||||
In a personal AI system, to build a user's own knowledge base, we first need to implement various spider programs to crawl and retrieve all user-related data. Modern web content is typically rich text, including text, images, videos, hyperlinks, etc. Organizing this rich text in a tree-like structure similar to HTML is necessary, hence the need to introduce an object structure to represent this content.
|
||||
|
||||
Different parts of this content cannot be vectorized using the same embedding model. For instance, text and images, as well as the content of an image and its EXIF information, need separate embeddings. This means that in the vector database, the same content may have multiple vector values, and a row can represent a whole content item or just a part of it.
|
||||
|
||||
We need a comprehensive object structure to represent the hierarchy and relationships of content, as well as to implement the indexing and storage of objects. In the Minimum Viable Product (MVP) version, we'll implement a specialized solution for email content. In future versions, we can generalize this to handle other types of content, such as Facebook posts, tweets, etc.
|
||||
|
||||
## Agent with knowleadge base
|
||||
At the same time, we also need to explore the paradigm of using the knowledge base in Agents and workflows, so that the agent can better complete tasks in interaction with users through the context provided by the knowledge base.
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 96 KiB |
@@ -0,0 +1,500 @@
|
||||
# Overview
|
||||
|
||||
The core goal of version 0.5.1 is to turn the concept of AIOS into code and get it up and running as quickly as possible. After three weeks of development, our plans have undergone some changes based on the actual progress of the system. Under the guidance of this goal, some components do not need to be fully implemented. Furthermore, based on the actual development experience from several demo Intelligent Applications, we intend to strengthen some components. This document will explain these changes and provide an update on the current development progress of MVP(0.5.1,0.5.2)
|
||||
|
||||
The previous plan, please see here: [MVP Plan](./mvp%20plan.md)
|
||||
|
||||
# Progress Status of MVP
|
||||
|
||||
- Each module includes whether the current version goals have been met, the current person in charge, and workload assessment.
|
||||
- Modules that are not marked for version 0.5.2 and do not have a designated person in charge are modules for which we are currently recruiting contributors.
|
||||
- Modules that have not been completed but already have a designated person in charge are modules that are currently in development.
|
||||
|
||||
- [x] AIOS Kernel
|
||||
- [x] Agent,@waterflier, A2
|
||||
- [ ] Optimization of system prompts,A2
|
||||
- [x] Workflow,@waterflier, A2
|
||||
- [x] AI Environments,@waterflier, A2
|
||||
- [x] Celender Environment,@waterflier, S2
|
||||
- [ ] Compatible with common Celender services(0.5.2),A2
|
||||
- [ ] Microsoft Outlook Calendar, S2
|
||||
- [ ] Google Calendar, S2
|
||||
- [ ] Apple Calendar, S2
|
||||
- [ ] Workspace Environment(0.5.2) @waterflier, A2
|
||||
- [ ] AI Functions,@waterflier,A2
|
||||
- [ ] Basic AI Functions(0.5.2)
|
||||
- [x] AI BUS,@waterflier, A2
|
||||
- [x] Chatsession,@waterflier, S2
|
||||
- [ ] Knowlege Base,@lurenpluto ,@photosssa, A8
|
||||
- [ ] Personal Models(>0.5.2),A8
|
||||
- [ ] Frame Services(0.5.2)
|
||||
- [ ] Kernel Service
|
||||
- [ ] System-Call Interface,A2
|
||||
- [ ] Name Service,A4
|
||||
- [ ] Node Daemon,A2
|
||||
- [ ] ACL Control,A4
|
||||
- [ ] Contact Manager,A2
|
||||
- [ ] Runtime Context (0.5.2),A4
|
||||
- [ ] Package System,@waterflier, A2+S4
|
||||
- [x] AI Compute System,@waterflier, A2
|
||||
- [ ] Scheduler,@streetycat, A2
|
||||
- [x] LLM
|
||||
- [x] GPT4 (Cloud),@waterflier, S1
|
||||
- [ ] LLaMa2,@streetycat, A2
|
||||
- [ ] Claude2, S2
|
||||
- [ ] Falcon2, S2
|
||||
- [ ] MPT-7B, S2
|
||||
- [ ] Vicuna, S2
|
||||
- [ ] Embeding,@photosssa,@lurenpluto , A4
|
||||
- [x] Txt2img,@glen0125,A4
|
||||
- [ ] Img2txt(0.5.2),A3
|
||||
- [ ] Txt2voice,A3
|
||||
- [ ] Voice2txt, @wugren,A3
|
||||
- [ ] Language Translate (Pending)
|
||||
- [x] Storage System
|
||||
- [ ] DFS (0.5.2),A4
|
||||
- [ ] Object Storage, @lurenpluto ,A2
|
||||
- [ ] D-RDB (0.5.2),A2
|
||||
- [ ] D-VDB,@lurenpluto , A4
|
||||
- [ ] Embeding Piplines,@photosssa, A2
|
||||
- [ ] Network Gateway,A6
|
||||
- [x] NDN Client, @waterflier, A1
|
||||
- [ ] Build-in Service
|
||||
- [x] Spider,@alexsunxl, A2
|
||||
- [x] E-mail Spider,@alexsunxl, S2
|
||||
- [ ] Telegram Spider,S2
|
||||
- [ ] Twitter Spider (0.5.2)
|
||||
- [ ] Facebook Spider (0.5.2)
|
||||
- [ ] Agent Message Tunnel (0.5.2)
|
||||
- [ ] E-mail Tunnel,A2
|
||||
- [ ] Telegram Tunnel,S2
|
||||
- [ ] Discord Tunnel,S2
|
||||
- [ ] Home IoT Environment (0.5.2), A4
|
||||
- [ ] Compatible Home Assistant (0.5.2), A4
|
||||
- [ ] Build-in Agents/Apps
|
||||
- [ ] Agent: Personal Information Assistant,@photosssa,@lurenpluto , A2
|
||||
- [ ] Agent: Bulter Jarvis,@waterflier, A2
|
||||
- [ ] App: Personal Station (0.5.2),A4+S4
|
||||
- [ ] UI
|
||||
- [x] CLI UI (aios_shell),@waterflier,S2
|
||||
- [ ] Web UI (0.5.2),A4+S4
|
||||
- [ ] 0.5.1 Integration Test (Senior*3)
|
||||
- [x] Workflow -> AI Agent -> AI Agent,@waterflier,S1
|
||||
- [ ] Spider -> Pipline -> Knowledge Base,@photosssa,S2
|
||||
- [ ] AI Agent <- Functions <- Knowledge Base,@lurenpluto,S2
|
||||
- [ ] SDK
|
||||
- [x] Workflow SDK,@waterflier, A2
|
||||
- [ ] AI Environments SDK (0.5.2), A2
|
||||
- [ ] Compute Kernel SDK (0.5.2), A2
|
||||
- [ ] Document (>0.5.2)
|
||||
- [ ] System design document, including the design document of each subsystem
|
||||
- [ ] Installation/use document for end users
|
||||
- [ ] SDK document for developers
|
||||
|
||||
|
||||
The following is the introduction of the adjustment of each component after the current implementation.
|
||||
|
||||
# AIOS Kernel
|
||||
|
||||
Define some of the important basic concepts of Intelligent Applications running on OpenDAN
|
||||
|
||||
## Agent
|
||||
|
||||
Agent is the core concept of the system, created through appropriate LLM, prompt words, and memory. Agents support our vision of a new relationship between humans and computation in the future:
|
||||
```
|
||||
Human <-> Agent <-> Compute
|
||||
```
|
||||
Agents form the basis of future intelligent applications. From the user's perspective, the strength of AIOS is primarily determined by "how many agents with different capabilities it has."
|
||||
The above process has now been implemented. In practice, I found a key issue is that we need to continuously seek the optimal solution. This issue directly relates to how application developers of OpenDAN build intelligent applications, so I think it has a high priority.
|
||||
|
||||
### Optimization of system prompts
|
||||
|
||||
The goal is to allow Agents to communicate with other Agents (forming a team), call Functions at the right time, and read/write status through the environment at the right time, using prompt words. The existing implementation is usually:
|
||||
|
||||
```
|
||||
When you decide to communicate with a work group, please use : sendmsg(group_name, content).
|
||||
```
|
||||
|
||||
Our optimization direction is:
|
||||
|
||||
1. To allow Agents to initiate calls accurately
|
||||
2. To use as few precious prompt word resources as possible.
|
||||
|
||||
If there are already systematic studies in this field, introductions are also welcome!
|
||||
|
||||
## Workflow
|
||||
|
||||
Workflow has realized the concept of allowing multiple Agents to play different roles and collaboratively solve complex problems within an organization. It is also the main form of intelligent applications on OpenDAN. Compared to a single Agent, building a team composed of Agents can effectively solve three inherent problems of LLM:
|
||||
|
||||
1. The prompt word window will grow, but it will remain limited for a long time.
|
||||
2. Like humans, Agents trained on different corpora and algorithms will have different personalities and will excel in different roles.
|
||||
3. The inference results of LLM are uncontrollable, so accuracy cannot be guaranteed. Just like humans make mistakes, the collaboration of multiple Agents is needed to improve accuracy.
|
||||
|
||||
The basic framework of Workflow has been completed (which is also the core of version 0.5.1). Following the subsequent SDK documentation, we now have a basic framework for third-party developers to develop applications on OpenDAN.
|
||||
|
||||
## AI Environments
|
||||
|
||||
Environments provide an abstraction for AI Agents to access the real world.
|
||||
|
||||
Environments include properties, events, methods (Env.Function), and come with natural language descriptions that Agents can understand. This allows AI Agents to understand the current environment and when to access it. For example, an Agent planning a trip needs to understand the real weather conditions at the destination in the future to make the right decisions. This weather condition needs to be provided to the Agent through Environments.
|
||||
|
||||
The events in Environments also provide logic for the autonomous work of Agents. For example, an Agent can track changes in the user's schedule and date, automatically helping the user plan and track the specific itinerary for the day.
|
||||
|
||||
### Celender Environment
|
||||
|
||||
The system's default environment, which can access the current time, the user's schedule, and the weather information at a specific location. It also contains some important and basic user information, including home address and office address.
|
||||
|
||||
#### Compatible with common Celender services. (0.5.2)
|
||||
|
||||
- Microsoft Outlook Calendar
|
||||
- Google Calendar
|
||||
- Apple Calendar
|
||||
|
||||
### Workspace Environment (0.5.2)
|
||||
|
||||
A file system-based workspace environment that allows the Agent to read/write files at appropriate times.
|
||||
|
||||
## AI Functions
|
||||
|
||||
Function is a core concept of AIOS, providing the Agent with descriptions of suitable callable Functions, allowing the Agent to invoke Functions at the right time. Through Functions, the Agent gains "execution power," rather than just being an advisor that only provides suggestions. The Function framework allows third-party developers to develop and publish Functions, supporting Agents and Workflows to have a list of available Functions, through which they can build appropriate prompt words, enabling the Agent to invoke Functions at the right time.
|
||||
|
||||
**Under development.**
|
||||
|
||||
### Basic AI Functions （0.5.2）
|
||||
|
||||
There are already a plethora of basic services in the world, such as querying the weather at a specific location at a specific time, checking hotel prices, or booking plane tickets. The system should separate the definition and implementation of Basic (generic) Functions, allowing Agent developers to implement common scenarios with generic logic. The definition of generic Functions is undoubtedly similar to standard setting work.
|
||||
|
||||
I know that many other projects have done a lot of work in this field, and ChatGPT also has dedicated function support. What we need to do is to find the open standards that are closest to our goals and then integrate them.
|
||||
|
||||
## AI BUS
|
||||
|
||||
The AI BUS connects various conceptual entities of OpenDAN. For example, if Agent A wants to send a message to another Agent B and wait for the processing result of the message, it can simply use the AI BUS:
|
||||
|
||||
```python
|
||||
resp = await AIBus.send_msg(agentA,agentB,msg)
|
||||
```
|
||||
|
||||
The abstraction of AI BUS allows different Agents to choose suitable physical hosts to run according to the system's needs. This is also why we define AIOS as a "Network OS". All entities registered on the AI BUS can be accessed via the AI BUS interface. As needed, we will also persist the messages in the AI BUS, so that when a distributed system experiences regular failures, it can continue to work after being pulled up again.
|
||||
|
||||
The concept of AI BUS has many similarities with traditional MessageQueues.
|
||||
|
||||
## Chatsession
|
||||
|
||||
Intuitively, ChatSession saves the "chat history". The chat history is currently the natural source of Agent Memory capability.
|
||||
Determining a ChatSession has three key attributes: Owner, Topic, and Remote. An operation where A sends a message to B and gets B's reply will generate two messages, and save them in two different ChatSessions.
|
||||
|
||||
Currently, ChatSession is saved based on sqlite. After the Zone-level D-RDB is set up in the future, it will be migrated to RD-DB.
|
||||
|
||||
## Knowlege Base
|
||||
|
||||
Provide a unified interface, support switching vector database kernel
|
||||
Integrate open source vector database (pay attention to Lience selection)
|
||||
When designing the interface, prepare for future access control
|
||||
|
||||
**Under development.**
|
||||
|
||||
## Personal Models （>0.5.2）
|
||||
|
||||
The goal of this subsystem is to support users in training models based on their own data, including subsequent usage, management, deployment, and other operations of the model. In the early stages, invoking this module and adding new models should be operations performed by advanced users.
|
||||
|
||||
It is still uncertain whether this module will be actively used in intelligent applications.
|
||||
|
||||
# Frame Services
|
||||
|
||||
The implementation offers a range of fundamental services for traditional Network OS. It connects users' devices to the same Zone via the network and provides a unified abstraction for application access. This component serves as a basic framework and computing resource for the operation of intelligent applications on the upper layer. On the lower layer, it connects various types of hardware through different protocols, integrates resources, and offers a unified abstraction for intelligent applications to access.
|
||||
|
||||
## Kernel Service (0.5.2)
|
||||
|
||||
The Kernel Service implements the System Calls for OpenDAN and provides a "kernel mode" abstraction. In version 0.5.1, since this component is not yet implemented, all code—whether system services or application code—runs in kernel mode.
|
||||
|
||||
In the future, we plan to maintain the system running in this mode for an extended period, as it facilitates debugging.
|
||||
|
||||
The Kernel Service is mainly composed of the following component:
|
||||
|
||||
### System-Call Interface
|
||||
|
||||
Centralizes the provision and management of system call interfaces.
|
||||
|
||||
### Name Service
|
||||
|
||||
It is the most crucial foundational state service in a cluster (Zone) comprised of all the user's devices. As the core service of the Zone, it provides the most basic guarantee for the availability and reliability of all services within the Zone. When a user needs to restore the Zone from a backup, the Name Service is the first service to be restored.
|
||||
|
||||
Its functionality is similar to that of `etcd` but includes a on-chain component. From a deployment standpoint, it needs to be operationally optimized for small clusters made up of consumer-level user devices.
|
||||
|
||||
### Node Daemon
|
||||
|
||||
It is a foundational service that runs on all devices that join the Zone, responding to essential kernel scheduling commands. It adjusts the services and data running on that particular device.
|
||||
|
||||
### ACL Control (>0.5.2)
|
||||
|
||||
Another essential foundational service of the kernel, it is responsible for the overall management of permissions related to users, applications, and data. The Runtime Context reads the relevant information and implements proper isolation.
|
||||
|
||||
### Contact Manager
|
||||
|
||||
From the perspectives of permission control and some early application scenarios, understanding the user's basic interpersonal relationships is an important component of OpenDAN's intelligent permission system. Therefore, we provide a contact management component at the system kernel layer. This component can be considered an upgraded version of the traditional operating system's "User Group" module.
|
||||
|
||||
## Runtime Context (0.5.2)
|
||||
|
||||
It serves as the runtime container for user-mode code, offering isolation guarantees for user-mode code.
|
||||
|
||||
Depending on the type of service, we offer three different Runtime Contexts. The most commonly used is Docker, followed by virtual machines, and finally, entire physical machines.
|
||||
|
||||
## Package System
|
||||
|
||||
The Package Manager is a fundamental component of the system for managing Packages. The sub system provides fundamental support for packaging, publishing, downloading, verifying, installing, and loading folders containing required packages under different scenarios. Based on relevant modules, it's easy to build a package management system similar to apt/pip/npm.
|
||||
|
||||
The system design has deeply referenced Git and NDN networks. The distinction between client and server is not that important. Through cryptography, it achieves decentralized trustworthy verification. Any client can become an effective repo server through simple configuration.
|
||||
|
||||
Based on the Package System, we can implement the publishing, downloading, and installation of extendable foundational entities such as Agents, Functions, and Environments. This enables the creation of an app store on OpenDAN.
|
||||
|
||||
**Under development.**
|
||||
|
||||
# AI Compute System
|
||||
|
||||
The purpose of designing Compute System is to enable our users to use their computational resources more efficiently. These computational resources can come from devices they own (such as their workstations and gaming laptops), as well as from cloud computing and decentralized computing networks.
|
||||
|
||||
[](compute_task.drawio)
|
||||
|
||||
The interface of this component is designed from the perspective of the model user rather than the model trainer. The basic form of its interface is:
|
||||
|
||||
```python
|
||||
compute_kernel.do_compute(function_name, model_name,args)
|
||||
```
|
||||
|
||||
## Scheduler
|
||||
|
||||
The goal of the Scheduler component is to select an appropriate ComputeNode to run tasks based on the tasks in the task queue and the known status of all ComputeNodes (which may be delayed). In the current version (0.5.1), the implementation of the Scheduler is only to get the system up and running. In the next version (0.5.2), the overall framework for computing resource scheduling needs to be established.
|
||||
|
||||
## LLM
|
||||
|
||||
LLM support is the system's most core functionality. OpenDAN requires that there be at least one available LLM computing node in the system. The supported interfaces are as follows:
|
||||
```
|
||||
def llm_completion(self,prompt:AgentPrompt,mode_name:Optional[str] = None,max_token:int = 0):
|
||||
```
|
||||
In the current era, many teams are working hard to develop new LLMs . We will also actively integrate these LLMs into OpenDAN.
|
||||
|
||||
- [x] GPT4 (Cloud)
|
||||
- [ ] LLaMa2 **Under development.**
|
||||
- [ ] Claude2
|
||||
- [ ] Falcon2
|
||||
- [ ] MPT-7B
|
||||
- [ ] Vicuna
|
||||
|
||||
|
||||
## Embeding
|
||||
|
||||
Provides computational support for the vectorization of different types of user data. The specific algorithms supported depend on the requirements of the entire pipeline.
|
||||
|
||||
***Under development.***
|
||||
|
||||
## Txt2img
|
||||
|
||||
Generate images based on text descriptions. According to the implementation mode, we can interface with a cloud-based implementation and a local implementation.
|
||||
|
||||
The local implementation will definitely use Stable Diffusion.
|
||||
|
||||
***Under development.***
|
||||
|
||||
## Img2txt (>0.5.2)
|
||||
|
||||
Generate appropriate text descriptions for the specified images.
|
||||
|
||||
## Txt2voice
|
||||
|
||||
Generate voice based on specified text, using a selected model (the focus is on personal models), and guided by certain emotional cue words.
|
||||
|
||||
***To be developed***
|
||||
|
||||
## Voice2txt
|
||||
|
||||
Extract text information from a segment of audio (or video) through speech recognition.
|
||||
|
||||
***To be developed***
|
||||
|
||||
## Language Translate
|
||||
|
||||
Translate a segment of text into a specified target language.
|
||||
|
||||
Since LLM itself is developed based on the foundation of translation, I am currently considering whether it is necessary to provide a text translation interface within the computing kernel. Following the principle of not adding entities if they are not needed, it can be postponed from development.
|
||||
***pending***
|
||||
|
||||
# Storage System
|
||||
|
||||
The file system (state storage) has always been a critical part of operating systems. Its implementation directly impacts the system's reliability and performance. The challenge of this section is how to transfer key technologies that are already mature in traditional cloud computing to clusters composed of consumer-level electronic devices with low operational maintenance, while still maintaining sufficient reliability and performance. The implementation of the subsystems in this section is of limited stability. Therefore, I believe the focus of OpenDAN in the early stages for this section should be on establishing stable interfaces to get the system running as quickly as possible, with independent improvements to be made in the future.
|
||||
|
||||
From the standpoint of trade-offs, our priorities are:
|
||||
|
||||
- Abandoning continuous consistency guarantees, the system only provides strong assurance for reliability up to "backup points." This means we allow the loss of some newly added data if the system experiences a failure.
|
||||
|
||||
- Allowing downtime, considering the consumer-level power supply, a short period of unavailability of the system itself will not have a significant impact. We can stop the service for backup/migration when necessary.
|
||||
|
||||
## DFS
|
||||
|
||||
Distributed file system, combining the public storage space on all devices to form a highly reliable, highly available file system.
|
||||
|
||||
## Object Storage
|
||||
|
||||
Distributed object storage, and based on MapObject, it implements trustworthy RootState management.
|
||||
|
||||
(MapObject and RootState is a concept from CYFS)
|
||||
|
||||
**Under development.**
|
||||
|
||||
## D-RDB
|
||||
|
||||
Distributed relational database, providing highly reliable and highly available relational database services (mainly used for OLTP - Online Transaction Processing). We do not encourage application developers to use RDB on a large scale; the main reason for offering this component is for compatibility considerations.
|
||||
|
||||
***Pending.***
|
||||
|
||||
## D-VDB
|
||||
|
||||
Distributed vector database, which currently appears to be the core foundational component of the Knowledge Base library.
|
||||
|
||||
***Under development.***
|
||||
|
||||
# Embeding Piplines
|
||||
|
||||
Read appropriate Raw Files and Meta Data from the specified location in the Storage System. After passing through a series of Embedding Pipelines, save the results to the Vector Database as defined by the Knowledge Base.
|
||||
|
||||
***Under development.***
|
||||
|
||||
# Network Gateway (0.5.2)
|
||||
|
||||
Obtain user data by recognizing network data.
|
||||
The Gateway also provides an external access entrance for the entire system, and access control can be unified.
|
||||
Provides the bus abstraction in the network operating system (the network cable is the bus), devices within the Zone are recognized by the system as plug-and-play devices, and can be called by applications/Agents
|
||||
|
||||
## NDN Client
|
||||
|
||||
AI-related models are all quite large, so we offer a download tool based on NDN (Named Data Networking) theory to replace curl. The NDN Client will continue to support new Content-Based protocols in the future, allowing OpenDAN developers to publish large packages more quickly, at lower costs, and more conveniently.
|
||||
|
||||
# Build-in Service
|
||||
|
||||
The basic functions of the system implemented by "user mode" can be regarded as pre -installed applications of the system.Let the system have basic availability without installing any intelligent applications.
|
||||
We should build built-in applications for 1,2 early preset scenarios, rather than all possible scenarios.This allows us to run the system faster and allow us to discover the shortcomings of the system faster, so as to improve the system faster.
|
||||
|
||||
## Spider
|
||||
|
||||
A series of reptiles are provided to help users import their data into the system.
|
||||
|
||||
### E-mail Spider
|
||||
|
||||
The most basic spider is used to capture user mail data.The main purpose of this is to determine the general data format(include text,image,contact) and location to save the grabbed data.
|
||||
|
||||
### Telegram Spider
|
||||
|
||||
Allow users to capture their own Telegram chat records and save them in the Knowlege Base
|
||||
|
||||
**To be developed.**
|
||||
|
||||
### Twitter Spider (0.5.2)
|
||||
|
||||
Allows users to scrape their own Twitter data and save it in the Knowledge Base.
|
||||
|
||||
### Facebook Spider (0.5.2)
|
||||
|
||||
Allows users to scrape their own Facebook data and save it in the Knowledge Base.
|
||||
|
||||
## Agent Message Tunnel （0.5.2）
|
||||
|
||||
The original ROBOT module, after considering its actual function, was renamed the Agent Message Tunnel.
|
||||
This is the default function supported by the system. It supports users to configure different message channels for different Agent/Workflow, so that users can interact with Agent/Worflow through existing software/services.From the perspective of product, the goal of this module can use the core function of OpenDAN without installing any new software on the one hand. On the other hand, it also creates a stronger mental model for users: My Agent can registered social account, so that Agent has his own identity in the virtual world.
|
||||
|
||||
### E-mail Tunnel
|
||||
|
||||
Let Agent have its own email account. After registration, users can interact with Agent through mail.
|
||||
|
||||
### Telegram Tunnel
|
||||
|
||||
Let Agent have his own Telegram account. After registration, users can interact with Agent through Telegram.
|
||||
|
||||
### Discord Tunnel
|
||||
|
||||
Let Agent have its own discord account. After registration, users can interact with agent through Discord.
|
||||
|
||||
## Home IoT Environment (0.5.2)
|
||||
|
||||
We've implemented a significant built-in environment: the Home Environment. Through this environment, the AI Agent can access real-time status of the home via installed IoT devices, including reading temperature and humidity information, accessing security camera data, and controlling smart devices in the home. This allows users to better manage a large number of smart IoT devices through AI technology. For instance, a user can simply tell the Agent, "Richard is coming over to watch a movie this afternoon," and the AI Agent will automatically read the security camera data, recognize Richard upon arrival, turn on the home projector, close the curtains, and turn on the wall lights.
|
||||
|
||||
Thanks to LLM's powerful natural language understanding, all we need to do is connect a smart microphone to the Home Environment and configure a simple voice-to-text feature. This makes it easy to implement a privately deployed and very intelligent version of Alexa.
|
||||
|
||||
In terms of system design, we use the Home Environment as an intermediary layer, freeing OpenDAN from having to spend energy on dealing with compatibility issues with various existing, complex IoT protocols. This keeps the system simple and makes it easier to expand.
|
||||
|
||||
### Compatible Home Assistant
|
||||
|
||||
Home Assistant is a well-known, open-source IoT system. We could consider implementing the Home Environment based on the Home Assistant's API.
|
||||
|
||||
# Build-in Agents/Apps
|
||||
|
||||
Once users have installed OpenDAN, it should have some basic functionalities, even without the installation of any third-party smart applications. These basic functions are provided via built-in Agents/Applications. Built-in applications have two important implications for OpenDAN:
|
||||
|
||||
1. They provide a developer's perspective to scrutinize whether our design is reasonable and the application development process is smooth.
|
||||
2. Through one or two scenarios, OpenDAN can be quickly put into use by real users in a production environment, and these scenarios can serve as a basis for driving system improvements in OpenDAN.
|
||||
|
||||
## Agent: Personal Information Assistant
|
||||
|
||||
Through interacting with this Agent, users can use natural language to query information that has already been saved in the Knowledge-Base. For example, "Please show me the photos from my meeting with Richard last week." They can also find their information more accurately based on some interactive questions.
|
||||
|
||||
***To be developed.***
|
||||
|
||||
## Agent: Bulter Jarvis （0.5.2）
|
||||
|
||||
The Butler Agent Jarvis can recognize certain special commands. Through these commands, it can communicate with other Agents in the system, check the system's status, and use all the system's functionalities. It can be seen as another entry point to AIOS_Shell.
|
||||
|
||||
Another important function of the Jarvis is to create sessions. When a user has many workflows/agents installed on their OpenDAN, they might not know which workflow/agent to talk to in order to solve a problem. I envision the future mode to be: "If you don't know who to turn to, ask the Jarvis." The Jarvis will create or find a suitable session based on a brief conversation with the user, and then guide the user into this session.
|
||||
|
||||
Based on these two functions, the Jarvis might be the only "special Agent" that requires custom development among all Agents, and it is a part of the system.
|
||||
|
||||
## App: Personal Station （0.5.2）
|
||||
|
||||
The Personal Station is a built-in application that provides a graphical user interface for users to interact with the system. It is a web application that can be accessed through a browser. It is also the first application that users will see after installing OpenDAN. It provides a simple interface for users to interact with the system, and it also provides a way for users to install new applications.
|
||||
|
||||
The main functions of Personal Station include:
|
||||
|
||||
1. Library, with the help of Personal Information Assistant, you can better manage your own photos, videos, music, documents, etc., and share them with friends more effectively. (For example, ask the assistant to share photos from an event, selecting from those you've starred, and distribute them to friends based on the people appearing in the photos.)
|
||||
2. HomePage, with functions similar to Facebook/Twitter, where you can post content you want to share. You can also open your Agent to friends and family, allowing them to interact with your Agent, discuss schedule arrangements, and query your KnowledgeBase for open content.
|
||||
|
||||
Home Station is a mobile-first WebApp.
|
||||
|
||||
# UI
|
||||
|
||||
## CLI UI (aios_shell)
|
||||
|
||||
The system provides the command line UI interface priority, facing developers and early senior users.
|
||||
|
||||
## Web UI (0.5.2)
|
||||
|
||||
Web UI interface for end users
|
||||
|
||||
# 0.5.1 Integration Test (Senior*3)
|
||||
|
||||
Can be divided into 3 parts
|
||||
1.Workflow -> AI Agent -> AI Agent
|
||||
2.Spider -> Pipline -> Knowledge Base
|
||||
3.AI Agent <- Functions <- Knowledge Base
|
||||
|
||||
|
||||
# SDK
|
||||
|
||||
## Workflow SDK
|
||||
|
||||
The SDK allows developers to expand the new workflow/agent to the system.
|
||||
At present, the SDK has completed the most original version. In ROOTFS/, the .tmol file is written according to the directory structure, and a new workflow/ agent can be added to the system.
|
||||
|
||||
## AI Environments SDK （>0.5.2）
|
||||
|
||||
The SDK allows developers to expand the system that can be called by AI, including
|
||||
- Expand the new environment
|
||||
- Expand the new function
|
||||
|
||||
## Compute Kernel SDK （>0.5.2）
|
||||
|
||||
This SDK allows developers to expand more core capabilities to the system
|
||||
|
||||
# Document (>0.5.2)
|
||||
|
||||
When we release 0.5.3, we must complete at least 3 documents:
|
||||
|
||||
1. OpenDan's complete system design document, including the design document of each subsystem.
|
||||
2. Installation/use document for end users.
|
||||
3. SDK document for developers.
|
||||
+169
@@ -0,0 +1,169 @@
|
||||
# OpenDAN Basic Planning of MVP
|
||||
0.5.1 Implement data capture into Knownlege-Base(KB) via Spider, followed by access by AI Agent (35%)
|
||||
0.5.2 Build a Personal-Center based on the KB and associate the AI Agent with accessible telgram accounts (30%)
|
||||
0.5.3 Release for waitlist （5%）
|
||||
0.5.4 First public release (10%)
|
||||
0.5.5 Incorporate modifications after the first public version, workload depends on feedback (15%)
|
||||
0.5.6 Official version of MVP (5%)
|
||||
|
||||
# Basic Architecture
|
||||
(TODO)
|
||||
|
||||
# R&D Process Management
|
||||
Based on the project management module provided by SourceDAO, explore a new open source R&D process of "open source is mining".
|
||||
0. Confirming Version Leader based on committee election.
|
||||
1. Module (task division): Divide the system's futures into independent modules as much as possible, the development workload should be at the level of 1 person for 2-3 weeks.
|
||||
2. Discussion: Discuss whether the module division is reasonable, and design it's *BUDGET* based on the difficulty of the module and its importance to the current version (most important step)
|
||||
3. Recruit module PM. The module PM is responsible for the module's test delivery: completing the set functions, constructing the basic tests, and passing the self-tests. Testing should retain at least 30% of development resources.
|
||||
4. For the completed module, PM should write and publicize the Project Proposal. It contains more detailed about module goals + design ideas, participating teams (if any, there should also be a preliminary division of work within the team and calculation of contribution value), and acceptance plan design.
|
||||
5. The PM completes development and self-testing. Mark the module is *DONE*.
|
||||
6. Version Leader organizes the acceptance of the module (a dedicated acceptor can be appointed).
|
||||
7. Version Leader organizes integration testing according to the completion situation, and the module PM fixes BUGs. The test results can be used in the nightly-channel of OpenDAN.
|
||||
8. After the test passes, the Version Leader announces the version release, anyone can use this from release-chanel.
|
||||
9. The committee accepts the version after the release effect. After acceptance, all participants can extract contribution rewards.
|
||||
|
||||
Difficulty is expressed in the mode of requirement engineer level * time (in weeks, less than 1 week is counted as 1 week). 1 week of work time is calculated as 20 hours.
|
||||
|
||||
|
||||
**Below is the module division design for version 0.5.1.**
|
||||
|
||||
# Package Management (Architect*2)
|
||||
The package management system provides basic package definition and local lookup functionality.
|
||||
|
||||
The types of packages are
|
||||
1. Agent
|
||||
2. Function
|
||||
3. Service
|
||||
4. App
|
||||
5. Models (various AI models)
|
||||
|
||||
The system itself is upgraded on this basis
|
||||
|
||||
# Agent Management (Senior*1)
|
||||
Define Agent package, and give it to the package management system for publishing/download/installation/running
|
||||
|
||||
## Some pure prompt word engineering Agent (ordinary*1)
|
||||
Can refer to the existing implementation of ChatGPT-Next-Web, mainly to let the system run as soon as possible
|
||||
|
||||
# Agent (AI) Runtime (Architecture*2)
|
||||
Implementing the Agent as a person to run is an important basic component
|
||||
The Agent is an instance of a person, and the contents of all its sessions are interconnected. We can clone a new person based on an existing Agent, and this new Agent runs in a new container inherited from the old agent.
|
||||
- Wrap LLM Kernel
|
||||
- Manage chat-session
|
||||
- Manage memory (one of the difficulties, different from Knowlege-Base)
|
||||
- Manage identity and permissions
|
||||
- Manage logs, for the convenience of development and debugging, we can see all the details of the key operation and self-thinking of the Agent.
|
||||
|
||||
# Agent Work Flow (Architecture*2)
|
||||
A team of AI Agents completes a complex task
|
||||
Understanding this goal can better design the Agent running container
|
||||
*Next version*
|
||||
|
||||
# LLM Kernel Packaging (Architecture*1)
|
||||
Encapsulate LLM, support Agent to use different LLM cores. We can support the GPT4 first.
|
||||
|
||||
## Local LLM Integration (Senior*2)
|
||||
If the integration is convenient, we can also integrate a local LLM core to see the effect as soon as possible.
|
||||
|
||||
# Function Management (Architecture*1)
|
||||
Define Function, Function itself is a stateless passive calculation. Easy to be recognized and called by Agent.
|
||||
Function management needs to design the basic structure of Function, implement cloth/download/installation/running based on package management
|
||||
|
||||
## Some basic standard Functions (Senior*2)
|
||||
Can refer to some existing work, the main work in the early stage is standardization, as much as possible in information reading, the write operation that needs to be authorized by users needs to be promoted for consideration
|
||||
Get the physical information of the specified location: time, temperature, humidity
|
||||
|
||||
# Compute Runtime (Architecture*1)
|
||||
Except for the Agent, all components run in isolated running containers
|
||||
The Agent also runs in a container, and its isolation conditions may be stricter than the running container.
|
||||
Built based on docker,
|
||||
|
||||
# Compute Task Manager (Architecture*2)
|
||||
Manage computational tasks that require more computational resources.
|
||||
Compute tasks can be completed locally or on other devices.
|
||||
Insufficient computing resources is the norm, and scheduling work should be done here. Normal queue, give way to urgent short tasks when necessary.
|
||||
|
||||
# Spiders
|
||||
## Spider Basic Architecture (Architecture*1 )
|
||||
## Local Storage Format Based on NamedObject Theory (Architecture*1 )
|
||||
## Email-Spider Senior*1
|
||||
No threshold at all, mainly to run the framework
|
||||
## Twitter-Spider Senior*2
|
||||
For the convenience of ordinary users to install, do not use twitter api
|
||||
But if it is really impossible to complete, then use twitter api (difficulty drops to ordinary*2)
|
||||
## Facebook-Spider Senior*2
|
||||
For the convenience of ordinary users to install, do not use facebook api
|
||||
But if it is really impossible to complete, then use facebook api (difficulty drops to ordinary*2)
|
||||
|
||||
# Embedding Pipelines
|
||||
## Piplene Basic Architecture (Architecture*3)
|
||||
NamedObject+RawData => Vector DB
|
||||
## Image-Embedding pipeline (Senior*3)
|
||||
Implement Image vectorization pipeline (focus)
|
||||
## Doc-Embedding pipeline (Senior*3)
|
||||
Implement Text vectorization pipeline
|
||||
## Code-Embedding pipeline (Senior*2)
|
||||
## Video-Embedding pipeline (Senior*3)
|
||||
## Audio-Embedding pipeline (Senior*3)
|
||||
|
||||
# Knowledge Base (Architecture*2)
|
||||
Provide a unified interface, support switching vector database kernel
|
||||
Integrate open source vector database (pay attention to Lience selection)
|
||||
|
||||
When designing the interface, prepare for future access control
|
||||
|
||||
# 0.5.1 Integration Test (Senior*3)
|
||||
Can be divided into two parts
|
||||
1.Spider -> Pipline -> Knowledge Base
|
||||
2.AI Agent <- Functions <- Knowledge Base
|
||||
3.WebUI (not necessary)
|
||||
|
||||
## Basic OS System Construction (Architecture*1)
|
||||
The MVP version will not build a very serious system, the main goal is to set up an integrated test environment.
|
||||
|
||||
# AI Gateway(Network OS BUS)
|
||||
Obtain data by recognizing network data
|
||||
The Gateway also provides an external access entrance for the entire system, and access control can be unified.
|
||||
Provides the bus abstraction in the network operating system (the network cable is the bus), devices within the Zone are recognized by the system as plug-and-play devices, and can be called by applications/Agents
|
||||
*Next version*
|
||||
|
||||
# AI Browser
|
||||
By cooperating with the browser, save the web pages that users have seen to capture data
|
||||
*Next version*
|
||||
|
||||
# IoT Functions
|
||||
On the one hand, define standard Functions according to device types
|
||||
On the other hand, it is necessary to crack some existing IoT devices so that they can be connected to the AI Gateway and can be controlled by the Agent
|
||||
*Next version*
|
||||
|
||||
# Personal Lora Model Building
|
||||
*Next version*
|
||||
|
||||
## Personal Image Lora
|
||||
## Personal Voice Lora
|
||||
|
||||
# Permission and Privacy Management
|
||||
*Next version*
|
||||
|
||||
# AI Capability (Function) Integration
|
||||
*Next version*
|
||||
|
||||
## Text2Image
|
||||
Stable Diffusion core
|
||||
Stable Diffusion model management
|
||||
## Text2Voice
|
||||
|
||||
|
||||
# Telegram API Integration
|
||||
*Next version*
|
||||
|
||||
# Web Version Personal Center
|
||||
*Next version*
|
||||
|
||||
# Built-in Agents
|
||||
*Next version*
|
||||
|
||||
# Built-in APPs
|
||||
*Next version*
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user