975 lines
29 KiB
Markdown
975 lines
29 KiB
Markdown
# Memory Gateway
|
||
|
||
Memory Gateway 是一个通用记忆网关,用于给 AI agent / harness 提供统一的记忆检索、文档上传、LLM 总结和知识沉淀能力。
|
||
|
||
它的定位不是某个单一业务场景的垂直应用,而是一个可复用的本地 memory/context gateway:上层 agent 通过 REST、MCP 或 Hermes skill 调用它,底层由 OpenViking 承载 memory/resource,由 Obsidian 承载人工可维护的 Markdown 知识。
|
||
|
||
## 当前能力
|
||
|
||
- 搜索 OpenViking memory / resource。
|
||
- 写入普通 memory。
|
||
- 写入结构化 resource。
|
||
- 对任意文本调用 LLM 总结,并按需沉淀到 OpenViking。
|
||
- 上传文档,使用 MarkItDown 转 Markdown。
|
||
- 将上传文档保存到 Obsidian vault。
|
||
- 将文档摘要和结构化 artifact 写入 OpenViking knowledge。
|
||
- 给 Hermes 提供通用 `memory-gateway` skill。
|
||
- 新增通用 Memory Gateway v1 方案与 POC 骨架:多用户、namespace、visibility/ACL、episode、session commit、audit、skills 分层。
|
||
- v1 metadata 默认持久化到 SQLite,覆盖 users、memories、episodes、profiles、audit。
|
||
- `/v1/memory/search` 先做本地 ACL 过滤,再按可见 namespace 查询 OpenViking。
|
||
- v1 MCP tools 已接入现有 `/mcp/rpc`。
|
||
- `/v1/sessions/{session_id}/commit` 优先调用独立 EverMemOS HTTP 服务;服务不可用且允许 fallback 时,才使用 Gateway 进程内 POC worker。
|
||
|
||
完整方案见:
|
||
|
||
```text
|
||
docs/generic-memory-gateway-design.md
|
||
```
|
||
|
||
## 总体架构
|
||
|
||
```mermaid
|
||
flowchart LR
|
||
Agent[AI Agents / Hermes / OpenClaw / Nanobot] -->|HTTP / MCP / Plugin Tools / Skill Scripts| Gateway[Memory Gateway]
|
||
Gateway --> Auth[Auth + ACL + Namespace Router]
|
||
Gateway --> SQLite[(SQLite Metadata Store)]
|
||
Gateway --> OV[OpenViking Context Layer]
|
||
Gateway --> Ever[EverMemOS Consolidation Service]
|
||
Gateway --> Obsidian[Obsidian Markdown Vault]
|
||
Gateway --> LLM[OpenAI-compatible LLM]
|
||
OV --> OVStore[(OpenViking memory/resource/index)]
|
||
Ever -->|promote stable memory| SQLite
|
||
Ever -->|review drafts| Obsidian
|
||
Obsidian --> Human[Human Review / Manual Editing]
|
||
```
|
||
|
||
模块职责:
|
||
|
||
- **Memory Gateway**:统一入口,提供 REST、MCP RPC、Hermes/OpenClaw plugin 所需的 `/v1` API;负责 user context、namespace 展开、visibility/ACL、audit、短期 episode 写入和 session commit 编排。
|
||
- **OpenViking**:context/resource/memory 检索层;Gateway 会把允许访问的 namespace 展开后交给 OpenViking 搜索。
|
||
- **EverMemOS**:长期记忆整理层;从 session episodes 中提取稳定候选,去重、合并、冲突检测,并决定是否 promote 到长期记忆或进入 review。
|
||
- **Obsidian Vault**:人工可维护的 Markdown 前台;保存上传文档、review draft、高价值可审查知识。不要把所有原始对话直接写入 Obsidian。
|
||
- **Hermes skill**:脚本式、显式调用入口,适合人工/agent 明确执行搜索、上传、commit。
|
||
- **Agent plugin**:Hermes/OpenClaw runtime adapter,注册 `memory_gateway` toolset 和 hooks,可在真实对话中自动 search、append episode、可选 commit。
|
||
|
||
## 全新服务器部署总览
|
||
|
||
以下步骤假设目标服务器是 Linux,初始没有 Python 环境、OpenViking、EverMemOS、Memory Gateway、Obsidian Vault 或 Hermes Agent。示例路径使用 `/opt`,可按实际环境替换。
|
||
|
||
推荐目录:
|
||
|
||
```text
|
||
/opt/
|
||
├── OpenViking/ # OpenViking 项目和 venv
|
||
└── memory-gateway/ # 本项目
|
||
├── .venv/
|
||
├── config.yaml # 本机配置,包含密钥,不提交
|
||
├── data/
|
||
│ └── memory_gateway.sqlite3
|
||
└── obsidian-vault/
|
||
├── 01_Knowledge/Uploaded/
|
||
└── Reviews/Queue/
|
||
```
|
||
|
||
### 1. 安装系统依赖
|
||
|
||
```bash
|
||
sudo apt-get update
|
||
sudo apt-get install -y \
|
||
git curl rsync build-essential \
|
||
python3 python3-venv python3-pip
|
||
```
|
||
|
||
可选安装 `uv`,后续 Python 环境会更快:
|
||
|
||
```bash
|
||
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||
```
|
||
|
||
如果服务器不能访问公网,请用内部镜像源安装 Python 依赖,并把 OpenViking、Hermes 和本项目代码放到内网 Git/制品库。
|
||
|
||
### 2. 安装 OpenViking
|
||
|
||
OpenViking 不在本仓库内,需要按 OpenViking 项目的官方安装方式部署。最终目标是服务器上可以运行:
|
||
|
||
```bash
|
||
openviking-server --host 127.0.0.1 --port 1933
|
||
```
|
||
|
||
一种常见安装形态:
|
||
|
||
```bash
|
||
cd /opt
|
||
git clone https://github.com/volcengine/OpenViking.git
|
||
cd /opt/OpenViking
|
||
python3 -m venv .venv
|
||
source .venv/bin/activate
|
||
pip install -U pip
|
||
pip install -e .
|
||
```
|
||
|
||
启动:
|
||
|
||
```bash
|
||
source /opt/OpenViking/.venv/bin/activate
|
||
openviking-server --host 127.0.0.1 --port 1933
|
||
```
|
||
|
||
健康检查:
|
||
|
||
```bash
|
||
curl http://127.0.0.1:1933/health
|
||
```
|
||
|
||
说明:OpenViking 建议先只绑定 `127.0.0.1`,由 Memory Gateway 统一对外暴露 API。
|
||
|
||
### 3. 安装 Memory Gateway
|
||
|
||
```bash
|
||
cd /opt
|
||
git clone https://gitea.bwgdi.com/tomtan/memory-gateway.git memory-gateway
|
||
cd /opt/memory-gateway
|
||
python3 -m venv .venv
|
||
source .venv/bin/activate
|
||
pip install -U pip
|
||
pip install -e ".[dev]"
|
||
```
|
||
|
||
创建目录和配置:
|
||
|
||
```bash
|
||
mkdir -p /opt/memory-gateway/data
|
||
mkdir -p /opt/memory-gateway/obsidian-vault/01_Knowledge/Uploaded
|
||
mkdir -p /opt/memory-gateway/obsidian-vault/Reviews/Queue
|
||
cp config.example.yaml config.yaml
|
||
```
|
||
|
||
修改 `config.yaml`:
|
||
|
||
```yaml
|
||
server:
|
||
host: "0.0.0.0"
|
||
port: 1934
|
||
api_key: "<CHANGE_ME_LONG_RANDOM_KEY>"
|
||
|
||
openviking:
|
||
url: "http://127.0.0.1:1933"
|
||
api_key: "<OPENVIKING_KEY_IF_REQUIRED>"
|
||
|
||
evermemos:
|
||
enabled: true
|
||
url: "http://127.0.0.1:1995"
|
||
fallback_to_local: true
|
||
|
||
obsidian:
|
||
vault_path: "/opt/memory-gateway/obsidian-vault"
|
||
knowledge_dir: "01_Knowledge/Uploaded"
|
||
review_dir: "Reviews/Queue"
|
||
|
||
storage:
|
||
backend: "sqlite"
|
||
sqlite_path: "/opt/memory-gateway/data/memory_gateway.sqlite3"
|
||
```
|
||
|
||
`config.yaml` 被 `.gitignore` 忽略,不应提交。
|
||
|
||
### 4. 安装/启动 EverMemOS
|
||
|
||
本项目内置一个 POC 级 EverMemOS-compatible 服务,适合单机验证:
|
||
|
||
```bash
|
||
cd /opt/memory-gateway
|
||
source .venv/bin/activate
|
||
python -m memory_gateway.evermemos_service \
|
||
--config /opt/memory-gateway/config.yaml \
|
||
--host 127.0.0.1 \
|
||
--port 1995
|
||
```
|
||
|
||
健康检查:
|
||
|
||
```bash
|
||
curl http://127.0.0.1:1995/health
|
||
```
|
||
|
||
如果已有独立 EverMemOS 服务,把 `config.yaml` 中的 `evermemos.url`、`api_key`、`consolidate_path` 改为远程服务即可。Memory Gateway 会在 `/v1/sessions/{session_id}/commit` 时调用它。
|
||
|
||
### 5. 启动 Memory Gateway
|
||
|
||
```bash
|
||
cd /opt/memory-gateway
|
||
source .venv/bin/activate
|
||
python -m memory_gateway.server --config /opt/memory-gateway/config.yaml
|
||
```
|
||
|
||
健康检查:
|
||
|
||
```bash
|
||
curl -H "X-API-Key: <CHANGE_ME_LONG_RANDOM_KEY>" http://127.0.0.1:1934/health
|
||
curl -H "X-API-Key: <CHANGE_ME_LONG_RANDOM_KEY>" http://127.0.0.1:1934/v1/evermemos/health
|
||
```
|
||
|
||
生产建议用 systemd/supervisor 管理 OpenViking、EverMemOS 和 Memory Gateway 三个进程,并通过 Nginx/Caddy/内网负载均衡做 TLS 和访问控制。
|
||
|
||
### 6. Obsidian Vault
|
||
|
||
服务器端不要求安装 Obsidian 桌面应用。这里的 Obsidian Vault 本质是 Markdown 目录:
|
||
|
||
```text
|
||
/opt/memory-gateway/obsidian-vault/
|
||
├── 01_Knowledge/Uploaded/ # /api/knowledge/upload 转换后的 Markdown
|
||
└── Reviews/Queue/ # EverMemOS 高价值/冲突候选 review draft
|
||
```
|
||
|
||
如果需要人工维护,可以:
|
||
|
||
- 用 Obsidian 桌面端通过 SSH/Syncthing/Git 同步这个 vault。
|
||
- 或直接在服务器上编辑 Markdown。
|
||
- 高价值、冲突、低置信度的长期记忆候选优先进 `Reviews/Queue/`,不要直接污染长期记忆。
|
||
|
||
### 7. 安装 Hermes Agent、Skill 和 Plugin
|
||
|
||
Hermes Agent 不在本仓库内。先按 Hermes 官方方式安装,确认命令可用:
|
||
|
||
```bash
|
||
hermes --version
|
||
hermes chat --help
|
||
```
|
||
|
||
安装 Memory Gateway skill:
|
||
|
||
```bash
|
||
mkdir -p ~/.hermes/skills/memory-gateway
|
||
rsync -a --delete \
|
||
/opt/memory-gateway/integrations/hermes/memory-gateway/ \
|
||
~/.hermes/skills/memory-gateway/
|
||
```
|
||
|
||
skill 的特点:
|
||
|
||
- 通过脚本显式调用 Gateway API。
|
||
- 适合手动或 agent policy 主动执行:搜索、上传、append episode、commit session。
|
||
- skill 本身不会自动记忆每轮对话。
|
||
|
||
安装 Memory Gateway agent plugin:
|
||
|
||
```bash
|
||
mkdir -p ~/.hermes/plugins
|
||
ln -s /opt/memory-gateway/plugins/memory-gateway-agent \
|
||
~/.hermes/plugins/memory-gateway-agent
|
||
hermes plugins enable memory-gateway-agent
|
||
hermes plugins list
|
||
hermes tools list
|
||
```
|
||
|
||
plugin 环境变量:
|
||
|
||
```bash
|
||
export MEMORY_GATEWAY_URL=http://127.0.0.1:1934
|
||
export MEMORY_GATEWAY_API_KEY=<CHANGE_ME_LONG_RANDOM_KEY>
|
||
export MEMORY_GATEWAY_DEFAULT_USER_ID=user_demo
|
||
export MEMORY_GATEWAY_DEFAULT_AGENT_ID=agent_hermes
|
||
export MEMORY_GATEWAY_DEFAULT_WORKSPACE_ID=workspace_demo
|
||
export MEMORY_GATEWAY_AUTO_SEARCH=true
|
||
export MEMORY_GATEWAY_AUTO_APPEND_EPISODE=true
|
||
export MEMORY_GATEWAY_AUTO_COMMIT_SESSION=false
|
||
```
|
||
|
||
plugin 的特点:
|
||
|
||
- 向 Hermes 注册 `memory_gateway` toolset。
|
||
- 注册 hooks:`on_session_start`、`pre_llm_call`、`post_llm_call`、`on_session_end`。
|
||
- `pre_llm_call` 自动检索相关记忆。
|
||
- `post_llm_call` 只写摘要型 candidate episode。
|
||
- `on_session_end` 默认不 commit;只有 `MEMORY_GATEWAY_AUTO_COMMIT_SESSION=true` 才提交 session。
|
||
|
||
## 目录结构
|
||
|
||
```text
|
||
memory-gateway/
|
||
├── memory_gateway/ # Gateway 服务代码
|
||
│ ├── server.py # REST / MCP 接口
|
||
│ ├── openviking_client.py # OpenViking client
|
||
│ ├── llm.py # OpenAI-compatible LLM summary
|
||
│ ├── document_ingest.py # MarkItDown + Obsidian write helpers
|
||
│ ├── config.py
|
||
│ └── types.py
|
||
├── integrations/hermes/
|
||
│ └── memory-gateway/ # 通用 Hermes skill
|
||
├── plugins/
|
||
│ └── memory-gateway-agent/ # Hermes/OpenClaw agent plugin adapter
|
||
├── obsidian-vault/
|
||
│ ├── 01_Knowledge/Uploaded/ # 上传文档转成的 Markdown
|
||
│ ├── Reviews/Queue/ # EverMemOS review draft
|
||
│ └── 05_Templates/ # 通用知识模板
|
||
├── data/ # SQLite metadata store,生产建议保留备份
|
||
│ └── memory_gateway.sqlite3
|
||
├── docs/
|
||
│ └── generic-memory-gateway-design.md
|
||
├── tests/
|
||
├── config.example.yaml
|
||
└── pyproject.toml
|
||
```
|
||
|
||
## 用户隔离、存储与长短期记忆
|
||
|
||
### 访问上下文
|
||
|
||
v1 API 的所有核心读写都应该带上访问上下文:
|
||
|
||
```json
|
||
{
|
||
"user_id": "user_demo",
|
||
"agent_id": "agent_hermes",
|
||
"workspace_id": "workspace_demo",
|
||
"session_id": "session_001"
|
||
}
|
||
```
|
||
|
||
Gateway 根据这些字段做 namespace 展开和 ACL 判断。不同用户必须使用不同 `user_id`,不同 agent 使用不同 `agent_id`,不同项目或团队空间使用不同 `workspace_id`。
|
||
|
||
### Namespace 设计
|
||
|
||
默认可见 namespace 形态:
|
||
|
||
```text
|
||
user/{user_id}/profile
|
||
user/{user_id}/preferences
|
||
user/{user_id}/long_term
|
||
agent/{agent_id}/memory
|
||
workspace/{workspace_id}/shared
|
||
session/{session_id}/episodic
|
||
global/public
|
||
```
|
||
|
||
隔离规则:
|
||
|
||
- `private`:默认私有,只允许同一 `user_id` 在 ACL 允许范围内访问。
|
||
- `agent-only`:只允许指定 agent 或 agent namespace 使用。
|
||
- `workspace-shared`:允许 workspace 成员/允许的 agent 使用。
|
||
- `global`:公共知识,谨慎使用,不存放个人信息。
|
||
- `session/{session_id}/episodic`:短期 session 记忆,默认只在当前 session 上下文中使用。
|
||
|
||
### 用户记忆存储位置
|
||
|
||
| 数据 | 存储位置 | 说明 |
|
||
| --- | --- | --- |
|
||
| users/profiles/memories/episodes/audit metadata | SQLite `storage.sqlite_path` | Gateway v1 的主 metadata store,负责隔离、ACL、audit、短期 episode 和长期 memory 记录 |
|
||
| 可检索 context/resource/memory | OpenViking | Gateway 按允许 namespace 查询 OpenViking,适合跨 agent 的 context 检索 |
|
||
| 上传文档 Markdown | Obsidian `01_Knowledge/Uploaded/` | 由 `/api/knowledge/upload` 写入,适合人工审查和维护 |
|
||
| 高价值/冲突 review draft | Obsidian `Reviews/Queue/` | EverMemOS 发现高价值或冲突候选时写入,不直接污染长期记忆 |
|
||
| Hermes plugin trace | `plugins/memory-gateway-agent/.tmp/hook_trace.log` | 默认关闭,只存 hook 名称、短 session id、Gateway action 和状态 |
|
||
|
||
### 什么时候写短期记忆
|
||
|
||
短期记忆写入 `episodes`,通常来自:
|
||
|
||
- Hermes plugin `post_llm_call` 判断用户明确要求“remember/记住”或出现稳定偏好、长期约束、项目事实。
|
||
- agent 或 skill 显式调用 `memory_append_episode`。
|
||
- 任务执行过程中的关键结论、可复用 workflow、架构决策。
|
||
|
||
短期记忆不应该包含:
|
||
|
||
- 完整原始对话。
|
||
- password、token、API key、cookie、private key。
|
||
- 一次性验证码、大段日志、低价值临时内容。
|
||
- 模型 chain-of-thought。
|
||
|
||
### 什么时候提升为长期记忆
|
||
|
||
长期记忆写入 `memories`,通常发生在:
|
||
|
||
1. Agent/skill/plugin 先写 `session/{session_id}/episodic` episode。
|
||
2. 调用 `/v1/sessions/{session_id}/commit` 或 MCP/tool `memory_commit_session`。
|
||
3. Gateway 把 session episodes、可见长期记忆和访问上下文发给 EverMemOS。
|
||
4. EverMemOS 做提取、去重、合并、冲突检测、重要性判断。
|
||
5. 稳定且超过阈值的候选 promote 到 `user/{user_id}/long_term` 或目标 namespace。
|
||
6. 高价值、冲突或需要人工确认的候选进入 Obsidian `Reviews/Queue/`。
|
||
|
||
Hermes plugin 默认:
|
||
|
||
- `MEMORY_GATEWAY_AUTO_SEARCH=true`:对话前自动检索。
|
||
- `MEMORY_GATEWAY_AUTO_APPEND_EPISODE=true`:对话后按 policy 写 candidate episode。
|
||
- `MEMORY_GATEWAY_AUTO_COMMIT_SESSION=false`:默认不自动提升长期记忆。
|
||
|
||
因此,POC 推荐流程是“先短期、后 commit、再长期”,不要把所有 session 内容直接 upsert 成长期记忆。
|
||
|
||
### 如何再次查询和调用用户记忆
|
||
|
||
HTTP 查询:
|
||
|
||
```bash
|
||
curl -X POST http://127.0.0.1:1934/v1/memory/search \
|
||
-H "Content-Type: application/json" \
|
||
-H "X-API-Key: <CHANGE_ME_LONG_RANDOM_KEY>" \
|
||
-d '{
|
||
"user_id": "user_demo",
|
||
"agent_id": "agent_hermes",
|
||
"workspace_id": "workspace_demo",
|
||
"session_id": "session_001",
|
||
"query": "用户输出偏好和项目约束",
|
||
"limit": 5
|
||
}'
|
||
```
|
||
|
||
Hermes skill 查询:
|
||
|
||
```bash
|
||
python ~/.hermes/skills/memory-gateway/scripts/memory_search.py \
|
||
--user-id user_demo \
|
||
--agent-id agent_hermes \
|
||
--workspace-id workspace_demo \
|
||
--session-id session_001 \
|
||
--query "用户输出偏好和项目约束" \
|
||
--limit 5
|
||
```
|
||
|
||
Hermes plugin 查询:
|
||
|
||
- 用户正常对话时,`pre_llm_call` 会自动调用 `memory_search`。
|
||
- Agent 也可以显式调用 `memory_search` tool。
|
||
|
||
## 远程调用与安全
|
||
|
||
Memory Gateway API 可以远程调用。生产/远程部署时建议:
|
||
|
||
1. `server.host` 设置为 `0.0.0.0`。
|
||
2. `server.api_key` 设置为长随机值。
|
||
3. 防火墙只开放可信来源,或通过 VPN/内网访问。
|
||
4. 使用 Nginx/Caddy/负载均衡终止 TLS。
|
||
5. 远程客户端一律带 `X-API-Key`。
|
||
|
||
远程调用示例:
|
||
|
||
```bash
|
||
export MEMORY_GATEWAY_URL=https://memory.example.com
|
||
export MEMORY_GATEWAY_API_KEY=<CHANGE_ME_LONG_RANDOM_KEY>
|
||
|
||
curl -X POST "$MEMORY_GATEWAY_URL/v1/memory/search" \
|
||
-H "Content-Type: application/json" \
|
||
-H "X-API-Key: $MEMORY_GATEWAY_API_KEY" \
|
||
-d '{
|
||
"user_id": "user_demo",
|
||
"agent_id": "remote_agent",
|
||
"workspace_id": "workspace_demo",
|
||
"query": "长期偏好",
|
||
"limit": 5
|
||
}'
|
||
```
|
||
|
||
远程 Hermes plugin 配置:
|
||
|
||
```bash
|
||
export MEMORY_GATEWAY_URL=https://memory.example.com
|
||
export MEMORY_GATEWAY_API_KEY=<CHANGE_ME_LONG_RANDOM_KEY>
|
||
export MEMORY_GATEWAY_DEFAULT_USER_ID=user_demo
|
||
export MEMORY_GATEWAY_DEFAULT_AGENT_ID=agent_hermes_remote
|
||
export MEMORY_GATEWAY_DEFAULT_WORKSPACE_ID=workspace_demo
|
||
```
|
||
|
||
注意:远程 API 不应裸奔在公网。至少启用 API key 和 TLS;涉及个人记忆时建议放在内网或 VPN 后面。
|
||
|
||
## 本地开发快捷启动
|
||
|
||
全新服务器请优先按上面的 `/opt` 部署流程执行。本节用于已经 clone 当前仓库后的开发机快速启动,路径都以仓库根目录为准。
|
||
|
||
```bash
|
||
cd memory-gateway
|
||
python3 -m venv .venv
|
||
source .venv/bin/activate
|
||
pip install -U pip
|
||
pip install -e ".[dev]"
|
||
mkdir -p data obsidian-vault/01_Knowledge/Uploaded obsidian-vault/Reviews/Queue
|
||
cp config.example.yaml config.yaml
|
||
```
|
||
|
||
建议把 `config.yaml` 中的本地开发路径改成:
|
||
|
||
```yaml
|
||
obsidian:
|
||
vault_path: "./obsidian-vault"
|
||
|
||
storage:
|
||
backend: sqlite
|
||
sqlite_path: "./data/memory_gateway.sqlite3"
|
||
```
|
||
|
||
启动顺序:
|
||
|
||
```bash
|
||
# 终端 1:OpenViking,按 OpenViking 项目实际环境启动
|
||
openviking-server --host 127.0.0.1 --port 1933
|
||
|
||
# 终端 2:EverMemOS-compatible POC 服务
|
||
python -m memory_gateway.evermemos_service \
|
||
--config ./config.yaml \
|
||
--host 127.0.0.1 \
|
||
--port 1995
|
||
|
||
# 终端 3:Memory Gateway
|
||
python -m memory_gateway.server --config ./config.yaml
|
||
```
|
||
|
||
健康检查:
|
||
|
||
```bash
|
||
curl http://127.0.0.1:1934/health
|
||
curl http://127.0.0.1:1995/health
|
||
curl http://127.0.0.1:1934/v1/evermemos/health
|
||
```
|
||
|
||
## REST 接口
|
||
|
||
如果 `config.yaml` 设置了 `server.api_key`,以下所有 HTTP 调用都需要加:
|
||
|
||
```bash
|
||
-H "X-API-Key: <CHANGE_ME_LONG_RANDOM_KEY>"
|
||
```
|
||
|
||
### `GET /health`
|
||
|
||
检查 Gateway 和 OpenViking 状态。
|
||
|
||
### `POST /api/search`
|
||
|
||
搜索 OpenViking memory / resource。
|
||
|
||
```bash
|
||
curl -X POST http://127.0.0.1:1934/api/search \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"query": "memory gateway document upload summary",
|
||
"uri": "viking://resources",
|
||
"limit": 5
|
||
}'
|
||
```
|
||
|
||
### `POST /api/memory`
|
||
|
||
写入普通 memory。
|
||
|
||
```bash
|
||
curl -X POST http://127.0.0.1:1934/api/memory \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"namespace": "memory-gateway",
|
||
"memory_type": "preference",
|
||
"content": "The user prefers concise technical summaries."
|
||
}'
|
||
```
|
||
|
||
### `POST /api/resource`
|
||
|
||
写入结构化 resource。
|
||
|
||
```bash
|
||
curl -X POST http://127.0.0.1:1934/api/resource \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"uri": "viking://resources/memory-gateway/knowledge/example.json",
|
||
"resource_type": "json",
|
||
"content": "{\"title\":\"example\"}"
|
||
}'
|
||
```
|
||
|
||
### `POST /api/summary`
|
||
|
||
调用 LLM 总结任意文本,并按需沉淀到 OpenViking。
|
||
|
||
```bash
|
||
curl -X POST http://127.0.0.1:1934/api/summary \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"title": "Project decision summary",
|
||
"content": "需要总结和沉淀的内容...",
|
||
"namespace": "memory-gateway",
|
||
"memory_type": "decision",
|
||
"tags": ["project", "decision"],
|
||
"persist_as": "resource"
|
||
}'
|
||
```
|
||
|
||
`persist_as` 支持:`none`、`memory`、`resource`、`both`。
|
||
|
||
### `POST /api/knowledge/upload`
|
||
|
||
上传文档,MarkItDown 转 Markdown,保存到 Obsidian,LLM 总结后写入 OpenViking knowledge。
|
||
|
||
```bash
|
||
curl -X POST http://127.0.0.1:1934/api/knowledge/upload \
|
||
-F "file=@/path/to/document.pdf" \
|
||
-F "title=Design Notes" \
|
||
-F "namespace=memory-gateway" \
|
||
-F "knowledge_type=design_doc" \
|
||
-F "tags=project,design,reference" \
|
||
-F "persist_as=resource"
|
||
```
|
||
|
||
默认保存到:
|
||
|
||
```text
|
||
obsidian-vault/01_Knowledge/Uploaded/
|
||
```
|
||
|
||
## v1 通用 Memory API
|
||
|
||
v1 API 面向多 agent 框架,带 user / agent / workspace / session 上下文和基础 ACL。
|
||
|
||
### 创建用户
|
||
|
||
```bash
|
||
curl -X POST http://127.0.0.1:1934/v1/users \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"user_id":"user_tom","display_name":"Tom","preferences":{"language":"zh-CN"}}'
|
||
```
|
||
|
||
### 写入记忆
|
||
|
||
```bash
|
||
curl -X POST http://127.0.0.1:1934/v1/memory \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"user_id": "user_tom",
|
||
"agent_id": "agent_hermes",
|
||
"workspace_id": "ws_memory_gateway",
|
||
"memory_type": "preference",
|
||
"content": "用户偏好中文输出,结构化但不要过度工程化。",
|
||
"summary": "中文、结构化、轻量 POC 优先。",
|
||
"tags": ["preference", "style"],
|
||
"importance": 0.8,
|
||
"confidence": 0.9,
|
||
"visibility": "private",
|
||
"source": "manual"
|
||
}'
|
||
```
|
||
|
||
### 检索记忆
|
||
|
||
```bash
|
||
curl -X POST http://127.0.0.1:1934/v1/memory/search \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"user_id": "user_tom",
|
||
"agent_id": "agent_hermes",
|
||
"workspace_id": "ws_memory_gateway",
|
||
"query": "中文输出",
|
||
"limit": 5
|
||
}'
|
||
```
|
||
|
||
返回会包含:
|
||
|
||
- `local_total`:SQLite metadata 命中的记忆数量。
|
||
- `openviking_total`:按可见 namespace 查询 OpenViking 的命中数量。
|
||
- `searched_namespaces`:Gateway 展开并允许查询的 namespace。
|
||
|
||
### 修改记忆
|
||
|
||
```bash
|
||
curl -X PATCH "http://127.0.0.1:1934/v1/memory/MEMORY_ID?user_id=user_tom&agent_id=agent_hermes&workspace_id=ws_memory_gateway" \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"summary":"用户偏好中文、结构化、少废话。","importance":0.9}'
|
||
```
|
||
|
||
### 写入 episode 并 commit session
|
||
|
||
```bash
|
||
curl -X POST http://127.0.0.1:1934/v1/episodes \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"user_id": "user_tom",
|
||
"agent_id": "agent_hermes",
|
||
"workspace_id": "ws_memory_gateway",
|
||
"session_id": "sess_demo",
|
||
"content": "结论:这个项目必须保留用户隔离和 namespace ACL。",
|
||
"tags": ["decision"]
|
||
}'
|
||
|
||
curl -X POST http://127.0.0.1:1934/v1/sessions/sess_demo/commit \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"user_id": "user_tom",
|
||
"agent_id": "agent_hermes",
|
||
"workspace_id": "ws_memory_gateway",
|
||
"session_id": "sess_demo",
|
||
"promote": true,
|
||
"min_importance": 0.6
|
||
}'
|
||
```
|
||
|
||
流程说明:
|
||
|
||
- 短期记忆先写入 SQLite 的 `episodes`,namespace 通常是 `session/{session_id}/episodic`。
|
||
- commit session 时,Gateway 把当前 session episodes、可见长期记忆和访问上下文发给 `http://127.0.0.1:1995/v1/sessions/consolidate`。
|
||
- EverMemOS 返回候选记忆、可直接提升的长期记忆、重复/冲突信息和 review draft 路径。
|
||
- Gateway 只把正常稳定候选写入长期 memory;高价值或冲突候选不会直接进入长期记忆,会写入:
|
||
|
||
```text
|
||
obsidian-vault/Reviews/Queue/
|
||
```
|
||
|
||
## MCP Tools
|
||
|
||
`POST /mcp/rpc` 支持:
|
||
|
||
- `search`
|
||
- `add_memory`
|
||
- `add_resource`
|
||
- `commit_summary`
|
||
- `get_status`
|
||
- `list_memories`
|
||
- `list_resources`
|
||
- `memory_search`
|
||
- `memory_upsert`
|
||
- `memory_append_episode`
|
||
- `memory_commit_session`
|
||
- `memory_get_profile`
|
||
- `memory_list_namespaces`
|
||
- `memory_delete`
|
||
- `memory_feedback`
|
||
|
||
## Hermes Skill
|
||
|
||
通用 Hermes skill:
|
||
|
||
```text
|
||
~/.hermes/skills/memory-gateway/
|
||
```
|
||
|
||
仓库副本:
|
||
|
||
```text
|
||
integrations/hermes/memory-gateway/
|
||
```
|
||
|
||
安装或更新到本机 Hermes skill 目录:
|
||
|
||
```bash
|
||
mkdir -p ~/.hermes/skills/memory-gateway
|
||
rsync -a --delete \
|
||
./integrations/hermes/memory-gateway/ \
|
||
~/.hermes/skills/memory-gateway/
|
||
```
|
||
|
||
使用方式:
|
||
|
||
- Hermes 对话中可以加载 `memory-gateway` skill,让 agent 按 skill 文档主动调用脚本。
|
||
- skill 不等于自动记忆;只有 agent 根据 skill/policy 主动调用脚本时才会写入或检索记忆。
|
||
- 适合人工可控的显式操作:创建用户、检索记忆、追加 episode、commit session、上传知识和检查 EverMemOS。
|
||
|
||
主要脚本:
|
||
|
||
```text
|
||
scripts/evermemos_health.py
|
||
scripts/memory_create_user.py
|
||
scripts/memory_append_episode.py
|
||
scripts/memory_commit_session.py
|
||
scripts/memory_search.py
|
||
scripts/memory_upsert.py
|
||
scripts/retrieve_memory.py
|
||
scripts/commit_summary.py
|
||
scripts/upload_knowledge.py
|
||
scripts/search_obsidian.py
|
||
```
|
||
|
||
检查 EverMemOS:
|
||
|
||
```bash
|
||
python ~/.hermes/skills/memory-gateway/scripts/evermemos_health.py
|
||
```
|
||
|
||
检索记忆:
|
||
|
||
```bash
|
||
python ~/.hermes/skills/memory-gateway/scripts/retrieve_memory.py \
|
||
--query "document upload summary memory gateway" \
|
||
--uri viking://resources \
|
||
--limit 5
|
||
```
|
||
|
||
总结沉淀:
|
||
|
||
```bash
|
||
python ~/.hermes/skills/memory-gateway/scripts/commit_summary.py \
|
||
--title "Reusable conclusion" \
|
||
--namespace memory-gateway \
|
||
--memory-type decision \
|
||
--tag project \
|
||
--persist-as resource \
|
||
--text "最终结论或可复用知识..."
|
||
```
|
||
|
||
上传知识:
|
||
|
||
```bash
|
||
python ~/.hermes/skills/memory-gateway/scripts/upload_knowledge.py \
|
||
--file /path/to/document.md \
|
||
--title "Knowledge note" \
|
||
--namespace memory-gateway \
|
||
--knowledge-type reference \
|
||
--tags project,reference \
|
||
--persist-as resource
|
||
```
|
||
|
||
完整长短期记忆测试:
|
||
|
||
```bash
|
||
python ~/.hermes/skills/memory-gateway/scripts/memory_create_user.py \
|
||
--user-id user_tom \
|
||
--display-name "Tom" \
|
||
--preference language=zh-CN
|
||
|
||
python ~/.hermes/skills/memory-gateway/scripts/memory_append_episode.py \
|
||
--user-id user_tom \
|
||
--agent-id agent_hermes \
|
||
--workspace-id ws_memory_gateway \
|
||
--session-id sess_demo \
|
||
--tag decision \
|
||
--text "结论:本机 EverMemOS 服务负责从 session episode 中整理稳定长期记忆。"
|
||
|
||
python ~/.hermes/skills/memory-gateway/scripts/memory_append_episode.py \
|
||
--user-id user_tom \
|
||
--agent-id agent_hermes \
|
||
--workspace-id ws_memory_gateway \
|
||
--session-id sess_demo \
|
||
--tag review \
|
||
--tag high-value \
|
||
--text "重要:高价值记忆应该进入 Obsidian review queue,避免错误记忆污染长期系统。"
|
||
|
||
python ~/.hermes/skills/memory-gateway/scripts/memory_commit_session.py \
|
||
--user-id user_tom \
|
||
--agent-id agent_hermes \
|
||
--workspace-id ws_memory_gateway \
|
||
--session-id sess_demo \
|
||
--min-importance 0.6
|
||
|
||
python ~/.hermes/skills/memory-gateway/scripts/memory_search.py \
|
||
--user-id user_tom \
|
||
--agent-id agent_hermes \
|
||
--workspace-id ws_memory_gateway \
|
||
--session-id sess_demo \
|
||
--query "EverMemOS 服务负责" \
|
||
--limit 5
|
||
```
|
||
|
||
## Hermes / OpenClaw Agent Plugin
|
||
|
||
通用 Agent plugin 位于:
|
||
|
||
```text
|
||
plugins/memory-gateway-agent/
|
||
```
|
||
|
||
它是独立 adapter,不 import Gateway 内部 `services/repositories/server`,所有调用都通过现有 `/v1` HTTP API。它面向 Hermes/OpenClaw 这类 agent runtime 暴露统一工具:
|
||
|
||
- `memory_search`
|
||
- `memory_append_episode`
|
||
- `memory_commit_session`
|
||
- `memory_upsert`
|
||
- `memory_feedback`
|
||
|
||
Hermes 本机安装:
|
||
|
||
```bash
|
||
mkdir -p ~/.hermes/plugins
|
||
ln -s "$(pwd)/plugins/memory-gateway-agent" \
|
||
~/.hermes/plugins/memory-gateway-agent
|
||
hermes plugins enable memory-gateway-agent
|
||
hermes plugins list
|
||
hermes tools list
|
||
```
|
||
|
||
如果软链接已存在,先确认它指向当前仓库:
|
||
|
||
```bash
|
||
ls -l ~/.hermes/plugins/memory-gateway-agent
|
||
```
|
||
|
||
运行配置:
|
||
|
||
```bash
|
||
export MEMORY_GATEWAY_URL=http://127.0.0.1:1934
|
||
export MEMORY_GATEWAY_API_KEY=
|
||
export MEMORY_GATEWAY_DEFAULT_USER_ID=test_user_memory_gateway_plugin
|
||
export MEMORY_GATEWAY_DEFAULT_AGENT_ID=test_hermes_memory_gateway_plugin
|
||
export MEMORY_GATEWAY_DEFAULT_WORKSPACE_ID=test_workspace_memory_gateway_plugin
|
||
export MEMORY_GATEWAY_AUTO_SEARCH=true
|
||
export MEMORY_GATEWAY_AUTO_APPEND_EPISODE=true
|
||
export MEMORY_GATEWAY_AUTO_COMMIT_SESSION=false
|
||
```
|
||
|
||
Hermes plugin 已验证:
|
||
|
||
- `hermes plugins list` 可发现并启用 `memory-gateway-agent`。
|
||
- `hermes tools list` 可看到 `memory_gateway` toolset。
|
||
- `pre_llm_call` 会自动检索 Memory Gateway。
|
||
- `post_llm_call` 会按 policy 写入摘要型 candidate episode。
|
||
- `on_session_end` 默认不会 commit;只有 `MEMORY_GATEWAY_AUTO_COMMIT_SESSION=true` 才会 commit。
|
||
|
||
真实 Hermes chat 验证:
|
||
|
||
```bash
|
||
PYTHONPATH="$(pwd)/plugins/memory-gateway-agent" \
|
||
python plugins/memory-gateway-agent/scripts/hermes_interactive_session_check.py
|
||
```
|
||
|
||
插件 E2E 验证:
|
||
|
||
```bash
|
||
PYTHONPATH="$(pwd)/plugins/memory-gateway-agent" \
|
||
python plugins/memory-gateway-agent/scripts/gateway_e2e_check.py
|
||
|
||
PYTHONPATH="$(pwd)/plugins/memory-gateway-agent" \
|
||
python plugins/memory-gateway-agent/scripts/hermes_hook_probe.py
|
||
```
|
||
|
||
清理测试数据:
|
||
|
||
```bash
|
||
PYTHONPATH="$(pwd)/plugins/memory-gateway-agent" \
|
||
python plugins/memory-gateway-agent/scripts/cleanup_test_memories.py
|
||
```
|
||
|
||
安全边界:
|
||
|
||
- plugin 不保存完整原始对话,只写摘要型 episode。
|
||
- 默认拒绝 password、token、API key、cookie、private key、完整 transcript 和大段日志。
|
||
- `memory_upsert` 是高风险长期记忆写入,不会自动触发。
|
||
- 用户要求 forget/delete 时,应走 `memory_feedback` 或 delete 能力。
|
||
- hook trace 默认关闭;需要排查时设置 `MEMORY_GATEWAY_PLUGIN_TRACE_HOOKS=true`,只会写入 hook 名称、短 session id、Gateway action 和状态到 `plugins/memory-gateway-agent/.tmp/hook_trace.log`。
|
||
|
||
OpenClaw manifest 目前是 best-effort 草案:
|
||
|
||
```text
|
||
plugins/memory-gateway-agent/openclaw.plugin.yaml
|
||
```
|
||
|
||
需要等 OpenClaw runtime 可用后再做第五阶段实测。
|
||
|
||
## 测试
|
||
|
||
```bash
|
||
cd memory-gateway
|
||
source .venv/bin/activate
|
||
PYTHONPATH="$(pwd)" pytest -q
|
||
|
||
PYTHONPATH="$(pwd)/plugins/memory-gateway-agent" \
|
||
pytest -q plugins/memory-gateway-agent/tests
|
||
```
|
||
|
||
当前测试覆盖:
|
||
|
||
- API key 校验。
|
||
- MCP tools/list。
|
||
- OpenViking search 透传。
|
||
- LLM summary artifact 构建。
|
||
- document upload -> markdown -> Obsidian -> OpenViking resource。
|
||
|
||
## 下一步
|
||
|
||
- 在 Gateway 层加强 `/api/search` 的 URI prefix 过滤和去重。
|
||
- 给 `/api/knowledge/upload` 增加文件大小限制、类型白名单和 dry-run。
|
||
- 增加 Obsidian -> OpenViking 增量同步脚本。
|
||
- 给 Memory Gateway skill 增加更稳定的“回答时引用 memory/resource/Obsidian note”输出约束。
|
||
- 增加更多文档解析格式和异常处理测试。
|