Add generic memory gateway v1
This commit is contained in:
767
docs/generic-memory-gateway-design.md
Normal file
767
docs/generic-memory-gateway-design.md
Normal file
@ -0,0 +1,767 @@
|
||||
# 通用 Memory Gateway 方案与 POC 骨架
|
||||
|
||||
本文基于当前仓库的轻量 FastAPI + MCP + OpenViking + Obsidian 能力扩展，不把系统设计成重平台。第一阶段目标是先跑通多用户隔离、namespace routing、记忆检索、写入、session commit 和人工 review 草稿，后续再替换持久化、向量索引和 EverMemOS worker。
|
||||
|
||||
## A. 总体架构图
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Agents["Agent Frameworks"]
|
||||
Nanobot[Nanobot]
|
||||
Hermes[Hermes Agent]
|
||||
OpenClaw[OpenClaw]
|
||||
Other[Other Agents]
|
||||
end
|
||||
|
||||
subgraph Gateway["Memory Gateway"]
|
||||
HTTP[HTTP API /v1]
|
||||
MCP[MCP tools]
|
||||
Auth[Auth / API Key / Future Login]
|
||||
ACL[ACL & Visibility Policy]
|
||||
Router[Namespace Router]
|
||||
Audit[Audit Log]
|
||||
Retrieval[Retrieval Orchestrator]
|
||||
Writeback[Writeback Orchestrator]
|
||||
end
|
||||
|
||||
subgraph Skills["Skills Layer"]
|
||||
Ingest[ingest]
|
||||
Extract[extract]
|
||||
Classify[classify]
|
||||
Retrieve[retrieve]
|
||||
Commit[commit]
|
||||
Merge[merge]
|
||||
Prune[prune]
|
||||
Summarize[summarize]
|
||||
end
|
||||
|
||||
subgraph OpenViking["OpenViking"]
|
||||
OVFS[context filesystem]
|
||||
OVMem[memory]
|
||||
OVRes[resources]
|
||||
OVSkills[skills]
|
||||
OVWorkspace[workspace]
|
||||
end
|
||||
|
||||
subgraph EverMemOS["EverMemOS"]
|
||||
LTE[long-term extraction]
|
||||
Consolidation[consolidation]
|
||||
Decay[decay]
|
||||
Dedup[dedup]
|
||||
Profile[profile evolution]
|
||||
end
|
||||
|
||||
subgraph Obsidian["Obsidian"]
|
||||
Vault[human editable memory vault]
|
||||
Reviews[review queue]
|
||||
Profiles[profiles]
|
||||
LongTerm[long-term notes]
|
||||
end
|
||||
|
||||
subgraph Storage["Storage"]
|
||||
DB[(metadata DB)]
|
||||
Vector[(vector index)]
|
||||
Files[(object / file storage)]
|
||||
end
|
||||
|
||||
Nanobot --> HTTP
|
||||
Hermes --> MCP
|
||||
OpenClaw --> HTTP
|
||||
Other --> HTTP
|
||||
Other --> MCP
|
||||
|
||||
HTTP --> Auth --> ACL --> Router
|
||||
MCP --> Auth
|
||||
Router --> Retrieval
|
||||
Router --> Writeback
|
||||
ACL --> Audit
|
||||
|
||||
Retrieval --> Skills
|
||||
Writeback --> Skills
|
||||
Skills --> OpenViking
|
||||
Skills --> EverMemOS
|
||||
Skills --> Obsidian
|
||||
|
||||
Gateway --> DB
|
||||
Gateway --> Vector
|
||||
Gateway --> Files
|
||||
OpenViking --> DB
|
||||
OpenViking --> Vector
|
||||
Obsidian --> Files
|
||||
EverMemOS --> DB
|
||||
EverMemOS --> Vector
|
||||
```
|
||||
|
||||
## B. 核心数据模型
|
||||
|
||||
代码骨架见 `memory_gateway/schemas.py`。核心模型如下。
|
||||
|
||||
### User
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "user_tom",
|
||||
"display_name": "Tom",
|
||||
"status": "active",
|
||||
"profile_namespace": "user/user_tom/profile",
|
||||
"preferences": {"language": "zh-CN"},
|
||||
"created_at": "2026-04-30T10:00:00Z",
|
||||
"updated_at": "2026-04-30T10:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
### Agent
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "agent_hermes_default",
|
||||
"name": "Hermes Default Agent",
|
||||
"framework": "hermes",
|
||||
"owner_user_id": "user_tom",
|
||||
"created_at": "2026-04-30T10:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
### Workspace
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "ws_memory_gateway",
|
||||
"name": "Memory Gateway POC",
|
||||
"owner_user_id": "user_tom",
|
||||
"member_user_ids": ["user_tom"],
|
||||
"allowed_agent_ids": ["agent_hermes_default"]
|
||||
}
|
||||
```
|
||||
|
||||
### Session
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "sess_20260430_001",
|
||||
"user_id": "user_tom",
|
||||
"agent_id": "agent_hermes_default",
|
||||
"workspace_id": "ws_memory_gateway",
|
||||
"status": "open",
|
||||
"expires_at": "2026-05-07T10:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
### MemoryRecord
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "mem_abc123",
|
||||
"user_id": "user_tom",
|
||||
"agent_id": "agent_hermes_default",
|
||||
"workspace_id": "ws_memory_gateway",
|
||||
"session_id": "sess_20260430_001",
|
||||
"namespace": "user/user_tom/long_term",
|
||||
"memory_type": "preference",
|
||||
"content": "用户偏好中文输出，结构化但不要过度平台化。",
|
||||
"summary": "中文、结构化、轻量 POC 优先。",
|
||||
"tags": ["preference", "style"],
|
||||
"importance": 0.8,
|
||||
"confidence": 0.9,
|
||||
"visibility": "private",
|
||||
"source": "conversation",
|
||||
"created_at": "2026-04-30T10:00:00Z",
|
||||
"updated_at": "2026-04-30T10:00:00Z",
|
||||
"expires_at": null,
|
||||
"version": 1
|
||||
}
|
||||
```
|
||||
|
||||
### EpisodeRecord
|
||||
|
||||
短期过程记录，默认不进入 Obsidian，不自动成为长期记忆。
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "epi_abc123",
|
||||
"user_id": "user_tom",
|
||||
"agent_id": "agent_hermes_default",
|
||||
"workspace_id": "ws_memory_gateway",
|
||||
"session_id": "sess_20260430_001",
|
||||
"namespace": "session/sess_20260430_001/episodic",
|
||||
"content": "本轮讨论了 Memory Gateway POC 范围。",
|
||||
"summary": "确认 POC 优先做隔离、检索、写入和整理。",
|
||||
"events": [],
|
||||
"tags": ["design"]
|
||||
}
|
||||
```
|
||||
|
||||
### ProfileRecord
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "profile_user_tom",
|
||||
"user_id": "user_tom",
|
||||
"namespace": "user/user_tom/profile",
|
||||
"display_name": "Tom",
|
||||
"stable_facts": ["正在设计通用 Memory Gateway"],
|
||||
"preferences": {"language": "Chinese"},
|
||||
"working_style": ["偏好可落地 POC"],
|
||||
"updated_from_memory_ids": ["mem_abc123"],
|
||||
"version": 3
|
||||
}
|
||||
```
|
||||
|
||||
### ACL / Visibility
|
||||
|
||||
`visibility` 四档：
|
||||
|
||||
- `private`：仅 `user_id` 相同可读写。
|
||||
- `agent-only`：同一 `user_id` 且同一 `agent_id` 可读写。
|
||||
- `workspace-shared`：在同一 `workspace_id` 且通过 workspace membership 授权后可读。
|
||||
- `global`：可公开检索，只能由受信任 actor 写入。
|
||||
|
||||
### AuditLog
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "audit_abc123",
|
||||
"actor_user_id": "user_tom",
|
||||
"actor_agent_id": "agent_hermes_default",
|
||||
"action": "memory_search",
|
||||
"target_type": "memory",
|
||||
"target_id": "mem_abc123",
|
||||
"namespace": "user/user_tom/long_term",
|
||||
"decision": "allow",
|
||||
"reason": "private owner",
|
||||
"created_at": "2026-04-30T10:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
## C. Namespace 与隔离设计
|
||||
|
||||
推荐 namespace：
|
||||
|
||||
```text
|
||||
user/{user_id}/profile
|
||||
user/{user_id}/preferences
|
||||
user/{user_id}/long_term
|
||||
agent/{agent_id}/memory
|
||||
workspace/{workspace_id}/shared
|
||||
session/{session_id}/episodic
|
||||
global/public
|
||||
```
|
||||
|
||||
隔离规则：
|
||||
|
||||
- 用户隔离：所有 `user/{user_id}/...` 默认只允许同一 `user_id` 访问。Gateway 先校验 actor，再把 namespace 映射到 OpenViking URI。
|
||||
- Agent 隔离：`agent/{agent_id}/memory` 用于某个 agent 的工具经验、失败教训、prompt working notes。默认 `agent-only`。
|
||||
- Workspace 共享：`workspace/{workspace_id}/shared` 必须检查用户是否属于 workspace，agent 是否在 `allowed_agent_ids` 内。
|
||||
- Session 过期：`session/{session_id}/episodic` 必须有 TTL。过期后不可检索；只保留必要 audit。
|
||||
- 可跨 agent 共享：用户显式确认的 profile、preferences、user long_term、workspace shared、global public。
|
||||
- 不可跨 agent 共享：agent-only memory、未 commit 的 session episodic、低置信度候选记忆、含敏感凭据或临时日志的内容。
|
||||
|
||||
OpenViking URI 映射：
|
||||
|
||||
```text
|
||||
viking://memory/user/{user_id}/long_term/{memory_id}.json
|
||||
viking://resources/workspace/{workspace_id}/shared/{slug}.md
|
||||
viking://skills/memory-gateway/{skill_name}
|
||||
```
|
||||
|
||||
## D. API 设计
|
||||
|
||||
第一阶段代码已挂载 `/v1` router，见 `memory_gateway/api_v1.py`。
|
||||
|
||||
### POST /v1/users
|
||||
|
||||
Request:
|
||||
|
||||
```json
|
||||
{"user_id": "user_tom", "display_name": "Tom", "preferences": {"language": "zh-CN"}}
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{"id": "user_tom", "display_name": "Tom", "profile_namespace": "user/user_tom/profile", "status": "active"}
|
||||
```
|
||||
|
||||
### GET /v1/users/{user_id}
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{"id": "user_tom", "display_name": "Tom", "status": "active"}
|
||||
```
|
||||
|
||||
### POST /v1/memory/search
|
||||
|
||||
Request:
|
||||
|
||||
```json
|
||||
{
|
||||
"user_id": "user_tom",
|
||||
"agent_id": "agent_hermes_default",
|
||||
"workspace_id": "ws_memory_gateway",
|
||||
"query": "中文输出偏好",
|
||||
"namespaces": ["user/user_tom/long_term"],
|
||||
"limit": 5
|
||||
}
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{
|
||||
"results": [
|
||||
{
|
||||
"memory": {
|
||||
"id": "mem_abc123",
|
||||
"namespace": "user/user_tom/long_term",
|
||||
"summary": "中文、结构化、轻量 POC 优先。"
|
||||
},
|
||||
"score": 2.7
|
||||
}
|
||||
],
|
||||
"total": 1
|
||||
}
|
||||
```
|
||||
|
||||
### POST /v1/memory
|
||||
|
||||
Request:
|
||||
|
||||
```json
|
||||
{
|
||||
"user_id": "user_tom",
|
||||
"agent_id": "agent_hermes_default",
|
||||
"workspace_id": "ws_memory_gateway",
|
||||
"memory_type": "preference",
|
||||
"content": "用户偏好中文输出。",
|
||||
"summary": "中文输出偏好",
|
||||
"tags": ["preference"],
|
||||
"importance": 0.8,
|
||||
"confidence": 0.9,
|
||||
"visibility": "private",
|
||||
"source": "manual"
|
||||
}
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{"id": "mem_abc123", "namespace": "user/user_tom/long_term", "version": 1}
|
||||
```
|
||||
|
||||
### GET /v1/memory/{memory_id}
|
||||
|
||||
Request query:
|
||||
|
||||
```text
|
||||
?user_id=user_tom&agent_id=agent_hermes_default&workspace_id=ws_memory_gateway
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{"id": "mem_abc123", "content": "用户偏好中文输出。", "visibility": "private"}
|
||||
```
|
||||
|
||||
### PATCH /v1/memory/{memory_id}
|
||||
|
||||
Request:
|
||||
|
||||
```json
|
||||
{"summary": "用户偏好中文、结构化、少废话。", "importance": 0.9}
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{"id": "mem_abc123", "version": 2, "importance": 0.9}
|
||||
```
|
||||
|
||||
### DELETE /v1/memory/{memory_id}
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{"deleted": true, "id": "mem_abc123"}
|
||||
```
|
||||
|
||||
### POST /v1/episodes
|
||||
|
||||
Request:
|
||||
|
||||
```json
|
||||
{
|
||||
"user_id": "user_tom",
|
||||
"agent_id": "agent_hermes_default",
|
||||
"workspace_id": "ws_memory_gateway",
|
||||
"session_id": "sess_001",
|
||||
"content": "本轮完成了 namespace 和 ACL 设计。",
|
||||
"tags": ["design"]
|
||||
}
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{"id": "epi_abc123", "namespace": "session/sess_001/episodic"}
|
||||
```
|
||||
|
||||
### POST /v1/sessions/{session_id}/commit
|
||||
|
||||
Request:
|
||||
|
||||
```json
|
||||
{
|
||||
"user_id": "user_tom",
|
||||
"agent_id": "agent_hermes_default",
|
||||
"workspace_id": "ws_memory_gateway",
|
||||
"promote": true,
|
||||
"min_importance": 0.6,
|
||||
"target_namespace": "user/user_tom/long_term"
|
||||
}
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{"session_id": "sess_001", "episodes": 3, "promoted": [{"id": "mem_def456"}]}
|
||||
```
|
||||
|
||||
### GET /v1/users/{user_id}/profile
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{"user_id": "user_tom", "namespace": "user/user_tom/profile", "preferences": {"language": "zh-CN"}}
|
||||
```
|
||||
|
||||
### POST /v1/memory/{memory_id}/feedback
|
||||
|
||||
Request:
|
||||
|
||||
```json
|
||||
{"user_id": "user_tom", "feedback": "incorrect", "comment": "这是一次临时偏好，不应长期保留。"}
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{"status": "ok", "memory_id": "mem_abc123", "feedback": "incorrect"}
|
||||
```
|
||||
|
||||
### GET /v1/namespaces
|
||||
|
||||
Request query:
|
||||
|
||||
```text
|
||||
?user_id=user_tom&agent_id=agent_hermes_default&workspace_id=ws_memory_gateway&session_id=sess_001
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
[
|
||||
{"namespace": "user/user_tom/profile", "visibility": "private"},
|
||||
{"namespace": "agent/agent_hermes_default/memory", "visibility": "agent-only"},
|
||||
{"namespace": "workspace/ws_memory_gateway/shared", "visibility": "workspace-shared"}
|
||||
]
|
||||
```
|
||||
|
||||
### GET /v1/audit
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
[{"action": "upsert_memory", "target_type": "memory", "decision": "allow"}]
|
||||
```
|
||||
|
||||
### MCP tools
|
||||
|
||||
目标 v1 tools 见 `memory_gateway/mcp_tools_v1.py`：
|
||||
|
||||
- `memory_search`
|
||||
- `memory_upsert`
|
||||
- `memory_append_episode`
|
||||
- `memory_commit_session`
|
||||
- `memory_get_profile`
|
||||
- `memory_list_namespaces`
|
||||
- `memory_delete`
|
||||
- `memory_feedback`
|
||||
|
||||
示例 MCP call:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "memory_search",
|
||||
"arguments": {
|
||||
"user_id": "user_tom",
|
||||
"agent_id": "agent_hermes_default",
|
||||
"workspace_id": "ws_memory_gateway",
|
||||
"query": "项目 POC 决策",
|
||||
"limit": 5
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## E. Skills 设计
|
||||
|
||||
代码骨架位于 `memory_gateway/skills/`。
|
||||
|
||||
| Skill | 功能 | 输入 | 输出 | 触发时机 | 组件 | 写长期记忆 |
|
||||
|---|---|---|---|---|---|---|
|
||||
| `ingest_skill` | 标准化对话、文件、任务事件 | raw text/file/events | normalized payload | agent 写入 episode 前 | Gateway, file storage | 否 |
|
||||
| `extract_memory_skill` | 从 episode/session 抽取候选记忆 | episode/session content | memory candidates | session commit / worker 定时 | LLM, EverMemOS | 否 |
|
||||
| `classify_memory_skill` | 判断 memory_type、visibility、namespace | candidate memory | classification | 写入前 | ACL, namespace router | 否 |
|
||||
| `retrieve_context_skill` | 聚合用户、agent、workspace 上下文 | query + context ids | ranked contexts | agent 调用前 | OpenViking, vector index | 否 |
|
||||
| `commit_memory_skill` | 写入长期记忆 | MemoryRecord | stored record | 人工确认或 commit 通过 | DB, OpenViking | 是 |
|
||||
| `summarize_episode_skill` | 压缩 episode | episode content | summary | session commit | LLM | 否 |
|
||||
| `merge_memory_skill` | 合并重复或相近记忆 | memory ids | merged memory | EverMemOS 整理 | DB, vector index | 是 |
|
||||
| `prune_memory_skill` | 衰减、归档、删除低质记忆 | policy + memory ids | archived/deleted list | 定时 worker | EverMemOS | 是 |
|
||||
| `export_to_obsidian_skill` | 生成 Obsidian review draft | high-value memory | markdown draft | 高价值或需人工确认 | Obsidian | 否 |
|
||||
| `import_from_obsidian_skill` | 从人工维护笔记导入记忆 | markdown path | MemoryRecord | vault sync | Obsidian, OpenViking | 是 |
|
||||
|
||||
## F. Obsidian Vault 设计
|
||||
|
||||
推荐目录：
|
||||
|
||||
```text
|
||||
obsidian-vault/
|
||||
├── Users/
|
||||
│ └── {user_id}/
|
||||
│ ├── Profile.md
|
||||
│ ├── Preferences.md
|
||||
│ └── LongTerm/
|
||||
├── Agents/
|
||||
│ └── {agent_id}/Experience.md
|
||||
├── Workspaces/
|
||||
│ └── {workspace_id}/Shared.md
|
||||
├── Memories/
|
||||
│ ├── LongTerm/
|
||||
│ └── Archived/
|
||||
├── Profiles/
|
||||
├── Reviews/
|
||||
│ ├── Queue/
|
||||
│ ├── Accepted/
|
||||
│ └── Rejected/
|
||||
├── Exports/
|
||||
└── Templates/
|
||||
```
|
||||
|
||||
进入 Obsidian 的内容：
|
||||
|
||||
- 人工可维护 profile、preferences、长期总结。
|
||||
- 高价值 workspace 知识、项目决策、复用经验。
|
||||
- EverMemOS 标记为 `needs_review` 的长期记忆草稿。
|
||||
|
||||
不进入 Obsidian 的内容：
|
||||
|
||||
- 全量原始对话。
|
||||
- 高频工具日志、临时 session trace。
|
||||
- 低置信度候选记忆。
|
||||
- 敏感凭据、token、临时错误栈。
|
||||
|
||||
标签体系：
|
||||
|
||||
```text
|
||||
#memory/profile
|
||||
#memory/preference
|
||||
#memory/long-term
|
||||
#memory/workspace
|
||||
#memory/agent-experience
|
||||
#memory/review
|
||||
#memory/conflict
|
||||
#memory/deprecated
|
||||
#source/evermemos
|
||||
#source/manual
|
||||
#visibility/private
|
||||
#visibility/workspace-shared
|
||||
```
|
||||
|
||||
模板文件已加入 `obsidian-vault/05_Templates/`。
|
||||
|
||||
## G. OpenViking 设计
|
||||
|
||||
OpenViking 作为统一 context 层，Gateway 不要求 agent 直接理解 OpenViking 内部结构。
|
||||
|
||||
组织方式：
|
||||
|
||||
```text
|
||||
viking://memory/user/{user_id}/profile
|
||||
viking://memory/user/{user_id}/preferences
|
||||
viking://memory/user/{user_id}/long_term
|
||||
viking://memory/agent/{agent_id}/memory
|
||||
viking://memory/workspace/{workspace_id}/shared
|
||||
viking://resources/user/{user_id}/obsidian/{note_id}.md
|
||||
viking://skills/memory-gateway/{skill_name}
|
||||
```
|
||||
|
||||
检索路径：
|
||||
|
||||
1. Agent 调用 Gateway `/v1/memory/search` 或 MCP `memory_search`。
|
||||
2. Gateway 执行 Auth、ACL、namespace expansion。
|
||||
3. Gateway 查询 metadata DB 和 vector index，必要时调用 OpenViking search。
|
||||
4. 返回统一 `MemoryRecord` 或 context chunk，不暴露底层差异。
|
||||
|
||||
同步：
|
||||
|
||||
- Obsidian accepted note 通过 `import_from_obsidian_skill` 写回 Gateway，再同步 OpenViking resource。
|
||||
- EverMemOS consolidation 后写入 `user/{user_id}/long_term` 或 `workspace/{workspace_id}/shared`。
|
||||
- Gateway 保存 `source_ref`，避免 OpenViking 与 Obsidian 互相重复导入。
|
||||
|
||||
## H. EverMemOS 设计
|
||||
|
||||
输入来源：
|
||||
|
||||
- `EpisodeRecord`：对话片段、任务执行摘要、agent 过程事件。
|
||||
- `SessionRecord`：session commit 包。
|
||||
- `MemoryFeedback`：incorrect、duplicate、outdated 等反馈。
|
||||
- Obsidian review 结果：accepted/rejected/edited。
|
||||
|
||||
整理流程：
|
||||
|
||||
1. 抽取：从 episode 中提炼候选事实、偏好、决策、经验。
|
||||
2. 打分：根据重要性、稳定性、重复出现次数、来源可信度打分。
|
||||
3. 去重：按 semantic hash + embedding 相似度查找近似 MemoryRecord。
|
||||
4. 合并：相同事实合并 evidence；更高置信度覆盖低置信度。
|
||||
5. 冲突检测：同一 subject 的相反陈述标记 `needs_review`，不自动覆盖。
|
||||
6. 衰减：长时间未命中且低反馈的记忆降低 importance。
|
||||
7. 归档：过期、错误、低置信度、被人工拒绝的记忆转 archived。
|
||||
8. profile evolution：只有稳定、重复、高置信偏好进入 ProfileRecord。
|
||||
|
||||
污染控制：
|
||||
|
||||
- session 临时内容不直接提升为长期记忆。
|
||||
- LLM 抽取结果默认是 candidate，需阈值或人工确认。
|
||||
- 每条长期记忆保留 source、confidence、version、feedback。
|
||||
- 对 profile 更新采用 evidence count，禁止一次对话永久改写强偏好。
|
||||
|
||||
## I. 工程目录结构
|
||||
|
||||
当前仓库保留 `memory_gateway/` 包名，目标结构如下：
|
||||
|
||||
```text
|
||||
memory-gateway/
|
||||
├── memory_gateway/
|
||||
│ ├── api_v1.py # v1 HTTP API
|
||||
│ ├── mcp_tools_v1.py # v1 MCP tool contract
|
||||
│ ├── schemas.py # User/Memory/Episode/Profile/ACL/Audit
|
||||
│ ├── namespace.py # namespace builder + ACL helpers
|
||||
│ ├── services.py # orchestration service
|
||||
│ ├── repositories.py # POC in-memory repo; later DB repo
|
||||
│ ├── security/ # future auth, RBAC, audit policy
|
||||
│ ├── skills/
|
||||
│ │ ├── ingest_skill.py
|
||||
│ │ ├── extract_memory_skill.py
|
||||
│ │ ├── classify_memory_skill.py
|
||||
│ │ ├── retrieve_context_skill.py
|
||||
│ │ ├── commit_memory_skill.py
|
||||
│ │ ├── summarize_episode_skill.py
|
||||
│ │ ├── merge_memory_skill.py
|
||||
│ │ ├── prune_memory_skill.py
|
||||
│ │ ├── export_to_obsidian_skill.py
|
||||
│ │ └── import_from_obsidian_skill.py
|
||||
│ ├── adapters/
|
||||
│ │ ├── openviking.py
|
||||
│ │ ├── evermemos.py
|
||||
│ │ └── obsidian.py
|
||||
│ └── workers/
|
||||
│ └── evermemos_worker.py
|
||||
├── obsidian-vault/
|
||||
├── integrations/
|
||||
│ ├── nanobot/
|
||||
│ ├── hermes/
|
||||
│ └── openclaw/
|
||||
└── tests/
|
||||
```
|
||||
|
||||
如果未来迁移到更标准的 `app/`，可把 `memory_gateway/api_v1.py` 对应到 `app/api`，`schemas.py` 对应到 `app/schemas`，`services.py` 对应到 `app/services`。
|
||||
|
||||
## J. 2 到 4 周 POC 实施计划
|
||||
|
||||
第一周：
|
||||
|
||||
- 完成 `/v1/users`、`/v1/memory`、`/v1/memory/search`、`/v1/episodes`。
|
||||
- 实现 namespace router、visibility、基础 audit。
|
||||
- 存储先用 SQLite 或当前内存 repo，搜索先用 lexical，OpenViking 作为可选后端。
|
||||
|
||||
第二周：
|
||||
|
||||
- 接入 OpenViking URI 写入和检索。
|
||||
- 实现 `retrieve_context_skill`、`commit_memory_skill`、`summarize_episode_skill`。
|
||||
- 给 Hermes/Nanobot/OpenClaw 提供最小 client 示例。
|
||||
|
||||
第三周：
|
||||
|
||||
- 加 EverMemOS worker 原型：session commit、candidate extraction、dedup、merge。
|
||||
- 增加 feedback 流程：incorrect、duplicate、outdated 影响 prune/merge。
|
||||
- 生成 Obsidian review draft，而不是直接写入最终知识库。
|
||||
|
||||
第四周：
|
||||
|
||||
- Obsidian import/export 双向同步。
|
||||
- 增加 profile evolution 的阈值和 evidence 机制。
|
||||
- 补充权限测试、污染测试、重复记忆测试、跨 agent 检索测试。
|
||||
|
||||
先做：
|
||||
|
||||
- 用户隔离、namespace、memory CRUD、episode append、session commit、basic search、audit。
|
||||
|
||||
暂不做：
|
||||
|
||||
- 完整登录系统、复杂 RBAC、多租户计费、实时同步、复杂 UI、全量向量数据库治理。
|
||||
|
||||
POC 成功指标：
|
||||
|
||||
- 不同 `user_id` 之间无法互相读写 private memory。
|
||||
- 同一 workspace 的共享记忆可被授权 agent 检索。
|
||||
- session 记忆不会自动污染长期记忆。
|
||||
- 10 条重复候选能合并到 1 到 2 条长期记忆。
|
||||
- 错误反馈后，该记忆不再进入默认 retrieval。
|
||||
- Hermes/Nanobot/OpenClaw 至少两个框架能通过统一 API 调用。
|
||||
|
||||
## K. 推荐默认方案
|
||||
|
||||
第一阶段最合理默认方案：
|
||||
|
||||
- FastAPI 提供 `/v1` 统一 HTTP API。
|
||||
- MCP 先保留现有 `/mcp/rpc`，新增 `memory_gateway/mcp_tools_v1.py` 作为目标 contract。
|
||||
- 存储使用 SQLite metadata + 本地文件存 object；当前代码先用 in-memory repo 验证接口。
|
||||
- 搜索先用 OpenViking search + 简单 lexical fallback；向量索引第二阶段引入。
|
||||
- Obsidian 只保存人工可读的高价值长期记忆和 review draft。
|
||||
- EverMemOS 第一阶段不做独立大系统，只做 worker 模块：extract、dedup、merge、prune、profile update。
|
||||
|
||||
第一阶段实现 API：
|
||||
|
||||
- `POST /v1/users`
|
||||
- `GET /v1/users/{user_id}`
|
||||
- `POST /v1/memory/search`
|
||||
- `POST /v1/memory`
|
||||
- `GET /v1/memory/{memory_id}`
|
||||
- `POST /v1/episodes`
|
||||
- `POST /v1/sessions/{session_id}/commit`
|
||||
- `GET /v1/users/{user_id}/profile`
|
||||
- `GET /v1/namespaces`
|
||||
|
||||
第一阶段实现 skills：
|
||||
|
||||
- `ingest_skill`
|
||||
- `summarize_episode_skill`
|
||||
- `retrieve_context_skill`
|
||||
- `commit_memory_skill`
|
||||
- `export_to_obsidian_skill`
|
||||
|
||||
第二阶段再补：
|
||||
|
||||
- `extract_memory_skill`
|
||||
- `classify_memory_skill`
|
||||
- `merge_memory_skill`
|
||||
- `prune_memory_skill`
|
||||
- `import_from_obsidian_skill`
|
||||
- 更完整的 EverMemOS consolidation 和 profile evolution。
|
||||
|
||||
角色分工：
|
||||
|
||||
- Obsidian 第一阶段：review draft、人类确认 profile/长期知识。第二阶段：双向同步。
|
||||
- OpenViking 第一阶段：统一 context/resource 检索入口。第二阶段：承载多 namespace context filesystem 和 skill registry。
|
||||
- EverMemOS 第一阶段：session commit worker。第二阶段：长期记忆治理、衰减、冲突检测、profile evolution。
|
||||
|
||||
Reference in New Issue
Block a user