283 lines
6.5 KiB
Markdown
283 lines
6.5 KiB
Markdown
# Memory Gateway
|
||
|
||
Memory Gateway 是一个通用记忆网关，用于给 AI agent / harness 提供统一的记忆检索、文档上传、LLM 总结和知识沉淀能力。
|
||
|
||
它的定位不是某个单一业务场景的垂直应用，而是一个可复用的本地 memory/context gateway：上层 agent 通过 REST、MCP 或 Hermes skill 调用它，底层由 OpenViking 承载 memory/resource，由 Obsidian 承载人工可维护的 Markdown 知识。
|
||
|
||
## 当前能力
|
||
|
||
- 搜索 OpenViking memory / resource。
|
||
- 写入普通 memory。
|
||
- 写入结构化 resource。
|
||
- 对任意文本调用 LLM 总结，并按需沉淀到 OpenViking。
|
||
- 上传文档，使用 MarkItDown 转 Markdown。
|
||
- 将上传文档保存到 Obsidian vault。
|
||
- 将文档摘要和结构化 artifact 写入 OpenViking knowledge。
|
||
- 给 Hermes 提供通用 `memory-gateway` skill。
|
||
|
||
## 架构
|
||
|
||
```text
|
||
Agent / Harness / CLI
|
||
-> Memory Gateway REST / MCP
|
||
-> OpenViking memory / resource
|
||
-> Obsidian markdown vault
|
||
-> OpenAI-compatible LLM summary
|
||
```
|
||
|
||
## 目录结构
|
||
|
||
```text
|
||
memory-gateway/
|
||
├── memory_gateway/ # Gateway 服务代码
|
||
│ ├── server.py # REST / MCP 接口
|
||
│ ├── openviking_client.py # OpenViking client
|
||
│ ├── llm.py # OpenAI-compatible LLM summary
|
||
│ ├── document_ingest.py # MarkItDown + Obsidian write helpers
|
||
│ ├── config.py
|
||
│ └── types.py
|
||
├── integrations/hermes/
|
||
│ └── memory-gateway/ # 通用 Hermes skill
|
||
├── obsidian-vault/
|
||
│ ├── 01_Knowledge/Uploaded/ # 上传文档转成的 Markdown
|
||
│ └── 05_Templates/ # 通用知识模板
|
||
├── tests/
|
||
├── config.example.yaml
|
||
└── pyproject.toml
|
||
```
|
||
|
||
## 环境
|
||
|
||
使用当前虚拟环境：
|
||
|
||
```bash
|
||
cd /home/tom/memory-gateway
|
||
source /home/tom/OpenViking/.venv/bin/activate
|
||
```
|
||
|
||
本地配置文件：
|
||
|
||
```text
|
||
/home/tom/memory-gateway/config.yaml
|
||
```
|
||
|
||
关键配置：
|
||
|
||
```yaml
|
||
memory:
|
||
default_namespace: memory-gateway
|
||
|
||
llm:
|
||
base_url: https://oai.bwgdi.com/v1
|
||
api_key: <local secret, git ignored>
|
||
model: MiniMaxAI
|
||
|
||
obsidian:
|
||
vault_path: /home/tom/memory-gateway/obsidian-vault
|
||
knowledge_dir: 01_Knowledge/Uploaded
|
||
```
|
||
|
||
`config.yaml` 已被 `.gitignore` 忽略，不会提交密钥。
|
||
|
||
## 启动
|
||
|
||
OpenViking 需要先运行在 `127.0.0.1:1933`。
|
||
|
||
```bash
|
||
source /home/tom/OpenViking/.venv/bin/activate
|
||
openviking-server --host 127.0.0.1 --port 1933
|
||
```
|
||
|
||
启动 Memory Gateway：
|
||
|
||
```bash
|
||
cd /home/tom/memory-gateway
|
||
source /home/tom/OpenViking/.venv/bin/activate
|
||
python -m memory_gateway.server --config /home/tom/memory-gateway/config.yaml
|
||
```
|
||
|
||
健康检查：
|
||
|
||
```bash
|
||
curl http://127.0.0.1:1934/health
|
||
```
|
||
|
||
## REST 接口
|
||
|
||
### `GET /health`
|
||
|
||
检查 Gateway 和 OpenViking 状态。
|
||
|
||
### `POST /api/search`
|
||
|
||
搜索 OpenViking memory / resource。
|
||
|
||
```bash
|
||
curl -X POST http://127.0.0.1:1934/api/search \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"query": "memory gateway document upload summary",
|
||
"uri": "viking://resources",
|
||
"limit": 5
|
||
}'
|
||
```
|
||
|
||
### `POST /api/memory`
|
||
|
||
写入普通 memory。
|
||
|
||
```bash
|
||
curl -X POST http://127.0.0.1:1934/api/memory \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"namespace": "memory-gateway",
|
||
"memory_type": "preference",
|
||
"content": "The user prefers concise technical summaries."
|
||
}'
|
||
```
|
||
|
||
### `POST /api/resource`
|
||
|
||
写入结构化 resource。
|
||
|
||
```bash
|
||
curl -X POST http://127.0.0.1:1934/api/resource \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"uri": "viking://resources/memory-gateway/knowledge/example.json",
|
||
"resource_type": "json",
|
||
"content": "{\"title\":\"example\"}"
|
||
}'
|
||
```
|
||
|
||
### `POST /api/summary`
|
||
|
||
调用 LLM 总结任意文本，并按需沉淀到 OpenViking。
|
||
|
||
```bash
|
||
curl -X POST http://127.0.0.1:1934/api/summary \
|
||
-H "Content-Type: application/json" \
|
||
-d '{
|
||
"title": "Project decision summary",
|
||
"content": "需要总结和沉淀的内容...",
|
||
"namespace": "memory-gateway",
|
||
"memory_type": "decision",
|
||
"tags": ["project", "decision"],
|
||
"persist_as": "resource"
|
||
}'
|
||
```
|
||
|
||
`persist_as` 支持：`none`、`memory`、`resource`、`both`。
|
||
|
||
### `POST /api/knowledge/upload`
|
||
|
||
上传文档，MarkItDown 转 Markdown，保存到 Obsidian，LLM 总结后写入 OpenViking knowledge。
|
||
|
||
```bash
|
||
curl -X POST http://127.0.0.1:1934/api/knowledge/upload \
|
||
-F "file=@/path/to/document.pdf" \
|
||
-F "title=Design Notes" \
|
||
-F "namespace=memory-gateway" \
|
||
-F "knowledge_type=design_doc" \
|
||
-F "tags=project,design,reference" \
|
||
-F "persist_as=resource"
|
||
```
|
||
|
||
默认保存到：
|
||
|
||
```text
|
||
obsidian-vault/01_Knowledge/Uploaded/
|
||
```
|
||
|
||
## MCP Tools
|
||
|
||
`POST /mcp/rpc` 支持：
|
||
|
||
- `search`
|
||
- `add_memory`
|
||
- `add_resource`
|
||
- `commit_summary`
|
||
- `get_status`
|
||
- `list_memories`
|
||
- `list_resources`
|
||
|
||
## Hermes Skill
|
||
|
||
通用 Hermes skill：
|
||
|
||
```text
|
||
/home/tom/.hermes/skills/memory-gateway/
|
||
```
|
||
|
||
仓库副本：
|
||
|
||
```text
|
||
integrations/hermes/memory-gateway/
|
||
```
|
||
|
||
主要脚本：
|
||
|
||
```text
|
||
scripts/retrieve_memory.py
|
||
scripts/commit_summary.py
|
||
scripts/upload_knowledge.py
|
||
scripts/search_obsidian.py
|
||
```
|
||
|
||
检索记忆：
|
||
|
||
```bash
|
||
python /home/tom/.hermes/skills/memory-gateway/scripts/retrieve_memory.py \
|
||
--query "document upload summary memory gateway" \
|
||
--uri viking://resources \
|
||
--limit 5
|
||
```
|
||
|
||
总结沉淀：
|
||
|
||
```bash
|
||
python /home/tom/.hermes/skills/memory-gateway/scripts/commit_summary.py \
|
||
--title "Reusable conclusion" \
|
||
--namespace memory-gateway \
|
||
--memory-type decision \
|
||
--tag project \
|
||
--persist-as resource \
|
||
--text "最终结论或可复用知识..."
|
||
```
|
||
|
||
上传知识：
|
||
|
||
```bash
|
||
python /home/tom/.hermes/skills/memory-gateway/scripts/upload_knowledge.py \
|
||
--file /path/to/document.md \
|
||
--title "Knowledge note" \
|
||
--namespace memory-gateway \
|
||
--knowledge-type reference \
|
||
--tags project,reference \
|
||
--persist-as resource
|
||
```
|
||
|
||
## 测试
|
||
|
||
```bash
|
||
cd /home/tom/memory-gateway
|
||
source /home/tom/OpenViking/.venv/bin/activate
|
||
PYTHONPATH=/home/tom/memory-gateway pytest -q
|
||
```
|
||
|
||
当前测试覆盖：
|
||
|
||
- API key 校验。
|
||
- MCP tools/list。
|
||
- OpenViking search 透传。
|
||
- LLM summary artifact 构建。
|
||
- document upload -> markdown -> Obsidian -> OpenViking resource。
|
||
|
||
## 下一步
|
||
|
||
- 在 Gateway 层加强 `/api/search` 的 URI prefix 过滤和去重。
|
||
- 给 `/api/knowledge/upload` 增加文件大小限制、类型白名单和 dry-run。
|
||
- 增加 Obsidian -> OpenViking 增量同步脚本。
|
||
- 给 Memory Gateway skill 增加更稳定的“回答时引用 memory/resource/Obsidian note”输出约束。
|
||
- 增加更多文档解析格式和异常处理测试。
|