diff --git a/README.md b/README.md index 3cc366c..e4412f9 100644 --- a/README.md +++ b/README.md @@ -1,974 +1,374 @@ # Memory Gateway -Memory Gateway 是一个通用记忆网关，用于给 AI agent / harness 提供统一的记忆检索、文档上传、LLM 总结和知识沉淀能力。 +Memory Gateway 是一个本地 memory/context gateway，用统一的 HTTP、MCP 和 Hermes skill 入口，把上层 agent 的记忆写入、上下文检索、会话提交和知识沉淀路由到 OpenViking、EverOS、SQLite metadata store 和可选 Obsidian vault。 -它的定位不是某个单一业务场景的垂直应用，而是一个可复用的本地 memory/context gateway：上层 agent 通过 REST、MCP 或 Hermes skill 调用它，底层由 OpenViking 承载 memory/resource，由 Obsidian 承载人工可维护的 Markdown 知识。 +当前项目的主线是 **OpenViking + EverOS 双后端**： -## 当前能力 +- OpenViking 运行在 `127.0.0.1:1933`，负责 session/resource/context 层能力。 +- EverOS/EverCore 运行在 `127.0.0.1:1995`，负责长期记忆、profile 和检索。 +- Memory Gateway 默认运行在 `127.0.0.1:1934`，提供统一 API、认证、metadata、outbox 和 adapter 编排。 -- 搜索 OpenViking memory / resource。 -- 写入普通 memory。 -- 写入结构化 resource。 -- 对任意文本调用 LLM 总结，并按需沉淀到 OpenViking。 -- 上传文档，使用 MarkItDown 转 Markdown。 -- 将上传文档保存到 Obsidian vault。 -- 将文档摘要和结构化 artifact 写入 OpenViking knowledge。 -- 给 Hermes 提供通用 `memory-gateway` skill。 -- 新增通用 Memory Gateway v1 方案与 POC 骨架：多用户、namespace、visibility/ACL、episode、session commit、audit、skills 分层。 -- v1 metadata 默认持久化到 SQLite，覆盖 users、memories、episodes、profiles、audit。 -- `/v1/memory/search` 先做本地 ACL 过滤，再按可见 namespace 查询 OpenViking。 -- v1 MCP tools 已接入现有 `/mcp/rpc`。 -- `/v1/sessions/{session_id}/commit` 优先调用独立 EverMemOS HTTP 服务；服务不可用且允许 fallback 时，才使用 Gateway 进程内 POC worker。 +## 核心能力 -完整方案见： +- `/v2/conversations/ingest`：把一轮对话写入 OpenViking 和 EverOS，并在本地保存 `memory_refs` 控制面引用。 +- `/v2/context/retrieve`：实际调用 OpenViking / EverOS 的 retrieve 接口，把两个后端返回的上下文合并到 `items`。 +- `/v2/conversations/{session_id}/commit`：创建 commit job 和 outbox events，用于异步生成长期 ref。 +- `/v2/admin/outbox/process`：处理 pending outbox，生成 OpenViking session ref、EverOS profile ref、EverOS long-term ref。 +- `/v2/memory/refs`：查询本地保存的后端引用元数据。 +- `/v1/*`：保留基础用户、memory、episode、session commit、audit 和 EverOS health 能力。 +- `/api/*`：保留旧版搜索、写 memory/resource、LLM summary、文档上传到 Obsidian/OpenViking 的兼容接口。 +- `/mcp/rpc` 与 `/mcp/sse`：提供 MCP 调用入口。 -```text -docs/generic-memory-gateway-design.md -``` - -## 总体架构 +## 架构 ```mermaid flowchart LR - Agent[AI Agents / Hermes / OpenClaw / Nanobot] -->|HTTP / MCP / Plugin Tools / Skill Scripts| Gateway[Memory Gateway] - Gateway --> Auth[Auth + ACL + Namespace Router] - Gateway --> SQLite[(SQLite Metadata Store)] - Gateway --> OV[OpenViking Context Layer] - Gateway --> Ever[EverMemOS Consolidation Service] + Agent[Agent / Hermes / MCP Client] --> Gateway[Memory Gateway] + Gateway --> SQLite[(SQLite metadata)] + Gateway --> OpenViking[OpenViking API] + Gateway --> EverOS[EverOS / EverCore API] Gateway --> Obsidian[Obsidian Markdown Vault] Gateway --> LLM[OpenAI-compatible LLM] - OV --> OVStore[(OpenViking memory/resource/index)] - Ever -->|promote stable memory| SQLite - Ever -->|review drafts| Obsidian - Obsidian --> Human[Human Review / Manual Editing] + + SQLite --> Refs[memory_refs / outbox_events / commit_jobs] + OpenViking --> OVStore[(session / resource / context index)] + EverOS --> EverStore[(memory / profile / long-term store)] ``` -模块职责： +Memory Gateway 不把完整对话正文长期保存到 SQLite。SQLite 主要保存控制面 metadata：用户、episode、audit、`memory_refs`、`commit_jobs`、`outbox_events`。真正的记忆正文和检索上下文在 OpenViking / EverOS 后端中。 -- **Memory Gateway**：统一入口，提供 REST、MCP RPC、Hermes/OpenClaw plugin 所需的 `/v1` API；负责 user context、namespace 展开、visibility/ACL、audit、短期 episode 写入和 session commit 编排。 -- **OpenViking**：context/resource/memory 检索层；Gateway 会把允许访问的 namespace 展开后交给 OpenViking 搜索。 -- **EverMemOS**：长期记忆整理层；从 session episodes 中提取稳定候选，去重、合并、冲突检测，并决定是否 promote 到长期记忆或进入 review。 -- **Obsidian Vault**：人工可维护的 Markdown 前台；保存上传文档、review draft、高价值可审查知识。不要把所有原始对话直接写入 Obsidian。 -- **Hermes skill**：脚本式、显式调用入口，适合人工/agent 明确执行搜索、上传、commit。 -- **Agent plugin**：Hermes/OpenClaw runtime adapter，注册 `memory_gateway` toolset 和 hooks，可在真实对话中自动 search、append episode、可选 commit。 - -## 全新服务器部署总览 - -以下步骤假设目标服务器是 Linux，初始没有 Python 环境、OpenViking、EverMemOS、Memory Gateway、Obsidian Vault 或 Hermes Agent。示例路径使用 `/opt`，可按实际环境替换。 - -推荐目录： +## 项目结构 ```text -/opt/ -├── OpenViking/ # OpenViking 项目和 venv -└── memory-gateway/ # 本项目 - ├── .venv/ - ├── config.yaml # 本机配置，包含密钥，不提交 - ├── data/ - │ └── memory_gateway.sqlite3 - └── obsidian-vault/ - ├── 01_Knowledge/Uploaded/ - └── Reviews/Queue/ +memory_gateway/ + api_v1.py # v1 REST API + api_v2.py # v2 workflow API + server.py # FastAPI + MCP + legacy /api entrypoint + services.py # v1 service + services_v2.py # v2 ingest/retrieve/commit/outbox orchestration + openviking_client.py # OpenViking adapter + everos_client.py # EverOS adapter + repositories.py # in-memory / SQLite metadata repository + schemas.py # v1 schemas + schemas_v2.py # v2 schemas + backend_contracts.py # backend adapter result contracts + backend_normalization.py # backend response normalization + backend_ref_mapping.py # native ref type -> MemoryRefType mapping + obsidian_review.py # Obsidian review draft support +integrations/hermes/memory-gateway/ + SKILL.md + scripts/ +plugins/memory-gateway-agent/ +tests/ +config.example.yaml +pyproject.toml ``` -### 1. 安装系统依赖 +## 安装 + +要求 Python 3.10+。 ```bash -sudo apt-get update -sudo apt-get install -y \ - git curl rsync build-essential \ - python3 python3-venv python3-pip -``` - -可选安装 `uv`，后续 Python 环境会更快： - -```bash -curl -LsSf https://astral.sh/uv/install.sh | sh -``` - -如果服务器不能访问公网，请用内部镜像源安装 Python 依赖，并把 OpenViking、Hermes 和本项目代码放到内网 Git/制品库。 - -### 2. 安装 OpenViking - -OpenViking 不在本仓库内，需要按 OpenViking 项目的官方安装方式部署。最终目标是服务器上可以运行： - -```bash -openviking-server --host 127.0.0.1 --port 1933 -``` - -一种常见安装形态： - -```bash -cd /opt -git clone https://github.com/volcengine/OpenViking.git -cd /opt/OpenViking -python3 -m venv .venv -source .venv/bin/activate -pip install -U pip -pip install -e . -``` - -启动： - -```bash -source /opt/OpenViking/.venv/bin/activate -openviking-server --host 127.0.0.1 --port 1933 -``` - -健康检查： - -```bash -curl http://127.0.0.1:1933/health -``` - -说明：OpenViking 建议先只绑定 `127.0.0.1`，由 Memory Gateway 统一对外暴露 API。 - -### 3. 安装 Memory Gateway - -```bash -cd /opt -git clone https://gitea.bwgdi.com/tomtan/memory-gateway.git memory-gateway -cd /opt/memory-gateway +cd /home/tom/memory-gateway python3 -m venv .venv source .venv/bin/activate pip install -U pip pip install -e ".[dev]" ``` -创建目录和配置： +如果使用 `uv`： ```bash -mkdir -p /opt/memory-gateway/data -mkdir -p /opt/memory-gateway/obsidian-vault/01_Knowledge/Uploaded -mkdir -p /opt/memory-gateway/obsidian-vault/Reviews/Queue +cd /home/tom/memory-gateway +uv sync --extra dev +``` + +## 依赖服务 + +### OpenViking + +参考 `/home/tom/OpenViking/CONTRIBUTING.md`。当前约定启动方式： + +```bash +openviking-server --host 127.0.0.1 --port 1933 +``` + +配置文件： + +```text +/home/tom/.openviking/ov.conf +``` + +### EverOS / EverCore + +参考 `/home/tom/EverOS/methods/EverCore/docs/installation/SETUP.md`。当前约定启动方式： + +```bash +cd /home/tom/EverOS/methods/EverCore +uv run python src/run.py --port 1995 +``` + +配置文件： + +```text +/home/tom/EverOS/methods/EverCore/.env +``` + +## 配置 + +复制示例配置： + +```bash +cd /home/tom/memory-gateway cp config.example.yaml config.yaml ``` -修改 `config.yaml`： +核心配置示例： ```yaml server: - host: "0.0.0.0" + host: "127.0.0.1" port: 1934 - api_key: "" + api_key: "" openviking: url: "http://127.0.0.1:1933" - api_key: "" + api_key: "" + timeout: 30 -evermemos: +everos: enabled: true + mode: "real" url: "http://127.0.0.1:1995" - fallback_to_local: true - -obsidian: - vault_path: "/opt/memory-gateway/obsidian-vault" - knowledge_dir: "01_Knowledge/Uploaded" - review_dir: "Reviews/Queue" + api_key: "" + timeout: 30 + health_path: "/health" + ingest_path: "/api/v1/memories" + search_path: "/api/v1/memories/search" + flush_path: "/api/v1/memories/flush" + retrieve_method: "keyword" storage: backend: "sqlite" - sqlite_path: "/opt/memory-gateway/data/memory_gateway.sqlite3" + sqlite_path: "/home/tom/memory-gateway/memory_gateway.sqlite3" ``` -`config.yaml` 被 `.gitignore` 忽略，不应提交。 - -### 4. 安装/启动 EverMemOS - -本项目内置一个 POC 级 EverMemOS-compatible 服务，适合单机验证： +也可以用环境变量覆盖后端配置，例如： ```bash -cd /opt/memory-gateway +export OPENVIKING_URL=http://127.0.0.1:1933 +export EVEROS_URL=http://127.0.0.1:1995 +export EVEROS_MODE=real +``` + +## 启动 + +```bash +cd /home/tom/memory-gateway source .venv/bin/activate -python -m memory_gateway.evermemos_service \ - --config /opt/memory-gateway/config.yaml \ - --host 127.0.0.1 \ - --port 1995 +python -m memory_gateway.server --config config.yaml ``` -健康检查： +也可以显式指定 host/port： ```bash -curl http://127.0.0.1:1995/health -``` - -如果已有独立 EverMemOS 服务，把 `config.yaml` 中的 `evermemos.url`、`api_key`、`consolidate_path` 改为远程服务即可。Memory Gateway 会在 `/v1/sessions/{session_id}/commit` 时调用它。 - -### 5. 启动 Memory Gateway - -```bash -cd /opt/memory-gateway -source .venv/bin/activate -python -m memory_gateway.server --config /opt/memory-gateway/config.yaml -``` - -健康检查： - -```bash -curl -H "X-API-Key: " http://127.0.0.1:1934/health -curl -H "X-API-Key: " http://127.0.0.1:1934/v1/evermemos/health -``` - -生产建议用 systemd/supervisor 管理 OpenViking、EverMemOS 和 Memory Gateway 三个进程，并通过 Nginx/Caddy/内网负载均衡做 TLS 和访问控制。 - -### 6. Obsidian Vault - -服务器端不要求安装 Obsidian 桌面应用。这里的 Obsidian Vault 本质是 Markdown 目录： - -```text -/opt/memory-gateway/obsidian-vault/ -├── 01_Knowledge/Uploaded/ # /api/knowledge/upload 转换后的 Markdown -└── Reviews/Queue/ # EverMemOS 高价值/冲突候选 review draft -``` - -如果需要人工维护，可以： - -- 用 Obsidian 桌面端通过 SSH/Syncthing/Git 同步这个 vault。 -- 或直接在服务器上编辑 Markdown。 -- 高价值、冲突、低置信度的长期记忆候选优先进 `Reviews/Queue/`，不要直接污染长期记忆。 - -### 7. 安装 Hermes Agent、Skill 和 Plugin - -Hermes Agent 不在本仓库内。先按 Hermes 官方方式安装，确认命令可用： - -```bash -hermes --version -hermes chat --help -``` - -安装 Memory Gateway skill： - -```bash -mkdir -p ~/.hermes/skills/memory-gateway -rsync -a --delete \ - /opt/memory-gateway/integrations/hermes/memory-gateway/ \ - ~/.hermes/skills/memory-gateway/ -``` - -skill 的特点： - -- 通过脚本显式调用 Gateway API。 -- 适合手动或 agent policy 主动执行：搜索、上传、append episode、commit session。 -- skill 本身不会自动记忆每轮对话。 - -安装 Memory Gateway agent plugin： - -```bash -mkdir -p ~/.hermes/plugins -ln -s /opt/memory-gateway/plugins/memory-gateway-agent \ - ~/.hermes/plugins/memory-gateway-agent -hermes plugins enable memory-gateway-agent -hermes plugins list -hermes tools list -``` - -plugin 环境变量： - -```bash -export MEMORY_GATEWAY_URL=http://127.0.0.1:1934 -export MEMORY_GATEWAY_API_KEY= -export MEMORY_GATEWAY_DEFAULT_USER_ID=user_demo -export MEMORY_GATEWAY_DEFAULT_AGENT_ID=agent_hermes -export MEMORY_GATEWAY_DEFAULT_WORKSPACE_ID=workspace_demo -export MEMORY_GATEWAY_AUTO_SEARCH=true -export MEMORY_GATEWAY_AUTO_APPEND_EPISODE=true -export MEMORY_GATEWAY_AUTO_COMMIT_SESSION=false -``` - -plugin 的特点： - -- 向 Hermes 注册 `memory_gateway` toolset。 -- 注册 hooks：`on_session_start`、`pre_llm_call`、`post_llm_call`、`on_session_end`。 -- `pre_llm_call` 自动检索相关记忆。 -- `post_llm_call` 只写摘要型 candidate episode。 -- `on_session_end` 默认不 commit；只有 `MEMORY_GATEWAY_AUTO_COMMIT_SESSION=true` 才提交 session。 - -## 目录结构 - -```text -memory-gateway/ -├── memory_gateway/ # Gateway 服务代码 -│ ├── server.py # REST / MCP 接口 -│ ├── openviking_client.py # OpenViking client -│ ├── llm.py # OpenAI-compatible LLM summary -│ ├── document_ingest.py # MarkItDown + Obsidian write helpers -│ ├── config.py -│ └── types.py -├── integrations/hermes/ -│ └── memory-gateway/ # 通用 Hermes skill -├── plugins/ -│ └── memory-gateway-agent/ # Hermes/OpenClaw agent plugin adapter -├── obsidian-vault/ -│ ├── 01_Knowledge/Uploaded/ # 上传文档转成的 Markdown -│ ├── Reviews/Queue/ # EverMemOS review draft -│ └── 05_Templates/ # 通用知识模板 -├── data/ # SQLite metadata store，生产建议保留备份 -│ └── memory_gateway.sqlite3 -├── docs/ -│ └── generic-memory-gateway-design.md -├── tests/ -├── config.example.yaml -└── pyproject.toml -``` - -## 用户隔离、存储与长短期记忆 - -### 访问上下文 - -v1 API 的所有核心读写都应该带上访问上下文： - -```json -{ - "user_id": "user_demo", - "agent_id": "agent_hermes", - "workspace_id": "workspace_demo", - "session_id": "session_001" -} -``` - -Gateway 根据这些字段做 namespace 展开和 ACL 判断。不同用户必须使用不同 `user_id`，不同 agent 使用不同 `agent_id`，不同项目或团队空间使用不同 `workspace_id`。 - -### Namespace 设计 - -默认可见 namespace 形态： - -```text -user/{user_id}/profile -user/{user_id}/preferences -user/{user_id}/long_term -agent/{agent_id}/memory -workspace/{workspace_id}/shared -session/{session_id}/episodic -global/public -``` - -隔离规则： - -- `private`：默认私有，只允许同一 `user_id` 在 ACL 允许范围内访问。 -- `agent-only`：只允许指定 agent 或 agent namespace 使用。 -- `workspace-shared`：允许 workspace 成员/允许的 agent 使用。 -- `global`：公共知识，谨慎使用，不存放个人信息。 -- `session/{session_id}/episodic`：短期 session 记忆，默认只在当前 session 上下文中使用。 - -### 用户记忆存储位置 - -| 数据 | 存储位置 | 说明 | -| --- | --- | --- | -| users/profiles/memories/episodes/audit metadata | SQLite `storage.sqlite_path` | Gateway v1 的主 metadata store，负责隔离、ACL、audit、短期 episode 和长期 memory 记录 | -| 可检索 context/resource/memory | OpenViking | Gateway 按允许 namespace 查询 OpenViking，适合跨 agent 的 context 检索 | -| 上传文档 Markdown | Obsidian `01_Knowledge/Uploaded/` | 由 `/api/knowledge/upload` 写入，适合人工审查和维护 | -| 高价值/冲突 review draft | Obsidian `Reviews/Queue/` | EverMemOS 发现高价值或冲突候选时写入，不直接污染长期记忆 | -| Hermes plugin trace | `plugins/memory-gateway-agent/.tmp/hook_trace.log` | 默认关闭，只存 hook 名称、短 session id、Gateway action 和状态 | - -### 什么时候写短期记忆 - -短期记忆写入 `episodes`，通常来自： - -- Hermes plugin `post_llm_call` 判断用户明确要求“remember/记住”或出现稳定偏好、长期约束、项目事实。 -- agent 或 skill 显式调用 `memory_append_episode`。 -- 任务执行过程中的关键结论、可复用 workflow、架构决策。 - -短期记忆不应该包含： - -- 完整原始对话。 -- password、token、API key、cookie、private key。 -- 一次性验证码、大段日志、低价值临时内容。 -- 模型 chain-of-thought。 - -### 什么时候提升为长期记忆 - -长期记忆写入 `memories`，通常发生在： - -1. Agent/skill/plugin 先写 `session/{session_id}/episodic` episode。 -2. 调用 `/v1/sessions/{session_id}/commit` 或 MCP/tool `memory_commit_session`。 -3. Gateway 把 session episodes、可见长期记忆和访问上下文发给 EverMemOS。 -4. EverMemOS 做提取、去重、合并、冲突检测、重要性判断。 -5. 稳定且超过阈值的候选 promote 到 `user/{user_id}/long_term` 或目标 namespace。 -6. 高价值、冲突或需要人工确认的候选进入 Obsidian `Reviews/Queue/`。 - -Hermes plugin 默认： - -- `MEMORY_GATEWAY_AUTO_SEARCH=true`：对话前自动检索。 -- `MEMORY_GATEWAY_AUTO_APPEND_EPISODE=true`：对话后按 policy 写 candidate episode。 -- `MEMORY_GATEWAY_AUTO_COMMIT_SESSION=false`：默认不自动提升长期记忆。 - -因此，POC 推荐流程是“先短期、后 commit、再长期”，不要把所有 session 内容直接 upsert 成长期记忆。 - -### 如何再次查询和调用用户记忆 - -HTTP 查询： - -```bash -curl -X POST http://127.0.0.1:1934/v1/memory/search \ - -H "Content-Type: application/json" \ - -H "X-API-Key: " \ - -d '{ - "user_id": "user_demo", - "agent_id": "agent_hermes", - "workspace_id": "workspace_demo", - "session_id": "session_001", - "query": "用户输出偏好和项目约束", - "limit": 5 - }' -``` - -Hermes skill 查询： - -```bash -python ~/.hermes/skills/memory-gateway/scripts/memory_search.py \ - --user-id user_demo \ - --agent-id agent_hermes \ - --workspace-id workspace_demo \ - --session-id session_001 \ - --query "用户输出偏好和项目约束" \ - --limit 5 -``` - -Hermes plugin 查询： - -- 用户正常对话时，`pre_llm_call` 会自动调用 `memory_search`。 -- Agent 也可以显式调用 `memory_search` tool。 - -## 远程调用与安全 - -Memory Gateway API 可以远程调用。生产/远程部署时建议： - -1. `server.host` 设置为 `0.0.0.0`。 -2. `server.api_key` 设置为长随机值。 -3. 防火墙只开放可信来源，或通过 VPN/内网访问。 -4. 使用 Nginx/Caddy/负载均衡终止 TLS。 -5. 远程客户端一律带 `X-API-Key`。 - -远程调用示例： - -```bash -export MEMORY_GATEWAY_URL=https://memory.example.com -export MEMORY_GATEWAY_API_KEY= - -curl -X POST "$MEMORY_GATEWAY_URL/v1/memory/search" \ - -H "Content-Type: application/json" \ - -H "X-API-Key: $MEMORY_GATEWAY_API_KEY" \ - -d '{ - "user_id": "user_demo", - "agent_id": "remote_agent", - "workspace_id": "workspace_demo", - "query": "长期偏好", - "limit": 5 - }' -``` - -远程 Hermes plugin 配置： - -```bash -export MEMORY_GATEWAY_URL=https://memory.example.com -export MEMORY_GATEWAY_API_KEY= -export MEMORY_GATEWAY_DEFAULT_USER_ID=user_demo -export MEMORY_GATEWAY_DEFAULT_AGENT_ID=agent_hermes_remote -export MEMORY_GATEWAY_DEFAULT_WORKSPACE_ID=workspace_demo -``` - -注意：远程 API 不应裸奔在公网。至少启用 API key 和 TLS；涉及个人记忆时建议放在内网或 VPN 后面。 - -## 本地开发快捷启动 - -全新服务器请优先按上面的 `/opt` 部署流程执行。本节用于已经 clone 当前仓库后的开发机快速启动，路径都以仓库根目录为准。 - -```bash -cd memory-gateway -python3 -m venv .venv -source .venv/bin/activate -pip install -U pip -pip install -e ".[dev]" -mkdir -p data obsidian-vault/01_Knowledge/Uploaded obsidian-vault/Reviews/Queue -cp config.example.yaml config.yaml -``` - -建议把 `config.yaml` 中的本地开发路径改成： - -```yaml -obsidian: - vault_path: "./obsidian-vault" - -storage: - backend: sqlite - sqlite_path: "./data/memory_gateway.sqlite3" -``` - -启动顺序： - -```bash -# 终端 1：OpenViking，按 OpenViking 项目实际环境启动 -openviking-server --host 127.0.0.1 --port 1933 - -# 终端 2：EverMemOS-compatible POC 服务 -python -m memory_gateway.evermemos_service \ - --config ./config.yaml \ - --host 127.0.0.1 \ - --port 1995 - -# 终端 3：Memory Gateway -python -m memory_gateway.server --config ./config.yaml +python -m memory_gateway.server --config config.yaml --host 127.0.0.1 --port 1934 ``` 健康检查： ```bash curl http://127.0.0.1:1934/health -curl http://127.0.0.1:1995/health -curl http://127.0.0.1:1934/v1/evermemos/health +curl http://127.0.0.1:1934/v1/everos/health ``` -## REST 接口 - -如果 `config.yaml` 设置了 `server.api_key`，以下所有 HTTP 调用都需要加： +如果设置了 `server.api_key`，请求需要带： ```bash --H "X-API-Key: " +-H "X-API-Key: " ``` -### `GET /health` +## v2 工作流 -检查 Gateway 和 OpenViking 状态。 - -### `POST /api/search` - -搜索 OpenViking memory / resource。 +### 1. Ingest 一轮对话 ```bash -curl -X POST http://127.0.0.1:1934/api/search \ - -H "Content-Type: application/json" \ +curl -s http://127.0.0.1:1934/v2/conversations/ingest \ + -H 'Content-Type: application/json' \ -d '{ - "query": "memory gateway document upload summary", - "uri": "viking://resources", - "limit": 5 + "workspace_id": "ws_1", + "user_id": "user_a", + "agent_id": "agent_cli", + "session_id": "sess_1", + "turn_id": "turn_1", + "namespace": "workspace/ws_1/user/user_a", + "role": "user", + "content": "Remember that the demo environment uses EverOS and OpenViking.", + "metadata": {"channel": "manual-test"} }' ``` -### `POST /api/memory` +结果中的 `refs` 是本地 `memory_refs` 控制面引用，通常包括： -写入普通 memory。 +- OpenViking `session_archive` ref +- EverOS `message_memory` ref + +这些 refs 保存的是 native id/uri、状态、hash、trace 等 metadata，不是完整记忆正文。 + +### 2. Retrieve 上下文 ```bash -curl -X POST http://127.0.0.1:1934/api/memory \ - -H "Content-Type: application/json" \ +curl -s http://127.0.0.1:1934/v2/context/retrieve \ + -H 'Content-Type: application/json' \ -d '{ - "namespace": "memory-gateway", - "memory_type": "preference", - "content": "The user prefers concise technical summaries." + "workspace_id": "ws_1", + "user_id": "user_a", + "agent_id": "agent_cli", + "session_id": "sess_1", + "namespace": "workspace/ws_1/user/user_a", + "query": "EverOS OpenViking demo environment", + "limit": 5, + "metadata": {"trace_id": "trace_manual_1"} }' ``` -### `POST /api/resource` +返回结构重点： -写入结构化 resource。 +- `items`：真实上下文，由 OpenViking / EverOS retrieve 返回后合并，包含 `text`、`source_backend`、`ref_id`、`score`、`memory_type`。 +- `refs`：本地已有的 `memory_refs` 视图，用于追踪哪些后端引用已保存。 +- `metadata.backend_results`：每个后端 retrieve 的状态、返回数量和错误信息。 + +### 3. Commit 一个 session ```bash -curl -X POST http://127.0.0.1:1934/api/resource \ - -H "Content-Type: application/json" \ +curl -s http://127.0.0.1:1934/v2/conversations/sess_1/commit \ + -H 'Content-Type: application/json' \ -d '{ - "uri": "viking://resources/memory-gateway/knowledge/example.json", - "resource_type": "json", - "content": "{\"title\":\"example\"}" + "workspace_id": "ws_1", + "user_id": "user_a", + "agent_id": "agent_cli", + "namespace": "workspace/ws_1/user/user_a" }' ``` -### `POST /api/summary` +该接口只创建 commit job 和 outbox events，不直接执行长期记忆生成。返回中会有 `job_id` 和 `metadata.gateway_id`。 -调用 LLM 总结任意文本，并按需沉淀到 OpenViking。 +### 4. Process outbox ```bash -curl -X POST http://127.0.0.1:1934/api/summary \ - -H "Content-Type: application/json" \ - -d '{ - "title": "Project decision summary", - "content": "需要总结和沉淀的内容...", - "namespace": "memory-gateway", - "memory_type": "decision", - "tags": ["project", "decision"], - "persist_as": "resource" - }' +curl -s -X POST 'http://127.0.0.1:1934/v2/admin/outbox/process?limit=20' ``` -`persist_as` 支持：`none`、`memory`、`resource`、`both`。 +处理成功后会生成长期 refs： -### `POST /api/knowledge/upload` +- OpenViking `session_archive` ref：session archive / summary 的 native 引用。 +- EverOS `profile` ref：用户 profile 的 native 引用。 +- EverOS `long_term_memory` ref：session 提炼出的长期记忆 native 引用。 -上传文档，MarkItDown 转 Markdown，保存到 Obsidian，LLM 总结后写入 OpenViking knowledge。 +这些 ref 保存在 SQLite 的 `memory_refs` 表中。 + +### 5. 查看 refs 和 job ```bash -curl -X POST http://127.0.0.1:1934/api/knowledge/upload \ - -F "file=@/path/to/document.pdf" \ - -F "title=Design Notes" \ - -F "namespace=memory-gateway" \ - -F "knowledge_type=design_doc" \ - -F "tags=project,design,reference" \ - -F "persist_as=resource" +curl -s 'http://127.0.0.1:1934/v2/memory/refs?workspace_id=ws_1&user_id=user_a&session_id=sess_1&limit=20' + +curl -s http://127.0.0.1:1934/v2/jobs/ ``` -默认保存到： +SQLite 默认路径取决于配置，例如： ```text -obsidian-vault/01_Knowledge/Uploaded/ +/home/tom/memory-gateway/memory_gateway.sqlite3 ``` -## v1 通用 Memory API +主要表： -v1 API 面向多 agent 框架，带 user / agent / workspace / session 上下文和基础 ACL。 +- `memory_refs` +- `outbox_events` +- `commit_jobs` +- `audit_logs` +- `users` / `memories` / `episodes` / `profiles` -### 创建用户 +## v1 和 legacy API -```bash -curl -X POST http://127.0.0.1:1934/v1/users \ - -H "Content-Type: application/json" \ - -d '{"user_id":"user_tom","display_name":"Tom","preferences":{"language":"zh-CN"}}' -``` - -### 写入记忆 - -```bash -curl -X POST http://127.0.0.1:1934/v1/memory \ - -H "Content-Type: application/json" \ - -d '{ - "user_id": "user_tom", - "agent_id": "agent_hermes", - "workspace_id": "ws_memory_gateway", - "memory_type": "preference", - "content": "用户偏好中文输出，结构化但不要过度工程化。", - "summary": "中文、结构化、轻量 POC 优先。", - "tags": ["preference", "style"], - "importance": 0.8, - "confidence": 0.9, - "visibility": "private", - "source": "manual" - }' -``` - -### 检索记忆 - -```bash -curl -X POST http://127.0.0.1:1934/v1/memory/search \ - -H "Content-Type: application/json" \ - -d '{ - "user_id": "user_tom", - "agent_id": "agent_hermes", - "workspace_id": "ws_memory_gateway", - "query": "中文输出", - "limit": 5 - }' -``` - -返回会包含： - -- `local_total`：SQLite metadata 命中的记忆数量。 -- `openviking_total`：按可见 namespace 查询 OpenViking 的命中数量。 -- `searched_namespaces`：Gateway 展开并允许查询的 namespace。 - -### 修改记忆 - -```bash -curl -X PATCH "http://127.0.0.1:1934/v1/memory/MEMORY_ID?user_id=user_tom&agent_id=agent_hermes&workspace_id=ws_memory_gateway" \ - -H "Content-Type: application/json" \ - -d '{"summary":"用户偏好中文、结构化、少废话。","importance":0.9}' -``` - -### 写入 episode 并 commit session - -```bash -curl -X POST http://127.0.0.1:1934/v1/episodes \ - -H "Content-Type: application/json" \ - -d '{ - "user_id": "user_tom", - "agent_id": "agent_hermes", - "workspace_id": "ws_memory_gateway", - "session_id": "sess_demo", - "content": "结论：这个项目必须保留用户隔离和 namespace ACL。", - "tags": ["decision"] - }' - -curl -X POST http://127.0.0.1:1934/v1/sessions/sess_demo/commit \ - -H "Content-Type: application/json" \ - -d '{ - "user_id": "user_tom", - "agent_id": "agent_hermes", - "workspace_id": "ws_memory_gateway", - "session_id": "sess_demo", - "promote": true, - "min_importance": 0.6 - }' -``` - -流程说明： - -- 短期记忆先写入 SQLite 的 `episodes`，namespace 通常是 `session/{session_id}/episodic`。 -- commit session 时，Gateway 把当前 session episodes、可见长期记忆和访问上下文发给 `http://127.0.0.1:1995/v1/sessions/consolidate`。 -- EverMemOS 返回候选记忆、可直接提升的长期记忆、重复/冲突信息和 review draft 路径。 -- Gateway 只把正常稳定候选写入长期 memory；高价值或冲突候选不会直接进入长期记忆，会写入： +v1 保留用户隔离、namespace、visibility/ACL、episode、session commit、audit 等基础能力： ```text -obsidian-vault/Reviews/Queue/ +POST /v1/users +GET /v1/users/{user_id} +POST /v1/memory/search +POST /v1/memory +GET /v1/memory/{memory_id} +PATCH /v1/memory/{memory_id} +DELETE /v1/memory/{memory_id} +POST /v1/episodes +POST /v1/sessions/{session_id}/commit +GET /v1/users/{user_id}/profile +POST /v1/memory/{memory_id}/feedback +GET /v1/namespaces +GET /v1/audit +GET /v1/everos/health ``` -## MCP Tools - -`POST /mcp/rpc` 支持： - -- `search` -- `add_memory` -- `add_resource` -- `commit_summary` -- `get_status` -- `list_memories` -- `list_resources` -- `memory_search` -- `memory_upsert` -- `memory_append_episode` -- `memory_commit_session` -- `memory_get_profile` -- `memory_list_namespaces` -- `memory_delete` -- `memory_feedback` - -## Hermes Skill - -通用 Hermes skill： +旧 `/api/*` 接口仍保留： ```text -~/.hermes/skills/memory-gateway/ +POST /api/search +POST /api/memory +POST /api/resource +POST /api/summary +POST /api/knowledge/upload ``` -仓库副本： +## MCP / Hermes + +MCP endpoints： + +```text +POST /mcp/rpc +GET /mcp/sse +``` + +Hermes skill 位于： ```text integrations/hermes/memory-gateway/ ``` -安装或更新到本机 Hermes skill 目录： +常用脚本示例： ```bash -mkdir -p ~/.hermes/skills/memory-gateway -rsync -a --delete \ - ./integrations/hermes/memory-gateway/ \ - ~/.hermes/skills/memory-gateway/ +python integrations/hermes/memory-gateway/scripts/everos_health.py +python integrations/hermes/memory-gateway/scripts/memory_commit_session.py --help ``` -使用方式： +## 开发与验证 -- Hermes 对话中可以加载 `memory-gateway` skill，让 agent 按 skill 文档主动调用脚本。 -- skill 不等于自动记忆；只有 agent 根据 skill/policy 主动调用脚本时才会写入或检索记忆。 -- 适合人工可控的显式操作：创建用户、检索记忆、追加 episode、commit session、上传知识和检查 EverMemOS。 - -主要脚本： - -```text -scripts/evermemos_health.py -scripts/memory_create_user.py -scripts/memory_append_episode.py -scripts/memory_commit_session.py -scripts/memory_search.py -scripts/memory_upsert.py -scripts/retrieve_memory.py -scripts/commit_summary.py -scripts/upload_knowledge.py -scripts/search_obsidian.py -``` - -检查 EverMemOS： +运行测试： ```bash -python ~/.hermes/skills/memory-gateway/scripts/evermemos_health.py +cd /home/tom/memory-gateway +PYTHONPATH=/home/tom/memory-gateway pytest -q ``` -检索记忆： +编译检查： ```bash -python ~/.hermes/skills/memory-gateway/scripts/retrieve_memory.py \ - --query "document upload summary memory gateway" \ - --uri viking://resources \ - --limit 5 +python -m compileall -q memory_gateway tests integrations/hermes/memory-gateway plugins/memory-gateway-agent ``` -总结沉淀： +Ruff 已在 `pyproject.toml` 中配置。如果本地环境安装了 ruff： ```bash -python ~/.hermes/skills/memory-gateway/scripts/commit_summary.py \ - --title "Reusable conclusion" \ - --namespace memory-gateway \ - --memory-type decision \ - --tag project \ - --persist-as resource \ - --text "最终结论或可复用知识..." +python -m ruff check . ``` -上传知识： +当前仓库不要求真实 OpenViking / EverOS 服务才能跑单元测试；真实服务流程需要先启动 `127.0.0.1:1933` 和 `127.0.0.1:1995`。 -```bash -python ~/.hermes/skills/memory-gateway/scripts/upload_knowledge.py \ - --file /path/to/document.md \ - --title "Knowledge note" \ - --namespace memory-gateway \ - --knowledge-type reference \ - --tags project,reference \ - --persist-as resource -``` +## 设计约束 -完整长短期记忆测试： - -```bash -python ~/.hermes/skills/memory-gateway/scripts/memory_create_user.py \ - --user-id user_tom \ - --display-name "Tom" \ - --preference language=zh-CN - -python ~/.hermes/skills/memory-gateway/scripts/memory_append_episode.py \ - --user-id user_tom \ - --agent-id agent_hermes \ - --workspace-id ws_memory_gateway \ - --session-id sess_demo \ - --tag decision \ - --text "结论：本机 EverMemOS 服务负责从 session episode 中整理稳定长期记忆。" - -python ~/.hermes/skills/memory-gateway/scripts/memory_append_episode.py \ - --user-id user_tom \ - --agent-id agent_hermes \ - --workspace-id ws_memory_gateway \ - --session-id sess_demo \ - --tag review \ - --tag high-value \ - --text "重要：高价值记忆应该进入 Obsidian review queue，避免错误记忆污染长期系统。" - -python ~/.hermes/skills/memory-gateway/scripts/memory_commit_session.py \ - --user-id user_tom \ - --agent-id agent_hermes \ - --workspace-id ws_memory_gateway \ - --session-id sess_demo \ - --min-importance 0.6 - -python ~/.hermes/skills/memory-gateway/scripts/memory_search.py \ - --user-id user_tom \ - --agent-id agent_hermes \ - --workspace-id ws_memory_gateway \ - --session-id sess_demo \ - --query "EverMemOS 服务负责" \ - --limit 5 -``` - -## Hermes / OpenClaw Agent Plugin - -通用 Agent plugin 位于： - -```text -plugins/memory-gateway-agent/ -``` - -它是独立 adapter，不 import Gateway 内部 `services/repositories/server`，所有调用都通过现有 `/v1` HTTP API。它面向 Hermes/OpenClaw 这类 agent runtime 暴露统一工具： - -- `memory_search` -- `memory_append_episode` -- `memory_commit_session` -- `memory_upsert` -- `memory_feedback` - -Hermes 本机安装： - -```bash -mkdir -p ~/.hermes/plugins -ln -s "$(pwd)/plugins/memory-gateway-agent" \ - ~/.hermes/plugins/memory-gateway-agent -hermes plugins enable memory-gateway-agent -hermes plugins list -hermes tools list -``` - -如果软链接已存在，先确认它指向当前仓库： - -```bash -ls -l ~/.hermes/plugins/memory-gateway-agent -``` - -运行配置： - -```bash -export MEMORY_GATEWAY_URL=http://127.0.0.1:1934 -export MEMORY_GATEWAY_API_KEY= -export MEMORY_GATEWAY_DEFAULT_USER_ID=test_user_memory_gateway_plugin -export MEMORY_GATEWAY_DEFAULT_AGENT_ID=test_hermes_memory_gateway_plugin -export MEMORY_GATEWAY_DEFAULT_WORKSPACE_ID=test_workspace_memory_gateway_plugin -export MEMORY_GATEWAY_AUTO_SEARCH=true -export MEMORY_GATEWAY_AUTO_APPEND_EPISODE=true -export MEMORY_GATEWAY_AUTO_COMMIT_SESSION=false -``` - -Hermes plugin 已验证： - -- `hermes plugins list` 可发现并启用 `memory-gateway-agent`。 -- `hermes tools list` 可看到 `memory_gateway` toolset。 -- `pre_llm_call` 会自动检索 Memory Gateway。 -- `post_llm_call` 会按 policy 写入摘要型 candidate episode。 -- `on_session_end` 默认不会 commit；只有 `MEMORY_GATEWAY_AUTO_COMMIT_SESSION=true` 才会 commit。 - -真实 Hermes chat 验证： - -```bash -PYTHONPATH="$(pwd)/plugins/memory-gateway-agent" \ -python plugins/memory-gateway-agent/scripts/hermes_interactive_session_check.py -``` - -插件 E2E 验证： - -```bash -PYTHONPATH="$(pwd)/plugins/memory-gateway-agent" \ -python plugins/memory-gateway-agent/scripts/gateway_e2e_check.py - -PYTHONPATH="$(pwd)/plugins/memory-gateway-agent" \ -python plugins/memory-gateway-agent/scripts/hermes_hook_probe.py -``` - -清理测试数据： - -```bash -PYTHONPATH="$(pwd)/plugins/memory-gateway-agent" \ -python plugins/memory-gateway-agent/scripts/cleanup_test_memories.py -``` - -安全边界： - -- plugin 不保存完整原始对话，只写摘要型 episode。 -- 默认拒绝 password、token、API key、cookie、private key、完整 transcript 和大段日志。 -- `memory_upsert` 是高风险长期记忆写入，不会自动触发。 -- 用户要求 forget/delete 时，应走 `memory_feedback` 或 delete 能力。 -- hook trace 默认关闭；需要排查时设置 `MEMORY_GATEWAY_PLUGIN_TRACE_HOOKS=true`，只会写入 hook 名称、短 session id、Gateway action 和状态到 `plugins/memory-gateway-agent/.tmp/hook_trace.log`。 - -OpenClaw manifest 目前是 best-effort 草案： - -```text -plugins/memory-gateway-agent/openclaw.plugin.yaml -``` - -需要等 OpenClaw runtime 可用后再做第五阶段实测。 - -## 测试 - -```bash -cd memory-gateway -source .venv/bin/activate -PYTHONPATH="$(pwd)" pytest -q - -PYTHONPATH="$(pwd)/plugins/memory-gateway-agent" \ - pytest -q plugins/memory-gateway-agent/tests -``` - -当前测试覆盖： - -- API key 校验。 -- MCP tools/list。 -- OpenViking search 透传。 -- LLM summary artifact 构建。 -- document upload -> markdown -> Obsidian -> OpenViking resource。 - -## 下一步 - -- 在 Gateway 层加强 `/api/search` 的 URI prefix 过滤和去重。 -- 给 `/api/knowledge/upload` 增加文件大小限制、类型白名单和 dry-run。 -- 增加 Obsidian -> OpenViking 增量同步脚本。 -- 给 Memory Gateway skill 增加更稳定的“回答时引用 memory/resource/Obsidian note”输出约束。 -- 增加更多文档解析格式和异常处理测试。 +- SQLite 保存控制面 metadata，不作为长期记忆正文数据库。 +- `refs` 是后端 native 对象引用，不等于上下文正文。 +- `retrieve.items` 才是运行时上下文内容。 +- `commit` 只创建 job/outbox；长期 refs 由 outbox process 生成。 +- 默认只绑定本机地址；远程暴露时必须设置 API key、TLS 和网络访问控制。 diff --git a/config.example.yaml b/config.example.yaml index 30405b6..d0c3d43 100644 --- a/config.example.yaml +++ b/config.example.yaml @@ -19,18 +19,19 @@ openviking: api_key: "" timeout: 30 -# EverMemOS 后台长期记忆整理服务 -evermemos: +# EverOS / EverCore 后台长期记忆整理服务 +everos: enabled: true - # 可以是本机 memory_gateway.evermemos_service，也可以是远程 EverMemOS 服务。 + mode: "real" + # 指向 /home/tom/EverOS/methods/EverCore 启动的 API。 url: "http://127.0.0.1:1995" api_key: "" timeout: 30 health_path: "/health" - # 如果远端服务实际 endpoint 不同，改这里即可，不需要改代码。 - consolidate_path: "/v1/sessions/consolidate" - # POC 默认允许远端不可用时用本地确定性 worker 降级，方便开发测试。 - fallback_to_local: true + ingest_path: "/api/v1/memories" + search_path: "/api/v1/memories/search" + flush_path: "/api/v1/memories/flush" + retrieve_method: "keyword" # 记忆配置 memory: diff --git a/docs/generic-memory-gateway-design.md b/docs/generic-memory-gateway-design.md index 9a0955c..340204c 100644 --- a/docs/generic-memory-gateway-design.md +++ b/docs/generic-memory-gateway-design.md @@ -1,6 +1,6 @@ # 通用 Memory Gateway 方案与 POC 骨架 -本文基于当前仓库的轻量 FastAPI + MCP + OpenViking + Obsidian 能力扩展，不把系统设计成重平台。第一阶段目标是先跑通多用户隔离、namespace routing、记忆检索、写入、session commit 和人工 review 草稿，后续再替换持久化、向量索引和 EverMemOS worker。 +本文基于当前仓库的轻量 FastAPI + MCP + OpenViking + Obsidian 能力扩展，不把系统设计成重平台。第一阶段目标是先跑通多用户隔离、namespace routing、记忆检索、写入、session commit 和人工 review 草稿，后续再替换持久化、向量索引和 EverOS worker。 ## A. 总体架构图 @@ -43,7 +43,7 @@ flowchart TB OVWorkspace[workspace] end - subgraph EverMemOS["EverMemOS"] + subgraph EverOS["EverOS"] LTE[long-term extraction] Consolidation[consolidation] Decay[decay] @@ -79,7 +79,7 @@ flowchart TB Retrieval --> Skills Writeback --> Skills Skills --> OpenViking - Skills --> EverMemOS + Skills --> EverOS Skills --> Obsidian Gateway --> DB @@ -88,8 +88,8 @@ flowchart TB OpenViking --> DB OpenViking --> Vector Obsidian --> Files - EverMemOS --> DB - EverMemOS --> Vector + EverOS --> DB + EverOS --> Vector ``` ## B. 核心数据模型 @@ -510,13 +510,13 @@ Response: | Skill | 功能 | 输入 | 输出 | 触发时机 | 组件 | 写长期记忆 | |---|---|---|---|---|---|---| | `ingest_skill` | 标准化对话、文件、任务事件 | raw text/file/events | normalized payload | agent 写入 episode 前 | Gateway, file storage | 否 | -| `extract_memory_skill` | 从 episode/session 抽取候选记忆 | episode/session content | memory candidates | session commit / worker 定时 | LLM, EverMemOS | 否 | +| `extract_memory_skill` | 从 episode/session 抽取候选记忆 | episode/session content | memory candidates | session commit / worker 定时 | LLM, EverOS | 否 | | `classify_memory_skill` | 判断 memory_type、visibility、namespace | candidate memory | classification | 写入前 | ACL, namespace router | 否 | | `retrieve_context_skill` | 聚合用户、agent、workspace 上下文 | query + context ids | ranked contexts | agent 调用前 | OpenViking, vector index | 否 | | `commit_memory_skill` | 写入长期记忆 | MemoryRecord | stored record | 人工确认或 commit 通过 | DB, OpenViking | 是 | | `summarize_episode_skill` | 压缩 episode | episode content | summary | session commit | LLM | 否 | -| `merge_memory_skill` | 合并重复或相近记忆 | memory ids | merged memory | EverMemOS 整理 | DB, vector index | 是 | -| `prune_memory_skill` | 衰减、归档、删除低质记忆 | policy + memory ids | archived/deleted list | 定时 worker | EverMemOS | 是 | +| `merge_memory_skill` | 合并重复或相近记忆 | memory ids | merged memory | EverOS 整理 | DB, vector index | 是 | +| `prune_memory_skill` | 衰减、归档、删除低质记忆 | policy + memory ids | archived/deleted list | 定时 worker | EverOS | 是 | | `export_to_obsidian_skill` | 生成 Obsidian review draft | high-value memory | markdown draft | 高价值或需人工确认 | Obsidian | 否 | | `import_from_obsidian_skill` | 从人工维护笔记导入记忆 | markdown path | MemoryRecord | vault sync | Obsidian, OpenViking | 是 | @@ -551,7 +551,7 @@ obsidian-vault/ - 人工可维护 profile、preferences、长期总结。 - 高价值 workspace 知识、项目决策、复用经验。 -- EverMemOS 标记为 `needs_review` 的长期记忆草稿。 +- EverOS 标记为 `needs_review` 的长期记忆草稿。 不进入 Obsidian 的内容： @@ -571,7 +571,7 @@ obsidian-vault/ #memory/review #memory/conflict #memory/deprecated -#source/evermemos +#source/everos #source/manual #visibility/private #visibility/workspace-shared @@ -605,10 +605,10 @@ viking://skills/memory-gateway/{skill_name} 同步： - Obsidian accepted note 通过 `import_from_obsidian_skill` 写回 Gateway，再同步 OpenViking resource。 -- EverMemOS consolidation 后写入 `user/{user_id}/long_term` 或 `workspace/{workspace_id}/shared`。 +- EverOS consolidation 后写入 `user/{user_id}/long_term` 或 `workspace/{workspace_id}/shared`。 - Gateway 保存 `source_ref`，避免 OpenViking 与 Obsidian 互相重复导入。 -## H. EverMemOS 设计 +## H. EverOS 设计 输入来源： @@ -662,10 +662,10 @@ memory-gateway/ │ │ └── import_from_obsidian_skill.py │ ├── adapters/ │ │ ├── openviking.py -│ │ ├── evermemos.py +│ │ ├── everos.py │ │ └── obsidian.py │ └── workers/ -│ └── evermemos_worker.py +│ └── everos_worker.py ├── obsidian-vault/ ├── integrations/ │ ├── nanobot/ @@ -692,7 +692,7 @@ memory-gateway/ 第三周： -- 加 EverMemOS worker 原型：session commit、candidate extraction、dedup、merge。 +- 加 EverOS worker 原型：session commit、candidate extraction、dedup、merge。 - 增加 feedback 流程：incorrect、duplicate、outdated 影响 prune/merge。 - 生成 Obsidian review draft，而不是直接写入最终知识库。 @@ -728,7 +728,7 @@ POC 成功指标： - 存储使用 SQLite metadata + 本地文件存 object；当前代码先用 in-memory repo 验证接口。 - 搜索先用 OpenViking search + 简单 lexical fallback；向量索引第二阶段引入。 - Obsidian 只保存人工可读的高价值长期记忆和 review draft。 -- EverMemOS 第一阶段不做独立大系统，只做 worker 模块：extract、dedup、merge、prune、profile update。 +- EverOS 第一阶段不做独立大系统，只做 worker 模块：extract、dedup、merge、prune、profile update。 第一阶段实现 API： @@ -757,11 +757,11 @@ POC 成功指标： - `merge_memory_skill` - `prune_memory_skill` - `import_from_obsidian_skill` -- 更完整的 EverMemOS consolidation 和 profile evolution。 +- 更完整的 EverOS consolidation 和 profile evolution。 角色分工： - Obsidian 第一阶段：review draft、人类确认 profile/长期知识。第二阶段：双向同步。 - OpenViking 第一阶段：统一 context/resource 检索入口。第二阶段：承载多 namespace context filesystem 和 skill registry。 -- EverMemOS 第一阶段：session commit worker。第二阶段：长期记忆治理、衰减、冲突检测、profile evolution。 +- EverOS 第一阶段：session commit worker。第二阶段：长期记忆治理、衰减、冲突检测、profile evolution。 diff --git a/integrations/hermes/memory-gateway/SKILL.md b/integrations/hermes/memory-gateway/SKILL.md index e7974ed..759a18b 100644 --- a/integrations/hermes/memory-gateway/SKILL.md +++ b/integrations/hermes/memory-gateway/SKILL.md @@ -1,10 +1,10 @@ --- name: memory-gateway -description: Use this skill when Hermes needs shared long-term memory, user-scoped preferences/profile, workspace memory, session episode capture, Memory Gateway retrieval, OpenViking context search, Obsidian document upload/review, or session commit through the standalone EverMemOS service. This skill is domain-neutral. +description: Use this skill when Hermes needs shared long-term memory, user-scoped preferences/profile, workspace memory, session episode capture, Memory Gateway retrieval, OpenViking context search, Obsidian document upload/review, or session commit through the standalone EverOS service. This skill is domain-neutral. version: 3.1.0 metadata: hermes: - tags: [memory, memory-gateway, openviking, obsidian, evermemos, long-term-memory, retrieval, agent-context] + tags: [memory, memory-gateway, openviking, obsidian, everos, long-term-memory, retrieval, agent-context] --- # Memory Gateway @@ -16,7 +16,7 @@ The gateway provides: - v1 user/agent/workspace/session aware memory APIs backed by SQLite metadata. - ACL and namespace routing before retrieval. - OpenViking fan-out search for visible namespaces. -- Session episode capture and commit through the standalone EverMemOS HTTP service, with Gateway local fallback only when configured. +- Session episode capture and commit through the standalone EverOS HTTP service, with Gateway local fallback only when configured. - Obsidian review drafts for high-value or conflicting long-term memory candidates. - Legacy summary/document upload endpoints for LLM summarization and Obsidian knowledge ingestion. @@ -25,7 +25,7 @@ The gateway provides: Defaults: - Memory Gateway URL: `http://127.0.0.1:1934` -- EverMemOS URL through Gateway config: `http://127.0.0.1:1995` +- EverOS URL through Gateway config: `http://127.0.0.1:1995` - Obsidian vault: `/home/tom/memory-gateway/obsidian-vault` - Default review queue: `/home/tom/memory-gateway/obsidian-vault/Reviews/Queue` @@ -41,7 +41,7 @@ For normal agent work: 1. Search memory before answering if prior context may matter. 2. Append important session episodes while working. -3. Commit the session at the end so EverMemOS can promote stable memories. +3. Commit the session at the end so EverOS can promote stable memories. 4. Use feedback to mark incorrect, duplicate, outdated, or useful memories. 5. Upload documents only when they are reusable knowledge, not raw noisy logs. @@ -49,13 +49,13 @@ Do not write full transcripts to long-term memory. Use episodes for temporary pr ## v1 Memory Commands -### Check EverMemOS +### Check EverOS ```bash -python /home/tom/.hermes/skills/memory-gateway/scripts/evermemos_health.py +python /home/tom/.hermes/skills/memory-gateway/scripts/everos_health.py ``` -Expected healthy response includes `status: ok` and `response.service: evermemos-local`. +Expected healthy response includes `status: ok` and `response.service: everos-local`. ### Create User @@ -122,10 +122,10 @@ python /home/tom/.hermes/skills/memory-gateway/scripts/memory_append_episode.py --text "结论：这个项目必须保留用户隔离和 namespace ACL。" ``` -### Commit Session Through EverMemOS +### Commit Session Through EverOS -This asks Memory Gateway to call the standalone EverMemOS service configured in `config.yaml`. -For local POC the default service is `http://127.0.0.1:1995`. If `evermemos.fallback_to_local` is true and the service is unavailable, Gateway returns `evermemos_backend: local-fallback`. +This asks Memory Gateway to call the standalone EverOS service configured in `config.yaml`. +For local POC the default service is `http://127.0.0.1:1995`. If `everos.fallback_to_local` is true and the service is unavailable, Gateway returns `everos_backend: local-fallback`. - extracts candidate memories from session episodes - deduplicates exact repeated candidates @@ -282,7 +282,7 @@ When using this skill, answer with: - Do not store raw noisy data as long-term memory. - Use `memory_append_episode.py` for temporary process notes. -- Use `memory_commit_session.py` at task end to let EverMemOS decide what should persist. +- Use `memory_commit_session.py` at task end to let EverOS decide what should persist. - Use `memory_upsert.py` directly only for stable, concise, user-approved memory. - Do not commit secrets, credentials, tokens, private keys, or unnecessary personal data. - If content is sensitive, summarize and redact before committing. diff --git a/integrations/hermes/memory-gateway/scripts/evermemos_health.py b/integrations/hermes/memory-gateway/scripts/evermemos_health.py deleted file mode 100644 index cee6cf9..0000000 --- a/integrations/hermes/memory-gateway/scripts/evermemos_health.py +++ /dev/null @@ -1,19 +0,0 @@ -#!/usr/bin/env python3 -from __future__ import annotations - -import argparse -import json - -from _client import DEFAULT_GATEWAY_API_KEY, DEFAULT_GATEWAY_URL, get_json - - -def main() -> None: - parser = argparse.ArgumentParser(description="Check standalone EverMemOS health through Memory Gateway.") - parser.add_argument("--gateway-url", default=DEFAULT_GATEWAY_URL) - parser.add_argument("--api-key", default=DEFAULT_GATEWAY_API_KEY) - args = parser.parse_args() - print(json.dumps(get_json("/v1/evermemos/health", gateway_url=args.gateway_url, api_key=args.api_key), ensure_ascii=False, indent=2)) - - -if __name__ == "__main__": - main() diff --git a/integrations/hermes/memory-gateway/scripts/everos_health.py b/integrations/hermes/memory-gateway/scripts/everos_health.py new file mode 100644 index 0000000..66c2004 --- /dev/null +++ b/integrations/hermes/memory-gateway/scripts/everos_health.py @@ -0,0 +1,21 @@ +#!/usr/bin/env python3 +"""Check EverOS health through Memory Gateway.""" + +from __future__ import annotations + +import argparse +import json + +from _client import get_json + + +def main() -> None: + parser = argparse.ArgumentParser(description="Check EverOS health through Memory Gateway.") + parser.add_argument("--gateway-url", default="http://127.0.0.1:1934") + parser.add_argument("--api-key", default=None) + args = parser.parse_args() + print(json.dumps(get_json("/v1/everos/health", gateway_url=args.gateway_url, api_key=args.api_key), ensure_ascii=False, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/integrations/hermes/memory-gateway/scripts/memory_commit_session.py b/integrations/hermes/memory-gateway/scripts/memory_commit_session.py index 8b7fa2c..ea0aab7 100644 --- a/integrations/hermes/memory-gateway/scripts/memory_commit_session.py +++ b/integrations/hermes/memory-gateway/scripts/memory_commit_session.py @@ -8,7 +8,7 @@ from _client import DEFAULT_GATEWAY_API_KEY, DEFAULT_GATEWAY_URL, post_json def main() -> None: - parser = argparse.ArgumentParser(description="Commit a session through the minimal EverMemOS consolidation worker.") + parser = argparse.ArgumentParser(description="Commit a session through the minimal EverOS consolidation worker.") parser.add_argument("--user-id", required=True) parser.add_argument("--session-id", required=True) parser.add_argument("--agent-id", default="") diff --git a/memory_gateway/api_v1.py b/memory_gateway/api_v1.py index 89ce3a3..9dafd8b 100644 --- a/memory_gateway/api_v1.py +++ b/memory_gateway/api_v1.py @@ -110,6 +110,6 @@ async def list_audit(limit: int = Query(default=100, ge=1, le=1000)): return service.list_audit(limit) -@router.get("/evermemos/health") -async def evermemos_health(): - return service.evermemos_health() +@router.get("/everos/health") +async def everos_health(): + return service.everos_health() diff --git a/memory_gateway/backend_adapter_mapping.py b/memory_gateway/backend_adapter_mapping.py index e59feeb..123ef50 100644 --- a/memory_gateway/backend_adapter_mapping.py +++ b/memory_gateway/backend_adapter_mapping.py @@ -1,6 +1,6 @@ """Contract-first mapping spec for future v2 backend adapters. -This module intentionally does not call OpenViking, EverMemOS, or Obsidian. +This module intentionally does not call OpenViking, EverOS, or Obsidian. It documents the stable Gateway control-plane fields that may be persisted in outbox payload refs, SQLite metadata_json, audit summaries, and related control records. It is not a validator for transient runtime adapter request objects: @@ -81,21 +81,21 @@ ADAPTER_MAPPING_SPECS: Final[tuple[AdapterMappingSpec, ...]] = ( result_model=BackendRetrieveResult, ), AdapterMappingSpec( - backend_type=BackendType.EVERMEMOS, + backend_type=BackendType.EVEROS, operation=BackendOperation.INGEST_TURN, adapter_method="ingest_message", backend_capability="message-level memory ingestion", result_model=BackendWriteResult, ), AdapterMappingSpec( - backend_type=BackendType.EVERMEMOS, + backend_type=BackendType.EVEROS, operation=BackendOperation.COMMIT_SESSION, adapter_method="extract_profile_long_term_v2", backend_capability="episodic/profile/long-term extraction", result_model=BackendCommitResult, ), AdapterMappingSpec( - backend_type=BackendType.EVERMEMOS, + backend_type=BackendType.EVEROS, operation=BackendOperation.RETRIEVE_CONTEXT, adapter_method="retrieve_context_v2", backend_capability="episodic/profile/long-term memory retrieval", diff --git a/memory_gateway/backend_normalization.py b/memory_gateway/backend_normalization.py index a3877a5..95f8e6d 100644 --- a/memory_gateway/backend_normalization.py +++ b/memory_gateway/backend_normalization.py @@ -63,16 +63,16 @@ def normalize_openviking_commit_response(raw: dict[str, Any]) -> BackendCommitRe ) -def normalize_evermemos_commit_response(raw: dict[str, Any]) -> BackendCommitResult: +def normalize_everos_commit_response(raw: dict[str, Any]) -> BackendCommitResult: status = _result_status(raw) - refs = [_produced_ref(BackendType.EVERMEMOS, item) for item in _extract_ref_items(raw)] + refs = [_produced_ref(BackendType.EVEROS, item) for item in _extract_ref_items(raw)] return BackendCommitResult( - backend_type=BackendType.EVERMEMOS, + backend_type=BackendType.EVEROS, operation=BackendOperation.COMMIT_SESSION, status=status, native_id=raw.get("native_id") or raw.get("session_id"), native_uri=raw.get("native_uri") or raw.get("uri"), - retryable=_retryable_from_raw(BackendType.EVERMEMOS, raw), + retryable=_retryable_from_raw(BackendType.EVEROS, raw), error_code=raw.get("error_code"), error_message=raw.get("error") or raw.get("error_message"), latency_ms=raw.get("latency_ms"), @@ -85,16 +85,16 @@ def normalize_openviking_ingest_response(raw: dict[str, Any]) -> BackendWriteRes return _write_result(BackendType.OPENVIKING, raw) -def normalize_evermemos_ingest_response(raw: dict[str, Any]) -> BackendWriteResult: - return _write_result(BackendType.EVERMEMOS, raw) +def normalize_everos_ingest_response(raw: dict[str, Any]) -> BackendWriteResult: + return _write_result(BackendType.EVEROS, raw) def normalize_openviking_retrieve_response(raw: dict[str, Any]) -> BackendRetrieveResult: return _retrieve_result(BackendType.OPENVIKING, raw) -def normalize_evermemos_retrieve_response(raw: dict[str, Any]) -> BackendRetrieveResult: - return _retrieve_result(BackendType.EVERMEMOS, raw) +def normalize_everos_retrieve_response(raw: dict[str, Any]) -> BackendRetrieveResult: + return _retrieve_result(BackendType.EVEROS, raw) def map_backend_error_to_retryable( @@ -129,10 +129,12 @@ def _write_result(backend_type: BackendType, raw: dict[str, Any]) -> BackendWrit raw.get("native_id") or raw.get("id") or raw.get("memory_id") + or raw.get("request_id") or raw.get("session_id") or data.get("native_id") or data.get("id") or data.get("memory_id") + or data.get("request_id") or data.get("session_id") ) native_uri = ( @@ -152,8 +154,8 @@ def _write_result(backend_type: BackendType, raw: dict[str, Any]) -> BackendWrit native_id=native_id, native_uri=native_uri, retryable=_retryable_from_raw(backend_type, raw), - error_code=raw.get("error_code"), - error_message=raw.get("error") or raw.get("error_message"), + error_code=raw.get("error_code") or raw.get("code"), + error_message=raw.get("error") or raw.get("error_message") or raw.get("message"), latency_ms=raw.get("latency_ms"), metadata=safe_backend_metadata(raw.get("metadata") or raw), ) @@ -174,8 +176,8 @@ def _retrieve_result(backend_type: BackendType, raw: dict[str, Any]) -> BackendR native_id=raw.get("native_id") or raw.get("session_id"), native_uri=raw.get("native_uri") or raw.get("uri"), retryable=_retryable_from_raw(backend_type, raw), - error_code=raw.get("error_code"), - error_message=raw.get("error") or raw.get("error_message"), + error_code=raw.get("error_code") or raw.get("code"), + error_message=raw.get("error") or raw.get("error_message") or raw.get("message"), latency_ms=raw.get("latency_ms"), items=[_retrieve_item(backend_type, item) for item in _extract_retrieve_items(raw)], metadata=safe_backend_metadata(raw.get("metadata") or raw), diff --git a/memory_gateway/backend_ref_mapping.py b/memory_gateway/backend_ref_mapping.py index 3212f94..178cfb2 100644 --- a/memory_gateway/backend_ref_mapping.py +++ b/memory_gateway/backend_ref_mapping.py @@ -11,7 +11,7 @@ OPENVIKING_REF_TYPE_MAP = { "session_summary": MemoryRefType.SESSION_ARCHIVE, } -EVERMEMOS_REF_TYPE_MAP = { +EVEROS_REF_TYPE_MAP = { "message_memory": MemoryRefType.MESSAGE_MEMORY, "episodic_memory": MemoryRefType.EPISODIC_MEMORY, "episode": MemoryRefType.EPISODIC_MEMORY, @@ -41,8 +41,8 @@ def map_backend_ref_type( if backend_type == BackendType.OPENVIKING: mapped = OPENVIKING_REF_TYPE_MAP.get(normalized, MemoryRefType.SESSION_ARCHIVE) - elif backend_type == BackendType.EVERMEMOS: - mapped = EVERMEMOS_REF_TYPE_MAP.get(normalized, MemoryRefType.LONG_TERM_MEMORY) + elif backend_type == BackendType.EVEROS: + mapped = EVEROS_REF_TYPE_MAP.get(normalized, MemoryRefType.LONG_TERM_MEMORY) elif backend_type == BackendType.OBSIDIAN: mapped = OBSIDIAN_REF_TYPE_MAP.get(normalized, MemoryRefType.DRAFT_REVIEW) else: @@ -59,8 +59,8 @@ def map_backend_ref_type( def _known_backend_ref_types(backend_type: BackendType) -> set[str]: if backend_type == BackendType.OPENVIKING: return set(OPENVIKING_REF_TYPE_MAP) - if backend_type == BackendType.EVERMEMOS: - return set(EVERMEMOS_REF_TYPE_MAP) + if backend_type == BackendType.EVEROS: + return set(EVEROS_REF_TYPE_MAP) if backend_type == BackendType.OBSIDIAN: return set(OBSIDIAN_REF_TYPE_MAP) return set() diff --git a/memory_gateway/config.py b/memory_gateway/config.py index 852184a..ad233e8 100644 --- a/memory_gateway/config.py +++ b/memory_gateway/config.py @@ -6,7 +6,7 @@ from typing import Optional import yaml from pydantic import ValidationError -from .types import Config, ServerConfig, OpenVikingConfig, EverMemOSConfig, MemoryConfig, LoggingConfig, LLMConfig, ObsidianConfig, StorageConfig +from .types import Config, ServerConfig, OpenVikingConfig, EverOSConfig, MemoryConfig, LoggingConfig, LLMConfig, ObsidianConfig, StorageConfig def load_config(config_path: Optional[str] = None) -> Config: @@ -30,7 +30,7 @@ def load_config(config_path: Optional[str] = None) -> Config: config = Config( server=ServerConfig(**data.get("server", {})), openviking=OpenVikingConfig(**data.get("openviking", {})), - evermemos=EverMemOSConfig(**data.get("evermemos", {})), + everos=EverOSConfig(**data.get("everos", {})), memory=MemoryConfig(**data.get("memory", {})), logging=LoggingConfig(**data.get("logging", {})), llm=LLMConfig(**data.get("llm", {})), @@ -62,11 +62,11 @@ _config: Optional[Config] = None def _apply_env_overrides(config: Config) -> Config: openviking_updates = _backend_env_updates("OPENVIKING") - evermemos_updates = _backend_env_updates("EVERMEMOS") + everos_updates = _backend_env_updates("EVEROS") if openviking_updates: config.openviking = config.openviking.model_copy(update=openviking_updates) - if evermemos_updates: - config.evermemos = config.evermemos.model_copy(update=evermemos_updates) + if everos_updates: + config.everos = config.everos.model_copy(update=everos_updates) return config @@ -83,6 +83,9 @@ def _backend_env_updates(prefix: str) -> dict: "TIMEOUT_SECONDS": "timeout", "VERIFY_SSL": "verify_ssl", "INGEST_PATH": "ingest_path", + "SEARCH_PATH": "search_path", + "FLUSH_PATH": "flush_path", + "RETRIEVE_METHOD": "retrieve_method", } for env_name, field_name in env_map.items(): value = os.environ.get(f"{prefix}_{env_name}") diff --git a/memory_gateway/evermemos_client.py b/memory_gateway/evermemos_client.py deleted file mode 100644 index c0b1b7f..0000000 --- a/memory_gateway/evermemos_client.py +++ /dev/null @@ -1,313 +0,0 @@ -"""Client for the external EverMemOS consolidation service.""" -from __future__ import annotations - -from json import JSONDecodeError -from typing import Any - -import httpx - -from .backend_contracts import BackendCommitResult, BackendOperation, BackendResultStatus, BackendRetrieveResult, BackendWriteResult -from .backend_normalization import ( - map_backend_error_to_retryable, - normalize_evermemos_commit_response, - normalize_evermemos_ingest_response, - normalize_evermemos_retrieve_response, -) -from .config import get_config -from .schemas import AccessContext, EpisodeRecord, MemoryRecord -from .schemas_v2 import BackendType - - -class EverMemOSError(RuntimeError): - """Raised when the external EverMemOS service cannot consolidate.""" - - -class EverMemOSClient: - """Small HTTP client with a tolerant response normalizer. - - The deployed EverMemOS API may evolve independently from Memory Gateway. - Gateway sends a stable payload and accepts several common response shapes: - `result`, `data`, or the raw top-level object with `candidates/promoted`. - """ - - def __init__( - self, - base_url: str | None = None, - api_key: str | None = None, - timeout: int | None = None, - enabled: bool | None = None, - mode: str | None = None, - verify_ssl: bool | None = None, - health_path: str | None = None, - ingest_path: str | None = None, - consolidate_path: str | None = None, - transport: httpx.BaseTransport | None = None, - ) -> None: - config = get_config().evermemos - self.base_url = (base_url if base_url is not None else config.url).rstrip("/") - self.api_key = api_key if api_key is not None else config.api_key - self.timeout = timeout or config.timeout - self.enabled = config.enabled if enabled is None else enabled - self.mode = mode or config.mode - self.verify_ssl = config.verify_ssl if verify_ssl is None else verify_ssl - self.health_path = health_path or config.health_path - self.ingest_path = ingest_path or config.ingest_path - self.consolidate_path = consolidate_path or config.consolidate_path - self.transport = transport - - def _headers(self) -> dict[str, str]: - headers = {"Content-Type": "application/json"} - if self.api_key: - headers["X-API-Key"] = self.api_key - headers["Authorization"] = f"Bearer {self.api_key}" - return headers - - def health(self) -> dict[str, Any]: - url = self.base_url + self.health_path - try: - health_timeout = httpx.Timeout(min(self.timeout, 2.0), connect=min(self.timeout, 0.5)) - with httpx.Client(timeout=health_timeout, headers=self._headers()) as client: - response = client.get(url) - response.raise_for_status() - return {"status": "ok", "url": self.base_url, "response": response.json()} - except Exception as exc: # noqa: BLE001 - return {"status": "error", "url": self.base_url, "error": str(exc)} - - def ingest_message(self, payload: dict[str, Any]) -> BackendWriteResult: - """v2 adapter placeholder for message-level EverMemOS ingestion. - - Mapping spec: `backend_adapter_mapping.AdapterMappingSpec` maps - EverMemOS ingest_turn to this method and requires BackendWriteResult. - Payloads must contain only control-plane fields; raw request bodies are - not persisted by the Gateway control-plane store. - - TODO(v2): bind this to EverMemOS `/api/v1/memories` or its stable - message ingestion API after the external contract settles. - """ - runtime_payload = self._build_ingest_payload(payload) - if self._use_real_api: - return self._ingest_message_real(runtime_payload) - raw = { - "status": "skipped", - "memory_id": runtime_payload.get("turn_id"), - "metadata": { - "reason": "evermemos_v2_ingest_adapter_not_configured", - "schema_version": "evermemos.fixture.ingest.v2", - }, - } - return self._normalize_ingest_response(raw) - - @property - def _use_real_api(self) -> bool: - # Real ingest is strictly gated by mode=real. The legacy `enabled` - # field is retained for config compatibility, but must not trigger - # network traffic by itself. - return self.mode == "real" - - def _ingest_message_real(self, runtime_payload: dict[str, Any]) -> BackendWriteResult: - if not self.base_url: - return self._failed_ingest_result( - error_code="config_error", - error_message="EverMemOS real ingest is enabled but base_url is missing", - retryable=False, - ) - try: - with httpx.Client( - base_url=self.base_url, - headers=self._headers(), - timeout=self.timeout, - verify=self.verify_ssl, - transport=self.transport, - ) as client: - response = client.post(self.ingest_path, json=runtime_payload) - if response.status_code >= 400: - return self._failed_ingest_result( - error_code=f"http_{response.status_code}", - error_message=f"EverMemOS ingest failed with HTTP {response.status_code}", - retryable=self._map_error(response), - ) - try: - raw = response.json() - except (JSONDecodeError, ValueError): - return self._failed_ingest_result( - error_code="invalid_json", - error_message="EverMemOS ingest returned invalid JSON", - retryable=True, - ) - if not isinstance(raw, dict): - return self._failed_ingest_result( - error_code="unexpected_response", - error_message="EverMemOS ingest returned an unexpected response shape", - retryable=True, - ) - return self._normalize_ingest_response(raw) - except httpx.TimeoutException as exc: - return self._failed_ingest_result("timeout", self._safe_error_message(exc), retryable=self._map_error(exc)) - except httpx.RequestError as exc: - return self._failed_ingest_result("network_error", self._safe_error_message(exc), retryable=self._map_error(exc)) - except Exception as exc: # noqa: BLE001 - return self._failed_ingest_result("unexpected_error", self._safe_error_message(exc), retryable=self._map_error(exc)) - - def extract_profile_long_term_v2(self, payload: dict[str, Any]) -> BackendCommitResult: - """v2 adapter placeholder for profile / long-term extraction. - - Mapping spec: commit_session returns BackendCommitResult and should - produce native episodic/profile/long-term refs once the real API is stable. - """ - runtime_payload = self._build_commit_payload(payload) - raw = { - "status": "success", - "session_id": runtime_payload.get("session_id"), - "metadata": { - "reason": "evermemos_v2_commit_fixture", - "schema_version": "evermemos.fixture.commit.v2", - }, - "data": { - "produced_refs": [ - { - "ref_type": "profile", - "profile_id": f"em_profile:{runtime_payload.get('user_id') or 'unknown'}", - "metadata": {"schema_version": "evermemos.fixture.profile.v2"}, - }, - { - "ref_type": "long_term_memory", - "memory_id": f"em_long_term:{runtime_payload.get('session_id')}", - "metadata": {"schema_version": "evermemos.fixture.long_term.v2"}, - }, - ] - }, - } - return self._normalize_commit_response(raw) - - def retrieve_context_v2(self, payload: dict[str, Any]) -> BackendRetrieveResult: - """v2 adapter placeholder for episodic/profile/long-term retrieval. - - Mapping spec: retrieve_context returns BackendRetrieveResult with - normalized context items, not raw backend payload dumps. - """ - raw = { - "status": "success", - "metadata": { - "reason": "evermemos_v2_retrieve_fixture", - "schema_version": "evermemos.fixture.retrieve.v2", - }, - "data": { - "items": [ - { - "text": "EverMemOS fixture profile context.", - "profile_id": f"em_profile:{payload.get('user_id') or 'unknown'}", - "score": 0.72, - "memory_type": "profile", - "metadata": {"schema_version": "evermemos.fixture.retrieve.item.v2"}, - }, - { - "text": "EverMemOS fixture long-term memory context.", - "memory_id": f"em_long_term:{payload.get('session_id') or 'unknown'}", - "score": 0.69, - "memory_type": "long_term_memory", - "metadata": {"schema_version": "evermemos.fixture.retrieve.item.v2"}, - }, - ] - }, - } - return self._normalize_retrieve_response(raw) - - def _build_ingest_payload(self, payload: dict[str, Any]) -> dict[str, Any]: - # Runtime-only adapter payload. It may include conversation content for - # the current request lifecycle; callers must not persist it to SQLite. - return dict(payload) - - def _build_commit_payload(self, payload: dict[str, Any]) -> dict[str, Any]: - return dict(payload) - - def _normalize_ingest_response(self, raw: dict[str, Any]) -> BackendWriteResult: - return normalize_evermemos_ingest_response(raw) - - def _normalize_commit_response(self, raw: dict[str, Any]) -> BackendCommitResult: - return normalize_evermemos_commit_response(raw) - - def _normalize_retrieve_response(self, raw: dict[str, Any]) -> BackendRetrieveResult: - return normalize_evermemos_retrieve_response(raw) - - def _map_error(self, exc_or_response: Any) -> bool: - status_code = getattr(exc_or_response, "status_code", None) - error_code = getattr(exc_or_response, "error_code", None) - error_message = str(exc_or_response) if exc_or_response is not None else None - return map_backend_error_to_retryable( - BackendType.EVERMEMOS, - status_code=status_code, - error_code=error_code, - error_message=error_message, - ) - - def _failed_ingest_result(self, error_code: str, error_message: str, retryable: bool) -> BackendWriteResult: - return BackendWriteResult( - backend_type=BackendType.EVERMEMOS, - operation=BackendOperation.INGEST_TURN, - status=BackendResultStatus.FAILED, - retryable=retryable, - error_code=error_code, - error_message=error_message, - metadata={"error_code": error_code}, - ) - - def _safe_error_message(self, exc: Exception) -> str: - return exc.__class__.__name__ - - def consolidate_session( - self, - session_id: str, - ctx: AccessContext, - episodes: list[EpisodeRecord], - existing_memories: list[MemoryRecord], - min_importance: float, - target_namespace: str | None, - ) -> dict[str, Any]: - payload = { - "schema_version": "memory-gateway.evermemos.consolidate.v1", - "session_id": session_id, - "context": ctx.model_dump(mode="json"), - "min_importance": min_importance, - "target_namespace": target_namespace, - "episodes": [episode.model_dump(mode="json") for episode in episodes], - "existing_memories": [memory.model_dump(mode="json") for memory in existing_memories], - } - paths = [ - self.consolidate_path, - "/v1/sessions/consolidate", - "/v1/memory/consolidate", - "/api/v1/sessions/consolidate", - "/api/consolidate", - "/consolidate", - ] - errors: list[str] = [] - for path in dict.fromkeys(paths): - try: - with httpx.Client(timeout=self.timeout, headers=self._headers()) as client: - response = client.post(self.base_url + path, json=payload) - if response.status_code == 404: - errors.append(f"{path}: 404") - continue - response.raise_for_status() - return self._normalize_response(response.json(), path) - except Exception as exc: # noqa: BLE001 - errors.append(f"{path}: {exc}") - if "Connection refused" in str(exc) or "timed out" in str(exc): - break - raise EverMemOSError("; ".join(errors) or "EverMemOS consolidation failed") - - def _normalize_response(self, payload: dict[str, Any], path: str) -> dict[str, Any]: - data = payload.get("result") or payload.get("data") or payload - return { - "backend": "external", - "service_url": self.base_url, - "endpoint": path, - "raw": payload, - "session_id": data.get("session_id"), - "episodes": data.get("episodes"), - "candidates": data.get("candidates") or data.get("candidate_memories") or [], - "promoted": data.get("promoted") or data.get("promoted_memories") or data.get("memories") or [], - "duplicates": data.get("duplicates") or [], - "conflicts": data.get("conflicts") or [], - "review_drafts": data.get("review_drafts") or [], - } diff --git a/memory_gateway/evermemos_service.py b/memory_gateway/evermemos_service.py deleted file mode 100644 index b0ac058..0000000 --- a/memory_gateway/evermemos_service.py +++ /dev/null @@ -1,149 +0,0 @@ -"""Standalone EverMemOS-compatible consolidation service. - -This is a lightweight local service for POC use. It intentionally exposes the -same HTTP contract that Memory Gateway calls: - -POST /v1/sessions/consolidate - -The service does not own Memory Gateway's metadata database. It receives -episodes and existing memories in the request, returns candidate/promoted -MemoryRecord payloads, and creates Obsidian review drafts for high-value or -conflicting candidates. -""" -from __future__ import annotations - -import argparse -import hashlib -import logging -from typing import Any - -from fastapi import FastAPI -from pydantic import BaseModel, Field - -from .config import load_config, set_config -from .repositories import InMemoryRepository -from .schemas import AccessContext, EpisodeRecord, MemoryRecord -from .workers.evermemos_worker import EverMemOSWorker - -logger = logging.getLogger(__name__) - - -class ConsolidateRequest(BaseModel): - schema_version: str = "memory-gateway.evermemos.consolidate.v1" - session_id: str - context: dict[str, Any] - min_importance: float = 0.6 - target_namespace: str | None = None - episodes: list[dict[str, Any]] = Field(default_factory=list) - existing_memories: list[dict[str, Any]] = Field(default_factory=list) - - -class MemoryIngestRequest(BaseModel): - workspace_id: str | None = None - user_id: str - session_id: str - turn_id: str - role: str = "user" - content: str - metadata: dict[str, Any] = Field(default_factory=dict) - source_type: str | None = None - source_event_id: str | None = None - - -app = FastAPI(title="Local EverMemOS POC Service", version="0.1.0") - - -@app.get("/health") -async def health() -> dict[str, Any]: - return { - "status": "ok", - "service": "evermemos-local", - "version": "0.1.0", - "contract": "memory-gateway.evermemos.consolidate.v1", - } - - -@app.post("/api/v1/memories") -async def ingest_memory(request: MemoryIngestRequest) -> dict[str, Any]: - """Accept message-level ingest for local real-adapter smoke tests. - - This POC endpoint intentionally does not persist raw conversation content. - It only returns a stable backend reference that Memory Gateway can store as - control-plane metadata. - """ - seed = "|".join( - [ - request.workspace_id or "", - request.user_id, - request.session_id, - request.turn_id, - request.source_event_id or "", - ] - ) - memory_id = "em_" + hashlib.sha256(seed.encode("utf-8")).hexdigest()[:24] - return { - "status": "success", - "memory_id": memory_id, - "native_uri": f"evermemos://memories/{memory_id}", - "metadata": { - "schema_version": "evermemos.local.ingest.v1", - "source_channel": request.metadata.get("source_channel") or request.metadata.get("channel"), - }, - } - - -@app.post("/v1/sessions/consolidate") -async def consolidate_session(request: ConsolidateRequest) -> dict[str, Any]: - repo = InMemoryRepository() - ctx = AccessContext.model_validate(request.context) - - for item in request.existing_memories: - try: - repo.upsert_memory(MemoryRecord.model_validate(item)) - except Exception as exc: # noqa: BLE001 - logger.warning("Skipping invalid existing memory: %s", exc) - - for item in request.episodes: - try: - repo.append_episode(EpisodeRecord.model_validate(item)) - except Exception as exc: # noqa: BLE001 - logger.warning("Skipping invalid episode: %s", exc) - - worker = EverMemOSWorker(repo) - result = worker.consolidate_session( - session_id=request.session_id, - ctx=ctx, - min_importance=request.min_importance, - target_namespace=request.target_namespace, - ) - return { - "status": "ok", - "backend": "evermemos-local", - "result": { - "session_id": result.session_id, - "episodes": result.episodes, - "candidates": [memory.model_dump(mode="json") for memory in result.candidates], - "promoted": [memory.model_dump(mode="json") for memory in result.promoted], - "duplicates": result.duplicates, - "conflicts": result.conflicts, - "review_drafts": result.review_drafts, - }, - } - - -def main() -> None: - import uvicorn - - parser = argparse.ArgumentParser(description="Run the local EverMemOS POC service.") - parser.add_argument("--config", default="config.yaml") - parser.add_argument("--host", default="127.0.0.1") - parser.add_argument("--port", type=int, default=1995) - args = parser.parse_args() - - config = load_config(args.config) - set_config(config) - uvicorn.run(app, host=args.host, port=args.port, log_level=config.logging.level.lower()) - - -if __name__ == "__main__": - main() diff --git a/memory_gateway/everos_client.py b/memory_gateway/everos_client.py new file mode 100644 index 0000000..2f5da74 --- /dev/null +++ b/memory_gateway/everos_client.py @@ -0,0 +1,496 @@ +"""Client for the external EverOS memory service.""" +from __future__ import annotations + +from datetime import datetime, timezone +from json import JSONDecodeError +from typing import Any + +import httpx + +from .backend_contracts import BackendCommitResult, BackendOperation, BackendResultStatus, BackendRetrieveResult, BackendWriteResult +from .backend_normalization import ( + map_backend_error_to_retryable, + normalize_everos_commit_response, + normalize_everos_ingest_response, + normalize_everos_retrieve_response, +) +from .config import get_config +from .schemas import AccessContext, EpisodeRecord, MemoryRecord +from .schemas_v2 import BackendType + + +class EverOSError(RuntimeError): + """Raised when the external EverOS service cannot process a request.""" + + +class EverOSClient: + """Small HTTP client with a tolerant response normalizer. + + The deployed EverOS API may evolve independently from Memory Gateway. + Gateway sends a stable payload and accepts several common response shapes: + `result`, `data`, or the raw top-level object with `candidates/promoted`. + """ + + def __init__( + self, + base_url: str | None = None, + api_key: str | None = None, + timeout: int | None = None, + enabled: bool | None = None, + mode: str | None = None, + verify_ssl: bool | None = None, + health_path: str | None = None, + ingest_path: str | None = None, + search_path: str | None = None, + flush_path: str | None = None, + retrieve_method: str | None = None, + transport: httpx.BaseTransport | None = None, + ) -> None: + config = get_config().everos + self.base_url = (base_url if base_url is not None else config.url).rstrip("/") + self.api_key = api_key if api_key is not None else config.api_key + self.timeout = timeout or config.timeout + self.enabled = config.enabled if enabled is None else enabled + self.mode = mode or config.mode + self.verify_ssl = config.verify_ssl if verify_ssl is None else verify_ssl + self.health_path = health_path or config.health_path + self.ingest_path = ingest_path or config.ingest_path + self.search_path = search_path or config.search_path + self.flush_path = flush_path or config.flush_path + self.retrieve_method = retrieve_method or config.retrieve_method + self.transport = transport + + def _headers(self) -> dict[str, str]: + headers = {"Content-Type": "application/json"} + if self.api_key: + headers["X-API-Key"] = self.api_key + headers["Authorization"] = f"Bearer {self.api_key}" + return headers + + def health(self) -> dict[str, Any]: + url = self.base_url + self.health_path + try: + health_timeout = httpx.Timeout(min(self.timeout, 2.0), connect=min(self.timeout, 0.5)) + with httpx.Client(timeout=health_timeout, headers=self._headers()) as client: + response = client.get(url) + response.raise_for_status() + return {"status": "ok", "url": self.base_url, "response": response.json()} + except Exception as exc: # noqa: BLE001 + return {"status": "error", "url": self.base_url, "error": str(exc)} + + def ingest_message(self, payload: dict[str, Any]) -> BackendWriteResult: + """Write one Gateway turn to EverOS.""" + runtime_payload = self._build_ingest_payload(payload) + if self._use_real_api: + return self._ingest_message_real(runtime_payload) + raw = { + "status": "skipped", + "memory_id": (runtime_payload.get("messages") or [{}])[0].get("message_id"), + "metadata": { + "reason": "everos_v2_ingest_adapter_not_configured", + "schema_version": "everos.fixture.ingest.v2", + }, + } + return self._normalize_ingest_response(raw) + + @property + def _use_real_api(self) -> bool: + # Real ingest is strictly gated by mode=real. The legacy `enabled` + # field is retained for config compatibility, but must not trigger + # network traffic by itself. + return self.mode == "real" + + def _ingest_message_real(self, runtime_payload: dict[str, Any]) -> BackendWriteResult: + if not self.base_url: + return self._failed_ingest_result( + error_code="config_error", + error_message="EverOS real ingest is enabled but base_url is missing", + retryable=False, + ) + try: + with httpx.Client( + base_url=self.base_url, + headers=self._headers(), + timeout=self.timeout, + verify=self.verify_ssl, + transport=self.transport, + ) as client: + response = client.post(self.ingest_path, json=runtime_payload) + if response.status_code >= 400: + return self._failed_ingest_result( + error_code=f"http_{response.status_code}", + error_message=f"EverOS ingest failed with HTTP {response.status_code}", + retryable=self._map_error(response), + ) + try: + raw = response.json() + except (JSONDecodeError, ValueError): + return self._failed_ingest_result( + error_code="invalid_json", + error_message="EverOS ingest returned invalid JSON", + retryable=True, + ) + if not isinstance(raw, dict): + return self._failed_ingest_result( + error_code="unexpected_response", + error_message="EverOS ingest returned an unexpected response shape", + retryable=True, + ) + return self._normalize_ingest_response(raw) + except httpx.TimeoutException as exc: + return self._failed_ingest_result("timeout", self._safe_error_message(exc), retryable=self._map_error(exc)) + except httpx.RequestError as exc: + return self._failed_ingest_result("network_error", self._safe_error_message(exc), retryable=self._map_error(exc)) + except Exception as exc: # noqa: BLE001 + return self._failed_ingest_result("unexpected_error", self._safe_error_message(exc), retryable=self._map_error(exc)) + + def extract_profile_long_term_v2(self, payload: dict[str, Any]) -> BackendCommitResult: + """v2 adapter placeholder for profile / long-term extraction. + + Mapping spec: commit_session returns BackendCommitResult and should + produce native episodic/profile/long-term refs once the real API is stable. + """ + runtime_payload = self._build_commit_payload(payload) + raw = { + "status": "success", + "session_id": runtime_payload.get("session_id"), + "metadata": { + "reason": "everos_v2_commit_fixture", + "schema_version": "everos.fixture.commit.v2", + }, + "data": { + "produced_refs": [ + { + "ref_type": "profile", + "profile_id": f"everos_profile:{runtime_payload.get('user_id') or 'unknown'}", + "metadata": {"schema_version": "everos.fixture.profile.v2"}, + }, + { + "ref_type": "long_term_memory", + "memory_id": f"everos_long_term:{runtime_payload.get('session_id')}", + "metadata": {"schema_version": "everos.fixture.long_term.v2"}, + }, + ] + }, + } + return self._normalize_commit_response(raw) + + def retrieve_context_v2(self, payload: dict[str, Any]) -> BackendRetrieveResult: + """ + Calls EverOS native API to retrieve memories. + """ + if not self._use_real_api: + return BackendRetrieveResult( + backend_type=BackendType.EVEROS, + operation=BackendOperation.RETRIEVE_CONTEXT, + status=BackendResultStatus.SKIPPED, + items=[], + metadata={"reason": "everos_retrieve_requires_real_mode"}, + ) + + query = payload.get("query", "") + user_id = payload.get("user_id", "") + try: + with httpx.Client( + base_url=self.base_url, + headers=self._headers(), + timeout=self.timeout, + verify=self.verify_ssl, + transport=self.transport, + ) as client: + resp = client.post( + self.search_path, + json={ + "query": query, + "method": self.retrieve_method, + "memory_types": ["episodic_memory", "profile", "raw_message"], + "top_k": payload.get("limit", 10), + "filters": self._search_filters(user_id=user_id, session_id=payload.get("session_id")), + }, + ) + if resp.status_code >= 400: + return BackendRetrieveResult( + backend_type=BackendType.EVEROS, + operation=BackendOperation.RETRIEVE_CONTEXT, + status=BackendResultStatus.FAILED, + items=[], + error_code=f"http_{resp.status_code}", + error_message=f"EverOS retrieve failed: {resp.text}", + retryable=False + ) + + items = self._items_from_search_response(resp.json()) + + raw = { + "status": "success", + "data": { + "items": items + } + } + return self._normalize_retrieve_response(raw) + except Exception as exc: + return BackendRetrieveResult( + backend_type=BackendType.EVEROS, + operation=BackendOperation.RETRIEVE_CONTEXT, + status=BackendResultStatus.FAILED, + items=[], + error_code="request_error", + error_message=str(exc), + retryable=True + ) + + def _build_ingest_payload(self, payload: dict[str, Any]) -> dict[str, Any]: + """ + Builds the payload according to EverOS native message schema. + """ + return { + "user_id": payload.get("user_id") or "gateway_user", + "session_id": payload.get("session_id"), + "messages": [ + { + "message_id": payload.get("turn_id") or f"msg_{int(datetime.now(timezone.utc).timestamp() * 1000)}", + "sender_id": payload.get("user_id") or "gateway_user", + "sender_name": payload.get("user_id") or "gateway_user", + "role": self._everos_role(payload.get("role", "user")), + "timestamp": self._timestamp_ms(payload), + "content": payload.get("content", ""), + } + ], + } + + def _build_commit_payload(self, payload: dict[str, Any]) -> dict[str, Any]: + return dict(payload) + + def _normalize_ingest_response(self, raw: dict[str, Any]) -> BackendWriteResult: + return normalize_everos_ingest_response(raw) + + def _normalize_commit_response(self, raw: dict[str, Any]) -> BackendCommitResult: + return normalize_everos_commit_response(raw) + + def _normalize_retrieve_response(self, raw: dict[str, Any]) -> BackendRetrieveResult: + return normalize_everos_retrieve_response(raw) + + def _map_error(self, exc_or_response: Any) -> bool: + status_code = getattr(exc_or_response, "status_code", None) + error_code = getattr(exc_or_response, "error_code", None) + error_message = str(exc_or_response) if exc_or_response is not None else None + return map_backend_error_to_retryable( + BackendType.EVEROS, + status_code=status_code, + error_code=error_code, + error_message=error_message, + ) + + def _failed_ingest_result(self, error_code: str, error_message: str, retryable: bool) -> BackendWriteResult: + return BackendWriteResult( + backend_type=BackendType.EVEROS, + operation=BackendOperation.INGEST_TURN, + status=BackendResultStatus.FAILED, + retryable=retryable, + error_code=error_code, + error_message=error_message, + metadata={"error_code": error_code}, + ) + + def _safe_error_message(self, exc: Exception) -> str: + return exc.__class__.__name__ + + def consolidate_session( + self, + session_id: str, + ctx: AccessContext, + episodes: list[EpisodeRecord], + existing_memories: list[MemoryRecord], + min_importance: float, + target_namespace: str | None, + ) -> dict[str, Any]: + if not self.base_url: + raise EverOSError("EverOS real mode requires base_url") + + user_id = ctx.user_id or "gateway_user" + agent_id = ctx.agent_id or "gateway_agent" + + with httpx.Client( + base_url=self.base_url, + timeout=self.timeout, + headers=self._headers(), + verify=self.verify_ssl, + transport=self.transport, + ) as client: + for episode in episodes: + self._memorize_episode(client, episode=episode, session_id=session_id, user_id=user_id, agent_id=agent_id) + self._flush_session(client, session_id=session_id, user_id=user_id) + promoted = self._fetch_session_memories(client, session_id=session_id, user_id=user_id, target_namespace=target_namespace) + + return { + "backend": "external", + "service_url": self.base_url, + "endpoint": self.ingest_path, + "raw": {"result": {"memories": promoted}}, + "session_id": session_id, + "episodes": len(episodes), + "candidates": promoted, + "promoted": promoted, + "duplicates": [], + "conflicts": [], + "review_drafts": [], + } + + def _memorize_episode( + self, + client: httpx.Client, + *, + episode: dict[str, Any], + session_id: str, + user_id: str, + agent_id: str, + ) -> None: + episode_data = episode.model_dump(mode="json") if hasattr(episode, "model_dump") else dict(episode) + episode_id = str(episode_data.get("id") or f"epi_{int(datetime.now(timezone.utc).timestamp())}") + sender = agent_id if episode_data.get("source") == "agent" else user_id + role = "assistant" if sender == agent_id else "user" + created_at = episode_data.get("created_at") or datetime.now(timezone.utc).isoformat() + payload = { + "user_id": user_id, + "session_id": session_id, + "messages": [ + { + "message_id": episode_id, + "sender_id": sender, + "sender_name": sender, + "role": role, + "timestamp": self._datetime_to_ms(created_at), + "content": episode_data.get("content") or "", + } + ], + } + response = client.post(self.ingest_path, json=payload) + response.raise_for_status() + + def _flush_session(self, client: httpx.Client, *, session_id: str, user_id: str) -> None: + response = client.post(self.flush_path, json={"user_id": user_id, "session_id": session_id}) + response.raise_for_status() + + def _fetch_session_memories( + self, + client: httpx.Client, + *, + session_id: str, + user_id: str, + target_namespace: str | None, + ) -> list[dict[str, Any]]: + response = client.post( + self.search_path, + json={ + "query": "memory", + "method": self.retrieve_method, + "memory_types": ["episodic_memory"], + "top_k": 20, + "filters": self._search_filters(user_id=user_id, session_id=session_id), + }, + ) + response.raise_for_status() + memories = self._items_from_search_response(response.json()) + normalized: list[dict[str, Any]] = [] + for index, memory in enumerate(memories, start=1): + content = memory.get("text") or memory.get("content") or memory.get("summary") or "" + if not content: + continue + normalized.append( + { + "id": memory.get("memory_id") or memory.get("id") or f"everos_{session_id}_{index}", + "namespace": target_namespace or f"user/{user_id}/long_term", + "memory_type": memory.get("memory_type") or "episodic_memory", + "content": content, + "summary": memory.get("summary") or content[:180], + "tags": ["everos-real", "memory-gateway"], + "importance": 0.7, + "confidence": 0.7, + "source": "everos", + "source_ref": memory.get("memory_id") or memory.get("id"), + } + ) + return normalized + + def _items_from_search_response(self, payload: dict[str, Any]) -> list[dict[str, Any]]: + data = payload.get("data") if isinstance(payload.get("data"), dict) else payload + items: list[dict[str, Any]] = [] + for memory_type, key in ( + ("episodic_memory", "episodes"), + ("profile", "profiles"), + ("raw_message", "raw_messages"), + ): + for item in data.get(key, []) or []: + if isinstance(item, dict): + items.append({**item, "memory_type": item.get("memory_type") or memory_type, "text": self._memory_text(item)}) + agent_memory = data.get("agent_memory") or {} + if isinstance(agent_memory, dict): + for item in agent_memory.get("cases", []) or []: + if isinstance(item, dict): + items.append({**item, "memory_type": "agent_case", "text": self._memory_text(item)}) + for item in agent_memory.get("skills", []) or []: + if isinstance(item, dict): + items.append({**item, "memory_type": "agent_skill", "text": self._memory_text(item)}) + return items + + def _memory_text(self, item: dict[str, Any]) -> str: + content_items = item.get("content_items") + if isinstance(content_items, list): + content_text = "\n".join( + str(content.get("text") or content.get("content") or "") + for content in content_items + if isinstance(content, dict) + ).strip() + else: + content_text = "" + profile_data = item.get("profile_data") + if isinstance(profile_data, dict): + profile_text = str(profile_data) + else: + profile_text = "" + return ( + item.get("episode") + or item.get("summary") + or item.get("subject") + or item.get("atomic_fact") + or item.get("task_intent") + or item.get("approach") + or item.get("content") + or content_text + or item.get("description") + or profile_text + or "" + ) + + def _search_filters(self, *, user_id: str | None, session_id: str | None = None) -> dict[str, Any]: + filters: dict[str, Any] = {"user_id": user_id or "gateway_user"} + if session_id: + filters["session_id"] = session_id + return filters + + def _timestamp_ms(self, payload: dict[str, Any]) -> int: + trace = payload.get("trace") if isinstance(payload.get("trace"), dict) else {} + timestamp = trace.get("timestamp") or payload.get("created_at") + if timestamp: + return self._datetime_to_ms(timestamp) + return int(datetime.now(timezone.utc).timestamp() * 1000) + + def _datetime_to_ms(self, value: Any) -> int: + if isinstance(value, (int, float)): + return int(value if value > 1_000_000_000_000 else value * 1000) + if isinstance(value, str): + text = value.replace("Z", "+00:00") + try: + return int(datetime.fromisoformat(text).timestamp() * 1000) + except ValueError: + return int(datetime.now(timezone.utc).timestamp() * 1000) + if isinstance(value, datetime): + return int(value.timestamp() * 1000) + return int(datetime.now(timezone.utc).timestamp() * 1000) + + def _everos_role(self, role: str) -> str: + if role in {"assistant", "agent"}: + return "assistant" + if role == "tool": + return "assistant" + return "user" diff --git a/memory_gateway/obsidian_review.py b/memory_gateway/obsidian_review.py index 4764615..c12ba44 100644 --- a/memory_gateway/obsidian_review.py +++ b/memory_gateway/obsidian_review.py @@ -43,7 +43,7 @@ def write_review_draft(memory: MemoryRecord, reason: str, conflict_ids: list[str f"created_at: {datetime.now(timezone.utc).isoformat()}", "tags:", " - memory/review", - " - source/evermemos", + " - source/everos", "---", "", f"# Memory Review - {title}", diff --git a/memory_gateway/openviking_client.py b/memory_gateway/openviking_client.py index e3de4dd..d714c48 100644 --- a/memory_gateway/openviking_client.py +++ b/memory_gateway/openviking_client.py @@ -1,7 +1,6 @@ """OpenViking client wrapper used by Memory Gateway.""" from __future__ import annotations -import json import logging import mimetypes import tempfile @@ -58,6 +57,7 @@ class OpenVikingClient: headers = {} if self.api_key: headers["X-API-Key"] = self.api_key + headers["Authorization"] = f"Bearer {self.api_key}" headers["X-OpenViking-Account"] = self.account headers["X-OpenViking-User"] = self.user return headers @@ -190,36 +190,64 @@ class OpenVikingClient: return self._normalize_commit_response(raw) async def retrieve_context_v2(self, payload: dict[str, Any]) -> BackendRetrieveResult: - """v2 adapter placeholder for OpenViking runtime context retrieval. - - Mapping spec: retrieve_context returns BackendRetrieveResult with - runtime context items, not raw backend payload dumps. """ - raw = { - "status": "ok", - "session_id": payload.get("session_id"), - "metadata": { - "reason": "openviking_v2_retrieve_fixture", - "schema_version": "openviking.fixture.retrieve.v2", - }, - "result": { - "items": [ - { - "text": "OpenViking fixture runtime context.", - "ref_id": f"ov_context:{payload.get('session_id') or 'unknown'}", - "score": 0.75, - "memory_type": "context_resource", - "metadata": {"schema_version": "openviking.fixture.retrieve.item.v2"}, - } - ] - }, - } - return self._normalize_retrieve_response(raw) + Calls OpenViking native API to retrieve context. + Uses POST /search + """ + if not self._use_real_api: + return BackendRetrieveResult( + backend_type=BackendType.OPENVIKING, + operation=BackendOperation.RETRIEVE_CONTEXT, + status=BackendResultStatus.SKIPPED, + items=[], + metadata={"reason": "openviking_retrieve_requires_real_mode"}, + ) + + query = payload.get("query", "") + session_id = payload.get("session_id") + + request_data = {"query": query, "limit": 10} + if session_id: + request_data["session_id"] = session_id + + try: + client = await self._get_client() + response = await client.post("/api/v1/search/search", json=request_data) + + if response.status_code >= 400: + return BackendRetrieveResult( + backend_type=BackendType.OPENVIKING, + operation=BackendOperation.RETRIEVE_CONTEXT, + status=BackendResultStatus.FAILED, + items=[], + error_code=f"http_{response.status_code}", + error_message=f"OpenViking search failed: {response.text}", + retryable=False + ) + + return self._normalize_retrieve_response(response.json()) + except Exception as exc: + return BackendRetrieveResult( + backend_type=BackendType.OPENVIKING, + operation=BackendOperation.RETRIEVE_CONTEXT, + status=BackendResultStatus.FAILED, + items=[], + error_code="request_error", + error_message=str(exc), + retryable=True + ) def _build_ingest_payload(self, payload: dict[str, Any]) -> dict[str, Any]: - # Runtime-only adapter payload. It may include conversation content for - # the current request lifecycle; callers must not persist it to SQLite. - return dict(payload) + """ + Build payload for native OpenViking AddMessageRequest. + OpenViking only expects role and content, and maybe metadata. + """ + return { + "role": payload.get("role", "user"), + "content": payload.get("content", ""), + "metadata": payload.get("metadata", {}), + "session_id": payload.get("session_id") # kept so format_ingest_path can use it + } def _format_ingest_path(self, payload: dict[str, Any]) -> str: session_id = str(payload.get("session_id") or "unknown") @@ -277,9 +305,9 @@ class OpenVikingClient: payload["limit"] = limit if uri: - payload["uri"] = uri + payload["target_uri"] = uri elif namespace: - payload["uri"] = f"viking://{namespace}" + payload["target_uri"] = f"viking://{namespace}" try: response = await client.post("/api/v1/search/search", json=payload) @@ -321,7 +349,7 @@ class OpenVikingClient: ns = namespace or self.config.memory.default_namespace or "user/default/memories" try: - response = await client.post("/api/v1/sessions", json={"mode": "interactive"}) + response = await client.post("/api/v1/sessions") response.raise_for_status() session_data = response.json() @@ -329,17 +357,15 @@ class OpenVikingClient: return session_data session_id = session_data["result"]["session_id"] - commit_response = await client.post( - f"/api/v1/sessions/{session_id}/commit", + message_response = await client.post( + f"/api/v1/sessions/{session_id}/messages", json={ - "messages": [ - { - "role": "user", - "content": f"[{ns}/{memory_type}] {content}", - } - ] + "role": "user", + "content": f"[{ns}/{memory_type}] {content}", }, ) + message_response.raise_for_status() + commit_response = await client.post(f"/api/v1/sessions/{session_id}/commit") commit_response.raise_for_status() return commit_response.json() except httpx.HTTPError as e: @@ -396,7 +422,6 @@ class OpenVikingClient: "temp_path": temp_ref, "to": uri, "wait": wait, - "source_name": Path(uri).name or tmp_path.name, "strict": False, } response = await client.post("/api/v1/resources", json=payload) @@ -425,7 +450,7 @@ class OpenVikingClient: try: response = await client.post( "/api/v1/search/search", - json={"query": "", "uri": f"viking://{ns}", "limit": limit or 10}, + json={"query": "", "target_uri": f"viking://{ns}", "limit": limit or 10}, ) response.raise_for_status() data = response.json() @@ -458,7 +483,7 @@ class OpenVikingClient: try: response = await client.post( "/api/v1/search/search", - json={"query": "", "uri": uri, "limit": limit or 10}, + json={"query": "", "target_uri": uri, "limit": limit or 10}, ) response.raise_for_status() data = response.json() diff --git a/memory_gateway/schemas.py b/memory_gateway/schemas.py index 6c81fce..ab5398a 100644 --- a/memory_gateway/schemas.py +++ b/memory_gateway/schemas.py @@ -38,7 +38,7 @@ class SourceType(str, Enum): AGENT = "agent" OBSIDIAN = "obsidian" OPENVIKING = "openviking" - EVERMEMOS = "evermemos" + EVEROS = "everos" MANUAL = "manual" @@ -224,4 +224,3 @@ class NamespaceInfo(BaseModel): owner_user_id: Optional[str] = None visibility: Visibility description: str - diff --git a/memory_gateway/schemas_v2.py b/memory_gateway/schemas_v2.py index 58c8418..d078c50 100644 --- a/memory_gateway/schemas_v2.py +++ b/memory_gateway/schemas_v2.py @@ -30,7 +30,7 @@ class BackendRefStatus(str, Enum): class BackendType(str, Enum): OPENVIKING = "openviking" - EVERMEMOS = "evermemos" + EVEROS = "everos" OBSIDIAN = "obsidian" @@ -54,7 +54,7 @@ class TraceContext(BaseModel): class IngestPolicy(BaseModel): allow_openviking: bool = True - allow_evermemos: bool = True + allow_everos: bool = True allow_obsidian_review: bool = False redact_sensitive: bool = True require_human_review: bool = False diff --git a/memory_gateway/server.py b/memory_gateway/server.py index 1793a4b..a1c4f38 100644 --- a/memory_gateway/server.py +++ b/memory_gateway/server.py @@ -478,12 +478,12 @@ async def health_check(): try: ov_client = await get_openviking_client() ov_status = await ov_client.health_check() - evermemos_status = v1_service.evermemos_health() + everos_status = v1_service.everos_health() return { "status": "ok", "gateway": "memory-gateway", "openviking": ov_status, - "evermemos": evermemos_status, + "everos": everos_status, } except Exception as e: return { diff --git a/memory_gateway/services.py b/memory_gateway/services.py index 07dee39..10412ad 100644 --- a/memory_gateway/services.py +++ b/memory_gateway/services.py @@ -1,12 +1,13 @@ """Application services for the generic Memory Gateway v1 API.""" from __future__ import annotations +from dataclasses import dataclass, field from datetime import datetime, timezone from fastapi import HTTPException, status from .config import get_config -from .evermemos_client import EverMemOSError, EverMemOSClient +from .everos_client import EverOSError, EverOSClient from .namespace import can_access_memory, default_namespace_for_context, user_long_term_namespace, visible_namespaces from .openviking_client import get_openviking_client from .repositories import MetadataRepository, repository @@ -29,13 +30,23 @@ from .schemas import ( UserRecord, Visibility, ) -from .workers.evermemos_worker import EverMemOSWorker + + +@dataclass +class ConsolidationResult: + session_id: str + episodes: int + candidates: list[MemoryRecord] = field(default_factory=list) + promoted: list[MemoryRecord] = field(default_factory=list) + duplicates: list[dict] = field(default_factory=list) + review_drafts: list[str] = field(default_factory=list) + conflicts: list[dict] = field(default_factory=list) class MemoryGatewayService: - def __init__(self, repo: MetadataRepository = repository, evermemos_client: EverMemOSClient | None = None) -> None: + def __init__(self, repo: MetadataRepository = repository, everos_client: EverOSClient | None = None) -> None: self.repo = repo - self.evermemos_client = evermemos_client + self.everos_client = everos_client def create_user(self, request: CreateUserRequest) -> UserRecord: user = UserRecord( @@ -204,10 +215,10 @@ class MemoryGatewayService: session_id=session_id, ) target_namespace = request.target_namespace or user_long_term_namespace(request.user_id) - config = get_config().evermemos + config = get_config().everos if config.enabled: try: - external_result = (self.evermemos_client or EverMemOSClient()).consolidate_session( + external_result = (self.everos_client or EverOSClient()).consolidate_session( session_id=session_id, ctx=ctx, episodes=episodes, @@ -217,32 +228,29 @@ class MemoryGatewayService: ) result = self._persist_external_consolidation(external_result, ctx, session_id) backend = "external" - except EverMemOSError as exc: + except EverOSError as exc: error = str(exc) - if not config.fallback_to_local: - self._audit( - "evermemos_commit_failed", - "session", - session_id, - actor_user_id=request.user_id, - actor_agent_id=request.agent_id, - decision="deny", - metadata={"error": error}, - ) - raise HTTPException(status_code=status.HTTP_502_BAD_GATEWAY, detail=f"EverMemOS failed: {error}") from exc - result = self._commit_session_locally(session_id, ctx, request) - backend = "local-fallback" + self._audit( + "everos_commit_failed", + "session", + session_id, + actor_user_id=request.user_id, + actor_agent_id=request.agent_id, + decision="deny", + metadata={"error": error}, + ) + raise HTTPException(status_code=status.HTTP_502_BAD_GATEWAY, detail=f"EverOS failed: {error}") from exc else: - result = self._commit_session_locally(session_id, ctx, request) - backend = "local-disabled" + result = None + backend = "disabled" else: result = None self._audit("commit_session", "session", session_id, actor_user_id=request.user_id, actor_agent_id=request.agent_id) if not result: - return {"session_id": session_id, "episodes": len(episodes), "promoted": [], "evermemos_backend": backend} + return {"session_id": session_id, "episodes": len(episodes), "promoted": [], "everos_backend": backend} return { - "evermemos_backend": backend, - "evermemos_error": error, + "everos_backend": backend, + "everos_error": error, "session_id": session_id, "episodes": result.episodes, "candidates": result.candidates, @@ -252,24 +260,13 @@ class MemoryGatewayService: "review_drafts": result.review_drafts, } - def evermemos_health(self) -> dict: - config = get_config().evermemos + def everos_health(self) -> dict: + config = get_config().everos if not config.enabled: return {"status": "disabled", "url": config.url} - return (self.evermemos_client or EverMemOSClient()).health() - - def _commit_session_locally(self, session_id: str, ctx: AccessContext, request: CommitSessionRequest): - worker = EverMemOSWorker(self.repo) - return worker.consolidate_session( - session_id=session_id, - ctx=ctx, - min_importance=request.min_importance, - target_namespace=request.target_namespace or user_long_term_namespace(request.user_id), - ) + return (self.everos_client or EverOSClient()).health() def _persist_external_consolidation(self, external_result: dict, ctx: AccessContext, session_id: str): - from .workers.evermemos_worker import ConsolidationResult - result = ConsolidationResult( session_id=session_id, episodes=external_result.get("episodes") or len(self.repo.list_session_episodes(session_id)), @@ -302,11 +299,11 @@ class MemoryGatewayService: data.setdefault("memory_type", MemoryType.SUMMARY.value) data.setdefault("content", data.get("text") or data.get("summary") or "") data.setdefault("summary", data.get("content", "")[:180]) - data.setdefault("tags", ["evermemos-external"]) + data.setdefault("tags", ["everos-external"]) data.setdefault("importance", 0.7) data.setdefault("confidence", 0.65) data.setdefault("visibility", Visibility.PRIVATE.value) - data.setdefault("source", SourceType.EVERMEMOS.value) + data.setdefault("source", SourceType.EVEROS.value) if not data["content"]: return None return MemoryRecord.model_validate(data) diff --git a/memory_gateway/services_v2.py b/memory_gateway/services_v2.py index 05559c8..0e31c1b 100644 --- a/memory_gateway/services_v2.py +++ b/memory_gateway/services_v2.py @@ -12,13 +12,14 @@ from .backend_contracts import ( BackendOperation, BackendCommitResult, BackendProducedRef, + BackendRetrieveResult, BackendResultStatus, BackendWriteResult, CommitJob, OutboxEvent, OutboxEventStatus, ) -from .evermemos_client import EverMemOSClient +from .everos_client import EverOSClient from .openviking_client import get_openviking_client from .repositories import MetadataRepository, repository from .schemas import AuditLog @@ -52,11 +53,11 @@ class MemoryGatewayV2Service: self, repo: MetadataRepository = repository, openviking_client_factory: OpenVikingClientFactory = get_openviking_client, - evermemos_client: Any | None = None, + everos_client: Any | None = None, ) -> None: self.repo = repo self.openviking_client_factory = openviking_client_factory - self.evermemos_client = evermemos_client + self.everos_client = everos_client async def ingest_conversation_turn(self, request: IngestRequest) -> IngestResponse: normalized = self._normalize_ingest_request(request) @@ -92,9 +93,9 @@ class MemoryGatewayV2Service: ) ) - if normalized.policy.allow_evermemos: + if normalized.policy.allow_everos: refs.append( - await self._write_evermemos_message( + await self._write_everos_message( normalized, payload, gateway_id=gateway_id, @@ -108,7 +109,7 @@ class MemoryGatewayV2Service: normalized, gateway_id, provenance_id, - BackendType.EVERMEMOS, + BackendType.EVEROS, MemoryRefType.MESSAGE_MEMORY, BackendRefStatus.SKIPPED, content_hash=content_hash, @@ -188,8 +189,21 @@ class MemoryGatewayV2Service: ) async def retrieve_context(self, request: RetrieveRequest) -> RetrieveResponse: - # TODO(v2): expand namespace ACL, fan out concurrently to OpenViking and - # EverMemOS, then apply lightweight merge/rerank before returning. + payload = { + "workspace_id": request.workspace_id, + "user_id": request.user_id, + "agent_id": request.agent_id, + "session_id": request.session_id, + "namespace": request.namespace, + "query": request.query, + "limit": request.limit, + "metadata": request.metadata, + } + results = [ + await self._retrieve_openviking_context(payload), + await self._retrieve_everos_context(payload), + ] + items = self._merge_retrieve_items(results, limit=request.limit) refs = self.repo.list_memory_refs( workspace_id=request.workspace_id, user_id=request.user_id, @@ -198,21 +212,6 @@ class MemoryGatewayV2Service: namespace=request.namespace, limit=request.limit, ) - items = [ - ContextItem( - text=None, - source_backend=ref.backend_type, - ref_id=ref.id, - score=0.0, - memory_type=ref.ref_type.value, - metadata={ - "status": ref.status.value, - "native_id": ref.native_id, - "native_uri": ref.native_uri, - }, - ) - for ref in refs - ] trace_id = request.metadata.get("trace_id") if request.metadata else None return RetrieveResponse( status=OperationStatus.SUCCESS, @@ -220,7 +219,7 @@ class MemoryGatewayV2Service: refs=self._view_refs(refs), conflicts=[], trace_id=trace_id, - metadata={"skeleton": True}, + metadata=self._retrieve_metadata(results), ) async def record_memory_feedback(self, request: FeedbackRequest) -> FeedbackResponse: @@ -386,6 +385,83 @@ class MemoryGatewayV2Service: limit=limit, ) + async def _retrieve_openviking_context(self, payload: dict[str, Any]) -> BackendRetrieveResult: + try: + client = await self.openviking_client_factory() + if not hasattr(client, "retrieve_context_v2"): + return BackendRetrieveResult( + backend_type=BackendType.OPENVIKING, + status=BackendResultStatus.SKIPPED, + metadata={"reason": "adapter_method_missing"}, + ) + result = client.retrieve_context_v2(payload) + if hasattr(result, "__await__"): + result = await result + return result + except Exception as exc: # noqa: BLE001 + return BackendRetrieveResult( + backend_type=BackendType.OPENVIKING, + status=BackendResultStatus.FAILED, + error_code="adapter_exception", + error_message=str(exc), + retryable=True, + ) + + async def _retrieve_everos_context(self, payload: dict[str, Any]) -> BackendRetrieveResult: + try: + client = self.everos_client or EverOSClient() + if not hasattr(client, "retrieve_context_v2"): + return BackendRetrieveResult( + backend_type=BackendType.EVEROS, + status=BackendResultStatus.SKIPPED, + metadata={"reason": "adapter_method_missing"}, + ) + result = client.retrieve_context_v2(payload) + if hasattr(result, "__await__"): + result = await result + return result + except Exception as exc: # noqa: BLE001 + return BackendRetrieveResult( + backend_type=BackendType.EVEROS, + status=BackendResultStatus.FAILED, + error_code="adapter_exception", + error_message=str(exc), + retryable=True, + ) + + def _merge_retrieve_items(self, results: list[BackendRetrieveResult], limit: int) -> list[ContextItem]: + items: list[ContextItem] = [] + for result in results: + if result.status != BackendResultStatus.SUCCESS: + continue + for item in result.items: + items.append( + ContextItem( + text=item.text, + source_backend=item.source_backend, + ref_id=item.ref_id, + score=item.score, + memory_type=item.memory_type, + metadata=item.metadata, + ) + ) + items.sort(key=lambda item: item.score, reverse=True) + return items[:limit] + + def _retrieve_metadata(self, results: list[BackendRetrieveResult]) -> dict[str, Any]: + return { + "backend_results": [ + { + "backend_type": result.backend_type.value, + "status": result.status.value, + "items": len(result.items), + "error_code": result.error_code, + "error_message": result.error_message, + } + for result in results + ] + } + async def _execute_outbox_event(self, event: OutboxEvent) -> BackendCommitResult | BackendWriteResult: payload = self._outbox_payload(event) if event.operation != BackendOperation.COMMIT_SESSION: @@ -406,11 +482,11 @@ class MemoryGatewayV2Service: ) result = await client.commit_session_v2(payload) return result - if event.backend_type == BackendType.EVERMEMOS: - client = self.evermemos_client or EverMemOSClient() + if event.backend_type == BackendType.EVEROS: + client = self.everos_client or EverOSClient() if not hasattr(client, "extract_profile_long_term_v2"): return BackendCommitResult( - backend_type=BackendType.EVERMEMOS, + backend_type=event.backend_type, operation=BackendOperation.COMMIT_SESSION, status=BackendResultStatus.SKIPPED, metadata={"reason": "adapter_method_missing"}, @@ -557,7 +633,7 @@ class MemoryGatewayV2Service: pass if event.backend_type == BackendType.OPENVIKING: return MemoryRefType.SESSION_ARCHIVE - if event.backend_type == BackendType.EVERMEMOS: + if event.backend_type == BackendType.EVEROS: return MemoryRefType.LONG_TERM_MEMORY return MemoryRefType.DRAFT_REVIEW @@ -712,7 +788,7 @@ class MemoryGatewayV2Service: metadata=self._control_metadata(request, content_hash), ) - async def _write_evermemos_message( + async def _write_everos_message( self, request: IngestRequest, payload: dict[str, Any], @@ -721,13 +797,13 @@ class MemoryGatewayV2Service: content_hash: str, ) -> MemoryRef: try: - client = self.evermemos_client or EverMemOSClient() + client = self.everos_client or EverOSClient() if not hasattr(client, "ingest_message"): return self._save_ref( request, gateway_id, provenance_id, - BackendType.EVERMEMOS, + BackendType.EVEROS, MemoryRefType.MESSAGE_MEMORY, BackendRefStatus.SKIPPED, content_hash=content_hash, @@ -740,7 +816,7 @@ class MemoryGatewayV2Service: request, gateway_id, provenance_id, - BackendType.EVERMEMOS, + BackendType.EVEROS, MemoryRefType.MESSAGE_MEMORY, result, content_hash, @@ -750,7 +826,7 @@ class MemoryGatewayV2Service: request, gateway_id, provenance_id, - BackendType.EVERMEMOS, + BackendType.EVEROS, MemoryRefType.MESSAGE_MEMORY, BackendRefStatus.FAILED, content_hash=content_hash, @@ -946,7 +1022,7 @@ class MemoryGatewayV2Service: "idempotency_key": request.idempotency_key, "request_id": request.request_id, } - for backend_type in (BackendType.OPENVIKING, BackendType.EVERMEMOS): + for backend_type in (BackendType.OPENVIKING, BackendType.EVEROS): event = OutboxEvent( id=self._outbox_event_id(gateway_id, backend_type, BackendOperation.COMMIT_SESSION), event_type="commit_session", diff --git a/memory_gateway/types.py b/memory_gateway/types.py index 5c6826b..1ff29ba 100644 --- a/memory_gateway/types.py +++ b/memory_gateway/types.py @@ -21,8 +21,8 @@ class OpenVikingConfig(BaseModel): ingest_path: str = "/api/v1/sessions/{session_id}/messages" -class EverMemOSConfig(BaseModel): - """External EverMemOS consolidation service configuration.""" +class EverOSConfig(BaseModel): + """External EverOS memory service configuration.""" enabled: bool = False mode: Literal["offline", "skeleton", "real"] = "offline" url: str = "http://127.0.0.1:1995" @@ -31,9 +31,9 @@ class EverMemOSConfig(BaseModel): verify_ssl: bool = True health_path: str = "/health" ingest_path: str = "/api/v1/memories" - consolidate_path: str = "/v1/sessions/consolidate" - fallback_to_local: bool = True - + search_path: str = "/api/v1/memories/search" + flush_path: str = "/api/v1/memories/flush" + retrieve_method: Literal["keyword", "vector", "hybrid", "rrf", "agentic"] = "keyword" class MemoryConfig(BaseModel): """记忆配置""" @@ -71,16 +71,18 @@ class LoggingConfig(BaseModel): class Config(BaseModel): """完整配置""" + def __init__(self, **data: Any) -> None: + super().__init__(**data) + server: ServerConfig = Field(default_factory=ServerConfig) openviking: OpenVikingConfig = Field(default_factory=OpenVikingConfig) - evermemos: EverMemOSConfig = Field(default_factory=EverMemOSConfig) + everos: EverOSConfig = Field(default_factory=EverOSConfig) memory: MemoryConfig = Field(default_factory=MemoryConfig) logging: LoggingConfig = Field(default_factory=LoggingConfig) llm: LLMConfig = Field(default_factory=LLMConfig) obsidian: ObsidianConfig = Field(default_factory=ObsidianConfig) storage: StorageConfig = Field(default_factory=StorageConfig) - class SearchRequest(BaseModel): """搜索请求""" query: str diff --git a/memory_gateway/workers/__init__.py b/memory_gateway/workers/__init__.py deleted file mode 100644 index be8d2d5..0000000 --- a/memory_gateway/workers/__init__.py +++ /dev/null @@ -1,2 +0,0 @@ -"""Background worker skeletons.""" - diff --git a/memory_gateway/workers/evermemos_worker.py b/memory_gateway/workers/evermemos_worker.py deleted file mode 100644 index 3b1ef08..0000000 --- a/memory_gateway/workers/evermemos_worker.py +++ /dev/null @@ -1,186 +0,0 @@ -"""Minimal EverMemOS-style consolidation worker. - -This worker is deliberately deterministic for the POC. It extracts stable -candidate memories from session episodes, deduplicates them against existing -records, promotes eligible records, and sends high-risk/high-value candidates -to Obsidian review rather than blindly polluting long-term memory. -""" -from __future__ import annotations - -import hashlib -import re -from dataclasses import dataclass, field - -from memory_gateway.namespace import default_namespace_for_context -from memory_gateway.obsidian_review import write_review_draft -from memory_gateway.repositories import MetadataRepository -from memory_gateway.schemas import ( - AccessContext, - EpisodeRecord, - MemoryRecord, - MemoryType, - SourceType, - Visibility, -) - - -_SENTENCE_RE = re.compile(r"(?<=[。！？.!?])\s+|\n+") -_NOISE_RE = re.compile(r"\s+") - - -@dataclass -class ConsolidationResult: - session_id: str - episodes: int - candidates: list[MemoryRecord] = field(default_factory=list) - promoted: list[MemoryRecord] = field(default_factory=list) - duplicates: list[dict] = field(default_factory=list) - review_drafts: list[str] = field(default_factory=list) - conflicts: list[dict] = field(default_factory=list) - - -class EverMemOSWorker: - def __init__(self, repo: MetadataRepository) -> None: - self.repo = repo - - def consolidate_session( - self, - session_id: str, - ctx: AccessContext, - min_importance: float = 0.6, - target_namespace: str | None = None, - ) -> ConsolidationResult: - episodes = self.repo.list_session_episodes(session_id) - result = ConsolidationResult(session_id=session_id, episodes=len(episodes)) - existing = list(self.repo.list_memories()) - seen_fingerprints = {self._fingerprint(memory.content): memory for memory in existing} - - for episode in episodes: - for candidate in self._extract_candidates(episode, ctx, min_importance, target_namespace): - result.candidates.append(candidate) - fingerprint = self._fingerprint(candidate.content) - duplicate = seen_fingerprints.get(fingerprint) - if duplicate: - result.duplicates.append({"candidate_id": candidate.id, "existing_id": duplicate.id}) - continue - - conflict_ids = self._find_conflicts(candidate, existing) - if conflict_ids: - draft = write_review_draft(candidate, reason="conflict", conflict_ids=conflict_ids) - result.review_drafts.append(str(draft)) - result.conflicts.append({"candidate_id": candidate.id, "conflict_ids": conflict_ids}) - continue - - if candidate.importance >= 0.85: - draft = write_review_draft(candidate, reason="high_value") - result.review_drafts.append(str(draft)) - continue - - if candidate.importance >= min_importance and candidate.confidence >= 0.55: - self.repo.upsert_memory(candidate) - result.promoted.append(candidate) - seen_fingerprints[fingerprint] = candidate - existing.append(candidate) - - return result - - def _extract_candidates( - self, - episode: EpisodeRecord, - ctx: AccessContext, - min_importance: float, - target_namespace: str | None, - ) -> list[MemoryRecord]: - text = episode.summary or episode.content - parts = [self._normalize(part) for part in _SENTENCE_RE.split(text) if self._normalize(part)] - candidates: list[MemoryRecord] = [] - for part in parts: - if len(part) < 20: - continue - memory_type = self._classify_type(part, episode.tags) - importance = self._estimate_importance(part, episode.tags, min_importance) - confidence = 0.65 if episode.summary else 0.58 - visibility = Visibility.WORKSPACE_SHARED if "workspace" in episode.tags and ctx.workspace_id else Visibility.PRIVATE - memory_ctx = AccessContext( - user_id=ctx.user_id, - agent_id=ctx.agent_id, - workspace_id=ctx.workspace_id, - session_id=ctx.session_id, - ) - candidates.append( - MemoryRecord( - user_id=ctx.user_id, - agent_id=ctx.agent_id, - workspace_id=ctx.workspace_id, - session_id=episode.session_id, - namespace=target_namespace or default_namespace_for_context(memory_ctx, visibility), - memory_type=memory_type, - content=part, - summary=part[:180], - tags=list(set(episode.tags + ["promoted-from-session", "evermemos-candidate"])), - importance=importance, - confidence=confidence, - visibility=visibility, - source=SourceType.EVERMEMOS, - source_ref=episode.id, - ) - ) - return candidates - - def _classify_type(self, text: str, tags: list[str]) -> MemoryType: - lowered = text.lower() - if "preference" in tags or "偏好" in text: - return MemoryType.PREFERENCE - if "decision" in tags or "决定" in text or "决策" in text: - return MemoryType.DECISION - if "procedure" in tags or "步骤" in text or "流程" in text: - return MemoryType.PROCEDURE - if "经验" in text or "worked" in lowered or "failed" in lowered: - return MemoryType.EXPERIENCE - return MemoryType.SUMMARY - - def _estimate_importance(self, text: str, tags: list[str], min_importance: float) -> float: - importance = max(min_importance, 0.6) - signal_words = ["必须", "不要", "偏好", "长期", "决策", "结论", "重要", "preference", "decision", "must"] - if any(word in text.lower() for word in signal_words): - importance += 0.15 - if "review" in tags or "high-value" in tags: - importance += 0.2 - return min(1.0, importance) - - def _find_conflicts(self, candidate: MemoryRecord, existing: list[MemoryRecord]) -> list[str]: - candidate_text = candidate.content.lower() - negation_signals = ["不要", "不再", "禁止", "not ", "never", "disable"] - positive_signals = ["需要", "必须", "启用", "prefer", "always", "enable"] - has_negative = any(signal in candidate_text for signal in negation_signals) - has_positive = any(signal in candidate_text for signal in positive_signals) - if not has_negative and not has_positive: - return [] - - candidate_tokens = self._tokens(candidate.content) - conflicts = [] - for memory in existing: - if memory.user_id != candidate.user_id: - continue - if memory.memory_type != candidate.memory_type: - continue - overlap = candidate_tokens.intersection(self._tokens(memory.content)) - if len(overlap) < 2: - continue - memory_text = memory.content.lower() - memory_negative = any(signal in memory_text for signal in negation_signals) - memory_positive = any(signal in memory_text for signal in positive_signals) - if has_negative != memory_negative or has_positive != memory_positive: - conflicts.append(memory.id) - return conflicts - - def _tokens(self, text: str) -> set[str]: - return {token for token in re.split(r"[^a-zA-Z0-9\u4e00-\u9fff]+", text.lower()) if len(token) >= 2} - - def _normalize(self, text: str) -> str: - return _NOISE_RE.sub(" ", text).strip(" -_*#\t") - - def _fingerprint(self, text: str) -> str: - normalized = self._normalize(text).lower() - return hashlib.sha1(normalized.encode("utf-8")).hexdigest() - diff --git a/plugins/memory-gateway-agent/README.md b/plugins/memory-gateway-agent/README.md index 24e16cd..d82d89b 100644 --- a/plugins/memory-gateway-agent/README.md +++ b/plugins/memory-gateway-agent/README.md @@ -117,7 +117,7 @@ Verified boundaries: The plugin rejects memory writes containing passwords, API keys, bearer tokens, cookies, private keys, SSH keys, one-time verification codes, large logs, full raw transcripts, and chain-of-thought. -The plugin writes summarized candidate episodes. It does not store full raw conversations. Long-term memory should normally be produced by `memory_commit_session`, allowing Memory Gateway and EverMemOS to deduplicate, detect conflicts, and route review drafts. +The plugin writes summarized candidate episodes. It does not store full raw conversations. Long-term memory should normally be produced by `memory_commit_session`, allowing Memory Gateway and EverOS to deduplicate, detect conflicts, and route review drafts. Direct long-term `memory_upsert` is high risk and is not called automatically. If a user asks to forget or delete a memory, the agent should call `memory_feedback` or a delete-capable tool instead of silently keeping the memory. diff --git a/plugins/memory-gateway-agent/hermes.plugin.yaml b/plugins/memory-gateway-agent/hermes.plugin.yaml index e5a9358..8213ce0 100644 --- a/plugins/memory-gateway-agent/hermes.plugin.yaml +++ b/plugins/memory-gateway-agent/hermes.plugin.yaml @@ -25,7 +25,7 @@ tools: memory_append_episode: description: Append a safe summarized candidate episode. memory_commit_session: - description: Ask Gateway/EverMemOS to consolidate session episodes. + description: Ask Gateway/EverOS to consolidate session episodes. memory_upsert: description: Upsert a stable memory through Gateway. memory_feedback: diff --git a/plugins/memory-gateway-agent/openclaw.plugin.yaml b/plugins/memory-gateway-agent/openclaw.plugin.yaml index 4c3e8f3..e6bc1cf 100644 --- a/plugins/memory-gateway-agent/openclaw.plugin.yaml +++ b/plugins/memory-gateway-agent/openclaw.plugin.yaml @@ -22,5 +22,5 @@ hooks: safety: stores_full_raw_conversation: false rejects_secrets: true - long_term_commit_via_evermemos: true + long_term_commit_via_everos: true diff --git a/plugins/memory-gateway-agent/policies/memory_policy.md b/plugins/memory-gateway-agent/policies/memory_policy.md index 9fe61cd..34c12e9 100644 --- a/plugins/memory-gateway-agent/policies/memory_policy.md +++ b/plugins/memory-gateway-agent/policies/memory_policy.md @@ -16,7 +16,7 @@ During a task: At task or session completion: -- Use `memory_commit_session` to let Memory Gateway and EverMemOS decide what can be promoted. +- Use `memory_commit_session` to let Memory Gateway and EverOS decide what can be promoted. - Do not promote all episodes directly to long-term memory. - Conflicting or high-value memories should enter review rather than overwrite existing memory. diff --git a/plugins/memory-gateway-agent/policies/safety_filter.md b/plugins/memory-gateway-agent/policies/safety_filter.md index e7e8a68..df630d5 100644 --- a/plugins/memory-gateway-agent/policies/safety_filter.md +++ b/plugins/memory-gateway-agent/policies/safety_filter.md @@ -20,5 +20,5 @@ The plugin must reject memory writes that contain: The plugin stores summaries rather than raw messages. If a message is useful but contains sensitive detail, redact the sensitive detail before writing. If redaction would remove the meaning, reject the write. -Long-term memory should normally be created by session commit and EverMemOS consolidation, not by direct upsert. +Long-term memory should normally be created by session commit and EverOS consolidation, not by direct upsert. diff --git a/plugins/memory-gateway-agent/schemas.py b/plugins/memory-gateway-agent/schemas.py index ed90068..210b6fd 100644 --- a/plugins/memory-gateway-agent/schemas.py +++ b/plugins/memory-gateway-agent/schemas.py @@ -42,7 +42,7 @@ MEMORY_APPEND_EPISODE = { MEMORY_COMMIT_SESSION = { "name": "memory_commit_session", - "description": "Commit a session through Memory Gateway and EverMemOS. Promotes only what consolidation accepts.", + "description": "Commit a session through Memory Gateway and EverOS. Promotes only what consolidation accepts.", "parameters": { "type": "object", "properties": { diff --git a/pyproject.toml b/pyproject.toml index f3838d8..7ecdbc3 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -12,7 +12,6 @@ dependencies = [ "pydantic>=2.5.0", "pyyaml>=6.0", "uvicorn>=0.27.0", - "tenacity>=8.2.0", "markitdown[all]>=0.1.5", "python-multipart>=0.0.9", ] @@ -28,8 +27,5 @@ dev = [ requires = ["hatchling"] build-backend = "hatchling.build" -[tool.uv] -dev-dependencies = [] - [tool.ruff] target-version = "py310" diff --git a/tests/integration/test_real_ingest.py b/tests/integration/test_real_ingest.py index 7dc1fc8..58edc2a 100644 --- a/tests/integration/test_real_ingest.py +++ b/tests/integration/test_real_ingest.py @@ -7,7 +7,7 @@ from fastapi import FastAPI from httpx import ASGITransport, AsyncClient import memory_gateway.api_v2 as api_v2 -from memory_gateway.evermemos_client import EverMemOSClient +from memory_gateway.everos_client import EverOSClient from memory_gateway.openviking_client import OpenVikingClient from memory_gateway.repositories import InMemoryRepository from memory_gateway.schemas_v2 import BackendRefStatus, BackendType, IngestRequest, IngestResponse, OperationStatus @@ -28,13 +28,13 @@ def _env(name: str) -> str: return value -def test_real_openviking_and_evermemos_ingest_writes_memory_refs(): +def test_real_openviking_and_everos_ingest_writes_memory_refs(): openviking_base_url = _env("OPENVIKING_BASE_URL") - evermemos_base_url = _env("EVERMEMOS_BASE_URL") + everos_base_url = _env("EVEROS_BASE_URL") openviking_api_key = os.environ.get("OPENVIKING_API_KEY", "") - evermemos_api_key = os.environ.get("EVERMEMOS_API_KEY", "") + everos_api_key = os.environ.get("EVEROS_API_KEY", "") openviking_ingest_path = os.environ.get("OPENVIKING_INGEST_PATH") - evermemos_ingest_path = os.environ.get("EVERMEMOS_INGEST_PATH") + everos_ingest_path = os.environ.get("EVEROS_INGEST_PATH") async def openviking_factory(): return OpenVikingClient( @@ -48,11 +48,11 @@ def test_real_openviking_and_evermemos_ingest_writes_memory_refs(): service = MemoryGatewayV2Service( repo=repo, openviking_client_factory=openviking_factory, - evermemos_client=EverMemOSClient( + everos_client=EverOSClient( mode="real", - base_url=evermemos_base_url, - api_key=evermemos_api_key, - ingest_path=evermemos_ingest_path, + base_url=everos_base_url, + api_key=everos_api_key, + ingest_path=everos_ingest_path, ), ) run_id = uuid4().hex[:12] @@ -60,20 +60,20 @@ def test_real_openviking_and_evermemos_ingest_writes_memory_refs(): response = asyncio.run(post_ingest(service, run_id)) refs = repo.list_memory_refs(session_id=f"real_ingest_sess_{run_id}", limit=10) - assert {ref.backend_type for ref in refs} == {BackendType.OPENVIKING, BackendType.EVERMEMOS} + assert {ref.backend_type for ref in refs} == {BackendType.OPENVIKING, BackendType.EVEROS} assert all(ref.content_hash for ref in refs) openviking_ref = next(ref for ref in refs if ref.backend_type == BackendType.OPENVIKING) - evermemos_ref = next(ref for ref in refs if ref.backend_type == BackendType.EVERMEMOS) + everos_ref = next(ref for ref in refs if ref.backend_type == BackendType.EVEROS) assert openviking_ref.status == BackendRefStatus.SUCCESS - if evermemos_ref.status == BackendRefStatus.SUCCESS: + if everos_ref.status == BackendRefStatus.SUCCESS: assert response.status == OperationStatus.SUCCESS - assert evermemos_ref.native_id - assert evermemos_ref.native_uri + assert everos_ref.native_id + assert everos_ref.native_uri else: - assert evermemos_ref.status == BackendRefStatus.FAILED + assert everos_ref.status == BackendRefStatus.FAILED assert response.status == OperationStatus.PARTIAL_SUCCESS - assert evermemos_ref.error_message + assert everos_ref.error_message async def post_ingest(service: MemoryGatewayV2Service, run_id: str): diff --git a/tests/test_evermemos_service.py b/tests/test_evermemos_service.py deleted file mode 100644 index 2db3a0c..0000000 --- a/tests/test_evermemos_service.py +++ /dev/null @@ -1,53 +0,0 @@ -import asyncio - -from memory_gateway.evermemos_service import ConsolidateRequest, consolidate_session - - -def test_evermemos_service_consolidates_session(monkeypatch, tmp_path): - monkeypatch.setattr( - "memory_gateway.obsidian_review.get_config", - lambda: type( - "Config", - (), - { - "obsidian": type( - "Obsidian", - (), - {"vault_path": str(tmp_path / "vault"), "review_dir": "Reviews/Queue"}, - )() - }, - )(), - ) - payload = { - "session_id": "sess_service", - "context": {"user_id": "user_a", "agent_id": "agent_a", "workspace_id": "ws_a", "session_id": "sess_service"}, - "episodes": [ - { - "user_id": "user_a", - "agent_id": "agent_a", - "workspace_id": "ws_a", - "session_id": "sess_service", - "namespace": "session/sess_service/episodic", - "content": "结论：EverMemOS 本地服务负责整理稳定长期记忆。", - "tags": ["decision"], - }, - { - "user_id": "user_a", - "agent_id": "agent_a", - "workspace_id": "ws_a", - "session_id": "sess_service", - "namespace": "session/sess_service/episodic", - "content": "重要：高价值记忆应该进入 Obsidian review queue。", - "tags": ["review", "high-value"], - }, - ], - } - - response = asyncio.run(consolidate_session(ConsolidateRequest.model_validate(payload))) - - assert response["status"] == "ok" - result = response["result"] - assert result["episodes"] == 2 - assert len(result["candidates"]) == 2 - assert len(result["promoted"]) == 1 - assert len(result["review_drafts"]) == 1 diff --git a/tests/test_server.py b/tests/test_server.py index 0b2fdbb..470ef32 100644 --- a/tests/test_server.py +++ b/tests/test_server.py @@ -137,7 +137,7 @@ def test_health_requires_api_key(monkeypatch): fake_get_openviking_client, ) monkeypatch.setattr("memory_gateway.server.summarize_with_llm", fake_summarize_with_llm) - monkeypatch.setattr("memory_gateway.server.v1_service.evermemos_health", lambda: {"status": "disabled"}) + monkeypatch.setattr("memory_gateway.server.v1_service.everos_health", lambda: {"status": "disabled"}) with pytest.raises(HTTPException) as exc_info: server.verify_api_key() diff --git a/tests/test_v1_service.py b/tests/test_v1_service.py index f2d2018..4b1df12 100644 --- a/tests/test_v1_service.py +++ b/tests/test_v1_service.py @@ -11,7 +11,7 @@ from memory_gateway.schemas import ( Visibility, ) from memory_gateway.services import MemoryGatewayService -from memory_gateway.types import Config, EverMemOSConfig, ObsidianConfig +from memory_gateway.types import Config, EverOSConfig, ObsidianConfig def test_private_memory_is_isolated_by_user(): @@ -67,10 +67,10 @@ def test_sqlite_repository_persists_memory(tmp_path): assert reloaded.content == "持久化 SQLite memory" -def test_commit_session_promotes_dedupes_and_creates_review_draft(monkeypatch, tmp_path): +def test_commit_session_disabled_does_not_use_local_fallback(monkeypatch, tmp_path): monkeypatch.setattr( "memory_gateway.services.get_config", - lambda: Config(evermemos=EverMemOSConfig(enabled=False)), + lambda: Config(everos=EverOSConfig(enabled=False)), ) monkeypatch.setattr( "memory_gateway.obsidian_review.get_config", @@ -103,29 +103,27 @@ def test_commit_session_promotes_dedupes_and_creates_review_draft(monkeypatch, t ), ) - assert len(result["promoted"]) == 1 - assert result["evermemos_backend"] == "local-disabled" - assert len(result["review_drafts"]) == 1 - assert (tmp_path / "vault" / "Reviews" / "Queue").exists() + assert result["promoted"] == [] + assert result["everos_backend"] == "disabled" -def test_commit_session_uses_external_evermemos(monkeypatch): +def test_commit_session_uses_external_everos(monkeypatch): monkeypatch.setattr( "memory_gateway.services.get_config", - lambda: Config(evermemos=EverMemOSConfig(enabled=True, fallback_to_local=False)), + lambda: Config(everos=EverOSConfig(enabled=True)), ) - class FakeEverMemOSClient: + class FakeEverOSClient: def consolidate_session(self, **kwargs): return { "episodes": 1, "candidates": [], "promoted": [ { - "content": "外部 EverMemOS 总结出的长期记忆", - "summary": "外部 EverMemOS 长期记忆", + "content": "外部 EverOS 总结出的长期记忆", + "summary": "外部 EverOS 长期记忆", "memory_type": "summary", - "tags": ["external-evermemos"], + "tags": ["external-everos"], } ], "duplicates": [], @@ -136,12 +134,12 @@ def test_commit_session_uses_external_evermemos(monkeypatch): def health(self): return {"status": "ok"} - service = MemoryGatewayService(InMemoryRepository(), evermemos_client=FakeEverMemOSClient()) + service = MemoryGatewayService(InMemoryRepository(), everos_client=FakeEverOSClient()) service.append_episode( EpisodeAppendRequest( user_id="user_a", session_id="sess_external", - content="这条 episode 应该交给外部 EverMemOS。", + content="这条 episode 应该交给外部 EverOS。", ) ) result = service.commit_session( @@ -149,9 +147,9 @@ def test_commit_session_uses_external_evermemos(monkeypatch): CommitSessionRequest(user_id="user_a", session_id="sess_external"), ) - assert result["evermemos_backend"] == "external" + assert result["everos_backend"] == "external" assert len(result["promoted"]) == 1 - search = service.search_memory(MemorySearchRequest(user_id="user_a", query="外部 EverMemOS")) + search = service.search_memory(MemorySearchRequest(user_id="user_a", query="外部 EverOS")) assert search["total"] == 1 diff --git a/tests/test_v2_api.py b/tests/test_v2_api.py index 6763109..bb8e0ae 100644 --- a/tests/test_v2_api.py +++ b/tests/test_v2_api.py @@ -16,9 +16,9 @@ from memory_gateway.backend_adapter_mapping import ( ) from memory_gateway.backend_normalization import ( map_backend_error_to_retryable, - normalize_evermemos_commit_response, - normalize_evermemos_ingest_response, - normalize_evermemos_retrieve_response, + normalize_everos_commit_response, + normalize_everos_ingest_response, + normalize_everos_retrieve_response, normalize_openviking_commit_response, normalize_openviking_ingest_response, normalize_openviking_retrieve_response, @@ -27,13 +27,14 @@ from memory_gateway.backend_contracts import ( BackendCommitResult, BackendOperation, BackendProducedRef, + BackendRetrieveItem, BackendResultStatus, BackendRetrieveResult, BackendWriteResult, OutboxEventStatus, ) from memory_gateway.backend_ref_mapping import map_backend_ref_type -from memory_gateway.evermemos_client import EverMemOSClient +from memory_gateway.everos_client import EverOSClient from memory_gateway.obsidian_review_client import ObsidianReviewClient from memory_gateway.openviking_client import OpenVikingClient from memory_gateway.repositories import InMemoryRepository, SQLiteRepository @@ -51,12 +52,75 @@ from memory_gateway.server_auth import verify_api_key_compat from memory_gateway.services_v2 import MemoryGatewayV2Service -FIXTURE_DIR = Path(__file__).parent / "fixtures" / "backend_responses" DOCS_DIR = Path(__file__).parent.parent / "docs" -def load_backend_fixture(name: str): - return json.loads((FIXTURE_DIR / name).read_text()) +def backend_response(name: str): + responses = { + "openviking_ingest_success.json": { + "status": "created", + "id": "ov_turn_fixture_1", + "uri": "viking://sessions/sess_fixture/turns/ov_turn_fixture_1", + "metadata": {"schema_version": "openviking.fixture.ingest.v2", "conversation": "SECRET"}, + }, + "openviking_ingest_real_success.json": { + "status": "created", + "id": "ov_real_turn_fixture_1", + "uri": "viking://sessions/ov_real_sess_fixture_1/turns/ov_real_turn_fixture_1", + "metadata": {"backend_request_id": "ov_req_real_1", "content": "SECRET"}, + }, + "openviking_ingest_real_error_401.json": {"status": "failed", "error": "unauthorized", "error_code": "unauthorized"}, + "openviking_ingest_real_error_422.json": {"status": "failed", "error": "validation failed", "error_code": "validation_error"}, + "openviking_ingest_real_error_500.json": {"status": "failed", "error": "server error", "error_code": "server_error"}, + "openviking_commit_success.json": { + "status": "ok", + "session_id": "sess_fixture", + "result": { + "refs": [ + {"type": "session_archive", "id": "ov_archive_fixture_1"}, + {"type": "context_resource", "id": "ov_resource_fixture_1"}, + ] + }, + "metadata": {"schema_version": "openviking.fixture.commit.v2", "messages": ["SECRET"]}, + }, + "openviking_retrieve_success.json": { + "status": "ok", + "result": { + "items": [ + {"text": "Relevant session summary", "id": "ov_archive_fixture_1", "score": 0.91, "type": "session_archive"}, + {"text": "Relevant resource", "id": "ov_resource_fixture_1", "score": 0.84, "type": "context_resource"}, + ] + }, + "metadata": {"schema_version": "openviking.fixture.retrieve.v2", "transcript": "SECRET"}, + }, + "everos_ingest_success.json": { + "status": "success", + "memory_id": "em_memory_fixture_1", + "metadata": {"schema_version": "everos.fixture.ingest.v2", "transcript": "SECRET"}, + }, + "everos_commit_success_multiple_refs.json": { + "status": "success", + "data": { + "produced_refs": [ + {"ref_type": "episodic_memory", "memory_id": "em_episode_fixture_1"}, + {"ref_type": "profile", "profile_id": "em_profile_fixture_1"}, + {"ref_type": "unknown_kind", "id": "em_long_fixture_1"}, + ] + }, + "metadata": {"schema_version": "everos.fixture.commit.v2", "messages": ["SECRET"]}, + }, + "everos_retrieve_success.json": { + "status": "success", + "data": { + "items": [ + {"text": "Relevant episodic memory", "memory_id": "em_episode_fixture_1", "score": 0.88, "memory_type": "episodic_memory"}, + {"text": "Relevant profile", "profile_id": "em_profile_fixture_1", "score": 0.73, "memory_type": "profile"}, + ] + }, + "metadata": {"schema_version": "everos.fixture.retrieve.v2", "conversation": "SECRET"}, + }, + } + return responses[name] def build_ingest_payload(**overrides): @@ -86,23 +150,53 @@ class FakeOpenVikingClient: "native_uri": f"viking://sessions/{payload['session_id']}/{payload['turn_id']}", } + async def retrieve_context_v2(self, payload): + return BackendRetrieveResult( + backend_type=BackendType.OPENVIKING, + status=BackendResultStatus.SUCCESS, + items=[ + BackendRetrieveItem( + text="OpenViking context for remember", + source_backend=BackendType.OPENVIKING, + ref_id="ov_ctx_1", + score=0.82, + memory_type="context_resource", + ) + ], + ) + async def fake_openviking_factory(): return FakeOpenVikingClient() -class FakeEverMemOSClient: +class FakeEverOSClient: def ingest_message(self, payload): return { "status": "success", "native_id": f"em_{payload['turn_id']}", - "native_uri": f"evermemos://memories/{payload['turn_id']}", + "native_uri": f"everos://memories/{payload['turn_id']}", } + def retrieve_context_v2(self, payload): + return BackendRetrieveResult( + backend_type=BackendType.EVEROS, + status=BackendResultStatus.SUCCESS, + items=[ + BackendRetrieveItem( + text="EverOS memory for remember", + source_backend=BackendType.EVEROS, + ref_id="em_ctx_1", + score=0.91, + memory_type="episodic_memory", + ) + ], + ) -class FailingEverMemOSClient: + +class FailingEverOSClient: def ingest_message(self, payload): - raise RuntimeError("evermemos unavailable") + raise RuntimeError("everos unavailable") class FakeCommitOpenVikingClient: @@ -120,7 +214,7 @@ def fake_commit_openviking_factory(result: BackendCommitResult): return factory -class FakeCommitEverMemOSClient: +class FakeCommitEverOSClient: def __init__(self, result: BackendCommitResult) -> None: self.result = result @@ -149,7 +243,7 @@ def commit_result( def test_v2_adapters_return_backend_write_result_contract(): ov_result = asyncio.run( - OpenVikingClient().ingest_conversation_turn( + OpenVikingClient(mode="offline").ingest_conversation_turn( { "workspace_id": "ws_1", "session_id": "sess_1", @@ -157,7 +251,7 @@ def test_v2_adapters_return_backend_write_result_contract(): } ) ) - em_result = EverMemOSClient().ingest_message( + em_result = EverOSClient(mode="offline").ingest_message( { "workspace_id": "ws_1", "session_id": "sess_1", @@ -168,7 +262,7 @@ def test_v2_adapters_return_backend_write_result_contract(): assert isinstance(ov_result, BackendWriteResult) assert isinstance(em_result, BackendWriteResult) assert ov_result.backend_type == BackendType.OPENVIKING - assert em_result.backend_type == BackendType.EVERMEMOS + assert em_result.backend_type == BackendType.EVEROS assert ov_result.operation == BackendOperation.INGEST_TURN assert em_result.operation == BackendOperation.INGEST_TURN assert ov_result.status == BackendResultStatus.SKIPPED @@ -180,10 +274,10 @@ def test_backend_env_overrides_enable_real_modes(monkeypatch, tmp_path): monkeypatch.setenv("OPENVIKING_BASE_URL", "http://openviking.env.test") monkeypatch.setenv("OPENVIKING_API_KEY", "ov-env-token") monkeypatch.setenv("OPENVIKING_TIMEOUT_SECONDS", "17") - monkeypatch.setenv("EVERMEMOS_MODE", "real") - monkeypatch.setenv("EVERMEMOS_BASE_URL", "http://evermemos.env.test") - monkeypatch.setenv("EVERMEMOS_API_KEY", "em-env-token") - monkeypatch.setenv("EVERMEMOS_INGEST_PATH", "/api/v1/memories") + monkeypatch.setenv("EVEROS_MODE", "real") + monkeypatch.setenv("EVEROS_BASE_URL", "http://everos.env.test") + monkeypatch.setenv("EVEROS_API_KEY", "em-env-token") + monkeypatch.setenv("EVEROS_INGEST_PATH", "/api/v1/memories") config = load_config(str(tmp_path / "missing.yaml")) @@ -191,10 +285,10 @@ def test_backend_env_overrides_enable_real_modes(monkeypatch, tmp_path): assert config.openviking.url == "http://openviking.env.test" assert config.openviking.api_key == "ov-env-token" assert config.openviking.timeout == 17 - assert config.evermemos.mode == "real" - assert config.evermemos.url == "http://evermemos.env.test" - assert config.evermemos.api_key == "em-env-token" - assert config.evermemos.ingest_path == "/api/v1/memories" + assert config.everos.mode == "real" + assert config.everos.url == "http://everos.env.test" + assert config.everos.api_key == "em-env-token" + assert config.everos.ingest_path == "/api/v1/memories" def test_openviking_default_ingest_does_not_touch_network(): @@ -202,6 +296,7 @@ def test_openviking_default_ingest_does_not_touch_network(): raise AssertionError("offline OpenViking ingest should not perform HTTP") client = OpenVikingClient( + mode="offline", base_url="http://openviking.test", transport=httpx.MockTransport(handler), ) @@ -263,7 +358,7 @@ def test_openviking_mode_real_with_base_url_uses_mock_http(): def handler(request): calls["count"] += 1 - return httpx.Response(200, json=load_backend_fixture("openviking_ingest_real_success.json")) + return httpx.Response(200, json=backend_response("openviking_ingest_real_success.json")) client = OpenVikingClient( mode="real", @@ -312,7 +407,7 @@ def test_openviking_real_ingest_mode_real_without_base_url_returns_config_error( def test_openviking_real_ingest_success_uses_mock_http_and_normalization(): seen_payload = {} seen_headers = {} - fixture = load_backend_fixture("openviking_ingest_real_success.json") + fixture = backend_response("openviking_ingest_real_success.json") def handler(request): seen_payload.update(json.loads(request.content.decode())) @@ -375,7 +470,7 @@ def test_openviking_real_ingest_http_retryable_and_nonretryable_statuses(): mode="real", base_url="http://openviking.test", api_key="super-secret-token", - transport=httpx.MockTransport(lambda request: httpx.Response(status_code, json=load_backend_fixture(name))), + transport=httpx.MockTransport(lambda request: httpx.Response(status_code, json=backend_response(name))), ) result_429 = asyncio.run(client_for_fixture("openviking_ingest_real_error_500.json", 429).ingest_conversation_turn({"session_id": "sess_http"})) @@ -415,14 +510,14 @@ def test_openviking_real_ingest_invalid_json_returns_failed_retryable(): assert "SECRET_JSON" not in json.dumps(result.model_dump(mode="json"), ensure_ascii=False) -def test_evermemos_default_ingest_does_not_touch_network_even_if_enabled(): +def test_everos_default_ingest_does_not_touch_network_even_if_enabled(): def handler(request): - raise AssertionError("EverMemOS ingest should not perform HTTP unless mode=real") + raise AssertionError("EverOS ingest should not perform HTTP unless mode=real") - client = EverMemOSClient( + client = EverOSClient( enabled=True, mode="offline", - base_url="http://evermemos.test", + base_url="http://everos.test", transport=httpx.MockTransport(handler), ) @@ -431,8 +526,8 @@ def test_evermemos_default_ingest_does_not_touch_network_even_if_enabled(): assert result.status == BackendResultStatus.SKIPPED -def test_evermemos_real_ingest_mode_real_without_base_url_returns_config_error(): - client = EverMemOSClient(mode="real", base_url="") +def test_everos_real_ingest_mode_real_without_base_url_returns_config_error(): + client = EverOSClient(mode="real", base_url="") result = client.ingest_message({"session_id": "sess_missing_url", "content": "SECRET"}) @@ -442,19 +537,19 @@ def test_evermemos_real_ingest_mode_real_without_base_url_returns_config_error() assert "SECRET" not in json.dumps(result.model_dump(mode="json"), ensure_ascii=False) -def test_evermemos_real_ingest_success_uses_mock_http_and_normalization(): +def test_everos_real_ingest_success_uses_mock_http_and_normalization(): seen_payload = {} seen_headers = {} - fixture = load_backend_fixture("evermemos_ingest_success.json") + fixture = backend_response("everos_ingest_success.json") def handler(request): seen_payload.update(json.loads(request.content.decode())) seen_headers.update(dict(request.headers)) return httpx.Response(200, json=fixture) - client = EverMemOSClient( + client = EverOSClient( mode="real", - base_url="http://evermemos.test", + base_url="http://everos.test", api_key="em-token", transport=httpx.MockTransport(handler), ) @@ -472,9 +567,9 @@ def test_evermemos_real_ingest_success_uses_mock_http_and_normalization(): "metadata": {"channel": "test"}, } ) - expected = normalize_evermemos_ingest_response(fixture) + expected = normalize_everos_ingest_response(fixture) - assert seen_payload["content"] == "SECRET_EM_CONTENT" + assert seen_payload["messages"][0]["content"] == "SECRET_EM_CONTENT" assert seen_headers["x-api-key"] == "em-token" assert seen_headers["authorization"] == "Bearer em-token" assert result == expected @@ -484,11 +579,11 @@ def test_evermemos_real_ingest_success_uses_mock_http_and_normalization(): assert "em-token" not in serialized -def test_evermemos_real_ingest_errors_are_backend_write_results_and_safe(): +def test_everos_real_ingest_errors_are_backend_write_results_and_safe(): def client_for_response(status_code, body=None, content=None): - return EverMemOSClient( + return EverOSClient( mode="real", - base_url="http://evermemos.test", + base_url="http://everos.test", api_key="em-super-secret-token", transport=httpx.MockTransport(lambda request: httpx.Response(status_code, json=body, content=content)), ) @@ -517,13 +612,13 @@ def test_evermemos_real_ingest_errors_are_backend_write_results_and_safe(): assert "em-super-secret-token" not in serialized -def test_evermemos_real_ingest_timeout_is_retryable_and_safe(): +def test_everos_real_ingest_timeout_is_retryable_and_safe(): def handler(request): raise httpx.ReadTimeout("timeout while sending SECRET_TIMEOUT_CONTENT") - client = EverMemOSClient( + client = EverOSClient( mode="real", - base_url="http://evermemos.test", + base_url="http://everos.test", transport=httpx.MockTransport(handler), ) @@ -540,9 +635,9 @@ def test_backend_adapter_mapping_spec_is_contract_first_and_control_plane_only() (BackendType.OPENVIKING, BackendOperation.INGEST_TURN), (BackendType.OPENVIKING, BackendOperation.COMMIT_SESSION), (BackendType.OPENVIKING, BackendOperation.RETRIEVE_CONTEXT), - (BackendType.EVERMEMOS, BackendOperation.INGEST_TURN), - (BackendType.EVERMEMOS, BackendOperation.COMMIT_SESSION), - (BackendType.EVERMEMOS, BackendOperation.RETRIEVE_CONTEXT), + (BackendType.EVEROS, BackendOperation.INGEST_TURN), + (BackendType.EVEROS, BackendOperation.COMMIT_SESSION), + (BackendType.EVEROS, BackendOperation.RETRIEVE_CONTEXT), (BackendType.OBSIDIAN, BackendOperation.CREATE_REVIEW_DRAFT), } @@ -551,12 +646,12 @@ def test_backend_adapter_mapping_spec_is_contract_first_and_control_plane_only() assert not DISALLOWED_PAYLOAD_FIELDS.intersection(spec.allowed_payload_fields) openviking_commit = get_adapter_mapping_spec(BackendType.OPENVIKING, BackendOperation.COMMIT_SESSION) - evermemos_ingest = get_adapter_mapping_spec(BackendType.EVERMEMOS, BackendOperation.INGEST_TURN) + everos_ingest = get_adapter_mapping_spec(BackendType.EVEROS, BackendOperation.INGEST_TURN) assert openviking_commit.adapter_method == "commit_session_v2" assert openviking_commit.result_model is BackendCommitResult - assert evermemos_ingest.adapter_method == "ingest_message" - assert evermemos_ingest.result_model is BackendWriteResult + assert everos_ingest.adapter_method == "ingest_message" + assert everos_ingest.result_model is BackendWriteResult def test_control_plane_persisted_payload_validator_rejects_content_and_raw_request(): @@ -590,26 +685,18 @@ def test_runtime_adapter_request_may_be_transient_but_outbox_payload_is_control_ validate_control_plane_persisted_payload(outbox_payload) -def test_commit_and_retrieve_adapter_skeletons_return_unified_contracts(): +def test_commit_adapter_skeletons_return_unified_contracts(): payload = {"workspace_id": "ws_1", "session_id": "sess_1", "gateway_id": "gw_1"} - ov_commit = asyncio.run(OpenVikingClient().commit_session_v2(payload)) - ov_retrieve = asyncio.run(OpenVikingClient().retrieve_context_v2(payload)) - em_commit = EverMemOSClient().extract_profile_long_term_v2(payload) - em_retrieve = EverMemOSClient().retrieve_context_v2(payload) + ov_commit = asyncio.run(OpenVikingClient(mode="skeleton").commit_session_v2(payload)) + em_commit = EverOSClient(mode="skeleton").extract_profile_long_term_v2(payload) assert isinstance(ov_commit, BackendCommitResult) assert isinstance(em_commit, BackendCommitResult) - assert isinstance(ov_retrieve, BackendRetrieveResult) - assert isinstance(em_retrieve, BackendRetrieveResult) assert ov_commit.status == BackendResultStatus.SUCCESS assert em_commit.status == BackendResultStatus.SUCCESS - assert ov_retrieve.status == BackendResultStatus.SUCCESS - assert em_retrieve.status == BackendResultStatus.SUCCESS assert ov_commit.refs[0].ref_type == MemoryRefType.SESSION_ARCHIVE assert {ref.ref_type for ref in em_commit.refs} == {MemoryRefType.PROFILE, MemoryRefType.LONG_TERM_MEMORY} - assert len(ov_retrieve.items) == 1 - assert len(em_retrieve.items) == 2 def test_client_skeletons_use_normalization_contracts_and_safe_metadata(): @@ -621,8 +708,8 @@ def test_client_skeletons_use_normalization_contracts_and_safe_metadata(): "content": "TRANSIENT_CONTENT_ONLY", "raw_request": {"content": "TRANSIENT_CONTENT_ONLY"}, } - ov_client = OpenVikingClient() - em_client = EverMemOSClient() + ov_client = OpenVikingClient(mode="skeleton") + em_client = EverOSClient(mode="skeleton") ov_ingest = asyncio.run(ov_client.ingest_conversation_turn(payload)) ov_commit = asyncio.run(ov_client.commit_session_v2(payload)) @@ -649,8 +736,8 @@ def test_client_skeletons_use_normalization_contracts_and_safe_metadata(): "status": "skipped", "memory_id": "turn_contract", "metadata": { - "reason": "evermemos_v2_ingest_adapter_not_configured", - "schema_version": "evermemos.fixture.ingest.v2", + "reason": "everos_v2_ingest_adapter_not_configured", + "schema_version": "everos.fixture.ingest.v2", }, } ) @@ -667,57 +754,30 @@ def test_client_skeletons_use_normalization_contracts_and_safe_metadata(): assert blocked not in serialized -def test_retrieve_skeletons_use_retrieve_normalization_and_safe_metadata(): - payload = { - "workspace_id": "ws_1", - "user_id": "user_a", - "session_id": "sess_retrieve_contract", - "query": "fixture query", - "content": "TRANSIENT_RETRIEVE_CONTENT", - } - ov_result = asyncio.run(OpenVikingClient().retrieve_context_v2(payload)) - em_result = EverMemOSClient().retrieve_context_v2(payload) - - assert isinstance(ov_result, BackendRetrieveResult) - assert isinstance(em_result, BackendRetrieveResult) - assert ov_result.status == BackendResultStatus.SUCCESS - assert em_result.status == BackendResultStatus.SUCCESS - assert ov_result.items[0].source_backend == BackendType.OPENVIKING - assert em_result.items[0].source_backend == BackendType.EVERMEMOS - assert ov_result.items[0].text - assert em_result.items[0].ref_id - serialized = json.dumps( - {"ov": ov_result.model_dump(mode="json"), "em": em_result.model_dump(mode="json")}, - ensure_ascii=False, - ) - for blocked in ("TRANSIENT_RETRIEVE_CONTENT", "content", "raw_request", "messages", "conversation", "transcript"): - assert blocked not in serialized - - def test_openviking_commit_skeleton_ref_type_is_mapped_from_fixture(): - result = asyncio.run(OpenVikingClient().commit_session_v2({"session_id": "sess_ov_map"})) + result = asyncio.run(OpenVikingClient(mode="skeleton").commit_session_v2({"session_id": "sess_ov_map"})) assert result.refs assert result.refs[0].ref_type == MemoryRefType.SESSION_ARCHIVE assert result.refs[0].native_id == "ov_session_summary:sess_ov_map" -def test_evermemos_skeleton_multiple_refs_are_written_by_process_outbox_event(): +def test_everos_skeleton_multiple_refs_are_written_by_process_outbox_event(): repo = InMemoryRepository() service = MemoryGatewayV2Service( repo=repo, openviking_client_factory=fake_commit_openviking_factory( commit_result(BackendType.OPENVIKING, BackendResultStatus.SKIPPED) ), - evermemos_client=EverMemOSClient(), + everos_client=EverOSClient(), ) response = asyncio.run( service.commit_session("sess_em_skeleton", CommitRequest(workspace_id="ws_1", user_id="user_a", agent_id="agent_cli")) ) - event = next(event for event in repo.list_outbox_events_by_job(response.job_id) if event.backend_type == BackendType.EVERMEMOS) + event = next(event for event in repo.list_outbox_events_by_job(response.job_id) if event.backend_type == BackendType.EVEROS) updated = asyncio.run(service.process_outbox_event(event.id)) - refs = repo.list_memory_refs(session_id="sess_em_skeleton", backend_type=BackendType.EVERMEMOS, status=BackendRefStatus.SUCCESS) + refs = repo.list_memory_refs(session_id="sess_em_skeleton", backend_type=BackendType.EVEROS, status=BackendRefStatus.SUCCESS) assert updated.status == OutboxEventStatus.SUCCESS assert len(refs) == 2 @@ -735,18 +795,18 @@ def test_obsidian_review_adapter_skeleton_returns_skipped_write_result(): def test_backend_commit_result_supports_multiple_produced_refs(): result = BackendCommitResult( - backend_type=BackendType.EVERMEMOS, + backend_type=BackendType.EVEROS, status=BackendResultStatus.SUCCESS, refs=[ BackendProducedRef(ref_type=MemoryRefType.PROFILE, native_id="profile_1"), - BackendProducedRef(ref_type=MemoryRefType.LONG_TERM_MEMORY, native_uri="evermemos://memories/long_1"), + BackendProducedRef(ref_type=MemoryRefType.LONG_TERM_MEMORY, native_uri="everos://memories/long_1"), ], ) dumped = result.model_dump(mode="json") assert len(result.refs) == 2 assert dumped["refs"][0]["ref_type"] == "profile" - assert dumped["refs"][1]["native_uri"] == "evermemos://memories/long_1" + assert dumped["refs"][1]["native_uri"] == "everos://memories/long_1" def test_backend_ref_type_mapping_and_unknown_fallback_preserves_original_type(): @@ -758,11 +818,11 @@ def test_backend_ref_type_mapping_and_unknown_fallback_preserves_original_type() assert mapped == MemoryRefType.SESSION_ARCHIVE assert metadata == {} - mapped, metadata = map_backend_ref_type(BackendType.EVERMEMOS, "preference") + mapped, metadata = map_backend_ref_type(BackendType.EVEROS, "preference") assert mapped == MemoryRefType.PROFILE assert metadata == {} - mapped, metadata = map_backend_ref_type(BackendType.EVERMEMOS, "unknown_signal") + mapped, metadata = map_backend_ref_type(BackendType.EVEROS, "unknown_signal") assert mapped == MemoryRefType.LONG_TERM_MEMORY assert metadata["original_ref_type"] == "unknown_signal" @@ -798,29 +858,10 @@ def test_openviking_commit_fixture_normalizes_to_backend_commit_result_without_u assert "messages" not in serialized -def test_backend_response_fixture_files_exist_and_load(): - names = { - "openviking_ingest_success.json", - "openviking_ingest_real_success.json", - "openviking_ingest_real_error_401.json", - "openviking_ingest_real_error_422.json", - "openviking_ingest_real_error_500.json", - "openviking_commit_success.json", - "openviking_retrieve_success.json", - "evermemos_ingest_success.json", - "evermemos_commit_success_multiple_refs.json", - "evermemos_retrieve_success.json", - } - - for name in names: - payload = load_backend_fixture(name) - assert payload["status"] - - def test_openviking_success_fixtures_normalize_without_unsafe_metadata(): - ingest = normalize_openviking_ingest_response(load_backend_fixture("openviking_ingest_success.json")) - commit = normalize_openviking_commit_response(load_backend_fixture("openviking_commit_success.json")) - retrieve = normalize_openviking_retrieve_response(load_backend_fixture("openviking_retrieve_success.json")) + ingest = normalize_openviking_ingest_response(backend_response("openviking_ingest_success.json")) + commit = normalize_openviking_commit_response(backend_response("openviking_commit_success.json")) + retrieve = normalize_openviking_retrieve_response(backend_response("openviking_retrieve_success.json")) assert ingest.status == BackendResultStatus.SUCCESS assert ingest.native_id == "ov_turn_fixture_1" @@ -841,7 +882,7 @@ def test_openviking_success_fixtures_normalize_without_unsafe_metadata(): assert blocked not in serialized -def test_evermemos_commit_fixture_normalizes_multiple_produced_refs_and_unknown_type(): +def test_everos_commit_fixture_normalizes_multiple_produced_refs_and_unknown_type(): raw = { "status": "success", "data": { @@ -853,7 +894,7 @@ def test_evermemos_commit_fixture_normalizes_multiple_produced_refs_and_unknown_ }, } - result = normalize_evermemos_commit_response(raw) + result = normalize_everos_commit_response(raw) assert result.status == BackendResultStatus.SUCCESS assert len(result.refs) == 3 @@ -866,10 +907,10 @@ def test_evermemos_commit_fixture_normalizes_multiple_produced_refs_and_unknown_ assert "SECRET_PROFILE" not in json.dumps(result.model_dump(mode="json"), ensure_ascii=False) -def test_evermemos_success_fixtures_normalize_without_unsafe_metadata(): - ingest = normalize_evermemos_ingest_response(load_backend_fixture("evermemos_ingest_success.json")) - commit = normalize_evermemos_commit_response(load_backend_fixture("evermemos_commit_success_multiple_refs.json")) - retrieve = normalize_evermemos_retrieve_response(load_backend_fixture("evermemos_retrieve_success.json")) +def test_everos_success_fixtures_normalize_without_unsafe_metadata(): + ingest = normalize_everos_ingest_response(backend_response("everos_ingest_success.json")) + commit = normalize_everos_commit_response(backend_response("everos_commit_success_multiple_refs.json")) + retrieve = normalize_everos_retrieve_response(backend_response("everos_retrieve_success.json")) assert ingest.status == BackendResultStatus.SUCCESS assert ingest.native_id == "em_memory_fixture_1" @@ -881,7 +922,7 @@ def test_evermemos_success_fixtures_normalize_without_unsafe_metadata(): } assert retrieve.status == BackendResultStatus.SUCCESS assert len(retrieve.items) == 2 - assert retrieve.items[0].source_backend == BackendType.EVERMEMOS + assert retrieve.items[0].source_backend == BackendType.EVEROS assert retrieve.items[0].memory_type == "episodic_memory" serialized = json.dumps( { @@ -897,7 +938,7 @@ def test_evermemos_success_fixtures_normalize_without_unsafe_metadata(): def test_malformed_retrieve_response_returns_skipped_empty_result(): ov = normalize_openviking_retrieve_response({}) - em = normalize_evermemos_retrieve_response({"data": {"unexpected": "shape"}}) + em = normalize_everos_retrieve_response({"data": {"unexpected": "shape"}}) assert ov.status == BackendResultStatus.SKIPPED assert ov.items == [] @@ -914,7 +955,7 @@ def test_ingest_response_normalizers_return_write_results_and_sanitize_metadata( "metadata": {"backend_request_id": "ov_req", "conversation": "SECRET"}, } ) - em = normalize_evermemos_ingest_response( + em = normalize_everos_ingest_response( { "status": "success", "memory_id": "em_turn_1", @@ -935,12 +976,12 @@ def test_ingest_response_normalizers_return_write_results_and_sanitize_metadata( def test_backend_error_retryable_mapping(): for status_code in (429, 500, 502, 503, 504): assert map_backend_error_to_retryable(BackendType.OPENVIKING, status_code=status_code) is True - assert map_backend_error_to_retryable(BackendType.EVERMEMOS, error_code="timeout") is True - assert map_backend_error_to_retryable(BackendType.EVERMEMOS, error_message="network_error: reset") is True + assert map_backend_error_to_retryable(BackendType.EVEROS, error_code="timeout") is True + assert map_backend_error_to_retryable(BackendType.EVEROS, error_message="network_error: reset") is True assert map_backend_error_to_retryable(BackendType.OPENVIKING, error_code="mystery") is True for status_code in (400, 401, 403, 404, 422): - assert map_backend_error_to_retryable(BackendType.EVERMEMOS, status_code=status_code) is False + assert map_backend_error_to_retryable(BackendType.EVEROS, status_code=status_code) is False def test_client_map_error_contracts_for_future_http_integration(): @@ -951,8 +992,8 @@ def test_client_map_error_contracts_for_future_http_integration(): def __str__(self): return f"response {self.status_code}" - ov_client = OpenVikingClient() - em_client = EverMemOSClient() + ov_client = OpenVikingClient(mode="skeleton") + em_client = EverOSClient(mode="skeleton") for status_code in (429, 500, 502, 503, 504): assert ov_client._map_error(ResponseLike(status_code)) is True @@ -979,20 +1020,20 @@ def test_ingest_service_records_two_success_refs(): service = MemoryGatewayV2Service( repo=repo, openviking_client_factory=fake_openviking_factory, - evermemos_client=FakeEverMemOSClient(), + everos_client=FakeEverOSClient(), ) response = asyncio.run(service.ingest_conversation_turn(IngestRequest(**build_ingest_payload()))) assert response.status == "success" assert len(response.refs) == 2 - assert {ref.backend_type.value for ref in response.refs} == {"openviking", "evermemos"} + assert {ref.backend_type.value for ref in response.refs} == {"openviking", "everos"} assert {ref.status for ref in repo.list_memory_refs()} == {BackendRefStatus.SUCCESS} assert len(repo.list_memory_refs(backend_type="openviking", status=BackendRefStatus.SUCCESS)) == 1 def test_v2_ingest_service_openviking_real_mock_success_writes_safe_memory_ref(): - fixture = load_backend_fixture("openviking_ingest_real_success.json") + fixture = backend_response("openviking_ingest_real_success.json") def handler(request): payload = json.loads(request.content.decode()) @@ -1012,7 +1053,7 @@ def test_v2_ingest_service_openviking_real_mock_success_writes_safe_memory_ref() service = MemoryGatewayV2Service( repo=repo, openviking_client_factory=real_openviking_factory, - evermemos_client=FakeEverMemOSClient(), + everos_client=FakeEverOSClient(), ) response = asyncio.run( @@ -1034,10 +1075,10 @@ def test_v2_ingest_service_openviking_real_mock_success_writes_safe_memory_ref() assert "ov-super-secret-token" not in audit_json -def test_v2_ingest_service_real_mock_success_writes_openviking_and_evermemos_refs_safely(): - ov_fixture = load_backend_fixture("openviking_ingest_real_success.json") - em_fixture = load_backend_fixture("evermemos_ingest_success.json") - seen = {"openviking": 0, "evermemos": 0} +def test_v2_ingest_service_real_mock_success_writes_openviking_and_everos_refs_safely(): + ov_fixture = backend_response("openviking_ingest_real_success.json") + em_fixture = backend_response("everos_ingest_success.json") + seen = {"openviking": 0, "everos": 0} def openviking_handler(request): payload = json.loads(request.content.decode()) @@ -1046,12 +1087,12 @@ def test_v2_ingest_service_real_mock_success_writes_openviking_and_evermemos_ref seen["openviking"] += 1 return httpx.Response(200, json=ov_fixture) - def evermemos_handler(request): + def everos_handler(request): payload = json.loads(request.content.decode()) - assert payload["content"] == "SECRET_DUAL_REAL_CONTENT" + assert payload["messages"][0]["content"] == "SECRET_DUAL_REAL_CONTENT" assert request.headers["x-api-key"] == "em-dual-token" assert request.headers["authorization"] == "Bearer em-dual-token" - seen["evermemos"] += 1 + seen["everos"] += 1 return httpx.Response(200, json=em_fixture) async def real_openviking_factory(): @@ -1066,11 +1107,11 @@ def test_v2_ingest_service_real_mock_success_writes_openviking_and_evermemos_ref service = MemoryGatewayV2Service( repo=repo, openviking_client_factory=real_openviking_factory, - evermemos_client=EverMemOSClient( + everos_client=EverOSClient( mode="real", - base_url="http://evermemos.test", + base_url="http://everos.test", api_key="em-dual-token", - transport=httpx.MockTransport(evermemos_handler), + transport=httpx.MockTransport(everos_handler), ), ) @@ -1092,8 +1133,8 @@ def test_v2_ingest_service_real_mock_success_writes_openviking_and_evermemos_ref audit_json = json.dumps([entry.model_dump(mode="json") for entry in repo.list_audit()], ensure_ascii=False) assert response.status == OperationStatus.SUCCESS - assert seen == {"openviking": 1, "evermemos": 1} - assert {ref.backend_type for ref in refs} == {BackendType.OPENVIKING, BackendType.EVERMEMOS} + assert seen == {"openviking": 1, "everos": 1} + assert {ref.backend_type for ref in refs} == {BackendType.OPENVIKING, BackendType.EVEROS} assert {ref.status for ref in refs} == {BackendRefStatus.SUCCESS} assert {ref.content_hash for ref in refs} assert "trace_dual_real" in serialized_refs @@ -1108,7 +1149,7 @@ def test_ingest_service_backend_failure_is_partial_success(): service = MemoryGatewayV2Service( repo=repo, openviking_client_factory=fake_openviking_factory, - evermemos_client=FailingEverMemOSClient(), + everos_client=FailingEverOSClient(), ) response = asyncio.run(service.ingest_conversation_turn(IngestRequest(**build_ingest_payload()))) @@ -1117,8 +1158,8 @@ def test_ingest_service_backend_failure_is_partial_success(): assert len(response.refs) == 2 failed = [ref for ref in response.refs if ref.status == BackendRefStatus.FAILED] assert len(failed) == 1 - assert failed[0].backend_type.value == "evermemos" - assert "evermemos unavailable" in failed[0].error_message + assert failed[0].backend_type.value == "everos" + assert "everos unavailable" in failed[0].error_message def test_ingest_service_records_two_skipped_refs_when_policy_disables_backends(): @@ -1126,7 +1167,7 @@ def test_ingest_service_records_two_skipped_refs_when_policy_disables_backends() service = MemoryGatewayV2Service( repo=repo, openviking_client_factory=fake_openviking_factory, - evermemos_client=FakeEverMemOSClient(), + everos_client=FakeEverOSClient(), ) response = asyncio.run( @@ -1135,7 +1176,7 @@ def test_ingest_service_records_two_skipped_refs_when_policy_disables_backends() **build_ingest_payload( policy={ "allow_openviking": False, - "allow_evermemos": False, + "allow_everos": False, } ) ) @@ -1153,7 +1194,7 @@ def test_duplicate_idempotency_key_upserts_memory_refs_without_duplicates(): service = MemoryGatewayV2Service( repo=repo, openviking_client_factory=fake_openviking_factory, - evermemos_client=FakeEverMemOSClient(), + everos_client=FakeEverOSClient(), ) first = asyncio.run( @@ -1185,7 +1226,7 @@ def test_memory_ref_metadata_does_not_store_conversation_content_or_raw_request( service = MemoryGatewayV2Service( repo=repo, openviking_client_factory=fake_openviking_factory, - evermemos_client=FakeEverMemOSClient(), + everos_client=FakeEverOSClient(), ) sensitive_content = "SECRET_CONVERSATION_CONTENT_SHOULD_NOT_BE_STORED" @@ -1213,7 +1254,7 @@ def test_sqlite_repository_persists_v2_memory_refs(tmp_path): service = MemoryGatewayV2Service( repo=repo, openviking_client_factory=fake_openviking_factory, - evermemos_client=FakeEverMemOSClient(), + everos_client=FakeEverOSClient(), ) asyncio.run(service.ingest_conversation_turn(IngestRequest(**build_ingest_payload(turn_id="turn_sqlite")))) @@ -1253,7 +1294,7 @@ def test_commit_session_creates_commit_job_and_outbox_events(): assert job.session_id == "sess_commit" assert job.status.value == "accepted" assert len(events) == 2 - assert {event.backend_type for event in events} == {BackendType.OPENVIKING, BackendType.EVERMEMOS} + assert {event.backend_type for event in events} == {BackendType.OPENVIKING, BackendType.EVEROS} assert {event.operation for event in events} == {BackendOperation.COMMIT_SESSION} assert {event.status for event in events} == {OutboxEventStatus.PENDING} @@ -1336,7 +1377,7 @@ def test_retrieve_response_contract_contains_items_refs_conflicts_trace_id_statu service = MemoryGatewayV2Service( repo=repo, openviking_client_factory=fake_openviking_factory, - evermemos_client=FakeEverMemOSClient(), + everos_client=FakeEverOSClient(), ) asyncio.run(service.ingest_conversation_turn(IngestRequest(**build_ingest_payload()))) @@ -1357,8 +1398,14 @@ def test_retrieve_response_contract_contains_items_refs_conflicts_trace_id_statu assert set(["items", "refs", "conflicts", "trace_id", "status"]).issubset(dumped) assert response.trace_id == "trace_1" assert response.status.value == "success" - assert len(response.items) == len(response.refs) + assert [item.source_backend for item in response.items] == [BackendType.EVEROS, BackendType.OPENVIKING] + assert [item.score for item in response.items] == [0.91, 0.82] + assert len(response.refs) == 2 assert response.conflicts == [] + assert response.metadata["backend_results"] == [ + {"backend_type": "openviking", "status": "success", "items": 1, "error_code": None, "error_message": None}, + {"backend_type": "everos", "status": "success", "items": 1, "error_code": None, "error_message": None}, + ] def test_process_commit_job_success_updates_job_and_writes_memory_refs(): @@ -1368,8 +1415,8 @@ def test_process_commit_job_success_updates_job_and_writes_memory_refs(): openviking_client_factory=fake_commit_openviking_factory( commit_result(BackendType.OPENVIKING, BackendResultStatus.SUCCESS, native_id="ov_commit_1", native_uri="viking://sessions/sess_commit") ), - evermemos_client=FakeCommitEverMemOSClient( - commit_result(BackendType.EVERMEMOS, BackendResultStatus.SUCCESS, native_id="em_commit_1", native_uri="evermemos://memories/em_commit_1") + everos_client=FakeCommitEverOSClient( + commit_result(BackendType.EVEROS, BackendResultStatus.SUCCESS, native_id="em_commit_1", native_uri="everos://memories/em_commit_1") ), ) response = asyncio.run( @@ -1389,7 +1436,7 @@ def test_process_commit_job_success_updates_job_and_writes_memory_refs(): assert job.created_refs_count == 2 assert {event.status for event in events} == {OutboxEventStatus.SUCCESS} assert len(refs) == 2 - assert {ref.backend_type for ref in refs} == {BackendType.OPENVIKING, BackendType.EVERMEMOS} + assert {ref.backend_type for ref in refs} == {BackendType.OPENVIKING, BackendType.EVEROS} def test_process_outbox_event_writes_multiple_produced_memory_refs(): @@ -1529,8 +1576,8 @@ def test_process_commit_job_one_success_one_failed_is_partial_success(): openviking_client_factory=fake_commit_openviking_factory( commit_result(BackendType.OPENVIKING, BackendResultStatus.SUCCESS, native_id="ov_commit_1") ), - evermemos_client=FakeCommitEverMemOSClient( - commit_result(BackendType.EVERMEMOS, BackendResultStatus.FAILED, retryable=False, error_message="evermemos failed") + everos_client=FakeCommitEverOSClient( + commit_result(BackendType.EVEROS, BackendResultStatus.FAILED, retryable=False, error_message="everos failed") ), ) response = asyncio.run( @@ -1542,7 +1589,7 @@ def test_process_commit_job_one_success_one_failed_is_partial_success(): assert job.status.value == "partial_success" assert job.created_refs_count == 1 - assert "evermemos failed" in job.error_message + assert "everos failed" in job.error_message assert {event.status for event in events} == {OutboxEventStatus.SUCCESS, OutboxEventStatus.DEAD_LETTER} @@ -1553,8 +1600,8 @@ def test_process_commit_job_two_failed_is_failed(): openviking_client_factory=fake_commit_openviking_factory( commit_result(BackendType.OPENVIKING, BackendResultStatus.FAILED, retryable=False, error_message="openviking failed") ), - evermemos_client=FakeCommitEverMemOSClient( - commit_result(BackendType.EVERMEMOS, BackendResultStatus.FAILED, retryable=False, error_message="evermemos failed") + everos_client=FakeCommitEverOSClient( + commit_result(BackendType.EVEROS, BackendResultStatus.FAILED, retryable=False, error_message="everos failed") ), ) response = asyncio.run( @@ -1566,7 +1613,7 @@ def test_process_commit_job_two_failed_is_failed(): assert job.status.value == "failed" assert job.created_refs_count == 0 assert "openviking failed" in job.error_message - assert "evermemos failed" in job.error_message + assert "everos failed" in job.error_message def test_retryable_failed_outbox_event_requeues_with_next_retry(): @@ -1597,8 +1644,8 @@ def test_process_pending_outbox_events_processes_pending_batch(): openviking_client_factory=fake_commit_openviking_factory( commit_result(BackendType.OPENVIKING, BackendResultStatus.SUCCESS, native_id="ov_commit_1") ), - evermemos_client=FakeCommitEverMemOSClient( - commit_result(BackendType.EVERMEMOS, BackendResultStatus.SUCCESS, native_id="em_commit_1") + everos_client=FakeCommitEverOSClient( + commit_result(BackendType.EVEROS, BackendResultStatus.SUCCESS, native_id="em_commit_1") ), ) asyncio.run( @@ -1641,8 +1688,8 @@ def test_commit_pipeline_metadata_does_not_store_content_or_raw_request(): openviking_client_factory=fake_commit_openviking_factory( commit_result(BackendType.OPENVIKING, BackendResultStatus.SUCCESS, native_id="ov_commit_1") ), - evermemos_client=FakeCommitEverMemOSClient( - commit_result(BackendType.EVERMEMOS, BackendResultStatus.SUCCESS, native_id="em_commit_1") + everos_client=FakeCommitEverOSClient( + commit_result(BackendType.EVEROS, BackendResultStatus.SUCCESS, native_id="em_commit_1") ), ) sensitive_content = "SECRET_COMMIT_PIPELINE_CONTENT_SHOULD_NOT_BE_STORED" @@ -1728,8 +1775,8 @@ def test_process_pending_outbox_events_uses_claim_and_does_not_process_existing_ openviking_client_factory=fake_commit_openviking_factory( commit_result(BackendType.OPENVIKING, BackendResultStatus.SUCCESS, native_id="ov_claimed") ), - evermemos_client=FakeCommitEverMemOSClient( - commit_result(BackendType.EVERMEMOS, BackendResultStatus.SUCCESS, native_id="em_claimed") + everos_client=FakeCommitEverOSClient( + commit_result(BackendType.EVEROS, BackendResultStatus.SUCCESS, native_id="em_claimed") ), ) response = asyncio.run( @@ -1754,8 +1801,8 @@ def test_terminal_outbox_statuses_clear_lock_fields(): openviking_client_factory=fake_commit_openviking_factory( commit_result(BackendType.OPENVIKING, BackendResultStatus.SUCCESS, native_id="ov_lock_clear") ), - evermemos_client=FakeCommitEverMemOSClient( - commit_result(BackendType.EVERMEMOS, BackendResultStatus.SKIPPED) + everos_client=FakeCommitEverOSClient( + commit_result(BackendType.EVEROS, BackendResultStatus.SKIPPED) ), ) response = asyncio.run( @@ -1848,8 +1895,8 @@ def test_admin_process_outbox_endpoint_triggers_pending_processing(monkeypatch): openviking_client_factory=fake_commit_openviking_factory( commit_result(BackendType.OPENVIKING, BackendResultStatus.SUCCESS, native_id="ov_admin") ), - evermemos_client=FakeCommitEverMemOSClient( - commit_result(BackendType.EVERMEMOS, BackendResultStatus.SUCCESS, native_id="em_admin") + everos_client=FakeCommitEverOSClient( + commit_result(BackendType.EVEROS, BackendResultStatus.SUCCESS, native_id="em_admin") ), ) asyncio.run( @@ -1907,7 +1954,7 @@ def test_v2_ingest_router_accepts_legal_request(monkeypatch): api_v2.v2_service = MemoryGatewayV2Service( repo=InMemoryRepository(), openviking_client_factory=fake_openviking_factory, - evermemos_client=FakeEverMemOSClient(), + everos_client=FakeEverOSClient(), ) app = FastAPI() app.dependency_overrides[verify_api_key_compat] = lambda: None