Replace EverMemOS with EverOS backend
This commit is contained in:
@ -19,18 +19,19 @@ openviking:
|
||||
api_key: ""
|
||||
timeout: 30
|
||||
|
||||
# EverMemOS 后台长期记忆整理服务
|
||||
evermemos:
|
||||
# EverOS / EverCore 后台长期记忆整理服务
|
||||
everos:
|
||||
enabled: true
|
||||
# 可以是本机 memory_gateway.evermemos_service,也可以是远程 EverMemOS 服务。
|
||||
mode: "real"
|
||||
# 指向 /home/tom/EverOS/methods/EverCore 启动的 API。
|
||||
url: "http://127.0.0.1:1995"
|
||||
api_key: ""
|
||||
timeout: 30
|
||||
health_path: "/health"
|
||||
# 如果远端服务实际 endpoint 不同,改这里即可,不需要改代码。
|
||||
consolidate_path: "/v1/sessions/consolidate"
|
||||
# POC 默认允许远端不可用时用本地确定性 worker 降级,方便开发测试。
|
||||
fallback_to_local: true
|
||||
ingest_path: "/api/v1/memories"
|
||||
search_path: "/api/v1/memories/search"
|
||||
flush_path: "/api/v1/memories/flush"
|
||||
retrieve_method: "keyword"
|
||||
|
||||
# 记忆配置
|
||||
memory:
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
# 通用 Memory Gateway 方案与 POC 骨架
|
||||
|
||||
本文基于当前仓库的轻量 FastAPI + MCP + OpenViking + Obsidian 能力扩展,不把系统设计成重平台。第一阶段目标是先跑通多用户隔离、namespace routing、记忆检索、写入、session commit 和人工 review 草稿,后续再替换持久化、向量索引和 EverMemOS worker。
|
||||
本文基于当前仓库的轻量 FastAPI + MCP + OpenViking + Obsidian 能力扩展,不把系统设计成重平台。第一阶段目标是先跑通多用户隔离、namespace routing、记忆检索、写入、session commit 和人工 review 草稿,后续再替换持久化、向量索引和 EverOS worker。
|
||||
|
||||
## A. 总体架构图
|
||||
|
||||
@ -43,7 +43,7 @@ flowchart TB
|
||||
OVWorkspace[workspace]
|
||||
end
|
||||
|
||||
subgraph EverMemOS["EverMemOS"]
|
||||
subgraph EverOS["EverOS"]
|
||||
LTE[long-term extraction]
|
||||
Consolidation[consolidation]
|
||||
Decay[decay]
|
||||
@ -79,7 +79,7 @@ flowchart TB
|
||||
Retrieval --> Skills
|
||||
Writeback --> Skills
|
||||
Skills --> OpenViking
|
||||
Skills --> EverMemOS
|
||||
Skills --> EverOS
|
||||
Skills --> Obsidian
|
||||
|
||||
Gateway --> DB
|
||||
@ -88,8 +88,8 @@ flowchart TB
|
||||
OpenViking --> DB
|
||||
OpenViking --> Vector
|
||||
Obsidian --> Files
|
||||
EverMemOS --> DB
|
||||
EverMemOS --> Vector
|
||||
EverOS --> DB
|
||||
EverOS --> Vector
|
||||
```
|
||||
|
||||
## B. 核心数据模型
|
||||
@ -510,13 +510,13 @@ Response:
|
||||
| Skill | 功能 | 输入 | 输出 | 触发时机 | 组件 | 写长期记忆 |
|
||||
|---|---|---|---|---|---|---|
|
||||
| `ingest_skill` | 标准化对话、文件、任务事件 | raw text/file/events | normalized payload | agent 写入 episode 前 | Gateway, file storage | 否 |
|
||||
| `extract_memory_skill` | 从 episode/session 抽取候选记忆 | episode/session content | memory candidates | session commit / worker 定时 | LLM, EverMemOS | 否 |
|
||||
| `extract_memory_skill` | 从 episode/session 抽取候选记忆 | episode/session content | memory candidates | session commit / worker 定时 | LLM, EverOS | 否 |
|
||||
| `classify_memory_skill` | 判断 memory_type、visibility、namespace | candidate memory | classification | 写入前 | ACL, namespace router | 否 |
|
||||
| `retrieve_context_skill` | 聚合用户、agent、workspace 上下文 | query + context ids | ranked contexts | agent 调用前 | OpenViking, vector index | 否 |
|
||||
| `commit_memory_skill` | 写入长期记忆 | MemoryRecord | stored record | 人工确认或 commit 通过 | DB, OpenViking | 是 |
|
||||
| `summarize_episode_skill` | 压缩 episode | episode content | summary | session commit | LLM | 否 |
|
||||
| `merge_memory_skill` | 合并重复或相近记忆 | memory ids | merged memory | EverMemOS 整理 | DB, vector index | 是 |
|
||||
| `prune_memory_skill` | 衰减、归档、删除低质记忆 | policy + memory ids | archived/deleted list | 定时 worker | EverMemOS | 是 |
|
||||
| `merge_memory_skill` | 合并重复或相近记忆 | memory ids | merged memory | EverOS 整理 | DB, vector index | 是 |
|
||||
| `prune_memory_skill` | 衰减、归档、删除低质记忆 | policy + memory ids | archived/deleted list | 定时 worker | EverOS | 是 |
|
||||
| `export_to_obsidian_skill` | 生成 Obsidian review draft | high-value memory | markdown draft | 高价值或需人工确认 | Obsidian | 否 |
|
||||
| `import_from_obsidian_skill` | 从人工维护笔记导入记忆 | markdown path | MemoryRecord | vault sync | Obsidian, OpenViking | 是 |
|
||||
|
||||
@ -551,7 +551,7 @@ obsidian-vault/
|
||||
|
||||
- 人工可维护 profile、preferences、长期总结。
|
||||
- 高价值 workspace 知识、项目决策、复用经验。
|
||||
- EverMemOS 标记为 `needs_review` 的长期记忆草稿。
|
||||
- EverOS 标记为 `needs_review` 的长期记忆草稿。
|
||||
|
||||
不进入 Obsidian 的内容:
|
||||
|
||||
@ -571,7 +571,7 @@ obsidian-vault/
|
||||
#memory/review
|
||||
#memory/conflict
|
||||
#memory/deprecated
|
||||
#source/evermemos
|
||||
#source/everos
|
||||
#source/manual
|
||||
#visibility/private
|
||||
#visibility/workspace-shared
|
||||
@ -605,10 +605,10 @@ viking://skills/memory-gateway/{skill_name}
|
||||
同步:
|
||||
|
||||
- Obsidian accepted note 通过 `import_from_obsidian_skill` 写回 Gateway,再同步 OpenViking resource。
|
||||
- EverMemOS consolidation 后写入 `user/{user_id}/long_term` 或 `workspace/{workspace_id}/shared`。
|
||||
- EverOS consolidation 后写入 `user/{user_id}/long_term` 或 `workspace/{workspace_id}/shared`。
|
||||
- Gateway 保存 `source_ref`,避免 OpenViking 与 Obsidian 互相重复导入。
|
||||
|
||||
## H. EverMemOS 设计
|
||||
## H. EverOS 设计
|
||||
|
||||
输入来源:
|
||||
|
||||
@ -662,10 +662,10 @@ memory-gateway/
|
||||
│ │ └── import_from_obsidian_skill.py
|
||||
│ ├── adapters/
|
||||
│ │ ├── openviking.py
|
||||
│ │ ├── evermemos.py
|
||||
│ │ ├── everos.py
|
||||
│ │ └── obsidian.py
|
||||
│ └── workers/
|
||||
│ └── evermemos_worker.py
|
||||
│ └── everos_worker.py
|
||||
├── obsidian-vault/
|
||||
├── integrations/
|
||||
│ ├── nanobot/
|
||||
@ -692,7 +692,7 @@ memory-gateway/
|
||||
|
||||
第三周:
|
||||
|
||||
- 加 EverMemOS worker 原型:session commit、candidate extraction、dedup、merge。
|
||||
- 加 EverOS worker 原型:session commit、candidate extraction、dedup、merge。
|
||||
- 增加 feedback 流程:incorrect、duplicate、outdated 影响 prune/merge。
|
||||
- 生成 Obsidian review draft,而不是直接写入最终知识库。
|
||||
|
||||
@ -728,7 +728,7 @@ POC 成功指标:
|
||||
- 存储使用 SQLite metadata + 本地文件存 object;当前代码先用 in-memory repo 验证接口。
|
||||
- 搜索先用 OpenViking search + 简单 lexical fallback;向量索引第二阶段引入。
|
||||
- Obsidian 只保存人工可读的高价值长期记忆和 review draft。
|
||||
- EverMemOS 第一阶段不做独立大系统,只做 worker 模块:extract、dedup、merge、prune、profile update。
|
||||
- EverOS 第一阶段不做独立大系统,只做 worker 模块:extract、dedup、merge、prune、profile update。
|
||||
|
||||
第一阶段实现 API:
|
||||
|
||||
@ -757,11 +757,11 @@ POC 成功指标:
|
||||
- `merge_memory_skill`
|
||||
- `prune_memory_skill`
|
||||
- `import_from_obsidian_skill`
|
||||
- 更完整的 EverMemOS consolidation 和 profile evolution。
|
||||
- 更完整的 EverOS consolidation 和 profile evolution。
|
||||
|
||||
角色分工:
|
||||
|
||||
- Obsidian 第一阶段:review draft、人类确认 profile/长期知识。第二阶段:双向同步。
|
||||
- OpenViking 第一阶段:统一 context/resource 检索入口。第二阶段:承载多 namespace context filesystem 和 skill registry。
|
||||
- EverMemOS 第一阶段:session commit worker。第二阶段:长期记忆治理、衰减、冲突检测、profile evolution。
|
||||
- EverOS 第一阶段:session commit worker。第二阶段:长期记忆治理、衰减、冲突检测、profile evolution。
|
||||
|
||||
|
||||
@ -1,10 +1,10 @@
|
||||
---
|
||||
name: memory-gateway
|
||||
description: Use this skill when Hermes needs shared long-term memory, user-scoped preferences/profile, workspace memory, session episode capture, Memory Gateway retrieval, OpenViking context search, Obsidian document upload/review, or session commit through the standalone EverMemOS service. This skill is domain-neutral.
|
||||
description: Use this skill when Hermes needs shared long-term memory, user-scoped preferences/profile, workspace memory, session episode capture, Memory Gateway retrieval, OpenViking context search, Obsidian document upload/review, or session commit through the standalone EverOS service. This skill is domain-neutral.
|
||||
version: 3.1.0
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [memory, memory-gateway, openviking, obsidian, evermemos, long-term-memory, retrieval, agent-context]
|
||||
tags: [memory, memory-gateway, openviking, obsidian, everos, long-term-memory, retrieval, agent-context]
|
||||
---
|
||||
|
||||
# Memory Gateway
|
||||
@ -16,7 +16,7 @@ The gateway provides:
|
||||
- v1 user/agent/workspace/session aware memory APIs backed by SQLite metadata.
|
||||
- ACL and namespace routing before retrieval.
|
||||
- OpenViking fan-out search for visible namespaces.
|
||||
- Session episode capture and commit through the standalone EverMemOS HTTP service, with Gateway local fallback only when configured.
|
||||
- Session episode capture and commit through the standalone EverOS HTTP service, with Gateway local fallback only when configured.
|
||||
- Obsidian review drafts for high-value or conflicting long-term memory candidates.
|
||||
- Legacy summary/document upload endpoints for LLM summarization and Obsidian knowledge ingestion.
|
||||
|
||||
@ -25,7 +25,7 @@ The gateway provides:
|
||||
Defaults:
|
||||
|
||||
- Memory Gateway URL: `http://127.0.0.1:1934`
|
||||
- EverMemOS URL through Gateway config: `http://127.0.0.1:1995`
|
||||
- EverOS URL through Gateway config: `http://127.0.0.1:1995`
|
||||
- Obsidian vault: `/home/tom/memory-gateway/obsidian-vault`
|
||||
- Default review queue: `/home/tom/memory-gateway/obsidian-vault/Reviews/Queue`
|
||||
|
||||
@ -41,7 +41,7 @@ For normal agent work:
|
||||
|
||||
1. Search memory before answering if prior context may matter.
|
||||
2. Append important session episodes while working.
|
||||
3. Commit the session at the end so EverMemOS can promote stable memories.
|
||||
3. Commit the session at the end so EverOS can promote stable memories.
|
||||
4. Use feedback to mark incorrect, duplicate, outdated, or useful memories.
|
||||
5. Upload documents only when they are reusable knowledge, not raw noisy logs.
|
||||
|
||||
@ -49,13 +49,13 @@ Do not write full transcripts to long-term memory. Use episodes for temporary pr
|
||||
|
||||
## v1 Memory Commands
|
||||
|
||||
### Check EverMemOS
|
||||
### Check EverOS
|
||||
|
||||
```bash
|
||||
python /home/tom/.hermes/skills/memory-gateway/scripts/evermemos_health.py
|
||||
python /home/tom/.hermes/skills/memory-gateway/scripts/everos_health.py
|
||||
```
|
||||
|
||||
Expected healthy response includes `status: ok` and `response.service: evermemos-local`.
|
||||
Expected healthy response includes `status: ok` and `response.service: everos-local`.
|
||||
|
||||
### Create User
|
||||
|
||||
@ -122,10 +122,10 @@ python /home/tom/.hermes/skills/memory-gateway/scripts/memory_append_episode.py
|
||||
--text "结论:这个项目必须保留用户隔离和 namespace ACL。"
|
||||
```
|
||||
|
||||
### Commit Session Through EverMemOS
|
||||
### Commit Session Through EverOS
|
||||
|
||||
This asks Memory Gateway to call the standalone EverMemOS service configured in `config.yaml`.
|
||||
For local POC the default service is `http://127.0.0.1:1995`. If `evermemos.fallback_to_local` is true and the service is unavailable, Gateway returns `evermemos_backend: local-fallback`.
|
||||
This asks Memory Gateway to call the standalone EverOS service configured in `config.yaml`.
|
||||
For local POC the default service is `http://127.0.0.1:1995`. If `everos.fallback_to_local` is true and the service is unavailable, Gateway returns `everos_backend: local-fallback`.
|
||||
|
||||
- extracts candidate memories from session episodes
|
||||
- deduplicates exact repeated candidates
|
||||
@ -282,7 +282,7 @@ When using this skill, answer with:
|
||||
|
||||
- Do not store raw noisy data as long-term memory.
|
||||
- Use `memory_append_episode.py` for temporary process notes.
|
||||
- Use `memory_commit_session.py` at task end to let EverMemOS decide what should persist.
|
||||
- Use `memory_commit_session.py` at task end to let EverOS decide what should persist.
|
||||
- Use `memory_upsert.py` directly only for stable, concise, user-approved memory.
|
||||
- Do not commit secrets, credentials, tokens, private keys, or unnecessary personal data.
|
||||
- If content is sensitive, summarize and redact before committing.
|
||||
|
||||
@ -1,19 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
|
||||
from _client import DEFAULT_GATEWAY_API_KEY, DEFAULT_GATEWAY_URL, get_json
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Check standalone EverMemOS health through Memory Gateway.")
|
||||
parser.add_argument("--gateway-url", default=DEFAULT_GATEWAY_URL)
|
||||
parser.add_argument("--api-key", default=DEFAULT_GATEWAY_API_KEY)
|
||||
args = parser.parse_args()
|
||||
print(json.dumps(get_json("/v1/evermemos/health", gateway_url=args.gateway_url, api_key=args.api_key), ensure_ascii=False, indent=2))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
21
integrations/hermes/memory-gateway/scripts/everos_health.py
Normal file
21
integrations/hermes/memory-gateway/scripts/everos_health.py
Normal file
@ -0,0 +1,21 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Check EverOS health through Memory Gateway."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
|
||||
from _client import get_json
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Check EverOS health through Memory Gateway.")
|
||||
parser.add_argument("--gateway-url", default="http://127.0.0.1:1934")
|
||||
parser.add_argument("--api-key", default=None)
|
||||
args = parser.parse_args()
|
||||
print(json.dumps(get_json("/v1/everos/health", gateway_url=args.gateway_url, api_key=args.api_key), ensure_ascii=False, indent=2))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@ -8,7 +8,7 @@ from _client import DEFAULT_GATEWAY_API_KEY, DEFAULT_GATEWAY_URL, post_json
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description="Commit a session through the minimal EverMemOS consolidation worker.")
|
||||
parser = argparse.ArgumentParser(description="Commit a session through the minimal EverOS consolidation worker.")
|
||||
parser.add_argument("--user-id", required=True)
|
||||
parser.add_argument("--session-id", required=True)
|
||||
parser.add_argument("--agent-id", default="")
|
||||
|
||||
@ -110,6 +110,6 @@ async def list_audit(limit: int = Query(default=100, ge=1, le=1000)):
|
||||
return service.list_audit(limit)
|
||||
|
||||
|
||||
@router.get("/evermemos/health")
|
||||
async def evermemos_health():
|
||||
return service.evermemos_health()
|
||||
@router.get("/everos/health")
|
||||
async def everos_health():
|
||||
return service.everos_health()
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
"""Contract-first mapping spec for future v2 backend adapters.
|
||||
|
||||
This module intentionally does not call OpenViking, EverMemOS, or Obsidian.
|
||||
This module intentionally does not call OpenViking, EverOS, or Obsidian.
|
||||
It documents the stable Gateway control-plane fields that may be persisted in
|
||||
outbox payload refs, SQLite metadata_json, audit summaries, and related control
|
||||
records. It is not a validator for transient runtime adapter request objects:
|
||||
@ -81,21 +81,21 @@ ADAPTER_MAPPING_SPECS: Final[tuple[AdapterMappingSpec, ...]] = (
|
||||
result_model=BackendRetrieveResult,
|
||||
),
|
||||
AdapterMappingSpec(
|
||||
backend_type=BackendType.EVERMEMOS,
|
||||
backend_type=BackendType.EVEROS,
|
||||
operation=BackendOperation.INGEST_TURN,
|
||||
adapter_method="ingest_message",
|
||||
backend_capability="message-level memory ingestion",
|
||||
result_model=BackendWriteResult,
|
||||
),
|
||||
AdapterMappingSpec(
|
||||
backend_type=BackendType.EVERMEMOS,
|
||||
backend_type=BackendType.EVEROS,
|
||||
operation=BackendOperation.COMMIT_SESSION,
|
||||
adapter_method="extract_profile_long_term_v2",
|
||||
backend_capability="episodic/profile/long-term extraction",
|
||||
result_model=BackendCommitResult,
|
||||
),
|
||||
AdapterMappingSpec(
|
||||
backend_type=BackendType.EVERMEMOS,
|
||||
backend_type=BackendType.EVEROS,
|
||||
operation=BackendOperation.RETRIEVE_CONTEXT,
|
||||
adapter_method="retrieve_context_v2",
|
||||
backend_capability="episodic/profile/long-term memory retrieval",
|
||||
|
||||
@ -63,16 +63,16 @@ def normalize_openviking_commit_response(raw: dict[str, Any]) -> BackendCommitRe
|
||||
)
|
||||
|
||||
|
||||
def normalize_evermemos_commit_response(raw: dict[str, Any]) -> BackendCommitResult:
|
||||
def normalize_everos_commit_response(raw: dict[str, Any]) -> BackendCommitResult:
|
||||
status = _result_status(raw)
|
||||
refs = [_produced_ref(BackendType.EVERMEMOS, item) for item in _extract_ref_items(raw)]
|
||||
refs = [_produced_ref(BackendType.EVEROS, item) for item in _extract_ref_items(raw)]
|
||||
return BackendCommitResult(
|
||||
backend_type=BackendType.EVERMEMOS,
|
||||
backend_type=BackendType.EVEROS,
|
||||
operation=BackendOperation.COMMIT_SESSION,
|
||||
status=status,
|
||||
native_id=raw.get("native_id") or raw.get("session_id"),
|
||||
native_uri=raw.get("native_uri") or raw.get("uri"),
|
||||
retryable=_retryable_from_raw(BackendType.EVERMEMOS, raw),
|
||||
retryable=_retryable_from_raw(BackendType.EVEROS, raw),
|
||||
error_code=raw.get("error_code"),
|
||||
error_message=raw.get("error") or raw.get("error_message"),
|
||||
latency_ms=raw.get("latency_ms"),
|
||||
@ -85,16 +85,16 @@ def normalize_openviking_ingest_response(raw: dict[str, Any]) -> BackendWriteRes
|
||||
return _write_result(BackendType.OPENVIKING, raw)
|
||||
|
||||
|
||||
def normalize_evermemos_ingest_response(raw: dict[str, Any]) -> BackendWriteResult:
|
||||
return _write_result(BackendType.EVERMEMOS, raw)
|
||||
def normalize_everos_ingest_response(raw: dict[str, Any]) -> BackendWriteResult:
|
||||
return _write_result(BackendType.EVEROS, raw)
|
||||
|
||||
|
||||
def normalize_openviking_retrieve_response(raw: dict[str, Any]) -> BackendRetrieveResult:
|
||||
return _retrieve_result(BackendType.OPENVIKING, raw)
|
||||
|
||||
|
||||
def normalize_evermemos_retrieve_response(raw: dict[str, Any]) -> BackendRetrieveResult:
|
||||
return _retrieve_result(BackendType.EVERMEMOS, raw)
|
||||
def normalize_everos_retrieve_response(raw: dict[str, Any]) -> BackendRetrieveResult:
|
||||
return _retrieve_result(BackendType.EVEROS, raw)
|
||||
|
||||
|
||||
def map_backend_error_to_retryable(
|
||||
@ -129,10 +129,12 @@ def _write_result(backend_type: BackendType, raw: dict[str, Any]) -> BackendWrit
|
||||
raw.get("native_id")
|
||||
or raw.get("id")
|
||||
or raw.get("memory_id")
|
||||
or raw.get("request_id")
|
||||
or raw.get("session_id")
|
||||
or data.get("native_id")
|
||||
or data.get("id")
|
||||
or data.get("memory_id")
|
||||
or data.get("request_id")
|
||||
or data.get("session_id")
|
||||
)
|
||||
native_uri = (
|
||||
@ -152,8 +154,8 @@ def _write_result(backend_type: BackendType, raw: dict[str, Any]) -> BackendWrit
|
||||
native_id=native_id,
|
||||
native_uri=native_uri,
|
||||
retryable=_retryable_from_raw(backend_type, raw),
|
||||
error_code=raw.get("error_code"),
|
||||
error_message=raw.get("error") or raw.get("error_message"),
|
||||
error_code=raw.get("error_code") or raw.get("code"),
|
||||
error_message=raw.get("error") or raw.get("error_message") or raw.get("message"),
|
||||
latency_ms=raw.get("latency_ms"),
|
||||
metadata=safe_backend_metadata(raw.get("metadata") or raw),
|
||||
)
|
||||
@ -174,8 +176,8 @@ def _retrieve_result(backend_type: BackendType, raw: dict[str, Any]) -> BackendR
|
||||
native_id=raw.get("native_id") or raw.get("session_id"),
|
||||
native_uri=raw.get("native_uri") or raw.get("uri"),
|
||||
retryable=_retryable_from_raw(backend_type, raw),
|
||||
error_code=raw.get("error_code"),
|
||||
error_message=raw.get("error") or raw.get("error_message"),
|
||||
error_code=raw.get("error_code") or raw.get("code"),
|
||||
error_message=raw.get("error") or raw.get("error_message") or raw.get("message"),
|
||||
latency_ms=raw.get("latency_ms"),
|
||||
items=[_retrieve_item(backend_type, item) for item in _extract_retrieve_items(raw)],
|
||||
metadata=safe_backend_metadata(raw.get("metadata") or raw),
|
||||
|
||||
@ -11,7 +11,7 @@ OPENVIKING_REF_TYPE_MAP = {
|
||||
"session_summary": MemoryRefType.SESSION_ARCHIVE,
|
||||
}
|
||||
|
||||
EVERMEMOS_REF_TYPE_MAP = {
|
||||
EVEROS_REF_TYPE_MAP = {
|
||||
"message_memory": MemoryRefType.MESSAGE_MEMORY,
|
||||
"episodic_memory": MemoryRefType.EPISODIC_MEMORY,
|
||||
"episode": MemoryRefType.EPISODIC_MEMORY,
|
||||
@ -41,8 +41,8 @@ def map_backend_ref_type(
|
||||
|
||||
if backend_type == BackendType.OPENVIKING:
|
||||
mapped = OPENVIKING_REF_TYPE_MAP.get(normalized, MemoryRefType.SESSION_ARCHIVE)
|
||||
elif backend_type == BackendType.EVERMEMOS:
|
||||
mapped = EVERMEMOS_REF_TYPE_MAP.get(normalized, MemoryRefType.LONG_TERM_MEMORY)
|
||||
elif backend_type == BackendType.EVEROS:
|
||||
mapped = EVEROS_REF_TYPE_MAP.get(normalized, MemoryRefType.LONG_TERM_MEMORY)
|
||||
elif backend_type == BackendType.OBSIDIAN:
|
||||
mapped = OBSIDIAN_REF_TYPE_MAP.get(normalized, MemoryRefType.DRAFT_REVIEW)
|
||||
else:
|
||||
@ -59,8 +59,8 @@ def map_backend_ref_type(
|
||||
def _known_backend_ref_types(backend_type: BackendType) -> set[str]:
|
||||
if backend_type == BackendType.OPENVIKING:
|
||||
return set(OPENVIKING_REF_TYPE_MAP)
|
||||
if backend_type == BackendType.EVERMEMOS:
|
||||
return set(EVERMEMOS_REF_TYPE_MAP)
|
||||
if backend_type == BackendType.EVEROS:
|
||||
return set(EVEROS_REF_TYPE_MAP)
|
||||
if backend_type == BackendType.OBSIDIAN:
|
||||
return set(OBSIDIAN_REF_TYPE_MAP)
|
||||
return set()
|
||||
|
||||
@ -6,7 +6,7 @@ from typing import Optional
|
||||
import yaml
|
||||
from pydantic import ValidationError
|
||||
|
||||
from .types import Config, ServerConfig, OpenVikingConfig, EverMemOSConfig, MemoryConfig, LoggingConfig, LLMConfig, ObsidianConfig, StorageConfig
|
||||
from .types import Config, ServerConfig, OpenVikingConfig, EverOSConfig, MemoryConfig, LoggingConfig, LLMConfig, ObsidianConfig, StorageConfig
|
||||
|
||||
|
||||
def load_config(config_path: Optional[str] = None) -> Config:
|
||||
@ -30,7 +30,7 @@ def load_config(config_path: Optional[str] = None) -> Config:
|
||||
config = Config(
|
||||
server=ServerConfig(**data.get("server", {})),
|
||||
openviking=OpenVikingConfig(**data.get("openviking", {})),
|
||||
evermemos=EverMemOSConfig(**data.get("evermemos", {})),
|
||||
everos=EverOSConfig(**data.get("everos", {})),
|
||||
memory=MemoryConfig(**data.get("memory", {})),
|
||||
logging=LoggingConfig(**data.get("logging", {})),
|
||||
llm=LLMConfig(**data.get("llm", {})),
|
||||
@ -62,11 +62,11 @@ _config: Optional[Config] = None
|
||||
|
||||
def _apply_env_overrides(config: Config) -> Config:
|
||||
openviking_updates = _backend_env_updates("OPENVIKING")
|
||||
evermemos_updates = _backend_env_updates("EVERMEMOS")
|
||||
everos_updates = _backend_env_updates("EVEROS")
|
||||
if openviking_updates:
|
||||
config.openviking = config.openviking.model_copy(update=openviking_updates)
|
||||
if evermemos_updates:
|
||||
config.evermemos = config.evermemos.model_copy(update=evermemos_updates)
|
||||
if everos_updates:
|
||||
config.everos = config.everos.model_copy(update=everos_updates)
|
||||
return config
|
||||
|
||||
|
||||
@ -83,6 +83,9 @@ def _backend_env_updates(prefix: str) -> dict:
|
||||
"TIMEOUT_SECONDS": "timeout",
|
||||
"VERIFY_SSL": "verify_ssl",
|
||||
"INGEST_PATH": "ingest_path",
|
||||
"SEARCH_PATH": "search_path",
|
||||
"FLUSH_PATH": "flush_path",
|
||||
"RETRIEVE_METHOD": "retrieve_method",
|
||||
}
|
||||
for env_name, field_name in env_map.items():
|
||||
value = os.environ.get(f"{prefix}_{env_name}")
|
||||
|
||||
@ -1,313 +0,0 @@
|
||||
"""Client for the external EverMemOS consolidation service."""
|
||||
from __future__ import annotations
|
||||
|
||||
from json import JSONDecodeError
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
from .backend_contracts import BackendCommitResult, BackendOperation, BackendResultStatus, BackendRetrieveResult, BackendWriteResult
|
||||
from .backend_normalization import (
|
||||
map_backend_error_to_retryable,
|
||||
normalize_evermemos_commit_response,
|
||||
normalize_evermemos_ingest_response,
|
||||
normalize_evermemos_retrieve_response,
|
||||
)
|
||||
from .config import get_config
|
||||
from .schemas import AccessContext, EpisodeRecord, MemoryRecord
|
||||
from .schemas_v2 import BackendType
|
||||
|
||||
|
||||
class EverMemOSError(RuntimeError):
|
||||
"""Raised when the external EverMemOS service cannot consolidate."""
|
||||
|
||||
|
||||
class EverMemOSClient:
|
||||
"""Small HTTP client with a tolerant response normalizer.
|
||||
|
||||
The deployed EverMemOS API may evolve independently from Memory Gateway.
|
||||
Gateway sends a stable payload and accepts several common response shapes:
|
||||
`result`, `data`, or the raw top-level object with `candidates/promoted`.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
base_url: str | None = None,
|
||||
api_key: str | None = None,
|
||||
timeout: int | None = None,
|
||||
enabled: bool | None = None,
|
||||
mode: str | None = None,
|
||||
verify_ssl: bool | None = None,
|
||||
health_path: str | None = None,
|
||||
ingest_path: str | None = None,
|
||||
consolidate_path: str | None = None,
|
||||
transport: httpx.BaseTransport | None = None,
|
||||
) -> None:
|
||||
config = get_config().evermemos
|
||||
self.base_url = (base_url if base_url is not None else config.url).rstrip("/")
|
||||
self.api_key = api_key if api_key is not None else config.api_key
|
||||
self.timeout = timeout or config.timeout
|
||||
self.enabled = config.enabled if enabled is None else enabled
|
||||
self.mode = mode or config.mode
|
||||
self.verify_ssl = config.verify_ssl if verify_ssl is None else verify_ssl
|
||||
self.health_path = health_path or config.health_path
|
||||
self.ingest_path = ingest_path or config.ingest_path
|
||||
self.consolidate_path = consolidate_path or config.consolidate_path
|
||||
self.transport = transport
|
||||
|
||||
def _headers(self) -> dict[str, str]:
|
||||
headers = {"Content-Type": "application/json"}
|
||||
if self.api_key:
|
||||
headers["X-API-Key"] = self.api_key
|
||||
headers["Authorization"] = f"Bearer {self.api_key}"
|
||||
return headers
|
||||
|
||||
def health(self) -> dict[str, Any]:
|
||||
url = self.base_url + self.health_path
|
||||
try:
|
||||
health_timeout = httpx.Timeout(min(self.timeout, 2.0), connect=min(self.timeout, 0.5))
|
||||
with httpx.Client(timeout=health_timeout, headers=self._headers()) as client:
|
||||
response = client.get(url)
|
||||
response.raise_for_status()
|
||||
return {"status": "ok", "url": self.base_url, "response": response.json()}
|
||||
except Exception as exc: # noqa: BLE001
|
||||
return {"status": "error", "url": self.base_url, "error": str(exc)}
|
||||
|
||||
def ingest_message(self, payload: dict[str, Any]) -> BackendWriteResult:
|
||||
"""v2 adapter placeholder for message-level EverMemOS ingestion.
|
||||
|
||||
Mapping spec: `backend_adapter_mapping.AdapterMappingSpec` maps
|
||||
EverMemOS ingest_turn to this method and requires BackendWriteResult.
|
||||
Payloads must contain only control-plane fields; raw request bodies are
|
||||
not persisted by the Gateway control-plane store.
|
||||
|
||||
TODO(v2): bind this to EverMemOS `/api/v1/memories` or its stable
|
||||
message ingestion API after the external contract settles.
|
||||
"""
|
||||
runtime_payload = self._build_ingest_payload(payload)
|
||||
if self._use_real_api:
|
||||
return self._ingest_message_real(runtime_payload)
|
||||
raw = {
|
||||
"status": "skipped",
|
||||
"memory_id": runtime_payload.get("turn_id"),
|
||||
"metadata": {
|
||||
"reason": "evermemos_v2_ingest_adapter_not_configured",
|
||||
"schema_version": "evermemos.fixture.ingest.v2",
|
||||
},
|
||||
}
|
||||
return self._normalize_ingest_response(raw)
|
||||
|
||||
@property
|
||||
def _use_real_api(self) -> bool:
|
||||
# Real ingest is strictly gated by mode=real. The legacy `enabled`
|
||||
# field is retained for config compatibility, but must not trigger
|
||||
# network traffic by itself.
|
||||
return self.mode == "real"
|
||||
|
||||
def _ingest_message_real(self, runtime_payload: dict[str, Any]) -> BackendWriteResult:
|
||||
if not self.base_url:
|
||||
return self._failed_ingest_result(
|
||||
error_code="config_error",
|
||||
error_message="EverMemOS real ingest is enabled but base_url is missing",
|
||||
retryable=False,
|
||||
)
|
||||
try:
|
||||
with httpx.Client(
|
||||
base_url=self.base_url,
|
||||
headers=self._headers(),
|
||||
timeout=self.timeout,
|
||||
verify=self.verify_ssl,
|
||||
transport=self.transport,
|
||||
) as client:
|
||||
response = client.post(self.ingest_path, json=runtime_payload)
|
||||
if response.status_code >= 400:
|
||||
return self._failed_ingest_result(
|
||||
error_code=f"http_{response.status_code}",
|
||||
error_message=f"EverMemOS ingest failed with HTTP {response.status_code}",
|
||||
retryable=self._map_error(response),
|
||||
)
|
||||
try:
|
||||
raw = response.json()
|
||||
except (JSONDecodeError, ValueError):
|
||||
return self._failed_ingest_result(
|
||||
error_code="invalid_json",
|
||||
error_message="EverMemOS ingest returned invalid JSON",
|
||||
retryable=True,
|
||||
)
|
||||
if not isinstance(raw, dict):
|
||||
return self._failed_ingest_result(
|
||||
error_code="unexpected_response",
|
||||
error_message="EverMemOS ingest returned an unexpected response shape",
|
||||
retryable=True,
|
||||
)
|
||||
return self._normalize_ingest_response(raw)
|
||||
except httpx.TimeoutException as exc:
|
||||
return self._failed_ingest_result("timeout", self._safe_error_message(exc), retryable=self._map_error(exc))
|
||||
except httpx.RequestError as exc:
|
||||
return self._failed_ingest_result("network_error", self._safe_error_message(exc), retryable=self._map_error(exc))
|
||||
except Exception as exc: # noqa: BLE001
|
||||
return self._failed_ingest_result("unexpected_error", self._safe_error_message(exc), retryable=self._map_error(exc))
|
||||
|
||||
def extract_profile_long_term_v2(self, payload: dict[str, Any]) -> BackendCommitResult:
|
||||
"""v2 adapter placeholder for profile / long-term extraction.
|
||||
|
||||
Mapping spec: commit_session returns BackendCommitResult and should
|
||||
produce native episodic/profile/long-term refs once the real API is stable.
|
||||
"""
|
||||
runtime_payload = self._build_commit_payload(payload)
|
||||
raw = {
|
||||
"status": "success",
|
||||
"session_id": runtime_payload.get("session_id"),
|
||||
"metadata": {
|
||||
"reason": "evermemos_v2_commit_fixture",
|
||||
"schema_version": "evermemos.fixture.commit.v2",
|
||||
},
|
||||
"data": {
|
||||
"produced_refs": [
|
||||
{
|
||||
"ref_type": "profile",
|
||||
"profile_id": f"em_profile:{runtime_payload.get('user_id') or 'unknown'}",
|
||||
"metadata": {"schema_version": "evermemos.fixture.profile.v2"},
|
||||
},
|
||||
{
|
||||
"ref_type": "long_term_memory",
|
||||
"memory_id": f"em_long_term:{runtime_payload.get('session_id')}",
|
||||
"metadata": {"schema_version": "evermemos.fixture.long_term.v2"},
|
||||
},
|
||||
]
|
||||
},
|
||||
}
|
||||
return self._normalize_commit_response(raw)
|
||||
|
||||
def retrieve_context_v2(self, payload: dict[str, Any]) -> BackendRetrieveResult:
|
||||
"""v2 adapter placeholder for episodic/profile/long-term retrieval.
|
||||
|
||||
Mapping spec: retrieve_context returns BackendRetrieveResult with
|
||||
normalized context items, not raw backend payload dumps.
|
||||
"""
|
||||
raw = {
|
||||
"status": "success",
|
||||
"metadata": {
|
||||
"reason": "evermemos_v2_retrieve_fixture",
|
||||
"schema_version": "evermemos.fixture.retrieve.v2",
|
||||
},
|
||||
"data": {
|
||||
"items": [
|
||||
{
|
||||
"text": "EverMemOS fixture profile context.",
|
||||
"profile_id": f"em_profile:{payload.get('user_id') or 'unknown'}",
|
||||
"score": 0.72,
|
||||
"memory_type": "profile",
|
||||
"metadata": {"schema_version": "evermemos.fixture.retrieve.item.v2"},
|
||||
},
|
||||
{
|
||||
"text": "EverMemOS fixture long-term memory context.",
|
||||
"memory_id": f"em_long_term:{payload.get('session_id') or 'unknown'}",
|
||||
"score": 0.69,
|
||||
"memory_type": "long_term_memory",
|
||||
"metadata": {"schema_version": "evermemos.fixture.retrieve.item.v2"},
|
||||
},
|
||||
]
|
||||
},
|
||||
}
|
||||
return self._normalize_retrieve_response(raw)
|
||||
|
||||
def _build_ingest_payload(self, payload: dict[str, Any]) -> dict[str, Any]:
|
||||
# Runtime-only adapter payload. It may include conversation content for
|
||||
# the current request lifecycle; callers must not persist it to SQLite.
|
||||
return dict(payload)
|
||||
|
||||
def _build_commit_payload(self, payload: dict[str, Any]) -> dict[str, Any]:
|
||||
return dict(payload)
|
||||
|
||||
def _normalize_ingest_response(self, raw: dict[str, Any]) -> BackendWriteResult:
|
||||
return normalize_evermemos_ingest_response(raw)
|
||||
|
||||
def _normalize_commit_response(self, raw: dict[str, Any]) -> BackendCommitResult:
|
||||
return normalize_evermemos_commit_response(raw)
|
||||
|
||||
def _normalize_retrieve_response(self, raw: dict[str, Any]) -> BackendRetrieveResult:
|
||||
return normalize_evermemos_retrieve_response(raw)
|
||||
|
||||
def _map_error(self, exc_or_response: Any) -> bool:
|
||||
status_code = getattr(exc_or_response, "status_code", None)
|
||||
error_code = getattr(exc_or_response, "error_code", None)
|
||||
error_message = str(exc_or_response) if exc_or_response is not None else None
|
||||
return map_backend_error_to_retryable(
|
||||
BackendType.EVERMEMOS,
|
||||
status_code=status_code,
|
||||
error_code=error_code,
|
||||
error_message=error_message,
|
||||
)
|
||||
|
||||
def _failed_ingest_result(self, error_code: str, error_message: str, retryable: bool) -> BackendWriteResult:
|
||||
return BackendWriteResult(
|
||||
backend_type=BackendType.EVERMEMOS,
|
||||
operation=BackendOperation.INGEST_TURN,
|
||||
status=BackendResultStatus.FAILED,
|
||||
retryable=retryable,
|
||||
error_code=error_code,
|
||||
error_message=error_message,
|
||||
metadata={"error_code": error_code},
|
||||
)
|
||||
|
||||
def _safe_error_message(self, exc: Exception) -> str:
|
||||
return exc.__class__.__name__
|
||||
|
||||
def consolidate_session(
|
||||
self,
|
||||
session_id: str,
|
||||
ctx: AccessContext,
|
||||
episodes: list[EpisodeRecord],
|
||||
existing_memories: list[MemoryRecord],
|
||||
min_importance: float,
|
||||
target_namespace: str | None,
|
||||
) -> dict[str, Any]:
|
||||
payload = {
|
||||
"schema_version": "memory-gateway.evermemos.consolidate.v1",
|
||||
"session_id": session_id,
|
||||
"context": ctx.model_dump(mode="json"),
|
||||
"min_importance": min_importance,
|
||||
"target_namespace": target_namespace,
|
||||
"episodes": [episode.model_dump(mode="json") for episode in episodes],
|
||||
"existing_memories": [memory.model_dump(mode="json") for memory in existing_memories],
|
||||
}
|
||||
paths = [
|
||||
self.consolidate_path,
|
||||
"/v1/sessions/consolidate",
|
||||
"/v1/memory/consolidate",
|
||||
"/api/v1/sessions/consolidate",
|
||||
"/api/consolidate",
|
||||
"/consolidate",
|
||||
]
|
||||
errors: list[str] = []
|
||||
for path in dict.fromkeys(paths):
|
||||
try:
|
||||
with httpx.Client(timeout=self.timeout, headers=self._headers()) as client:
|
||||
response = client.post(self.base_url + path, json=payload)
|
||||
if response.status_code == 404:
|
||||
errors.append(f"{path}: 404")
|
||||
continue
|
||||
response.raise_for_status()
|
||||
return self._normalize_response(response.json(), path)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
errors.append(f"{path}: {exc}")
|
||||
if "Connection refused" in str(exc) or "timed out" in str(exc):
|
||||
break
|
||||
raise EverMemOSError("; ".join(errors) or "EverMemOS consolidation failed")
|
||||
|
||||
def _normalize_response(self, payload: dict[str, Any], path: str) -> dict[str, Any]:
|
||||
data = payload.get("result") or payload.get("data") or payload
|
||||
return {
|
||||
"backend": "external",
|
||||
"service_url": self.base_url,
|
||||
"endpoint": path,
|
||||
"raw": payload,
|
||||
"session_id": data.get("session_id"),
|
||||
"episodes": data.get("episodes"),
|
||||
"candidates": data.get("candidates") or data.get("candidate_memories") or [],
|
||||
"promoted": data.get("promoted") or data.get("promoted_memories") or data.get("memories") or [],
|
||||
"duplicates": data.get("duplicates") or [],
|
||||
"conflicts": data.get("conflicts") or [],
|
||||
"review_drafts": data.get("review_drafts") or [],
|
||||
}
|
||||
@ -1,149 +0,0 @@
|
||||
"""Standalone EverMemOS-compatible consolidation service.
|
||||
|
||||
This is a lightweight local service for POC use. It intentionally exposes the
|
||||
same HTTP contract that Memory Gateway calls:
|
||||
|
||||
POST /v1/sessions/consolidate
|
||||
|
||||
The service does not own Memory Gateway's metadata database. It receives
|
||||
episodes and existing memories in the request, returns candidate/promoted
|
||||
MemoryRecord payloads, and creates Obsidian review drafts for high-value or
|
||||
conflicting candidates.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import logging
|
||||
from typing import Any
|
||||
|
||||
from fastapi import FastAPI
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from .config import load_config, set_config
|
||||
from .repositories import InMemoryRepository
|
||||
from .schemas import AccessContext, EpisodeRecord, MemoryRecord
|
||||
from .workers.evermemos_worker import EverMemOSWorker
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class ConsolidateRequest(BaseModel):
|
||||
schema_version: str = "memory-gateway.evermemos.consolidate.v1"
|
||||
session_id: str
|
||||
context: dict[str, Any]
|
||||
min_importance: float = 0.6
|
||||
target_namespace: str | None = None
|
||||
episodes: list[dict[str, Any]] = Field(default_factory=list)
|
||||
existing_memories: list[dict[str, Any]] = Field(default_factory=list)
|
||||
|
||||
|
||||
class MemoryIngestRequest(BaseModel):
|
||||
workspace_id: str | None = None
|
||||
user_id: str
|
||||
session_id: str
|
||||
turn_id: str
|
||||
role: str = "user"
|
||||
content: str
|
||||
metadata: dict[str, Any] = Field(default_factory=dict)
|
||||
source_type: str | None = None
|
||||
source_event_id: str | None = None
|
||||
|
||||
|
||||
app = FastAPI(title="Local EverMemOS POC Service", version="0.1.0")
|
||||
|
||||
|
||||
@app.get("/health")
|
||||
async def health() -> dict[str, Any]:
|
||||
return {
|
||||
"status": "ok",
|
||||
"service": "evermemos-local",
|
||||
"version": "0.1.0",
|
||||
"contract": "memory-gateway.evermemos.consolidate.v1",
|
||||
}
|
||||
|
||||
|
||||
@app.post("/api/v1/memories")
|
||||
async def ingest_memory(request: MemoryIngestRequest) -> dict[str, Any]:
|
||||
"""Accept message-level ingest for local real-adapter smoke tests.
|
||||
|
||||
This POC endpoint intentionally does not persist raw conversation content.
|
||||
It only returns a stable backend reference that Memory Gateway can store as
|
||||
control-plane metadata.
|
||||
"""
|
||||
seed = "|".join(
|
||||
[
|
||||
request.workspace_id or "",
|
||||
request.user_id,
|
||||
request.session_id,
|
||||
request.turn_id,
|
||||
request.source_event_id or "",
|
||||
]
|
||||
)
|
||||
memory_id = "em_" + hashlib.sha256(seed.encode("utf-8")).hexdigest()[:24]
|
||||
return {
|
||||
"status": "success",
|
||||
"memory_id": memory_id,
|
||||
"native_uri": f"evermemos://memories/{memory_id}",
|
||||
"metadata": {
|
||||
"schema_version": "evermemos.local.ingest.v1",
|
||||
"source_channel": request.metadata.get("source_channel") or request.metadata.get("channel"),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
@app.post("/v1/sessions/consolidate")
|
||||
async def consolidate_session(request: ConsolidateRequest) -> dict[str, Any]:
|
||||
repo = InMemoryRepository()
|
||||
ctx = AccessContext.model_validate(request.context)
|
||||
|
||||
for item in request.existing_memories:
|
||||
try:
|
||||
repo.upsert_memory(MemoryRecord.model_validate(item))
|
||||
except Exception as exc: # noqa: BLE001
|
||||
logger.warning("Skipping invalid existing memory: %s", exc)
|
||||
|
||||
for item in request.episodes:
|
||||
try:
|
||||
repo.append_episode(EpisodeRecord.model_validate(item))
|
||||
except Exception as exc: # noqa: BLE001
|
||||
logger.warning("Skipping invalid episode: %s", exc)
|
||||
|
||||
worker = EverMemOSWorker(repo)
|
||||
result = worker.consolidate_session(
|
||||
session_id=request.session_id,
|
||||
ctx=ctx,
|
||||
min_importance=request.min_importance,
|
||||
target_namespace=request.target_namespace,
|
||||
)
|
||||
return {
|
||||
"status": "ok",
|
||||
"backend": "evermemos-local",
|
||||
"result": {
|
||||
"session_id": result.session_id,
|
||||
"episodes": result.episodes,
|
||||
"candidates": [memory.model_dump(mode="json") for memory in result.candidates],
|
||||
"promoted": [memory.model_dump(mode="json") for memory in result.promoted],
|
||||
"duplicates": result.duplicates,
|
||||
"conflicts": result.conflicts,
|
||||
"review_drafts": result.review_drafts,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def main() -> None:
|
||||
import uvicorn
|
||||
|
||||
parser = argparse.ArgumentParser(description="Run the local EverMemOS POC service.")
|
||||
parser.add_argument("--config", default="config.yaml")
|
||||
parser.add_argument("--host", default="127.0.0.1")
|
||||
parser.add_argument("--port", type=int, default=1995)
|
||||
args = parser.parse_args()
|
||||
|
||||
config = load_config(args.config)
|
||||
set_config(config)
|
||||
uvicorn.run(app, host=args.host, port=args.port, log_level=config.logging.level.lower())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
496
memory_gateway/everos_client.py
Normal file
496
memory_gateway/everos_client.py
Normal file
@ -0,0 +1,496 @@
|
||||
"""Client for the external EverOS memory service."""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from json import JSONDecodeError
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
from .backend_contracts import BackendCommitResult, BackendOperation, BackendResultStatus, BackendRetrieveResult, BackendWriteResult
|
||||
from .backend_normalization import (
|
||||
map_backend_error_to_retryable,
|
||||
normalize_everos_commit_response,
|
||||
normalize_everos_ingest_response,
|
||||
normalize_everos_retrieve_response,
|
||||
)
|
||||
from .config import get_config
|
||||
from .schemas import AccessContext, EpisodeRecord, MemoryRecord
|
||||
from .schemas_v2 import BackendType
|
||||
|
||||
|
||||
class EverOSError(RuntimeError):
|
||||
"""Raised when the external EverOS service cannot process a request."""
|
||||
|
||||
|
||||
class EverOSClient:
|
||||
"""Small HTTP client with a tolerant response normalizer.
|
||||
|
||||
The deployed EverOS API may evolve independently from Memory Gateway.
|
||||
Gateway sends a stable payload and accepts several common response shapes:
|
||||
`result`, `data`, or the raw top-level object with `candidates/promoted`.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
base_url: str | None = None,
|
||||
api_key: str | None = None,
|
||||
timeout: int | None = None,
|
||||
enabled: bool | None = None,
|
||||
mode: str | None = None,
|
||||
verify_ssl: bool | None = None,
|
||||
health_path: str | None = None,
|
||||
ingest_path: str | None = None,
|
||||
search_path: str | None = None,
|
||||
flush_path: str | None = None,
|
||||
retrieve_method: str | None = None,
|
||||
transport: httpx.BaseTransport | None = None,
|
||||
) -> None:
|
||||
config = get_config().everos
|
||||
self.base_url = (base_url if base_url is not None else config.url).rstrip("/")
|
||||
self.api_key = api_key if api_key is not None else config.api_key
|
||||
self.timeout = timeout or config.timeout
|
||||
self.enabled = config.enabled if enabled is None else enabled
|
||||
self.mode = mode or config.mode
|
||||
self.verify_ssl = config.verify_ssl if verify_ssl is None else verify_ssl
|
||||
self.health_path = health_path or config.health_path
|
||||
self.ingest_path = ingest_path or config.ingest_path
|
||||
self.search_path = search_path or config.search_path
|
||||
self.flush_path = flush_path or config.flush_path
|
||||
self.retrieve_method = retrieve_method or config.retrieve_method
|
||||
self.transport = transport
|
||||
|
||||
def _headers(self) -> dict[str, str]:
|
||||
headers = {"Content-Type": "application/json"}
|
||||
if self.api_key:
|
||||
headers["X-API-Key"] = self.api_key
|
||||
headers["Authorization"] = f"Bearer {self.api_key}"
|
||||
return headers
|
||||
|
||||
def health(self) -> dict[str, Any]:
|
||||
url = self.base_url + self.health_path
|
||||
try:
|
||||
health_timeout = httpx.Timeout(min(self.timeout, 2.0), connect=min(self.timeout, 0.5))
|
||||
with httpx.Client(timeout=health_timeout, headers=self._headers()) as client:
|
||||
response = client.get(url)
|
||||
response.raise_for_status()
|
||||
return {"status": "ok", "url": self.base_url, "response": response.json()}
|
||||
except Exception as exc: # noqa: BLE001
|
||||
return {"status": "error", "url": self.base_url, "error": str(exc)}
|
||||
|
||||
def ingest_message(self, payload: dict[str, Any]) -> BackendWriteResult:
|
||||
"""Write one Gateway turn to EverOS."""
|
||||
runtime_payload = self._build_ingest_payload(payload)
|
||||
if self._use_real_api:
|
||||
return self._ingest_message_real(runtime_payload)
|
||||
raw = {
|
||||
"status": "skipped",
|
||||
"memory_id": (runtime_payload.get("messages") or [{}])[0].get("message_id"),
|
||||
"metadata": {
|
||||
"reason": "everos_v2_ingest_adapter_not_configured",
|
||||
"schema_version": "everos.fixture.ingest.v2",
|
||||
},
|
||||
}
|
||||
return self._normalize_ingest_response(raw)
|
||||
|
||||
@property
|
||||
def _use_real_api(self) -> bool:
|
||||
# Real ingest is strictly gated by mode=real. The legacy `enabled`
|
||||
# field is retained for config compatibility, but must not trigger
|
||||
# network traffic by itself.
|
||||
return self.mode == "real"
|
||||
|
||||
def _ingest_message_real(self, runtime_payload: dict[str, Any]) -> BackendWriteResult:
|
||||
if not self.base_url:
|
||||
return self._failed_ingest_result(
|
||||
error_code="config_error",
|
||||
error_message="EverOS real ingest is enabled but base_url is missing",
|
||||
retryable=False,
|
||||
)
|
||||
try:
|
||||
with httpx.Client(
|
||||
base_url=self.base_url,
|
||||
headers=self._headers(),
|
||||
timeout=self.timeout,
|
||||
verify=self.verify_ssl,
|
||||
transport=self.transport,
|
||||
) as client:
|
||||
response = client.post(self.ingest_path, json=runtime_payload)
|
||||
if response.status_code >= 400:
|
||||
return self._failed_ingest_result(
|
||||
error_code=f"http_{response.status_code}",
|
||||
error_message=f"EverOS ingest failed with HTTP {response.status_code}",
|
||||
retryable=self._map_error(response),
|
||||
)
|
||||
try:
|
||||
raw = response.json()
|
||||
except (JSONDecodeError, ValueError):
|
||||
return self._failed_ingest_result(
|
||||
error_code="invalid_json",
|
||||
error_message="EverOS ingest returned invalid JSON",
|
||||
retryable=True,
|
||||
)
|
||||
if not isinstance(raw, dict):
|
||||
return self._failed_ingest_result(
|
||||
error_code="unexpected_response",
|
||||
error_message="EverOS ingest returned an unexpected response shape",
|
||||
retryable=True,
|
||||
)
|
||||
return self._normalize_ingest_response(raw)
|
||||
except httpx.TimeoutException as exc:
|
||||
return self._failed_ingest_result("timeout", self._safe_error_message(exc), retryable=self._map_error(exc))
|
||||
except httpx.RequestError as exc:
|
||||
return self._failed_ingest_result("network_error", self._safe_error_message(exc), retryable=self._map_error(exc))
|
||||
except Exception as exc: # noqa: BLE001
|
||||
return self._failed_ingest_result("unexpected_error", self._safe_error_message(exc), retryable=self._map_error(exc))
|
||||
|
||||
def extract_profile_long_term_v2(self, payload: dict[str, Any]) -> BackendCommitResult:
|
||||
"""v2 adapter placeholder for profile / long-term extraction.
|
||||
|
||||
Mapping spec: commit_session returns BackendCommitResult and should
|
||||
produce native episodic/profile/long-term refs once the real API is stable.
|
||||
"""
|
||||
runtime_payload = self._build_commit_payload(payload)
|
||||
raw = {
|
||||
"status": "success",
|
||||
"session_id": runtime_payload.get("session_id"),
|
||||
"metadata": {
|
||||
"reason": "everos_v2_commit_fixture",
|
||||
"schema_version": "everos.fixture.commit.v2",
|
||||
},
|
||||
"data": {
|
||||
"produced_refs": [
|
||||
{
|
||||
"ref_type": "profile",
|
||||
"profile_id": f"everos_profile:{runtime_payload.get('user_id') or 'unknown'}",
|
||||
"metadata": {"schema_version": "everos.fixture.profile.v2"},
|
||||
},
|
||||
{
|
||||
"ref_type": "long_term_memory",
|
||||
"memory_id": f"everos_long_term:{runtime_payload.get('session_id')}",
|
||||
"metadata": {"schema_version": "everos.fixture.long_term.v2"},
|
||||
},
|
||||
]
|
||||
},
|
||||
}
|
||||
return self._normalize_commit_response(raw)
|
||||
|
||||
def retrieve_context_v2(self, payload: dict[str, Any]) -> BackendRetrieveResult:
|
||||
"""
|
||||
Calls EverOS native API to retrieve memories.
|
||||
"""
|
||||
if not self._use_real_api:
|
||||
return BackendRetrieveResult(
|
||||
backend_type=BackendType.EVEROS,
|
||||
operation=BackendOperation.RETRIEVE_CONTEXT,
|
||||
status=BackendResultStatus.SKIPPED,
|
||||
items=[],
|
||||
metadata={"reason": "everos_retrieve_requires_real_mode"},
|
||||
)
|
||||
|
||||
query = payload.get("query", "")
|
||||
user_id = payload.get("user_id", "")
|
||||
try:
|
||||
with httpx.Client(
|
||||
base_url=self.base_url,
|
||||
headers=self._headers(),
|
||||
timeout=self.timeout,
|
||||
verify=self.verify_ssl,
|
||||
transport=self.transport,
|
||||
) as client:
|
||||
resp = client.post(
|
||||
self.search_path,
|
||||
json={
|
||||
"query": query,
|
||||
"method": self.retrieve_method,
|
||||
"memory_types": ["episodic_memory", "profile", "raw_message"],
|
||||
"top_k": payload.get("limit", 10),
|
||||
"filters": self._search_filters(user_id=user_id, session_id=payload.get("session_id")),
|
||||
},
|
||||
)
|
||||
if resp.status_code >= 400:
|
||||
return BackendRetrieveResult(
|
||||
backend_type=BackendType.EVEROS,
|
||||
operation=BackendOperation.RETRIEVE_CONTEXT,
|
||||
status=BackendResultStatus.FAILED,
|
||||
items=[],
|
||||
error_code=f"http_{resp.status_code}",
|
||||
error_message=f"EverOS retrieve failed: {resp.text}",
|
||||
retryable=False
|
||||
)
|
||||
|
||||
items = self._items_from_search_response(resp.json())
|
||||
|
||||
raw = {
|
||||
"status": "success",
|
||||
"data": {
|
||||
"items": items
|
||||
}
|
||||
}
|
||||
return self._normalize_retrieve_response(raw)
|
||||
except Exception as exc:
|
||||
return BackendRetrieveResult(
|
||||
backend_type=BackendType.EVEROS,
|
||||
operation=BackendOperation.RETRIEVE_CONTEXT,
|
||||
status=BackendResultStatus.FAILED,
|
||||
items=[],
|
||||
error_code="request_error",
|
||||
error_message=str(exc),
|
||||
retryable=True
|
||||
)
|
||||
|
||||
def _build_ingest_payload(self, payload: dict[str, Any]) -> dict[str, Any]:
|
||||
"""
|
||||
Builds the payload according to EverOS native message schema.
|
||||
"""
|
||||
return {
|
||||
"user_id": payload.get("user_id") or "gateway_user",
|
||||
"session_id": payload.get("session_id"),
|
||||
"messages": [
|
||||
{
|
||||
"message_id": payload.get("turn_id") or f"msg_{int(datetime.now(timezone.utc).timestamp() * 1000)}",
|
||||
"sender_id": payload.get("user_id") or "gateway_user",
|
||||
"sender_name": payload.get("user_id") or "gateway_user",
|
||||
"role": self._everos_role(payload.get("role", "user")),
|
||||
"timestamp": self._timestamp_ms(payload),
|
||||
"content": payload.get("content", ""),
|
||||
}
|
||||
],
|
||||
}
|
||||
|
||||
def _build_commit_payload(self, payload: dict[str, Any]) -> dict[str, Any]:
|
||||
return dict(payload)
|
||||
|
||||
def _normalize_ingest_response(self, raw: dict[str, Any]) -> BackendWriteResult:
|
||||
return normalize_everos_ingest_response(raw)
|
||||
|
||||
def _normalize_commit_response(self, raw: dict[str, Any]) -> BackendCommitResult:
|
||||
return normalize_everos_commit_response(raw)
|
||||
|
||||
def _normalize_retrieve_response(self, raw: dict[str, Any]) -> BackendRetrieveResult:
|
||||
return normalize_everos_retrieve_response(raw)
|
||||
|
||||
def _map_error(self, exc_or_response: Any) -> bool:
|
||||
status_code = getattr(exc_or_response, "status_code", None)
|
||||
error_code = getattr(exc_or_response, "error_code", None)
|
||||
error_message = str(exc_or_response) if exc_or_response is not None else None
|
||||
return map_backend_error_to_retryable(
|
||||
BackendType.EVEROS,
|
||||
status_code=status_code,
|
||||
error_code=error_code,
|
||||
error_message=error_message,
|
||||
)
|
||||
|
||||
def _failed_ingest_result(self, error_code: str, error_message: str, retryable: bool) -> BackendWriteResult:
|
||||
return BackendWriteResult(
|
||||
backend_type=BackendType.EVEROS,
|
||||
operation=BackendOperation.INGEST_TURN,
|
||||
status=BackendResultStatus.FAILED,
|
||||
retryable=retryable,
|
||||
error_code=error_code,
|
||||
error_message=error_message,
|
||||
metadata={"error_code": error_code},
|
||||
)
|
||||
|
||||
def _safe_error_message(self, exc: Exception) -> str:
|
||||
return exc.__class__.__name__
|
||||
|
||||
def consolidate_session(
|
||||
self,
|
||||
session_id: str,
|
||||
ctx: AccessContext,
|
||||
episodes: list[EpisodeRecord],
|
||||
existing_memories: list[MemoryRecord],
|
||||
min_importance: float,
|
||||
target_namespace: str | None,
|
||||
) -> dict[str, Any]:
|
||||
if not self.base_url:
|
||||
raise EverOSError("EverOS real mode requires base_url")
|
||||
|
||||
user_id = ctx.user_id or "gateway_user"
|
||||
agent_id = ctx.agent_id or "gateway_agent"
|
||||
|
||||
with httpx.Client(
|
||||
base_url=self.base_url,
|
||||
timeout=self.timeout,
|
||||
headers=self._headers(),
|
||||
verify=self.verify_ssl,
|
||||
transport=self.transport,
|
||||
) as client:
|
||||
for episode in episodes:
|
||||
self._memorize_episode(client, episode=episode, session_id=session_id, user_id=user_id, agent_id=agent_id)
|
||||
self._flush_session(client, session_id=session_id, user_id=user_id)
|
||||
promoted = self._fetch_session_memories(client, session_id=session_id, user_id=user_id, target_namespace=target_namespace)
|
||||
|
||||
return {
|
||||
"backend": "external",
|
||||
"service_url": self.base_url,
|
||||
"endpoint": self.ingest_path,
|
||||
"raw": {"result": {"memories": promoted}},
|
||||
"session_id": session_id,
|
||||
"episodes": len(episodes),
|
||||
"candidates": promoted,
|
||||
"promoted": promoted,
|
||||
"duplicates": [],
|
||||
"conflicts": [],
|
||||
"review_drafts": [],
|
||||
}
|
||||
|
||||
def _memorize_episode(
|
||||
self,
|
||||
client: httpx.Client,
|
||||
*,
|
||||
episode: dict[str, Any],
|
||||
session_id: str,
|
||||
user_id: str,
|
||||
agent_id: str,
|
||||
) -> None:
|
||||
episode_data = episode.model_dump(mode="json") if hasattr(episode, "model_dump") else dict(episode)
|
||||
episode_id = str(episode_data.get("id") or f"epi_{int(datetime.now(timezone.utc).timestamp())}")
|
||||
sender = agent_id if episode_data.get("source") == "agent" else user_id
|
||||
role = "assistant" if sender == agent_id else "user"
|
||||
created_at = episode_data.get("created_at") or datetime.now(timezone.utc).isoformat()
|
||||
payload = {
|
||||
"user_id": user_id,
|
||||
"session_id": session_id,
|
||||
"messages": [
|
||||
{
|
||||
"message_id": episode_id,
|
||||
"sender_id": sender,
|
||||
"sender_name": sender,
|
||||
"role": role,
|
||||
"timestamp": self._datetime_to_ms(created_at),
|
||||
"content": episode_data.get("content") or "",
|
||||
}
|
||||
],
|
||||
}
|
||||
response = client.post(self.ingest_path, json=payload)
|
||||
response.raise_for_status()
|
||||
|
||||
def _flush_session(self, client: httpx.Client, *, session_id: str, user_id: str) -> None:
|
||||
response = client.post(self.flush_path, json={"user_id": user_id, "session_id": session_id})
|
||||
response.raise_for_status()
|
||||
|
||||
def _fetch_session_memories(
|
||||
self,
|
||||
client: httpx.Client,
|
||||
*,
|
||||
session_id: str,
|
||||
user_id: str,
|
||||
target_namespace: str | None,
|
||||
) -> list[dict[str, Any]]:
|
||||
response = client.post(
|
||||
self.search_path,
|
||||
json={
|
||||
"query": "memory",
|
||||
"method": self.retrieve_method,
|
||||
"memory_types": ["episodic_memory"],
|
||||
"top_k": 20,
|
||||
"filters": self._search_filters(user_id=user_id, session_id=session_id),
|
||||
},
|
||||
)
|
||||
response.raise_for_status()
|
||||
memories = self._items_from_search_response(response.json())
|
||||
normalized: list[dict[str, Any]] = []
|
||||
for index, memory in enumerate(memories, start=1):
|
||||
content = memory.get("text") or memory.get("content") or memory.get("summary") or ""
|
||||
if not content:
|
||||
continue
|
||||
normalized.append(
|
||||
{
|
||||
"id": memory.get("memory_id") or memory.get("id") or f"everos_{session_id}_{index}",
|
||||
"namespace": target_namespace or f"user/{user_id}/long_term",
|
||||
"memory_type": memory.get("memory_type") or "episodic_memory",
|
||||
"content": content,
|
||||
"summary": memory.get("summary") or content[:180],
|
||||
"tags": ["everos-real", "memory-gateway"],
|
||||
"importance": 0.7,
|
||||
"confidence": 0.7,
|
||||
"source": "everos",
|
||||
"source_ref": memory.get("memory_id") or memory.get("id"),
|
||||
}
|
||||
)
|
||||
return normalized
|
||||
|
||||
def _items_from_search_response(self, payload: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
data = payload.get("data") if isinstance(payload.get("data"), dict) else payload
|
||||
items: list[dict[str, Any]] = []
|
||||
for memory_type, key in (
|
||||
("episodic_memory", "episodes"),
|
||||
("profile", "profiles"),
|
||||
("raw_message", "raw_messages"),
|
||||
):
|
||||
for item in data.get(key, []) or []:
|
||||
if isinstance(item, dict):
|
||||
items.append({**item, "memory_type": item.get("memory_type") or memory_type, "text": self._memory_text(item)})
|
||||
agent_memory = data.get("agent_memory") or {}
|
||||
if isinstance(agent_memory, dict):
|
||||
for item in agent_memory.get("cases", []) or []:
|
||||
if isinstance(item, dict):
|
||||
items.append({**item, "memory_type": "agent_case", "text": self._memory_text(item)})
|
||||
for item in agent_memory.get("skills", []) or []:
|
||||
if isinstance(item, dict):
|
||||
items.append({**item, "memory_type": "agent_skill", "text": self._memory_text(item)})
|
||||
return items
|
||||
|
||||
def _memory_text(self, item: dict[str, Any]) -> str:
|
||||
content_items = item.get("content_items")
|
||||
if isinstance(content_items, list):
|
||||
content_text = "\n".join(
|
||||
str(content.get("text") or content.get("content") or "")
|
||||
for content in content_items
|
||||
if isinstance(content, dict)
|
||||
).strip()
|
||||
else:
|
||||
content_text = ""
|
||||
profile_data = item.get("profile_data")
|
||||
if isinstance(profile_data, dict):
|
||||
profile_text = str(profile_data)
|
||||
else:
|
||||
profile_text = ""
|
||||
return (
|
||||
item.get("episode")
|
||||
or item.get("summary")
|
||||
or item.get("subject")
|
||||
or item.get("atomic_fact")
|
||||
or item.get("task_intent")
|
||||
or item.get("approach")
|
||||
or item.get("content")
|
||||
or content_text
|
||||
or item.get("description")
|
||||
or profile_text
|
||||
or ""
|
||||
)
|
||||
|
||||
def _search_filters(self, *, user_id: str | None, session_id: str | None = None) -> dict[str, Any]:
|
||||
filters: dict[str, Any] = {"user_id": user_id or "gateway_user"}
|
||||
if session_id:
|
||||
filters["session_id"] = session_id
|
||||
return filters
|
||||
|
||||
def _timestamp_ms(self, payload: dict[str, Any]) -> int:
|
||||
trace = payload.get("trace") if isinstance(payload.get("trace"), dict) else {}
|
||||
timestamp = trace.get("timestamp") or payload.get("created_at")
|
||||
if timestamp:
|
||||
return self._datetime_to_ms(timestamp)
|
||||
return int(datetime.now(timezone.utc).timestamp() * 1000)
|
||||
|
||||
def _datetime_to_ms(self, value: Any) -> int:
|
||||
if isinstance(value, (int, float)):
|
||||
return int(value if value > 1_000_000_000_000 else value * 1000)
|
||||
if isinstance(value, str):
|
||||
text = value.replace("Z", "+00:00")
|
||||
try:
|
||||
return int(datetime.fromisoformat(text).timestamp() * 1000)
|
||||
except ValueError:
|
||||
return int(datetime.now(timezone.utc).timestamp() * 1000)
|
||||
if isinstance(value, datetime):
|
||||
return int(value.timestamp() * 1000)
|
||||
return int(datetime.now(timezone.utc).timestamp() * 1000)
|
||||
|
||||
def _everos_role(self, role: str) -> str:
|
||||
if role in {"assistant", "agent"}:
|
||||
return "assistant"
|
||||
if role == "tool":
|
||||
return "assistant"
|
||||
return "user"
|
||||
@ -43,7 +43,7 @@ def write_review_draft(memory: MemoryRecord, reason: str, conflict_ids: list[str
|
||||
f"created_at: {datetime.now(timezone.utc).isoformat()}",
|
||||
"tags:",
|
||||
" - memory/review",
|
||||
" - source/evermemos",
|
||||
" - source/everos",
|
||||
"---",
|
||||
"",
|
||||
f"# Memory Review - {title}",
|
||||
|
||||
@ -1,7 +1,6 @@
|
||||
"""OpenViking client wrapper used by Memory Gateway."""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import mimetypes
|
||||
import tempfile
|
||||
@ -58,6 +57,7 @@ class OpenVikingClient:
|
||||
headers = {}
|
||||
if self.api_key:
|
||||
headers["X-API-Key"] = self.api_key
|
||||
headers["Authorization"] = f"Bearer {self.api_key}"
|
||||
headers["X-OpenViking-Account"] = self.account
|
||||
headers["X-OpenViking-User"] = self.user
|
||||
return headers
|
||||
@ -190,36 +190,64 @@ class OpenVikingClient:
|
||||
return self._normalize_commit_response(raw)
|
||||
|
||||
async def retrieve_context_v2(self, payload: dict[str, Any]) -> BackendRetrieveResult:
|
||||
"""v2 adapter placeholder for OpenViking runtime context retrieval.
|
||||
|
||||
Mapping spec: retrieve_context returns BackendRetrieveResult with
|
||||
runtime context items, not raw backend payload dumps.
|
||||
"""
|
||||
raw = {
|
||||
"status": "ok",
|
||||
"session_id": payload.get("session_id"),
|
||||
"metadata": {
|
||||
"reason": "openviking_v2_retrieve_fixture",
|
||||
"schema_version": "openviking.fixture.retrieve.v2",
|
||||
},
|
||||
"result": {
|
||||
"items": [
|
||||
{
|
||||
"text": "OpenViking fixture runtime context.",
|
||||
"ref_id": f"ov_context:{payload.get('session_id') or 'unknown'}",
|
||||
"score": 0.75,
|
||||
"memory_type": "context_resource",
|
||||
"metadata": {"schema_version": "openviking.fixture.retrieve.item.v2"},
|
||||
}
|
||||
]
|
||||
},
|
||||
}
|
||||
return self._normalize_retrieve_response(raw)
|
||||
Calls OpenViking native API to retrieve context.
|
||||
Uses POST /search
|
||||
"""
|
||||
if not self._use_real_api:
|
||||
return BackendRetrieveResult(
|
||||
backend_type=BackendType.OPENVIKING,
|
||||
operation=BackendOperation.RETRIEVE_CONTEXT,
|
||||
status=BackendResultStatus.SKIPPED,
|
||||
items=[],
|
||||
metadata={"reason": "openviking_retrieve_requires_real_mode"},
|
||||
)
|
||||
|
||||
query = payload.get("query", "")
|
||||
session_id = payload.get("session_id")
|
||||
|
||||
request_data = {"query": query, "limit": 10}
|
||||
if session_id:
|
||||
request_data["session_id"] = session_id
|
||||
|
||||
try:
|
||||
client = await self._get_client()
|
||||
response = await client.post("/api/v1/search/search", json=request_data)
|
||||
|
||||
if response.status_code >= 400:
|
||||
return BackendRetrieveResult(
|
||||
backend_type=BackendType.OPENVIKING,
|
||||
operation=BackendOperation.RETRIEVE_CONTEXT,
|
||||
status=BackendResultStatus.FAILED,
|
||||
items=[],
|
||||
error_code=f"http_{response.status_code}",
|
||||
error_message=f"OpenViking search failed: {response.text}",
|
||||
retryable=False
|
||||
)
|
||||
|
||||
return self._normalize_retrieve_response(response.json())
|
||||
except Exception as exc:
|
||||
return BackendRetrieveResult(
|
||||
backend_type=BackendType.OPENVIKING,
|
||||
operation=BackendOperation.RETRIEVE_CONTEXT,
|
||||
status=BackendResultStatus.FAILED,
|
||||
items=[],
|
||||
error_code="request_error",
|
||||
error_message=str(exc),
|
||||
retryable=True
|
||||
)
|
||||
|
||||
def _build_ingest_payload(self, payload: dict[str, Any]) -> dict[str, Any]:
|
||||
# Runtime-only adapter payload. It may include conversation content for
|
||||
# the current request lifecycle; callers must not persist it to SQLite.
|
||||
return dict(payload)
|
||||
"""
|
||||
Build payload for native OpenViking AddMessageRequest.
|
||||
OpenViking only expects role and content, and maybe metadata.
|
||||
"""
|
||||
return {
|
||||
"role": payload.get("role", "user"),
|
||||
"content": payload.get("content", ""),
|
||||
"metadata": payload.get("metadata", {}),
|
||||
"session_id": payload.get("session_id") # kept so format_ingest_path can use it
|
||||
}
|
||||
|
||||
def _format_ingest_path(self, payload: dict[str, Any]) -> str:
|
||||
session_id = str(payload.get("session_id") or "unknown")
|
||||
@ -277,9 +305,9 @@ class OpenVikingClient:
|
||||
payload["limit"] = limit
|
||||
|
||||
if uri:
|
||||
payload["uri"] = uri
|
||||
payload["target_uri"] = uri
|
||||
elif namespace:
|
||||
payload["uri"] = f"viking://{namespace}"
|
||||
payload["target_uri"] = f"viking://{namespace}"
|
||||
|
||||
try:
|
||||
response = await client.post("/api/v1/search/search", json=payload)
|
||||
@ -321,7 +349,7 @@ class OpenVikingClient:
|
||||
ns = namespace or self.config.memory.default_namespace or "user/default/memories"
|
||||
|
||||
try:
|
||||
response = await client.post("/api/v1/sessions", json={"mode": "interactive"})
|
||||
response = await client.post("/api/v1/sessions")
|
||||
response.raise_for_status()
|
||||
session_data = response.json()
|
||||
|
||||
@ -329,17 +357,15 @@ class OpenVikingClient:
|
||||
return session_data
|
||||
|
||||
session_id = session_data["result"]["session_id"]
|
||||
commit_response = await client.post(
|
||||
f"/api/v1/sessions/{session_id}/commit",
|
||||
message_response = await client.post(
|
||||
f"/api/v1/sessions/{session_id}/messages",
|
||||
json={
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": f"[{ns}/{memory_type}] {content}",
|
||||
}
|
||||
]
|
||||
"role": "user",
|
||||
"content": f"[{ns}/{memory_type}] {content}",
|
||||
},
|
||||
)
|
||||
message_response.raise_for_status()
|
||||
commit_response = await client.post(f"/api/v1/sessions/{session_id}/commit")
|
||||
commit_response.raise_for_status()
|
||||
return commit_response.json()
|
||||
except httpx.HTTPError as e:
|
||||
@ -396,7 +422,6 @@ class OpenVikingClient:
|
||||
"temp_path": temp_ref,
|
||||
"to": uri,
|
||||
"wait": wait,
|
||||
"source_name": Path(uri).name or tmp_path.name,
|
||||
"strict": False,
|
||||
}
|
||||
response = await client.post("/api/v1/resources", json=payload)
|
||||
@ -425,7 +450,7 @@ class OpenVikingClient:
|
||||
try:
|
||||
response = await client.post(
|
||||
"/api/v1/search/search",
|
||||
json={"query": "", "uri": f"viking://{ns}", "limit": limit or 10},
|
||||
json={"query": "", "target_uri": f"viking://{ns}", "limit": limit or 10},
|
||||
)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
@ -458,7 +483,7 @@ class OpenVikingClient:
|
||||
try:
|
||||
response = await client.post(
|
||||
"/api/v1/search/search",
|
||||
json={"query": "", "uri": uri, "limit": limit or 10},
|
||||
json={"query": "", "target_uri": uri, "limit": limit or 10},
|
||||
)
|
||||
response.raise_for_status()
|
||||
data = response.json()
|
||||
|
||||
@ -38,7 +38,7 @@ class SourceType(str, Enum):
|
||||
AGENT = "agent"
|
||||
OBSIDIAN = "obsidian"
|
||||
OPENVIKING = "openviking"
|
||||
EVERMEMOS = "evermemos"
|
||||
EVEROS = "everos"
|
||||
MANUAL = "manual"
|
||||
|
||||
|
||||
@ -224,4 +224,3 @@ class NamespaceInfo(BaseModel):
|
||||
owner_user_id: Optional[str] = None
|
||||
visibility: Visibility
|
||||
description: str
|
||||
|
||||
|
||||
@ -30,7 +30,7 @@ class BackendRefStatus(str, Enum):
|
||||
|
||||
class BackendType(str, Enum):
|
||||
OPENVIKING = "openviking"
|
||||
EVERMEMOS = "evermemos"
|
||||
EVEROS = "everos"
|
||||
OBSIDIAN = "obsidian"
|
||||
|
||||
|
||||
@ -54,7 +54,7 @@ class TraceContext(BaseModel):
|
||||
|
||||
class IngestPolicy(BaseModel):
|
||||
allow_openviking: bool = True
|
||||
allow_evermemos: bool = True
|
||||
allow_everos: bool = True
|
||||
allow_obsidian_review: bool = False
|
||||
redact_sensitive: bool = True
|
||||
require_human_review: bool = False
|
||||
|
||||
@ -478,12 +478,12 @@ async def health_check():
|
||||
try:
|
||||
ov_client = await get_openviking_client()
|
||||
ov_status = await ov_client.health_check()
|
||||
evermemos_status = v1_service.evermemos_health()
|
||||
everos_status = v1_service.everos_health()
|
||||
return {
|
||||
"status": "ok",
|
||||
"gateway": "memory-gateway",
|
||||
"openviking": ov_status,
|
||||
"evermemos": evermemos_status,
|
||||
"everos": everos_status,
|
||||
}
|
||||
except Exception as e:
|
||||
return {
|
||||
|
||||
@ -1,12 +1,13 @@
|
||||
"""Application services for the generic Memory Gateway v1 API."""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from fastapi import HTTPException, status
|
||||
|
||||
from .config import get_config
|
||||
from .evermemos_client import EverMemOSError, EverMemOSClient
|
||||
from .everos_client import EverOSError, EverOSClient
|
||||
from .namespace import can_access_memory, default_namespace_for_context, user_long_term_namespace, visible_namespaces
|
||||
from .openviking_client import get_openviking_client
|
||||
from .repositories import MetadataRepository, repository
|
||||
@ -29,13 +30,23 @@ from .schemas import (
|
||||
UserRecord,
|
||||
Visibility,
|
||||
)
|
||||
from .workers.evermemos_worker import EverMemOSWorker
|
||||
|
||||
|
||||
@dataclass
|
||||
class ConsolidationResult:
|
||||
session_id: str
|
||||
episodes: int
|
||||
candidates: list[MemoryRecord] = field(default_factory=list)
|
||||
promoted: list[MemoryRecord] = field(default_factory=list)
|
||||
duplicates: list[dict] = field(default_factory=list)
|
||||
review_drafts: list[str] = field(default_factory=list)
|
||||
conflicts: list[dict] = field(default_factory=list)
|
||||
|
||||
|
||||
class MemoryGatewayService:
|
||||
def __init__(self, repo: MetadataRepository = repository, evermemos_client: EverMemOSClient | None = None) -> None:
|
||||
def __init__(self, repo: MetadataRepository = repository, everos_client: EverOSClient | None = None) -> None:
|
||||
self.repo = repo
|
||||
self.evermemos_client = evermemos_client
|
||||
self.everos_client = everos_client
|
||||
|
||||
def create_user(self, request: CreateUserRequest) -> UserRecord:
|
||||
user = UserRecord(
|
||||
@ -204,10 +215,10 @@ class MemoryGatewayService:
|
||||
session_id=session_id,
|
||||
)
|
||||
target_namespace = request.target_namespace or user_long_term_namespace(request.user_id)
|
||||
config = get_config().evermemos
|
||||
config = get_config().everos
|
||||
if config.enabled:
|
||||
try:
|
||||
external_result = (self.evermemos_client or EverMemOSClient()).consolidate_session(
|
||||
external_result = (self.everos_client or EverOSClient()).consolidate_session(
|
||||
session_id=session_id,
|
||||
ctx=ctx,
|
||||
episodes=episodes,
|
||||
@ -217,32 +228,29 @@ class MemoryGatewayService:
|
||||
)
|
||||
result = self._persist_external_consolidation(external_result, ctx, session_id)
|
||||
backend = "external"
|
||||
except EverMemOSError as exc:
|
||||
except EverOSError as exc:
|
||||
error = str(exc)
|
||||
if not config.fallback_to_local:
|
||||
self._audit(
|
||||
"evermemos_commit_failed",
|
||||
"session",
|
||||
session_id,
|
||||
actor_user_id=request.user_id,
|
||||
actor_agent_id=request.agent_id,
|
||||
decision="deny",
|
||||
metadata={"error": error},
|
||||
)
|
||||
raise HTTPException(status_code=status.HTTP_502_BAD_GATEWAY, detail=f"EverMemOS failed: {error}") from exc
|
||||
result = self._commit_session_locally(session_id, ctx, request)
|
||||
backend = "local-fallback"
|
||||
self._audit(
|
||||
"everos_commit_failed",
|
||||
"session",
|
||||
session_id,
|
||||
actor_user_id=request.user_id,
|
||||
actor_agent_id=request.agent_id,
|
||||
decision="deny",
|
||||
metadata={"error": error},
|
||||
)
|
||||
raise HTTPException(status_code=status.HTTP_502_BAD_GATEWAY, detail=f"EverOS failed: {error}") from exc
|
||||
else:
|
||||
result = self._commit_session_locally(session_id, ctx, request)
|
||||
backend = "local-disabled"
|
||||
result = None
|
||||
backend = "disabled"
|
||||
else:
|
||||
result = None
|
||||
self._audit("commit_session", "session", session_id, actor_user_id=request.user_id, actor_agent_id=request.agent_id)
|
||||
if not result:
|
||||
return {"session_id": session_id, "episodes": len(episodes), "promoted": [], "evermemos_backend": backend}
|
||||
return {"session_id": session_id, "episodes": len(episodes), "promoted": [], "everos_backend": backend}
|
||||
return {
|
||||
"evermemos_backend": backend,
|
||||
"evermemos_error": error,
|
||||
"everos_backend": backend,
|
||||
"everos_error": error,
|
||||
"session_id": session_id,
|
||||
"episodes": result.episodes,
|
||||
"candidates": result.candidates,
|
||||
@ -252,24 +260,13 @@ class MemoryGatewayService:
|
||||
"review_drafts": result.review_drafts,
|
||||
}
|
||||
|
||||
def evermemos_health(self) -> dict:
|
||||
config = get_config().evermemos
|
||||
def everos_health(self) -> dict:
|
||||
config = get_config().everos
|
||||
if not config.enabled:
|
||||
return {"status": "disabled", "url": config.url}
|
||||
return (self.evermemos_client or EverMemOSClient()).health()
|
||||
|
||||
def _commit_session_locally(self, session_id: str, ctx: AccessContext, request: CommitSessionRequest):
|
||||
worker = EverMemOSWorker(self.repo)
|
||||
return worker.consolidate_session(
|
||||
session_id=session_id,
|
||||
ctx=ctx,
|
||||
min_importance=request.min_importance,
|
||||
target_namespace=request.target_namespace or user_long_term_namespace(request.user_id),
|
||||
)
|
||||
return (self.everos_client or EverOSClient()).health()
|
||||
|
||||
def _persist_external_consolidation(self, external_result: dict, ctx: AccessContext, session_id: str):
|
||||
from .workers.evermemos_worker import ConsolidationResult
|
||||
|
||||
result = ConsolidationResult(
|
||||
session_id=session_id,
|
||||
episodes=external_result.get("episodes") or len(self.repo.list_session_episodes(session_id)),
|
||||
@ -302,11 +299,11 @@ class MemoryGatewayService:
|
||||
data.setdefault("memory_type", MemoryType.SUMMARY.value)
|
||||
data.setdefault("content", data.get("text") or data.get("summary") or "")
|
||||
data.setdefault("summary", data.get("content", "")[:180])
|
||||
data.setdefault("tags", ["evermemos-external"])
|
||||
data.setdefault("tags", ["everos-external"])
|
||||
data.setdefault("importance", 0.7)
|
||||
data.setdefault("confidence", 0.65)
|
||||
data.setdefault("visibility", Visibility.PRIVATE.value)
|
||||
data.setdefault("source", SourceType.EVERMEMOS.value)
|
||||
data.setdefault("source", SourceType.EVEROS.value)
|
||||
if not data["content"]:
|
||||
return None
|
||||
return MemoryRecord.model_validate(data)
|
||||
|
||||
@ -12,13 +12,14 @@ from .backend_contracts import (
|
||||
BackendOperation,
|
||||
BackendCommitResult,
|
||||
BackendProducedRef,
|
||||
BackendRetrieveResult,
|
||||
BackendResultStatus,
|
||||
BackendWriteResult,
|
||||
CommitJob,
|
||||
OutboxEvent,
|
||||
OutboxEventStatus,
|
||||
)
|
||||
from .evermemos_client import EverMemOSClient
|
||||
from .everos_client import EverOSClient
|
||||
from .openviking_client import get_openviking_client
|
||||
from .repositories import MetadataRepository, repository
|
||||
from .schemas import AuditLog
|
||||
@ -52,11 +53,11 @@ class MemoryGatewayV2Service:
|
||||
self,
|
||||
repo: MetadataRepository = repository,
|
||||
openviking_client_factory: OpenVikingClientFactory = get_openviking_client,
|
||||
evermemos_client: Any | None = None,
|
||||
everos_client: Any | None = None,
|
||||
) -> None:
|
||||
self.repo = repo
|
||||
self.openviking_client_factory = openviking_client_factory
|
||||
self.evermemos_client = evermemos_client
|
||||
self.everos_client = everos_client
|
||||
|
||||
async def ingest_conversation_turn(self, request: IngestRequest) -> IngestResponse:
|
||||
normalized = self._normalize_ingest_request(request)
|
||||
@ -92,9 +93,9 @@ class MemoryGatewayV2Service:
|
||||
)
|
||||
)
|
||||
|
||||
if normalized.policy.allow_evermemos:
|
||||
if normalized.policy.allow_everos:
|
||||
refs.append(
|
||||
await self._write_evermemos_message(
|
||||
await self._write_everos_message(
|
||||
normalized,
|
||||
payload,
|
||||
gateway_id=gateway_id,
|
||||
@ -108,7 +109,7 @@ class MemoryGatewayV2Service:
|
||||
normalized,
|
||||
gateway_id,
|
||||
provenance_id,
|
||||
BackendType.EVERMEMOS,
|
||||
BackendType.EVEROS,
|
||||
MemoryRefType.MESSAGE_MEMORY,
|
||||
BackendRefStatus.SKIPPED,
|
||||
content_hash=content_hash,
|
||||
@ -188,8 +189,21 @@ class MemoryGatewayV2Service:
|
||||
)
|
||||
|
||||
async def retrieve_context(self, request: RetrieveRequest) -> RetrieveResponse:
|
||||
# TODO(v2): expand namespace ACL, fan out concurrently to OpenViking and
|
||||
# EverMemOS, then apply lightweight merge/rerank before returning.
|
||||
payload = {
|
||||
"workspace_id": request.workspace_id,
|
||||
"user_id": request.user_id,
|
||||
"agent_id": request.agent_id,
|
||||
"session_id": request.session_id,
|
||||
"namespace": request.namespace,
|
||||
"query": request.query,
|
||||
"limit": request.limit,
|
||||
"metadata": request.metadata,
|
||||
}
|
||||
results = [
|
||||
await self._retrieve_openviking_context(payload),
|
||||
await self._retrieve_everos_context(payload),
|
||||
]
|
||||
items = self._merge_retrieve_items(results, limit=request.limit)
|
||||
refs = self.repo.list_memory_refs(
|
||||
workspace_id=request.workspace_id,
|
||||
user_id=request.user_id,
|
||||
@ -198,21 +212,6 @@ class MemoryGatewayV2Service:
|
||||
namespace=request.namespace,
|
||||
limit=request.limit,
|
||||
)
|
||||
items = [
|
||||
ContextItem(
|
||||
text=None,
|
||||
source_backend=ref.backend_type,
|
||||
ref_id=ref.id,
|
||||
score=0.0,
|
||||
memory_type=ref.ref_type.value,
|
||||
metadata={
|
||||
"status": ref.status.value,
|
||||
"native_id": ref.native_id,
|
||||
"native_uri": ref.native_uri,
|
||||
},
|
||||
)
|
||||
for ref in refs
|
||||
]
|
||||
trace_id = request.metadata.get("trace_id") if request.metadata else None
|
||||
return RetrieveResponse(
|
||||
status=OperationStatus.SUCCESS,
|
||||
@ -220,7 +219,7 @@ class MemoryGatewayV2Service:
|
||||
refs=self._view_refs(refs),
|
||||
conflicts=[],
|
||||
trace_id=trace_id,
|
||||
metadata={"skeleton": True},
|
||||
metadata=self._retrieve_metadata(results),
|
||||
)
|
||||
|
||||
async def record_memory_feedback(self, request: FeedbackRequest) -> FeedbackResponse:
|
||||
@ -386,6 +385,83 @@ class MemoryGatewayV2Service:
|
||||
limit=limit,
|
||||
)
|
||||
|
||||
async def _retrieve_openviking_context(self, payload: dict[str, Any]) -> BackendRetrieveResult:
|
||||
try:
|
||||
client = await self.openviking_client_factory()
|
||||
if not hasattr(client, "retrieve_context_v2"):
|
||||
return BackendRetrieveResult(
|
||||
backend_type=BackendType.OPENVIKING,
|
||||
status=BackendResultStatus.SKIPPED,
|
||||
metadata={"reason": "adapter_method_missing"},
|
||||
)
|
||||
result = client.retrieve_context_v2(payload)
|
||||
if hasattr(result, "__await__"):
|
||||
result = await result
|
||||
return result
|
||||
except Exception as exc: # noqa: BLE001
|
||||
return BackendRetrieveResult(
|
||||
backend_type=BackendType.OPENVIKING,
|
||||
status=BackendResultStatus.FAILED,
|
||||
error_code="adapter_exception",
|
||||
error_message=str(exc),
|
||||
retryable=True,
|
||||
)
|
||||
|
||||
async def _retrieve_everos_context(self, payload: dict[str, Any]) -> BackendRetrieveResult:
|
||||
try:
|
||||
client = self.everos_client or EverOSClient()
|
||||
if not hasattr(client, "retrieve_context_v2"):
|
||||
return BackendRetrieveResult(
|
||||
backend_type=BackendType.EVEROS,
|
||||
status=BackendResultStatus.SKIPPED,
|
||||
metadata={"reason": "adapter_method_missing"},
|
||||
)
|
||||
result = client.retrieve_context_v2(payload)
|
||||
if hasattr(result, "__await__"):
|
||||
result = await result
|
||||
return result
|
||||
except Exception as exc: # noqa: BLE001
|
||||
return BackendRetrieveResult(
|
||||
backend_type=BackendType.EVEROS,
|
||||
status=BackendResultStatus.FAILED,
|
||||
error_code="adapter_exception",
|
||||
error_message=str(exc),
|
||||
retryable=True,
|
||||
)
|
||||
|
||||
def _merge_retrieve_items(self, results: list[BackendRetrieveResult], limit: int) -> list[ContextItem]:
|
||||
items: list[ContextItem] = []
|
||||
for result in results:
|
||||
if result.status != BackendResultStatus.SUCCESS:
|
||||
continue
|
||||
for item in result.items:
|
||||
items.append(
|
||||
ContextItem(
|
||||
text=item.text,
|
||||
source_backend=item.source_backend,
|
||||
ref_id=item.ref_id,
|
||||
score=item.score,
|
||||
memory_type=item.memory_type,
|
||||
metadata=item.metadata,
|
||||
)
|
||||
)
|
||||
items.sort(key=lambda item: item.score, reverse=True)
|
||||
return items[:limit]
|
||||
|
||||
def _retrieve_metadata(self, results: list[BackendRetrieveResult]) -> dict[str, Any]:
|
||||
return {
|
||||
"backend_results": [
|
||||
{
|
||||
"backend_type": result.backend_type.value,
|
||||
"status": result.status.value,
|
||||
"items": len(result.items),
|
||||
"error_code": result.error_code,
|
||||
"error_message": result.error_message,
|
||||
}
|
||||
for result in results
|
||||
]
|
||||
}
|
||||
|
||||
async def _execute_outbox_event(self, event: OutboxEvent) -> BackendCommitResult | BackendWriteResult:
|
||||
payload = self._outbox_payload(event)
|
||||
if event.operation != BackendOperation.COMMIT_SESSION:
|
||||
@ -406,11 +482,11 @@ class MemoryGatewayV2Service:
|
||||
)
|
||||
result = await client.commit_session_v2(payload)
|
||||
return result
|
||||
if event.backend_type == BackendType.EVERMEMOS:
|
||||
client = self.evermemos_client or EverMemOSClient()
|
||||
if event.backend_type == BackendType.EVEROS:
|
||||
client = self.everos_client or EverOSClient()
|
||||
if not hasattr(client, "extract_profile_long_term_v2"):
|
||||
return BackendCommitResult(
|
||||
backend_type=BackendType.EVERMEMOS,
|
||||
backend_type=event.backend_type,
|
||||
operation=BackendOperation.COMMIT_SESSION,
|
||||
status=BackendResultStatus.SKIPPED,
|
||||
metadata={"reason": "adapter_method_missing"},
|
||||
@ -557,7 +633,7 @@ class MemoryGatewayV2Service:
|
||||
pass
|
||||
if event.backend_type == BackendType.OPENVIKING:
|
||||
return MemoryRefType.SESSION_ARCHIVE
|
||||
if event.backend_type == BackendType.EVERMEMOS:
|
||||
if event.backend_type == BackendType.EVEROS:
|
||||
return MemoryRefType.LONG_TERM_MEMORY
|
||||
return MemoryRefType.DRAFT_REVIEW
|
||||
|
||||
@ -712,7 +788,7 @@ class MemoryGatewayV2Service:
|
||||
metadata=self._control_metadata(request, content_hash),
|
||||
)
|
||||
|
||||
async def _write_evermemos_message(
|
||||
async def _write_everos_message(
|
||||
self,
|
||||
request: IngestRequest,
|
||||
payload: dict[str, Any],
|
||||
@ -721,13 +797,13 @@ class MemoryGatewayV2Service:
|
||||
content_hash: str,
|
||||
) -> MemoryRef:
|
||||
try:
|
||||
client = self.evermemos_client or EverMemOSClient()
|
||||
client = self.everos_client or EverOSClient()
|
||||
if not hasattr(client, "ingest_message"):
|
||||
return self._save_ref(
|
||||
request,
|
||||
gateway_id,
|
||||
provenance_id,
|
||||
BackendType.EVERMEMOS,
|
||||
BackendType.EVEROS,
|
||||
MemoryRefType.MESSAGE_MEMORY,
|
||||
BackendRefStatus.SKIPPED,
|
||||
content_hash=content_hash,
|
||||
@ -740,7 +816,7 @@ class MemoryGatewayV2Service:
|
||||
request,
|
||||
gateway_id,
|
||||
provenance_id,
|
||||
BackendType.EVERMEMOS,
|
||||
BackendType.EVEROS,
|
||||
MemoryRefType.MESSAGE_MEMORY,
|
||||
result,
|
||||
content_hash,
|
||||
@ -750,7 +826,7 @@ class MemoryGatewayV2Service:
|
||||
request,
|
||||
gateway_id,
|
||||
provenance_id,
|
||||
BackendType.EVERMEMOS,
|
||||
BackendType.EVEROS,
|
||||
MemoryRefType.MESSAGE_MEMORY,
|
||||
BackendRefStatus.FAILED,
|
||||
content_hash=content_hash,
|
||||
@ -946,7 +1022,7 @@ class MemoryGatewayV2Service:
|
||||
"idempotency_key": request.idempotency_key,
|
||||
"request_id": request.request_id,
|
||||
}
|
||||
for backend_type in (BackendType.OPENVIKING, BackendType.EVERMEMOS):
|
||||
for backend_type in (BackendType.OPENVIKING, BackendType.EVEROS):
|
||||
event = OutboxEvent(
|
||||
id=self._outbox_event_id(gateway_id, backend_type, BackendOperation.COMMIT_SESSION),
|
||||
event_type="commit_session",
|
||||
|
||||
@ -21,8 +21,8 @@ class OpenVikingConfig(BaseModel):
|
||||
ingest_path: str = "/api/v1/sessions/{session_id}/messages"
|
||||
|
||||
|
||||
class EverMemOSConfig(BaseModel):
|
||||
"""External EverMemOS consolidation service configuration."""
|
||||
class EverOSConfig(BaseModel):
|
||||
"""External EverOS memory service configuration."""
|
||||
enabled: bool = False
|
||||
mode: Literal["offline", "skeleton", "real"] = "offline"
|
||||
url: str = "http://127.0.0.1:1995"
|
||||
@ -31,9 +31,9 @@ class EverMemOSConfig(BaseModel):
|
||||
verify_ssl: bool = True
|
||||
health_path: str = "/health"
|
||||
ingest_path: str = "/api/v1/memories"
|
||||
consolidate_path: str = "/v1/sessions/consolidate"
|
||||
fallback_to_local: bool = True
|
||||
|
||||
search_path: str = "/api/v1/memories/search"
|
||||
flush_path: str = "/api/v1/memories/flush"
|
||||
retrieve_method: Literal["keyword", "vector", "hybrid", "rrf", "agentic"] = "keyword"
|
||||
|
||||
class MemoryConfig(BaseModel):
|
||||
"""记忆配置"""
|
||||
@ -71,16 +71,18 @@ class LoggingConfig(BaseModel):
|
||||
|
||||
class Config(BaseModel):
|
||||
"""完整配置"""
|
||||
def __init__(self, **data: Any) -> None:
|
||||
super().__init__(**data)
|
||||
|
||||
server: ServerConfig = Field(default_factory=ServerConfig)
|
||||
openviking: OpenVikingConfig = Field(default_factory=OpenVikingConfig)
|
||||
evermemos: EverMemOSConfig = Field(default_factory=EverMemOSConfig)
|
||||
everos: EverOSConfig = Field(default_factory=EverOSConfig)
|
||||
memory: MemoryConfig = Field(default_factory=MemoryConfig)
|
||||
logging: LoggingConfig = Field(default_factory=LoggingConfig)
|
||||
llm: LLMConfig = Field(default_factory=LLMConfig)
|
||||
obsidian: ObsidianConfig = Field(default_factory=ObsidianConfig)
|
||||
storage: StorageConfig = Field(default_factory=StorageConfig)
|
||||
|
||||
|
||||
class SearchRequest(BaseModel):
|
||||
"""搜索请求"""
|
||||
query: str
|
||||
|
||||
@ -1,2 +0,0 @@
|
||||
"""Background worker skeletons."""
|
||||
|
||||
@ -1,186 +0,0 @@
|
||||
"""Minimal EverMemOS-style consolidation worker.
|
||||
|
||||
This worker is deliberately deterministic for the POC. It extracts stable
|
||||
candidate memories from session episodes, deduplicates them against existing
|
||||
records, promotes eligible records, and sends high-risk/high-value candidates
|
||||
to Obsidian review rather than blindly polluting long-term memory.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from memory_gateway.namespace import default_namespace_for_context
|
||||
from memory_gateway.obsidian_review import write_review_draft
|
||||
from memory_gateway.repositories import MetadataRepository
|
||||
from memory_gateway.schemas import (
|
||||
AccessContext,
|
||||
EpisodeRecord,
|
||||
MemoryRecord,
|
||||
MemoryType,
|
||||
SourceType,
|
||||
Visibility,
|
||||
)
|
||||
|
||||
|
||||
_SENTENCE_RE = re.compile(r"(?<=[。!?.!?])\s+|\n+")
|
||||
_NOISE_RE = re.compile(r"\s+")
|
||||
|
||||
|
||||
@dataclass
|
||||
class ConsolidationResult:
|
||||
session_id: str
|
||||
episodes: int
|
||||
candidates: list[MemoryRecord] = field(default_factory=list)
|
||||
promoted: list[MemoryRecord] = field(default_factory=list)
|
||||
duplicates: list[dict] = field(default_factory=list)
|
||||
review_drafts: list[str] = field(default_factory=list)
|
||||
conflicts: list[dict] = field(default_factory=list)
|
||||
|
||||
|
||||
class EverMemOSWorker:
|
||||
def __init__(self, repo: MetadataRepository) -> None:
|
||||
self.repo = repo
|
||||
|
||||
def consolidate_session(
|
||||
self,
|
||||
session_id: str,
|
||||
ctx: AccessContext,
|
||||
min_importance: float = 0.6,
|
||||
target_namespace: str | None = None,
|
||||
) -> ConsolidationResult:
|
||||
episodes = self.repo.list_session_episodes(session_id)
|
||||
result = ConsolidationResult(session_id=session_id, episodes=len(episodes))
|
||||
existing = list(self.repo.list_memories())
|
||||
seen_fingerprints = {self._fingerprint(memory.content): memory for memory in existing}
|
||||
|
||||
for episode in episodes:
|
||||
for candidate in self._extract_candidates(episode, ctx, min_importance, target_namespace):
|
||||
result.candidates.append(candidate)
|
||||
fingerprint = self._fingerprint(candidate.content)
|
||||
duplicate = seen_fingerprints.get(fingerprint)
|
||||
if duplicate:
|
||||
result.duplicates.append({"candidate_id": candidate.id, "existing_id": duplicate.id})
|
||||
continue
|
||||
|
||||
conflict_ids = self._find_conflicts(candidate, existing)
|
||||
if conflict_ids:
|
||||
draft = write_review_draft(candidate, reason="conflict", conflict_ids=conflict_ids)
|
||||
result.review_drafts.append(str(draft))
|
||||
result.conflicts.append({"candidate_id": candidate.id, "conflict_ids": conflict_ids})
|
||||
continue
|
||||
|
||||
if candidate.importance >= 0.85:
|
||||
draft = write_review_draft(candidate, reason="high_value")
|
||||
result.review_drafts.append(str(draft))
|
||||
continue
|
||||
|
||||
if candidate.importance >= min_importance and candidate.confidence >= 0.55:
|
||||
self.repo.upsert_memory(candidate)
|
||||
result.promoted.append(candidate)
|
||||
seen_fingerprints[fingerprint] = candidate
|
||||
existing.append(candidate)
|
||||
|
||||
return result
|
||||
|
||||
def _extract_candidates(
|
||||
self,
|
||||
episode: EpisodeRecord,
|
||||
ctx: AccessContext,
|
||||
min_importance: float,
|
||||
target_namespace: str | None,
|
||||
) -> list[MemoryRecord]:
|
||||
text = episode.summary or episode.content
|
||||
parts = [self._normalize(part) for part in _SENTENCE_RE.split(text) if self._normalize(part)]
|
||||
candidates: list[MemoryRecord] = []
|
||||
for part in parts:
|
||||
if len(part) < 20:
|
||||
continue
|
||||
memory_type = self._classify_type(part, episode.tags)
|
||||
importance = self._estimate_importance(part, episode.tags, min_importance)
|
||||
confidence = 0.65 if episode.summary else 0.58
|
||||
visibility = Visibility.WORKSPACE_SHARED if "workspace" in episode.tags and ctx.workspace_id else Visibility.PRIVATE
|
||||
memory_ctx = AccessContext(
|
||||
user_id=ctx.user_id,
|
||||
agent_id=ctx.agent_id,
|
||||
workspace_id=ctx.workspace_id,
|
||||
session_id=ctx.session_id,
|
||||
)
|
||||
candidates.append(
|
||||
MemoryRecord(
|
||||
user_id=ctx.user_id,
|
||||
agent_id=ctx.agent_id,
|
||||
workspace_id=ctx.workspace_id,
|
||||
session_id=episode.session_id,
|
||||
namespace=target_namespace or default_namespace_for_context(memory_ctx, visibility),
|
||||
memory_type=memory_type,
|
||||
content=part,
|
||||
summary=part[:180],
|
||||
tags=list(set(episode.tags + ["promoted-from-session", "evermemos-candidate"])),
|
||||
importance=importance,
|
||||
confidence=confidence,
|
||||
visibility=visibility,
|
||||
source=SourceType.EVERMEMOS,
|
||||
source_ref=episode.id,
|
||||
)
|
||||
)
|
||||
return candidates
|
||||
|
||||
def _classify_type(self, text: str, tags: list[str]) -> MemoryType:
|
||||
lowered = text.lower()
|
||||
if "preference" in tags or "偏好" in text:
|
||||
return MemoryType.PREFERENCE
|
||||
if "decision" in tags or "决定" in text or "决策" in text:
|
||||
return MemoryType.DECISION
|
||||
if "procedure" in tags or "步骤" in text or "流程" in text:
|
||||
return MemoryType.PROCEDURE
|
||||
if "经验" in text or "worked" in lowered or "failed" in lowered:
|
||||
return MemoryType.EXPERIENCE
|
||||
return MemoryType.SUMMARY
|
||||
|
||||
def _estimate_importance(self, text: str, tags: list[str], min_importance: float) -> float:
|
||||
importance = max(min_importance, 0.6)
|
||||
signal_words = ["必须", "不要", "偏好", "长期", "决策", "结论", "重要", "preference", "decision", "must"]
|
||||
if any(word in text.lower() for word in signal_words):
|
||||
importance += 0.15
|
||||
if "review" in tags or "high-value" in tags:
|
||||
importance += 0.2
|
||||
return min(1.0, importance)
|
||||
|
||||
def _find_conflicts(self, candidate: MemoryRecord, existing: list[MemoryRecord]) -> list[str]:
|
||||
candidate_text = candidate.content.lower()
|
||||
negation_signals = ["不要", "不再", "禁止", "not ", "never", "disable"]
|
||||
positive_signals = ["需要", "必须", "启用", "prefer", "always", "enable"]
|
||||
has_negative = any(signal in candidate_text for signal in negation_signals)
|
||||
has_positive = any(signal in candidate_text for signal in positive_signals)
|
||||
if not has_negative and not has_positive:
|
||||
return []
|
||||
|
||||
candidate_tokens = self._tokens(candidate.content)
|
||||
conflicts = []
|
||||
for memory in existing:
|
||||
if memory.user_id != candidate.user_id:
|
||||
continue
|
||||
if memory.memory_type != candidate.memory_type:
|
||||
continue
|
||||
overlap = candidate_tokens.intersection(self._tokens(memory.content))
|
||||
if len(overlap) < 2:
|
||||
continue
|
||||
memory_text = memory.content.lower()
|
||||
memory_negative = any(signal in memory_text for signal in negation_signals)
|
||||
memory_positive = any(signal in memory_text for signal in positive_signals)
|
||||
if has_negative != memory_negative or has_positive != memory_positive:
|
||||
conflicts.append(memory.id)
|
||||
return conflicts
|
||||
|
||||
def _tokens(self, text: str) -> set[str]:
|
||||
return {token for token in re.split(r"[^a-zA-Z0-9\u4e00-\u9fff]+", text.lower()) if len(token) >= 2}
|
||||
|
||||
def _normalize(self, text: str) -> str:
|
||||
return _NOISE_RE.sub(" ", text).strip(" -_*#\t")
|
||||
|
||||
def _fingerprint(self, text: str) -> str:
|
||||
normalized = self._normalize(text).lower()
|
||||
return hashlib.sha1(normalized.encode("utf-8")).hexdigest()
|
||||
|
||||
@ -117,7 +117,7 @@ Verified boundaries:
|
||||
|
||||
The plugin rejects memory writes containing passwords, API keys, bearer tokens, cookies, private keys, SSH keys, one-time verification codes, large logs, full raw transcripts, and chain-of-thought.
|
||||
|
||||
The plugin writes summarized candidate episodes. It does not store full raw conversations. Long-term memory should normally be produced by `memory_commit_session`, allowing Memory Gateway and EverMemOS to deduplicate, detect conflicts, and route review drafts.
|
||||
The plugin writes summarized candidate episodes. It does not store full raw conversations. Long-term memory should normally be produced by `memory_commit_session`, allowing Memory Gateway and EverOS to deduplicate, detect conflicts, and route review drafts.
|
||||
|
||||
Direct long-term `memory_upsert` is high risk and is not called automatically. If a user asks to forget or delete a memory, the agent should call `memory_feedback` or a delete-capable tool instead of silently keeping the memory.
|
||||
|
||||
|
||||
@ -25,7 +25,7 @@ tools:
|
||||
memory_append_episode:
|
||||
description: Append a safe summarized candidate episode.
|
||||
memory_commit_session:
|
||||
description: Ask Gateway/EverMemOS to consolidate session episodes.
|
||||
description: Ask Gateway/EverOS to consolidate session episodes.
|
||||
memory_upsert:
|
||||
description: Upsert a stable memory through Gateway.
|
||||
memory_feedback:
|
||||
|
||||
@ -22,5 +22,5 @@ hooks:
|
||||
safety:
|
||||
stores_full_raw_conversation: false
|
||||
rejects_secrets: true
|
||||
long_term_commit_via_evermemos: true
|
||||
long_term_commit_via_everos: true
|
||||
|
||||
|
||||
@ -16,7 +16,7 @@ During a task:
|
||||
|
||||
At task or session completion:
|
||||
|
||||
- Use `memory_commit_session` to let Memory Gateway and EverMemOS decide what can be promoted.
|
||||
- Use `memory_commit_session` to let Memory Gateway and EverOS decide what can be promoted.
|
||||
- Do not promote all episodes directly to long-term memory.
|
||||
- Conflicting or high-value memories should enter review rather than overwrite existing memory.
|
||||
|
||||
|
||||
@ -20,5 +20,5 @@ The plugin must reject memory writes that contain:
|
||||
|
||||
The plugin stores summaries rather than raw messages. If a message is useful but contains sensitive detail, redact the sensitive detail before writing. If redaction would remove the meaning, reject the write.
|
||||
|
||||
Long-term memory should normally be created by session commit and EverMemOS consolidation, not by direct upsert.
|
||||
Long-term memory should normally be created by session commit and EverOS consolidation, not by direct upsert.
|
||||
|
||||
|
||||
@ -42,7 +42,7 @@ MEMORY_APPEND_EPISODE = {
|
||||
|
||||
MEMORY_COMMIT_SESSION = {
|
||||
"name": "memory_commit_session",
|
||||
"description": "Commit a session through Memory Gateway and EverMemOS. Promotes only what consolidation accepts.",
|
||||
"description": "Commit a session through Memory Gateway and EverOS. Promotes only what consolidation accepts.",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
|
||||
@ -12,7 +12,6 @@ dependencies = [
|
||||
"pydantic>=2.5.0",
|
||||
"pyyaml>=6.0",
|
||||
"uvicorn>=0.27.0",
|
||||
"tenacity>=8.2.0",
|
||||
"markitdown[all]>=0.1.5",
|
||||
"python-multipart>=0.0.9",
|
||||
]
|
||||
@ -28,8 +27,5 @@ dev = [
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[tool.uv]
|
||||
dev-dependencies = []
|
||||
|
||||
[tool.ruff]
|
||||
target-version = "py310"
|
||||
|
||||
@ -7,7 +7,7 @@ from fastapi import FastAPI
|
||||
from httpx import ASGITransport, AsyncClient
|
||||
|
||||
import memory_gateway.api_v2 as api_v2
|
||||
from memory_gateway.evermemos_client import EverMemOSClient
|
||||
from memory_gateway.everos_client import EverOSClient
|
||||
from memory_gateway.openviking_client import OpenVikingClient
|
||||
from memory_gateway.repositories import InMemoryRepository
|
||||
from memory_gateway.schemas_v2 import BackendRefStatus, BackendType, IngestRequest, IngestResponse, OperationStatus
|
||||
@ -28,13 +28,13 @@ def _env(name: str) -> str:
|
||||
return value
|
||||
|
||||
|
||||
def test_real_openviking_and_evermemos_ingest_writes_memory_refs():
|
||||
def test_real_openviking_and_everos_ingest_writes_memory_refs():
|
||||
openviking_base_url = _env("OPENVIKING_BASE_URL")
|
||||
evermemos_base_url = _env("EVERMEMOS_BASE_URL")
|
||||
everos_base_url = _env("EVEROS_BASE_URL")
|
||||
openviking_api_key = os.environ.get("OPENVIKING_API_KEY", "")
|
||||
evermemos_api_key = os.environ.get("EVERMEMOS_API_KEY", "")
|
||||
everos_api_key = os.environ.get("EVEROS_API_KEY", "")
|
||||
openviking_ingest_path = os.environ.get("OPENVIKING_INGEST_PATH")
|
||||
evermemos_ingest_path = os.environ.get("EVERMEMOS_INGEST_PATH")
|
||||
everos_ingest_path = os.environ.get("EVEROS_INGEST_PATH")
|
||||
|
||||
async def openviking_factory():
|
||||
return OpenVikingClient(
|
||||
@ -48,11 +48,11 @@ def test_real_openviking_and_evermemos_ingest_writes_memory_refs():
|
||||
service = MemoryGatewayV2Service(
|
||||
repo=repo,
|
||||
openviking_client_factory=openviking_factory,
|
||||
evermemos_client=EverMemOSClient(
|
||||
everos_client=EverOSClient(
|
||||
mode="real",
|
||||
base_url=evermemos_base_url,
|
||||
api_key=evermemos_api_key,
|
||||
ingest_path=evermemos_ingest_path,
|
||||
base_url=everos_base_url,
|
||||
api_key=everos_api_key,
|
||||
ingest_path=everos_ingest_path,
|
||||
),
|
||||
)
|
||||
run_id = uuid4().hex[:12]
|
||||
@ -60,20 +60,20 @@ def test_real_openviking_and_evermemos_ingest_writes_memory_refs():
|
||||
response = asyncio.run(post_ingest(service, run_id))
|
||||
|
||||
refs = repo.list_memory_refs(session_id=f"real_ingest_sess_{run_id}", limit=10)
|
||||
assert {ref.backend_type for ref in refs} == {BackendType.OPENVIKING, BackendType.EVERMEMOS}
|
||||
assert {ref.backend_type for ref in refs} == {BackendType.OPENVIKING, BackendType.EVEROS}
|
||||
assert all(ref.content_hash for ref in refs)
|
||||
openviking_ref = next(ref for ref in refs if ref.backend_type == BackendType.OPENVIKING)
|
||||
evermemos_ref = next(ref for ref in refs if ref.backend_type == BackendType.EVERMEMOS)
|
||||
everos_ref = next(ref for ref in refs if ref.backend_type == BackendType.EVEROS)
|
||||
|
||||
assert openviking_ref.status == BackendRefStatus.SUCCESS
|
||||
if evermemos_ref.status == BackendRefStatus.SUCCESS:
|
||||
if everos_ref.status == BackendRefStatus.SUCCESS:
|
||||
assert response.status == OperationStatus.SUCCESS
|
||||
assert evermemos_ref.native_id
|
||||
assert evermemos_ref.native_uri
|
||||
assert everos_ref.native_id
|
||||
assert everos_ref.native_uri
|
||||
else:
|
||||
assert evermemos_ref.status == BackendRefStatus.FAILED
|
||||
assert everos_ref.status == BackendRefStatus.FAILED
|
||||
assert response.status == OperationStatus.PARTIAL_SUCCESS
|
||||
assert evermemos_ref.error_message
|
||||
assert everos_ref.error_message
|
||||
|
||||
|
||||
async def post_ingest(service: MemoryGatewayV2Service, run_id: str):
|
||||
|
||||
@ -1,53 +0,0 @@
|
||||
import asyncio
|
||||
|
||||
from memory_gateway.evermemos_service import ConsolidateRequest, consolidate_session
|
||||
|
||||
|
||||
def test_evermemos_service_consolidates_session(monkeypatch, tmp_path):
|
||||
monkeypatch.setattr(
|
||||
"memory_gateway.obsidian_review.get_config",
|
||||
lambda: type(
|
||||
"Config",
|
||||
(),
|
||||
{
|
||||
"obsidian": type(
|
||||
"Obsidian",
|
||||
(),
|
||||
{"vault_path": str(tmp_path / "vault"), "review_dir": "Reviews/Queue"},
|
||||
)()
|
||||
},
|
||||
)(),
|
||||
)
|
||||
payload = {
|
||||
"session_id": "sess_service",
|
||||
"context": {"user_id": "user_a", "agent_id": "agent_a", "workspace_id": "ws_a", "session_id": "sess_service"},
|
||||
"episodes": [
|
||||
{
|
||||
"user_id": "user_a",
|
||||
"agent_id": "agent_a",
|
||||
"workspace_id": "ws_a",
|
||||
"session_id": "sess_service",
|
||||
"namespace": "session/sess_service/episodic",
|
||||
"content": "结论:EverMemOS 本地服务负责整理稳定长期记忆。",
|
||||
"tags": ["decision"],
|
||||
},
|
||||
{
|
||||
"user_id": "user_a",
|
||||
"agent_id": "agent_a",
|
||||
"workspace_id": "ws_a",
|
||||
"session_id": "sess_service",
|
||||
"namespace": "session/sess_service/episodic",
|
||||
"content": "重要:高价值记忆应该进入 Obsidian review queue。",
|
||||
"tags": ["review", "high-value"],
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
response = asyncio.run(consolidate_session(ConsolidateRequest.model_validate(payload)))
|
||||
|
||||
assert response["status"] == "ok"
|
||||
result = response["result"]
|
||||
assert result["episodes"] == 2
|
||||
assert len(result["candidates"]) == 2
|
||||
assert len(result["promoted"]) == 1
|
||||
assert len(result["review_drafts"]) == 1
|
||||
@ -137,7 +137,7 @@ def test_health_requires_api_key(monkeypatch):
|
||||
fake_get_openviking_client,
|
||||
)
|
||||
monkeypatch.setattr("memory_gateway.server.summarize_with_llm", fake_summarize_with_llm)
|
||||
monkeypatch.setattr("memory_gateway.server.v1_service.evermemos_health", lambda: {"status": "disabled"})
|
||||
monkeypatch.setattr("memory_gateway.server.v1_service.everos_health", lambda: {"status": "disabled"})
|
||||
|
||||
with pytest.raises(HTTPException) as exc_info:
|
||||
server.verify_api_key()
|
||||
|
||||
@ -11,7 +11,7 @@ from memory_gateway.schemas import (
|
||||
Visibility,
|
||||
)
|
||||
from memory_gateway.services import MemoryGatewayService
|
||||
from memory_gateway.types import Config, EverMemOSConfig, ObsidianConfig
|
||||
from memory_gateway.types import Config, EverOSConfig, ObsidianConfig
|
||||
|
||||
|
||||
def test_private_memory_is_isolated_by_user():
|
||||
@ -67,10 +67,10 @@ def test_sqlite_repository_persists_memory(tmp_path):
|
||||
assert reloaded.content == "持久化 SQLite memory"
|
||||
|
||||
|
||||
def test_commit_session_promotes_dedupes_and_creates_review_draft(monkeypatch, tmp_path):
|
||||
def test_commit_session_disabled_does_not_use_local_fallback(monkeypatch, tmp_path):
|
||||
monkeypatch.setattr(
|
||||
"memory_gateway.services.get_config",
|
||||
lambda: Config(evermemos=EverMemOSConfig(enabled=False)),
|
||||
lambda: Config(everos=EverOSConfig(enabled=False)),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"memory_gateway.obsidian_review.get_config",
|
||||
@ -103,29 +103,27 @@ def test_commit_session_promotes_dedupes_and_creates_review_draft(monkeypatch, t
|
||||
),
|
||||
)
|
||||
|
||||
assert len(result["promoted"]) == 1
|
||||
assert result["evermemos_backend"] == "local-disabled"
|
||||
assert len(result["review_drafts"]) == 1
|
||||
assert (tmp_path / "vault" / "Reviews" / "Queue").exists()
|
||||
assert result["promoted"] == []
|
||||
assert result["everos_backend"] == "disabled"
|
||||
|
||||
|
||||
def test_commit_session_uses_external_evermemos(monkeypatch):
|
||||
def test_commit_session_uses_external_everos(monkeypatch):
|
||||
monkeypatch.setattr(
|
||||
"memory_gateway.services.get_config",
|
||||
lambda: Config(evermemos=EverMemOSConfig(enabled=True, fallback_to_local=False)),
|
||||
lambda: Config(everos=EverOSConfig(enabled=True)),
|
||||
)
|
||||
|
||||
class FakeEverMemOSClient:
|
||||
class FakeEverOSClient:
|
||||
def consolidate_session(self, **kwargs):
|
||||
return {
|
||||
"episodes": 1,
|
||||
"candidates": [],
|
||||
"promoted": [
|
||||
{
|
||||
"content": "外部 EverMemOS 总结出的长期记忆",
|
||||
"summary": "外部 EverMemOS 长期记忆",
|
||||
"content": "外部 EverOS 总结出的长期记忆",
|
||||
"summary": "外部 EverOS 长期记忆",
|
||||
"memory_type": "summary",
|
||||
"tags": ["external-evermemos"],
|
||||
"tags": ["external-everos"],
|
||||
}
|
||||
],
|
||||
"duplicates": [],
|
||||
@ -136,12 +134,12 @@ def test_commit_session_uses_external_evermemos(monkeypatch):
|
||||
def health(self):
|
||||
return {"status": "ok"}
|
||||
|
||||
service = MemoryGatewayService(InMemoryRepository(), evermemos_client=FakeEverMemOSClient())
|
||||
service = MemoryGatewayService(InMemoryRepository(), everos_client=FakeEverOSClient())
|
||||
service.append_episode(
|
||||
EpisodeAppendRequest(
|
||||
user_id="user_a",
|
||||
session_id="sess_external",
|
||||
content="这条 episode 应该交给外部 EverMemOS。",
|
||||
content="这条 episode 应该交给外部 EverOS。",
|
||||
)
|
||||
)
|
||||
result = service.commit_session(
|
||||
@ -149,9 +147,9 @@ def test_commit_session_uses_external_evermemos(monkeypatch):
|
||||
CommitSessionRequest(user_id="user_a", session_id="sess_external"),
|
||||
)
|
||||
|
||||
assert result["evermemos_backend"] == "external"
|
||||
assert result["everos_backend"] == "external"
|
||||
assert len(result["promoted"]) == 1
|
||||
search = service.search_memory(MemorySearchRequest(user_id="user_a", query="外部 EverMemOS"))
|
||||
search = service.search_memory(MemorySearchRequest(user_id="user_a", query="外部 EverOS"))
|
||||
assert search["total"] == 1
|
||||
|
||||
|
||||
|
||||
@ -16,9 +16,9 @@ from memory_gateway.backend_adapter_mapping import (
|
||||
)
|
||||
from memory_gateway.backend_normalization import (
|
||||
map_backend_error_to_retryable,
|
||||
normalize_evermemos_commit_response,
|
||||
normalize_evermemos_ingest_response,
|
||||
normalize_evermemos_retrieve_response,
|
||||
normalize_everos_commit_response,
|
||||
normalize_everos_ingest_response,
|
||||
normalize_everos_retrieve_response,
|
||||
normalize_openviking_commit_response,
|
||||
normalize_openviking_ingest_response,
|
||||
normalize_openviking_retrieve_response,
|
||||
@ -27,13 +27,14 @@ from memory_gateway.backend_contracts import (
|
||||
BackendCommitResult,
|
||||
BackendOperation,
|
||||
BackendProducedRef,
|
||||
BackendRetrieveItem,
|
||||
BackendResultStatus,
|
||||
BackendRetrieveResult,
|
||||
BackendWriteResult,
|
||||
OutboxEventStatus,
|
||||
)
|
||||
from memory_gateway.backend_ref_mapping import map_backend_ref_type
|
||||
from memory_gateway.evermemos_client import EverMemOSClient
|
||||
from memory_gateway.everos_client import EverOSClient
|
||||
from memory_gateway.obsidian_review_client import ObsidianReviewClient
|
||||
from memory_gateway.openviking_client import OpenVikingClient
|
||||
from memory_gateway.repositories import InMemoryRepository, SQLiteRepository
|
||||
@ -51,12 +52,75 @@ from memory_gateway.server_auth import verify_api_key_compat
|
||||
from memory_gateway.services_v2 import MemoryGatewayV2Service
|
||||
|
||||
|
||||
FIXTURE_DIR = Path(__file__).parent / "fixtures" / "backend_responses"
|
||||
DOCS_DIR = Path(__file__).parent.parent / "docs"
|
||||
|
||||
|
||||
def load_backend_fixture(name: str):
|
||||
return json.loads((FIXTURE_DIR / name).read_text())
|
||||
def backend_response(name: str):
|
||||
responses = {
|
||||
"openviking_ingest_success.json": {
|
||||
"status": "created",
|
||||
"id": "ov_turn_fixture_1",
|
||||
"uri": "viking://sessions/sess_fixture/turns/ov_turn_fixture_1",
|
||||
"metadata": {"schema_version": "openviking.fixture.ingest.v2", "conversation": "SECRET"},
|
||||
},
|
||||
"openviking_ingest_real_success.json": {
|
||||
"status": "created",
|
||||
"id": "ov_real_turn_fixture_1",
|
||||
"uri": "viking://sessions/ov_real_sess_fixture_1/turns/ov_real_turn_fixture_1",
|
||||
"metadata": {"backend_request_id": "ov_req_real_1", "content": "SECRET"},
|
||||
},
|
||||
"openviking_ingest_real_error_401.json": {"status": "failed", "error": "unauthorized", "error_code": "unauthorized"},
|
||||
"openviking_ingest_real_error_422.json": {"status": "failed", "error": "validation failed", "error_code": "validation_error"},
|
||||
"openviking_ingest_real_error_500.json": {"status": "failed", "error": "server error", "error_code": "server_error"},
|
||||
"openviking_commit_success.json": {
|
||||
"status": "ok",
|
||||
"session_id": "sess_fixture",
|
||||
"result": {
|
||||
"refs": [
|
||||
{"type": "session_archive", "id": "ov_archive_fixture_1"},
|
||||
{"type": "context_resource", "id": "ov_resource_fixture_1"},
|
||||
]
|
||||
},
|
||||
"metadata": {"schema_version": "openviking.fixture.commit.v2", "messages": ["SECRET"]},
|
||||
},
|
||||
"openviking_retrieve_success.json": {
|
||||
"status": "ok",
|
||||
"result": {
|
||||
"items": [
|
||||
{"text": "Relevant session summary", "id": "ov_archive_fixture_1", "score": 0.91, "type": "session_archive"},
|
||||
{"text": "Relevant resource", "id": "ov_resource_fixture_1", "score": 0.84, "type": "context_resource"},
|
||||
]
|
||||
},
|
||||
"metadata": {"schema_version": "openviking.fixture.retrieve.v2", "transcript": "SECRET"},
|
||||
},
|
||||
"everos_ingest_success.json": {
|
||||
"status": "success",
|
||||
"memory_id": "em_memory_fixture_1",
|
||||
"metadata": {"schema_version": "everos.fixture.ingest.v2", "transcript": "SECRET"},
|
||||
},
|
||||
"everos_commit_success_multiple_refs.json": {
|
||||
"status": "success",
|
||||
"data": {
|
||||
"produced_refs": [
|
||||
{"ref_type": "episodic_memory", "memory_id": "em_episode_fixture_1"},
|
||||
{"ref_type": "profile", "profile_id": "em_profile_fixture_1"},
|
||||
{"ref_type": "unknown_kind", "id": "em_long_fixture_1"},
|
||||
]
|
||||
},
|
||||
"metadata": {"schema_version": "everos.fixture.commit.v2", "messages": ["SECRET"]},
|
||||
},
|
||||
"everos_retrieve_success.json": {
|
||||
"status": "success",
|
||||
"data": {
|
||||
"items": [
|
||||
{"text": "Relevant episodic memory", "memory_id": "em_episode_fixture_1", "score": 0.88, "memory_type": "episodic_memory"},
|
||||
{"text": "Relevant profile", "profile_id": "em_profile_fixture_1", "score": 0.73, "memory_type": "profile"},
|
||||
]
|
||||
},
|
||||
"metadata": {"schema_version": "everos.fixture.retrieve.v2", "conversation": "SECRET"},
|
||||
},
|
||||
}
|
||||
return responses[name]
|
||||
|
||||
|
||||
def build_ingest_payload(**overrides):
|
||||
@ -86,23 +150,53 @@ class FakeOpenVikingClient:
|
||||
"native_uri": f"viking://sessions/{payload['session_id']}/{payload['turn_id']}",
|
||||
}
|
||||
|
||||
async def retrieve_context_v2(self, payload):
|
||||
return BackendRetrieveResult(
|
||||
backend_type=BackendType.OPENVIKING,
|
||||
status=BackendResultStatus.SUCCESS,
|
||||
items=[
|
||||
BackendRetrieveItem(
|
||||
text="OpenViking context for remember",
|
||||
source_backend=BackendType.OPENVIKING,
|
||||
ref_id="ov_ctx_1",
|
||||
score=0.82,
|
||||
memory_type="context_resource",
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
async def fake_openviking_factory():
|
||||
return FakeOpenVikingClient()
|
||||
|
||||
|
||||
class FakeEverMemOSClient:
|
||||
class FakeEverOSClient:
|
||||
def ingest_message(self, payload):
|
||||
return {
|
||||
"status": "success",
|
||||
"native_id": f"em_{payload['turn_id']}",
|
||||
"native_uri": f"evermemos://memories/{payload['turn_id']}",
|
||||
"native_uri": f"everos://memories/{payload['turn_id']}",
|
||||
}
|
||||
|
||||
def retrieve_context_v2(self, payload):
|
||||
return BackendRetrieveResult(
|
||||
backend_type=BackendType.EVEROS,
|
||||
status=BackendResultStatus.SUCCESS,
|
||||
items=[
|
||||
BackendRetrieveItem(
|
||||
text="EverOS memory for remember",
|
||||
source_backend=BackendType.EVEROS,
|
||||
ref_id="em_ctx_1",
|
||||
score=0.91,
|
||||
memory_type="episodic_memory",
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
class FailingEverMemOSClient:
|
||||
|
||||
class FailingEverOSClient:
|
||||
def ingest_message(self, payload):
|
||||
raise RuntimeError("evermemos unavailable")
|
||||
raise RuntimeError("everos unavailable")
|
||||
|
||||
|
||||
class FakeCommitOpenVikingClient:
|
||||
@ -120,7 +214,7 @@ def fake_commit_openviking_factory(result: BackendCommitResult):
|
||||
return factory
|
||||
|
||||
|
||||
class FakeCommitEverMemOSClient:
|
||||
class FakeCommitEverOSClient:
|
||||
def __init__(self, result: BackendCommitResult) -> None:
|
||||
self.result = result
|
||||
|
||||
@ -149,7 +243,7 @@ def commit_result(
|
||||
|
||||
def test_v2_adapters_return_backend_write_result_contract():
|
||||
ov_result = asyncio.run(
|
||||
OpenVikingClient().ingest_conversation_turn(
|
||||
OpenVikingClient(mode="offline").ingest_conversation_turn(
|
||||
{
|
||||
"workspace_id": "ws_1",
|
||||
"session_id": "sess_1",
|
||||
@ -157,7 +251,7 @@ def test_v2_adapters_return_backend_write_result_contract():
|
||||
}
|
||||
)
|
||||
)
|
||||
em_result = EverMemOSClient().ingest_message(
|
||||
em_result = EverOSClient(mode="offline").ingest_message(
|
||||
{
|
||||
"workspace_id": "ws_1",
|
||||
"session_id": "sess_1",
|
||||
@ -168,7 +262,7 @@ def test_v2_adapters_return_backend_write_result_contract():
|
||||
assert isinstance(ov_result, BackendWriteResult)
|
||||
assert isinstance(em_result, BackendWriteResult)
|
||||
assert ov_result.backend_type == BackendType.OPENVIKING
|
||||
assert em_result.backend_type == BackendType.EVERMEMOS
|
||||
assert em_result.backend_type == BackendType.EVEROS
|
||||
assert ov_result.operation == BackendOperation.INGEST_TURN
|
||||
assert em_result.operation == BackendOperation.INGEST_TURN
|
||||
assert ov_result.status == BackendResultStatus.SKIPPED
|
||||
@ -180,10 +274,10 @@ def test_backend_env_overrides_enable_real_modes(monkeypatch, tmp_path):
|
||||
monkeypatch.setenv("OPENVIKING_BASE_URL", "http://openviking.env.test")
|
||||
monkeypatch.setenv("OPENVIKING_API_KEY", "ov-env-token")
|
||||
monkeypatch.setenv("OPENVIKING_TIMEOUT_SECONDS", "17")
|
||||
monkeypatch.setenv("EVERMEMOS_MODE", "real")
|
||||
monkeypatch.setenv("EVERMEMOS_BASE_URL", "http://evermemos.env.test")
|
||||
monkeypatch.setenv("EVERMEMOS_API_KEY", "em-env-token")
|
||||
monkeypatch.setenv("EVERMEMOS_INGEST_PATH", "/api/v1/memories")
|
||||
monkeypatch.setenv("EVEROS_MODE", "real")
|
||||
monkeypatch.setenv("EVEROS_BASE_URL", "http://everos.env.test")
|
||||
monkeypatch.setenv("EVEROS_API_KEY", "em-env-token")
|
||||
monkeypatch.setenv("EVEROS_INGEST_PATH", "/api/v1/memories")
|
||||
|
||||
config = load_config(str(tmp_path / "missing.yaml"))
|
||||
|
||||
@ -191,10 +285,10 @@ def test_backend_env_overrides_enable_real_modes(monkeypatch, tmp_path):
|
||||
assert config.openviking.url == "http://openviking.env.test"
|
||||
assert config.openviking.api_key == "ov-env-token"
|
||||
assert config.openviking.timeout == 17
|
||||
assert config.evermemos.mode == "real"
|
||||
assert config.evermemos.url == "http://evermemos.env.test"
|
||||
assert config.evermemos.api_key == "em-env-token"
|
||||
assert config.evermemos.ingest_path == "/api/v1/memories"
|
||||
assert config.everos.mode == "real"
|
||||
assert config.everos.url == "http://everos.env.test"
|
||||
assert config.everos.api_key == "em-env-token"
|
||||
assert config.everos.ingest_path == "/api/v1/memories"
|
||||
|
||||
|
||||
def test_openviking_default_ingest_does_not_touch_network():
|
||||
@ -202,6 +296,7 @@ def test_openviking_default_ingest_does_not_touch_network():
|
||||
raise AssertionError("offline OpenViking ingest should not perform HTTP")
|
||||
|
||||
client = OpenVikingClient(
|
||||
mode="offline",
|
||||
base_url="http://openviking.test",
|
||||
transport=httpx.MockTransport(handler),
|
||||
)
|
||||
@ -263,7 +358,7 @@ def test_openviking_mode_real_with_base_url_uses_mock_http():
|
||||
|
||||
def handler(request):
|
||||
calls["count"] += 1
|
||||
return httpx.Response(200, json=load_backend_fixture("openviking_ingest_real_success.json"))
|
||||
return httpx.Response(200, json=backend_response("openviking_ingest_real_success.json"))
|
||||
|
||||
client = OpenVikingClient(
|
||||
mode="real",
|
||||
@ -312,7 +407,7 @@ def test_openviking_real_ingest_mode_real_without_base_url_returns_config_error(
|
||||
def test_openviking_real_ingest_success_uses_mock_http_and_normalization():
|
||||
seen_payload = {}
|
||||
seen_headers = {}
|
||||
fixture = load_backend_fixture("openviking_ingest_real_success.json")
|
||||
fixture = backend_response("openviking_ingest_real_success.json")
|
||||
|
||||
def handler(request):
|
||||
seen_payload.update(json.loads(request.content.decode()))
|
||||
@ -375,7 +470,7 @@ def test_openviking_real_ingest_http_retryable_and_nonretryable_statuses():
|
||||
mode="real",
|
||||
base_url="http://openviking.test",
|
||||
api_key="super-secret-token",
|
||||
transport=httpx.MockTransport(lambda request: httpx.Response(status_code, json=load_backend_fixture(name))),
|
||||
transport=httpx.MockTransport(lambda request: httpx.Response(status_code, json=backend_response(name))),
|
||||
)
|
||||
|
||||
result_429 = asyncio.run(client_for_fixture("openviking_ingest_real_error_500.json", 429).ingest_conversation_turn({"session_id": "sess_http"}))
|
||||
@ -415,14 +510,14 @@ def test_openviking_real_ingest_invalid_json_returns_failed_retryable():
|
||||
assert "SECRET_JSON" not in json.dumps(result.model_dump(mode="json"), ensure_ascii=False)
|
||||
|
||||
|
||||
def test_evermemos_default_ingest_does_not_touch_network_even_if_enabled():
|
||||
def test_everos_default_ingest_does_not_touch_network_even_if_enabled():
|
||||
def handler(request):
|
||||
raise AssertionError("EverMemOS ingest should not perform HTTP unless mode=real")
|
||||
raise AssertionError("EverOS ingest should not perform HTTP unless mode=real")
|
||||
|
||||
client = EverMemOSClient(
|
||||
client = EverOSClient(
|
||||
enabled=True,
|
||||
mode="offline",
|
||||
base_url="http://evermemos.test",
|
||||
base_url="http://everos.test",
|
||||
transport=httpx.MockTransport(handler),
|
||||
)
|
||||
|
||||
@ -431,8 +526,8 @@ def test_evermemos_default_ingest_does_not_touch_network_even_if_enabled():
|
||||
assert result.status == BackendResultStatus.SKIPPED
|
||||
|
||||
|
||||
def test_evermemos_real_ingest_mode_real_without_base_url_returns_config_error():
|
||||
client = EverMemOSClient(mode="real", base_url="")
|
||||
def test_everos_real_ingest_mode_real_without_base_url_returns_config_error():
|
||||
client = EverOSClient(mode="real", base_url="")
|
||||
|
||||
result = client.ingest_message({"session_id": "sess_missing_url", "content": "SECRET"})
|
||||
|
||||
@ -442,19 +537,19 @@ def test_evermemos_real_ingest_mode_real_without_base_url_returns_config_error()
|
||||
assert "SECRET" not in json.dumps(result.model_dump(mode="json"), ensure_ascii=False)
|
||||
|
||||
|
||||
def test_evermemos_real_ingest_success_uses_mock_http_and_normalization():
|
||||
def test_everos_real_ingest_success_uses_mock_http_and_normalization():
|
||||
seen_payload = {}
|
||||
seen_headers = {}
|
||||
fixture = load_backend_fixture("evermemos_ingest_success.json")
|
||||
fixture = backend_response("everos_ingest_success.json")
|
||||
|
||||
def handler(request):
|
||||
seen_payload.update(json.loads(request.content.decode()))
|
||||
seen_headers.update(dict(request.headers))
|
||||
return httpx.Response(200, json=fixture)
|
||||
|
||||
client = EverMemOSClient(
|
||||
client = EverOSClient(
|
||||
mode="real",
|
||||
base_url="http://evermemos.test",
|
||||
base_url="http://everos.test",
|
||||
api_key="em-token",
|
||||
transport=httpx.MockTransport(handler),
|
||||
)
|
||||
@ -472,9 +567,9 @@ def test_evermemos_real_ingest_success_uses_mock_http_and_normalization():
|
||||
"metadata": {"channel": "test"},
|
||||
}
|
||||
)
|
||||
expected = normalize_evermemos_ingest_response(fixture)
|
||||
expected = normalize_everos_ingest_response(fixture)
|
||||
|
||||
assert seen_payload["content"] == "SECRET_EM_CONTENT"
|
||||
assert seen_payload["messages"][0]["content"] == "SECRET_EM_CONTENT"
|
||||
assert seen_headers["x-api-key"] == "em-token"
|
||||
assert seen_headers["authorization"] == "Bearer em-token"
|
||||
assert result == expected
|
||||
@ -484,11 +579,11 @@ def test_evermemos_real_ingest_success_uses_mock_http_and_normalization():
|
||||
assert "em-token" not in serialized
|
||||
|
||||
|
||||
def test_evermemos_real_ingest_errors_are_backend_write_results_and_safe():
|
||||
def test_everos_real_ingest_errors_are_backend_write_results_and_safe():
|
||||
def client_for_response(status_code, body=None, content=None):
|
||||
return EverMemOSClient(
|
||||
return EverOSClient(
|
||||
mode="real",
|
||||
base_url="http://evermemos.test",
|
||||
base_url="http://everos.test",
|
||||
api_key="em-super-secret-token",
|
||||
transport=httpx.MockTransport(lambda request: httpx.Response(status_code, json=body, content=content)),
|
||||
)
|
||||
@ -517,13 +612,13 @@ def test_evermemos_real_ingest_errors_are_backend_write_results_and_safe():
|
||||
assert "em-super-secret-token" not in serialized
|
||||
|
||||
|
||||
def test_evermemos_real_ingest_timeout_is_retryable_and_safe():
|
||||
def test_everos_real_ingest_timeout_is_retryable_and_safe():
|
||||
def handler(request):
|
||||
raise httpx.ReadTimeout("timeout while sending SECRET_TIMEOUT_CONTENT")
|
||||
|
||||
client = EverMemOSClient(
|
||||
client = EverOSClient(
|
||||
mode="real",
|
||||
base_url="http://evermemos.test",
|
||||
base_url="http://everos.test",
|
||||
transport=httpx.MockTransport(handler),
|
||||
)
|
||||
|
||||
@ -540,9 +635,9 @@ def test_backend_adapter_mapping_spec_is_contract_first_and_control_plane_only()
|
||||
(BackendType.OPENVIKING, BackendOperation.INGEST_TURN),
|
||||
(BackendType.OPENVIKING, BackendOperation.COMMIT_SESSION),
|
||||
(BackendType.OPENVIKING, BackendOperation.RETRIEVE_CONTEXT),
|
||||
(BackendType.EVERMEMOS, BackendOperation.INGEST_TURN),
|
||||
(BackendType.EVERMEMOS, BackendOperation.COMMIT_SESSION),
|
||||
(BackendType.EVERMEMOS, BackendOperation.RETRIEVE_CONTEXT),
|
||||
(BackendType.EVEROS, BackendOperation.INGEST_TURN),
|
||||
(BackendType.EVEROS, BackendOperation.COMMIT_SESSION),
|
||||
(BackendType.EVEROS, BackendOperation.RETRIEVE_CONTEXT),
|
||||
(BackendType.OBSIDIAN, BackendOperation.CREATE_REVIEW_DRAFT),
|
||||
}
|
||||
|
||||
@ -551,12 +646,12 @@ def test_backend_adapter_mapping_spec_is_contract_first_and_control_plane_only()
|
||||
assert not DISALLOWED_PAYLOAD_FIELDS.intersection(spec.allowed_payload_fields)
|
||||
|
||||
openviking_commit = get_adapter_mapping_spec(BackendType.OPENVIKING, BackendOperation.COMMIT_SESSION)
|
||||
evermemos_ingest = get_adapter_mapping_spec(BackendType.EVERMEMOS, BackendOperation.INGEST_TURN)
|
||||
everos_ingest = get_adapter_mapping_spec(BackendType.EVEROS, BackendOperation.INGEST_TURN)
|
||||
|
||||
assert openviking_commit.adapter_method == "commit_session_v2"
|
||||
assert openviking_commit.result_model is BackendCommitResult
|
||||
assert evermemos_ingest.adapter_method == "ingest_message"
|
||||
assert evermemos_ingest.result_model is BackendWriteResult
|
||||
assert everos_ingest.adapter_method == "ingest_message"
|
||||
assert everos_ingest.result_model is BackendWriteResult
|
||||
|
||||
|
||||
def test_control_plane_persisted_payload_validator_rejects_content_and_raw_request():
|
||||
@ -590,26 +685,18 @@ def test_runtime_adapter_request_may_be_transient_but_outbox_payload_is_control_
|
||||
validate_control_plane_persisted_payload(outbox_payload)
|
||||
|
||||
|
||||
def test_commit_and_retrieve_adapter_skeletons_return_unified_contracts():
|
||||
def test_commit_adapter_skeletons_return_unified_contracts():
|
||||
payload = {"workspace_id": "ws_1", "session_id": "sess_1", "gateway_id": "gw_1"}
|
||||
|
||||
ov_commit = asyncio.run(OpenVikingClient().commit_session_v2(payload))
|
||||
ov_retrieve = asyncio.run(OpenVikingClient().retrieve_context_v2(payload))
|
||||
em_commit = EverMemOSClient().extract_profile_long_term_v2(payload)
|
||||
em_retrieve = EverMemOSClient().retrieve_context_v2(payload)
|
||||
ov_commit = asyncio.run(OpenVikingClient(mode="skeleton").commit_session_v2(payload))
|
||||
em_commit = EverOSClient(mode="skeleton").extract_profile_long_term_v2(payload)
|
||||
|
||||
assert isinstance(ov_commit, BackendCommitResult)
|
||||
assert isinstance(em_commit, BackendCommitResult)
|
||||
assert isinstance(ov_retrieve, BackendRetrieveResult)
|
||||
assert isinstance(em_retrieve, BackendRetrieveResult)
|
||||
assert ov_commit.status == BackendResultStatus.SUCCESS
|
||||
assert em_commit.status == BackendResultStatus.SUCCESS
|
||||
assert ov_retrieve.status == BackendResultStatus.SUCCESS
|
||||
assert em_retrieve.status == BackendResultStatus.SUCCESS
|
||||
assert ov_commit.refs[0].ref_type == MemoryRefType.SESSION_ARCHIVE
|
||||
assert {ref.ref_type for ref in em_commit.refs} == {MemoryRefType.PROFILE, MemoryRefType.LONG_TERM_MEMORY}
|
||||
assert len(ov_retrieve.items) == 1
|
||||
assert len(em_retrieve.items) == 2
|
||||
|
||||
|
||||
def test_client_skeletons_use_normalization_contracts_and_safe_metadata():
|
||||
@ -621,8 +708,8 @@ def test_client_skeletons_use_normalization_contracts_and_safe_metadata():
|
||||
"content": "TRANSIENT_CONTENT_ONLY",
|
||||
"raw_request": {"content": "TRANSIENT_CONTENT_ONLY"},
|
||||
}
|
||||
ov_client = OpenVikingClient()
|
||||
em_client = EverMemOSClient()
|
||||
ov_client = OpenVikingClient(mode="skeleton")
|
||||
em_client = EverOSClient(mode="skeleton")
|
||||
|
||||
ov_ingest = asyncio.run(ov_client.ingest_conversation_turn(payload))
|
||||
ov_commit = asyncio.run(ov_client.commit_session_v2(payload))
|
||||
@ -649,8 +736,8 @@ def test_client_skeletons_use_normalization_contracts_and_safe_metadata():
|
||||
"status": "skipped",
|
||||
"memory_id": "turn_contract",
|
||||
"metadata": {
|
||||
"reason": "evermemos_v2_ingest_adapter_not_configured",
|
||||
"schema_version": "evermemos.fixture.ingest.v2",
|
||||
"reason": "everos_v2_ingest_adapter_not_configured",
|
||||
"schema_version": "everos.fixture.ingest.v2",
|
||||
},
|
||||
}
|
||||
)
|
||||
@ -667,57 +754,30 @@ def test_client_skeletons_use_normalization_contracts_and_safe_metadata():
|
||||
assert blocked not in serialized
|
||||
|
||||
|
||||
def test_retrieve_skeletons_use_retrieve_normalization_and_safe_metadata():
|
||||
payload = {
|
||||
"workspace_id": "ws_1",
|
||||
"user_id": "user_a",
|
||||
"session_id": "sess_retrieve_contract",
|
||||
"query": "fixture query",
|
||||
"content": "TRANSIENT_RETRIEVE_CONTENT",
|
||||
}
|
||||
ov_result = asyncio.run(OpenVikingClient().retrieve_context_v2(payload))
|
||||
em_result = EverMemOSClient().retrieve_context_v2(payload)
|
||||
|
||||
assert isinstance(ov_result, BackendRetrieveResult)
|
||||
assert isinstance(em_result, BackendRetrieveResult)
|
||||
assert ov_result.status == BackendResultStatus.SUCCESS
|
||||
assert em_result.status == BackendResultStatus.SUCCESS
|
||||
assert ov_result.items[0].source_backend == BackendType.OPENVIKING
|
||||
assert em_result.items[0].source_backend == BackendType.EVERMEMOS
|
||||
assert ov_result.items[0].text
|
||||
assert em_result.items[0].ref_id
|
||||
serialized = json.dumps(
|
||||
{"ov": ov_result.model_dump(mode="json"), "em": em_result.model_dump(mode="json")},
|
||||
ensure_ascii=False,
|
||||
)
|
||||
for blocked in ("TRANSIENT_RETRIEVE_CONTENT", "content", "raw_request", "messages", "conversation", "transcript"):
|
||||
assert blocked not in serialized
|
||||
|
||||
|
||||
def test_openviking_commit_skeleton_ref_type_is_mapped_from_fixture():
|
||||
result = asyncio.run(OpenVikingClient().commit_session_v2({"session_id": "sess_ov_map"}))
|
||||
result = asyncio.run(OpenVikingClient(mode="skeleton").commit_session_v2({"session_id": "sess_ov_map"}))
|
||||
|
||||
assert result.refs
|
||||
assert result.refs[0].ref_type == MemoryRefType.SESSION_ARCHIVE
|
||||
assert result.refs[0].native_id == "ov_session_summary:sess_ov_map"
|
||||
|
||||
|
||||
def test_evermemos_skeleton_multiple_refs_are_written_by_process_outbox_event():
|
||||
def test_everos_skeleton_multiple_refs_are_written_by_process_outbox_event():
|
||||
repo = InMemoryRepository()
|
||||
service = MemoryGatewayV2Service(
|
||||
repo=repo,
|
||||
openviking_client_factory=fake_commit_openviking_factory(
|
||||
commit_result(BackendType.OPENVIKING, BackendResultStatus.SKIPPED)
|
||||
),
|
||||
evermemos_client=EverMemOSClient(),
|
||||
everos_client=EverOSClient(),
|
||||
)
|
||||
response = asyncio.run(
|
||||
service.commit_session("sess_em_skeleton", CommitRequest(workspace_id="ws_1", user_id="user_a", agent_id="agent_cli"))
|
||||
)
|
||||
event = next(event for event in repo.list_outbox_events_by_job(response.job_id) if event.backend_type == BackendType.EVERMEMOS)
|
||||
event = next(event for event in repo.list_outbox_events_by_job(response.job_id) if event.backend_type == BackendType.EVEROS)
|
||||
|
||||
updated = asyncio.run(service.process_outbox_event(event.id))
|
||||
refs = repo.list_memory_refs(session_id="sess_em_skeleton", backend_type=BackendType.EVERMEMOS, status=BackendRefStatus.SUCCESS)
|
||||
refs = repo.list_memory_refs(session_id="sess_em_skeleton", backend_type=BackendType.EVEROS, status=BackendRefStatus.SUCCESS)
|
||||
|
||||
assert updated.status == OutboxEventStatus.SUCCESS
|
||||
assert len(refs) == 2
|
||||
@ -735,18 +795,18 @@ def test_obsidian_review_adapter_skeleton_returns_skipped_write_result():
|
||||
|
||||
def test_backend_commit_result_supports_multiple_produced_refs():
|
||||
result = BackendCommitResult(
|
||||
backend_type=BackendType.EVERMEMOS,
|
||||
backend_type=BackendType.EVEROS,
|
||||
status=BackendResultStatus.SUCCESS,
|
||||
refs=[
|
||||
BackendProducedRef(ref_type=MemoryRefType.PROFILE, native_id="profile_1"),
|
||||
BackendProducedRef(ref_type=MemoryRefType.LONG_TERM_MEMORY, native_uri="evermemos://memories/long_1"),
|
||||
BackendProducedRef(ref_type=MemoryRefType.LONG_TERM_MEMORY, native_uri="everos://memories/long_1"),
|
||||
],
|
||||
)
|
||||
|
||||
dumped = result.model_dump(mode="json")
|
||||
assert len(result.refs) == 2
|
||||
assert dumped["refs"][0]["ref_type"] == "profile"
|
||||
assert dumped["refs"][1]["native_uri"] == "evermemos://memories/long_1"
|
||||
assert dumped["refs"][1]["native_uri"] == "everos://memories/long_1"
|
||||
|
||||
|
||||
def test_backend_ref_type_mapping_and_unknown_fallback_preserves_original_type():
|
||||
@ -758,11 +818,11 @@ def test_backend_ref_type_mapping_and_unknown_fallback_preserves_original_type()
|
||||
assert mapped == MemoryRefType.SESSION_ARCHIVE
|
||||
assert metadata == {}
|
||||
|
||||
mapped, metadata = map_backend_ref_type(BackendType.EVERMEMOS, "preference")
|
||||
mapped, metadata = map_backend_ref_type(BackendType.EVEROS, "preference")
|
||||
assert mapped == MemoryRefType.PROFILE
|
||||
assert metadata == {}
|
||||
|
||||
mapped, metadata = map_backend_ref_type(BackendType.EVERMEMOS, "unknown_signal")
|
||||
mapped, metadata = map_backend_ref_type(BackendType.EVEROS, "unknown_signal")
|
||||
assert mapped == MemoryRefType.LONG_TERM_MEMORY
|
||||
assert metadata["original_ref_type"] == "unknown_signal"
|
||||
|
||||
@ -798,29 +858,10 @@ def test_openviking_commit_fixture_normalizes_to_backend_commit_result_without_u
|
||||
assert "messages" not in serialized
|
||||
|
||||
|
||||
def test_backend_response_fixture_files_exist_and_load():
|
||||
names = {
|
||||
"openviking_ingest_success.json",
|
||||
"openviking_ingest_real_success.json",
|
||||
"openviking_ingest_real_error_401.json",
|
||||
"openviking_ingest_real_error_422.json",
|
||||
"openviking_ingest_real_error_500.json",
|
||||
"openviking_commit_success.json",
|
||||
"openviking_retrieve_success.json",
|
||||
"evermemos_ingest_success.json",
|
||||
"evermemos_commit_success_multiple_refs.json",
|
||||
"evermemos_retrieve_success.json",
|
||||
}
|
||||
|
||||
for name in names:
|
||||
payload = load_backend_fixture(name)
|
||||
assert payload["status"]
|
||||
|
||||
|
||||
def test_openviking_success_fixtures_normalize_without_unsafe_metadata():
|
||||
ingest = normalize_openviking_ingest_response(load_backend_fixture("openviking_ingest_success.json"))
|
||||
commit = normalize_openviking_commit_response(load_backend_fixture("openviking_commit_success.json"))
|
||||
retrieve = normalize_openviking_retrieve_response(load_backend_fixture("openviking_retrieve_success.json"))
|
||||
ingest = normalize_openviking_ingest_response(backend_response("openviking_ingest_success.json"))
|
||||
commit = normalize_openviking_commit_response(backend_response("openviking_commit_success.json"))
|
||||
retrieve = normalize_openviking_retrieve_response(backend_response("openviking_retrieve_success.json"))
|
||||
|
||||
assert ingest.status == BackendResultStatus.SUCCESS
|
||||
assert ingest.native_id == "ov_turn_fixture_1"
|
||||
@ -841,7 +882,7 @@ def test_openviking_success_fixtures_normalize_without_unsafe_metadata():
|
||||
assert blocked not in serialized
|
||||
|
||||
|
||||
def test_evermemos_commit_fixture_normalizes_multiple_produced_refs_and_unknown_type():
|
||||
def test_everos_commit_fixture_normalizes_multiple_produced_refs_and_unknown_type():
|
||||
raw = {
|
||||
"status": "success",
|
||||
"data": {
|
||||
@ -853,7 +894,7 @@ def test_evermemos_commit_fixture_normalizes_multiple_produced_refs_and_unknown_
|
||||
},
|
||||
}
|
||||
|
||||
result = normalize_evermemos_commit_response(raw)
|
||||
result = normalize_everos_commit_response(raw)
|
||||
|
||||
assert result.status == BackendResultStatus.SUCCESS
|
||||
assert len(result.refs) == 3
|
||||
@ -866,10 +907,10 @@ def test_evermemos_commit_fixture_normalizes_multiple_produced_refs_and_unknown_
|
||||
assert "SECRET_PROFILE" not in json.dumps(result.model_dump(mode="json"), ensure_ascii=False)
|
||||
|
||||
|
||||
def test_evermemos_success_fixtures_normalize_without_unsafe_metadata():
|
||||
ingest = normalize_evermemos_ingest_response(load_backend_fixture("evermemos_ingest_success.json"))
|
||||
commit = normalize_evermemos_commit_response(load_backend_fixture("evermemos_commit_success_multiple_refs.json"))
|
||||
retrieve = normalize_evermemos_retrieve_response(load_backend_fixture("evermemos_retrieve_success.json"))
|
||||
def test_everos_success_fixtures_normalize_without_unsafe_metadata():
|
||||
ingest = normalize_everos_ingest_response(backend_response("everos_ingest_success.json"))
|
||||
commit = normalize_everos_commit_response(backend_response("everos_commit_success_multiple_refs.json"))
|
||||
retrieve = normalize_everos_retrieve_response(backend_response("everos_retrieve_success.json"))
|
||||
|
||||
assert ingest.status == BackendResultStatus.SUCCESS
|
||||
assert ingest.native_id == "em_memory_fixture_1"
|
||||
@ -881,7 +922,7 @@ def test_evermemos_success_fixtures_normalize_without_unsafe_metadata():
|
||||
}
|
||||
assert retrieve.status == BackendResultStatus.SUCCESS
|
||||
assert len(retrieve.items) == 2
|
||||
assert retrieve.items[0].source_backend == BackendType.EVERMEMOS
|
||||
assert retrieve.items[0].source_backend == BackendType.EVEROS
|
||||
assert retrieve.items[0].memory_type == "episodic_memory"
|
||||
serialized = json.dumps(
|
||||
{
|
||||
@ -897,7 +938,7 @@ def test_evermemos_success_fixtures_normalize_without_unsafe_metadata():
|
||||
|
||||
def test_malformed_retrieve_response_returns_skipped_empty_result():
|
||||
ov = normalize_openviking_retrieve_response({})
|
||||
em = normalize_evermemos_retrieve_response({"data": {"unexpected": "shape"}})
|
||||
em = normalize_everos_retrieve_response({"data": {"unexpected": "shape"}})
|
||||
|
||||
assert ov.status == BackendResultStatus.SKIPPED
|
||||
assert ov.items == []
|
||||
@ -914,7 +955,7 @@ def test_ingest_response_normalizers_return_write_results_and_sanitize_metadata(
|
||||
"metadata": {"backend_request_id": "ov_req", "conversation": "SECRET"},
|
||||
}
|
||||
)
|
||||
em = normalize_evermemos_ingest_response(
|
||||
em = normalize_everos_ingest_response(
|
||||
{
|
||||
"status": "success",
|
||||
"memory_id": "em_turn_1",
|
||||
@ -935,12 +976,12 @@ def test_ingest_response_normalizers_return_write_results_and_sanitize_metadata(
|
||||
def test_backend_error_retryable_mapping():
|
||||
for status_code in (429, 500, 502, 503, 504):
|
||||
assert map_backend_error_to_retryable(BackendType.OPENVIKING, status_code=status_code) is True
|
||||
assert map_backend_error_to_retryable(BackendType.EVERMEMOS, error_code="timeout") is True
|
||||
assert map_backend_error_to_retryable(BackendType.EVERMEMOS, error_message="network_error: reset") is True
|
||||
assert map_backend_error_to_retryable(BackendType.EVEROS, error_code="timeout") is True
|
||||
assert map_backend_error_to_retryable(BackendType.EVEROS, error_message="network_error: reset") is True
|
||||
assert map_backend_error_to_retryable(BackendType.OPENVIKING, error_code="mystery") is True
|
||||
|
||||
for status_code in (400, 401, 403, 404, 422):
|
||||
assert map_backend_error_to_retryable(BackendType.EVERMEMOS, status_code=status_code) is False
|
||||
assert map_backend_error_to_retryable(BackendType.EVEROS, status_code=status_code) is False
|
||||
|
||||
|
||||
def test_client_map_error_contracts_for_future_http_integration():
|
||||
@ -951,8 +992,8 @@ def test_client_map_error_contracts_for_future_http_integration():
|
||||
def __str__(self):
|
||||
return f"response {self.status_code}"
|
||||
|
||||
ov_client = OpenVikingClient()
|
||||
em_client = EverMemOSClient()
|
||||
ov_client = OpenVikingClient(mode="skeleton")
|
||||
em_client = EverOSClient(mode="skeleton")
|
||||
|
||||
for status_code in (429, 500, 502, 503, 504):
|
||||
assert ov_client._map_error(ResponseLike(status_code)) is True
|
||||
@ -979,20 +1020,20 @@ def test_ingest_service_records_two_success_refs():
|
||||
service = MemoryGatewayV2Service(
|
||||
repo=repo,
|
||||
openviking_client_factory=fake_openviking_factory,
|
||||
evermemos_client=FakeEverMemOSClient(),
|
||||
everos_client=FakeEverOSClient(),
|
||||
)
|
||||
|
||||
response = asyncio.run(service.ingest_conversation_turn(IngestRequest(**build_ingest_payload())))
|
||||
|
||||
assert response.status == "success"
|
||||
assert len(response.refs) == 2
|
||||
assert {ref.backend_type.value for ref in response.refs} == {"openviking", "evermemos"}
|
||||
assert {ref.backend_type.value for ref in response.refs} == {"openviking", "everos"}
|
||||
assert {ref.status for ref in repo.list_memory_refs()} == {BackendRefStatus.SUCCESS}
|
||||
assert len(repo.list_memory_refs(backend_type="openviking", status=BackendRefStatus.SUCCESS)) == 1
|
||||
|
||||
|
||||
def test_v2_ingest_service_openviking_real_mock_success_writes_safe_memory_ref():
|
||||
fixture = load_backend_fixture("openviking_ingest_real_success.json")
|
||||
fixture = backend_response("openviking_ingest_real_success.json")
|
||||
|
||||
def handler(request):
|
||||
payload = json.loads(request.content.decode())
|
||||
@ -1012,7 +1053,7 @@ def test_v2_ingest_service_openviking_real_mock_success_writes_safe_memory_ref()
|
||||
service = MemoryGatewayV2Service(
|
||||
repo=repo,
|
||||
openviking_client_factory=real_openviking_factory,
|
||||
evermemos_client=FakeEverMemOSClient(),
|
||||
everos_client=FakeEverOSClient(),
|
||||
)
|
||||
|
||||
response = asyncio.run(
|
||||
@ -1034,10 +1075,10 @@ def test_v2_ingest_service_openviking_real_mock_success_writes_safe_memory_ref()
|
||||
assert "ov-super-secret-token" not in audit_json
|
||||
|
||||
|
||||
def test_v2_ingest_service_real_mock_success_writes_openviking_and_evermemos_refs_safely():
|
||||
ov_fixture = load_backend_fixture("openviking_ingest_real_success.json")
|
||||
em_fixture = load_backend_fixture("evermemos_ingest_success.json")
|
||||
seen = {"openviking": 0, "evermemos": 0}
|
||||
def test_v2_ingest_service_real_mock_success_writes_openviking_and_everos_refs_safely():
|
||||
ov_fixture = backend_response("openviking_ingest_real_success.json")
|
||||
em_fixture = backend_response("everos_ingest_success.json")
|
||||
seen = {"openviking": 0, "everos": 0}
|
||||
|
||||
def openviking_handler(request):
|
||||
payload = json.loads(request.content.decode())
|
||||
@ -1046,12 +1087,12 @@ def test_v2_ingest_service_real_mock_success_writes_openviking_and_evermemos_ref
|
||||
seen["openviking"] += 1
|
||||
return httpx.Response(200, json=ov_fixture)
|
||||
|
||||
def evermemos_handler(request):
|
||||
def everos_handler(request):
|
||||
payload = json.loads(request.content.decode())
|
||||
assert payload["content"] == "SECRET_DUAL_REAL_CONTENT"
|
||||
assert payload["messages"][0]["content"] == "SECRET_DUAL_REAL_CONTENT"
|
||||
assert request.headers["x-api-key"] == "em-dual-token"
|
||||
assert request.headers["authorization"] == "Bearer em-dual-token"
|
||||
seen["evermemos"] += 1
|
||||
seen["everos"] += 1
|
||||
return httpx.Response(200, json=em_fixture)
|
||||
|
||||
async def real_openviking_factory():
|
||||
@ -1066,11 +1107,11 @@ def test_v2_ingest_service_real_mock_success_writes_openviking_and_evermemos_ref
|
||||
service = MemoryGatewayV2Service(
|
||||
repo=repo,
|
||||
openviking_client_factory=real_openviking_factory,
|
||||
evermemos_client=EverMemOSClient(
|
||||
everos_client=EverOSClient(
|
||||
mode="real",
|
||||
base_url="http://evermemos.test",
|
||||
base_url="http://everos.test",
|
||||
api_key="em-dual-token",
|
||||
transport=httpx.MockTransport(evermemos_handler),
|
||||
transport=httpx.MockTransport(everos_handler),
|
||||
),
|
||||
)
|
||||
|
||||
@ -1092,8 +1133,8 @@ def test_v2_ingest_service_real_mock_success_writes_openviking_and_evermemos_ref
|
||||
audit_json = json.dumps([entry.model_dump(mode="json") for entry in repo.list_audit()], ensure_ascii=False)
|
||||
|
||||
assert response.status == OperationStatus.SUCCESS
|
||||
assert seen == {"openviking": 1, "evermemos": 1}
|
||||
assert {ref.backend_type for ref in refs} == {BackendType.OPENVIKING, BackendType.EVERMEMOS}
|
||||
assert seen == {"openviking": 1, "everos": 1}
|
||||
assert {ref.backend_type for ref in refs} == {BackendType.OPENVIKING, BackendType.EVEROS}
|
||||
assert {ref.status for ref in refs} == {BackendRefStatus.SUCCESS}
|
||||
assert {ref.content_hash for ref in refs}
|
||||
assert "trace_dual_real" in serialized_refs
|
||||
@ -1108,7 +1149,7 @@ def test_ingest_service_backend_failure_is_partial_success():
|
||||
service = MemoryGatewayV2Service(
|
||||
repo=repo,
|
||||
openviking_client_factory=fake_openviking_factory,
|
||||
evermemos_client=FailingEverMemOSClient(),
|
||||
everos_client=FailingEverOSClient(),
|
||||
)
|
||||
|
||||
response = asyncio.run(service.ingest_conversation_turn(IngestRequest(**build_ingest_payload())))
|
||||
@ -1117,8 +1158,8 @@ def test_ingest_service_backend_failure_is_partial_success():
|
||||
assert len(response.refs) == 2
|
||||
failed = [ref for ref in response.refs if ref.status == BackendRefStatus.FAILED]
|
||||
assert len(failed) == 1
|
||||
assert failed[0].backend_type.value == "evermemos"
|
||||
assert "evermemos unavailable" in failed[0].error_message
|
||||
assert failed[0].backend_type.value == "everos"
|
||||
assert "everos unavailable" in failed[0].error_message
|
||||
|
||||
|
||||
def test_ingest_service_records_two_skipped_refs_when_policy_disables_backends():
|
||||
@ -1126,7 +1167,7 @@ def test_ingest_service_records_two_skipped_refs_when_policy_disables_backends()
|
||||
service = MemoryGatewayV2Service(
|
||||
repo=repo,
|
||||
openviking_client_factory=fake_openviking_factory,
|
||||
evermemos_client=FakeEverMemOSClient(),
|
||||
everos_client=FakeEverOSClient(),
|
||||
)
|
||||
|
||||
response = asyncio.run(
|
||||
@ -1135,7 +1176,7 @@ def test_ingest_service_records_two_skipped_refs_when_policy_disables_backends()
|
||||
**build_ingest_payload(
|
||||
policy={
|
||||
"allow_openviking": False,
|
||||
"allow_evermemos": False,
|
||||
"allow_everos": False,
|
||||
}
|
||||
)
|
||||
)
|
||||
@ -1153,7 +1194,7 @@ def test_duplicate_idempotency_key_upserts_memory_refs_without_duplicates():
|
||||
service = MemoryGatewayV2Service(
|
||||
repo=repo,
|
||||
openviking_client_factory=fake_openviking_factory,
|
||||
evermemos_client=FakeEverMemOSClient(),
|
||||
everos_client=FakeEverOSClient(),
|
||||
)
|
||||
|
||||
first = asyncio.run(
|
||||
@ -1185,7 +1226,7 @@ def test_memory_ref_metadata_does_not_store_conversation_content_or_raw_request(
|
||||
service = MemoryGatewayV2Service(
|
||||
repo=repo,
|
||||
openviking_client_factory=fake_openviking_factory,
|
||||
evermemos_client=FakeEverMemOSClient(),
|
||||
everos_client=FakeEverOSClient(),
|
||||
)
|
||||
sensitive_content = "SECRET_CONVERSATION_CONTENT_SHOULD_NOT_BE_STORED"
|
||||
|
||||
@ -1213,7 +1254,7 @@ def test_sqlite_repository_persists_v2_memory_refs(tmp_path):
|
||||
service = MemoryGatewayV2Service(
|
||||
repo=repo,
|
||||
openviking_client_factory=fake_openviking_factory,
|
||||
evermemos_client=FakeEverMemOSClient(),
|
||||
everos_client=FakeEverOSClient(),
|
||||
)
|
||||
|
||||
asyncio.run(service.ingest_conversation_turn(IngestRequest(**build_ingest_payload(turn_id="turn_sqlite"))))
|
||||
@ -1253,7 +1294,7 @@ def test_commit_session_creates_commit_job_and_outbox_events():
|
||||
assert job.session_id == "sess_commit"
|
||||
assert job.status.value == "accepted"
|
||||
assert len(events) == 2
|
||||
assert {event.backend_type for event in events} == {BackendType.OPENVIKING, BackendType.EVERMEMOS}
|
||||
assert {event.backend_type for event in events} == {BackendType.OPENVIKING, BackendType.EVEROS}
|
||||
assert {event.operation for event in events} == {BackendOperation.COMMIT_SESSION}
|
||||
assert {event.status for event in events} == {OutboxEventStatus.PENDING}
|
||||
|
||||
@ -1336,7 +1377,7 @@ def test_retrieve_response_contract_contains_items_refs_conflicts_trace_id_statu
|
||||
service = MemoryGatewayV2Service(
|
||||
repo=repo,
|
||||
openviking_client_factory=fake_openviking_factory,
|
||||
evermemos_client=FakeEverMemOSClient(),
|
||||
everos_client=FakeEverOSClient(),
|
||||
)
|
||||
asyncio.run(service.ingest_conversation_turn(IngestRequest(**build_ingest_payload())))
|
||||
|
||||
@ -1357,8 +1398,14 @@ def test_retrieve_response_contract_contains_items_refs_conflicts_trace_id_statu
|
||||
assert set(["items", "refs", "conflicts", "trace_id", "status"]).issubset(dumped)
|
||||
assert response.trace_id == "trace_1"
|
||||
assert response.status.value == "success"
|
||||
assert len(response.items) == len(response.refs)
|
||||
assert [item.source_backend for item in response.items] == [BackendType.EVEROS, BackendType.OPENVIKING]
|
||||
assert [item.score for item in response.items] == [0.91, 0.82]
|
||||
assert len(response.refs) == 2
|
||||
assert response.conflicts == []
|
||||
assert response.metadata["backend_results"] == [
|
||||
{"backend_type": "openviking", "status": "success", "items": 1, "error_code": None, "error_message": None},
|
||||
{"backend_type": "everos", "status": "success", "items": 1, "error_code": None, "error_message": None},
|
||||
]
|
||||
|
||||
|
||||
def test_process_commit_job_success_updates_job_and_writes_memory_refs():
|
||||
@ -1368,8 +1415,8 @@ def test_process_commit_job_success_updates_job_and_writes_memory_refs():
|
||||
openviking_client_factory=fake_commit_openviking_factory(
|
||||
commit_result(BackendType.OPENVIKING, BackendResultStatus.SUCCESS, native_id="ov_commit_1", native_uri="viking://sessions/sess_commit")
|
||||
),
|
||||
evermemos_client=FakeCommitEverMemOSClient(
|
||||
commit_result(BackendType.EVERMEMOS, BackendResultStatus.SUCCESS, native_id="em_commit_1", native_uri="evermemos://memories/em_commit_1")
|
||||
everos_client=FakeCommitEverOSClient(
|
||||
commit_result(BackendType.EVEROS, BackendResultStatus.SUCCESS, native_id="em_commit_1", native_uri="everos://memories/em_commit_1")
|
||||
),
|
||||
)
|
||||
response = asyncio.run(
|
||||
@ -1389,7 +1436,7 @@ def test_process_commit_job_success_updates_job_and_writes_memory_refs():
|
||||
assert job.created_refs_count == 2
|
||||
assert {event.status for event in events} == {OutboxEventStatus.SUCCESS}
|
||||
assert len(refs) == 2
|
||||
assert {ref.backend_type for ref in refs} == {BackendType.OPENVIKING, BackendType.EVERMEMOS}
|
||||
assert {ref.backend_type for ref in refs} == {BackendType.OPENVIKING, BackendType.EVEROS}
|
||||
|
||||
|
||||
def test_process_outbox_event_writes_multiple_produced_memory_refs():
|
||||
@ -1529,8 +1576,8 @@ def test_process_commit_job_one_success_one_failed_is_partial_success():
|
||||
openviking_client_factory=fake_commit_openviking_factory(
|
||||
commit_result(BackendType.OPENVIKING, BackendResultStatus.SUCCESS, native_id="ov_commit_1")
|
||||
),
|
||||
evermemos_client=FakeCommitEverMemOSClient(
|
||||
commit_result(BackendType.EVERMEMOS, BackendResultStatus.FAILED, retryable=False, error_message="evermemos failed")
|
||||
everos_client=FakeCommitEverOSClient(
|
||||
commit_result(BackendType.EVEROS, BackendResultStatus.FAILED, retryable=False, error_message="everos failed")
|
||||
),
|
||||
)
|
||||
response = asyncio.run(
|
||||
@ -1542,7 +1589,7 @@ def test_process_commit_job_one_success_one_failed_is_partial_success():
|
||||
|
||||
assert job.status.value == "partial_success"
|
||||
assert job.created_refs_count == 1
|
||||
assert "evermemos failed" in job.error_message
|
||||
assert "everos failed" in job.error_message
|
||||
assert {event.status for event in events} == {OutboxEventStatus.SUCCESS, OutboxEventStatus.DEAD_LETTER}
|
||||
|
||||
|
||||
@ -1553,8 +1600,8 @@ def test_process_commit_job_two_failed_is_failed():
|
||||
openviking_client_factory=fake_commit_openviking_factory(
|
||||
commit_result(BackendType.OPENVIKING, BackendResultStatus.FAILED, retryable=False, error_message="openviking failed")
|
||||
),
|
||||
evermemos_client=FakeCommitEverMemOSClient(
|
||||
commit_result(BackendType.EVERMEMOS, BackendResultStatus.FAILED, retryable=False, error_message="evermemos failed")
|
||||
everos_client=FakeCommitEverOSClient(
|
||||
commit_result(BackendType.EVEROS, BackendResultStatus.FAILED, retryable=False, error_message="everos failed")
|
||||
),
|
||||
)
|
||||
response = asyncio.run(
|
||||
@ -1566,7 +1613,7 @@ def test_process_commit_job_two_failed_is_failed():
|
||||
assert job.status.value == "failed"
|
||||
assert job.created_refs_count == 0
|
||||
assert "openviking failed" in job.error_message
|
||||
assert "evermemos failed" in job.error_message
|
||||
assert "everos failed" in job.error_message
|
||||
|
||||
|
||||
def test_retryable_failed_outbox_event_requeues_with_next_retry():
|
||||
@ -1597,8 +1644,8 @@ def test_process_pending_outbox_events_processes_pending_batch():
|
||||
openviking_client_factory=fake_commit_openviking_factory(
|
||||
commit_result(BackendType.OPENVIKING, BackendResultStatus.SUCCESS, native_id="ov_commit_1")
|
||||
),
|
||||
evermemos_client=FakeCommitEverMemOSClient(
|
||||
commit_result(BackendType.EVERMEMOS, BackendResultStatus.SUCCESS, native_id="em_commit_1")
|
||||
everos_client=FakeCommitEverOSClient(
|
||||
commit_result(BackendType.EVEROS, BackendResultStatus.SUCCESS, native_id="em_commit_1")
|
||||
),
|
||||
)
|
||||
asyncio.run(
|
||||
@ -1641,8 +1688,8 @@ def test_commit_pipeline_metadata_does_not_store_content_or_raw_request():
|
||||
openviking_client_factory=fake_commit_openviking_factory(
|
||||
commit_result(BackendType.OPENVIKING, BackendResultStatus.SUCCESS, native_id="ov_commit_1")
|
||||
),
|
||||
evermemos_client=FakeCommitEverMemOSClient(
|
||||
commit_result(BackendType.EVERMEMOS, BackendResultStatus.SUCCESS, native_id="em_commit_1")
|
||||
everos_client=FakeCommitEverOSClient(
|
||||
commit_result(BackendType.EVEROS, BackendResultStatus.SUCCESS, native_id="em_commit_1")
|
||||
),
|
||||
)
|
||||
sensitive_content = "SECRET_COMMIT_PIPELINE_CONTENT_SHOULD_NOT_BE_STORED"
|
||||
@ -1728,8 +1775,8 @@ def test_process_pending_outbox_events_uses_claim_and_does_not_process_existing_
|
||||
openviking_client_factory=fake_commit_openviking_factory(
|
||||
commit_result(BackendType.OPENVIKING, BackendResultStatus.SUCCESS, native_id="ov_claimed")
|
||||
),
|
||||
evermemos_client=FakeCommitEverMemOSClient(
|
||||
commit_result(BackendType.EVERMEMOS, BackendResultStatus.SUCCESS, native_id="em_claimed")
|
||||
everos_client=FakeCommitEverOSClient(
|
||||
commit_result(BackendType.EVEROS, BackendResultStatus.SUCCESS, native_id="em_claimed")
|
||||
),
|
||||
)
|
||||
response = asyncio.run(
|
||||
@ -1754,8 +1801,8 @@ def test_terminal_outbox_statuses_clear_lock_fields():
|
||||
openviking_client_factory=fake_commit_openviking_factory(
|
||||
commit_result(BackendType.OPENVIKING, BackendResultStatus.SUCCESS, native_id="ov_lock_clear")
|
||||
),
|
||||
evermemos_client=FakeCommitEverMemOSClient(
|
||||
commit_result(BackendType.EVERMEMOS, BackendResultStatus.SKIPPED)
|
||||
everos_client=FakeCommitEverOSClient(
|
||||
commit_result(BackendType.EVEROS, BackendResultStatus.SKIPPED)
|
||||
),
|
||||
)
|
||||
response = asyncio.run(
|
||||
@ -1848,8 +1895,8 @@ def test_admin_process_outbox_endpoint_triggers_pending_processing(monkeypatch):
|
||||
openviking_client_factory=fake_commit_openviking_factory(
|
||||
commit_result(BackendType.OPENVIKING, BackendResultStatus.SUCCESS, native_id="ov_admin")
|
||||
),
|
||||
evermemos_client=FakeCommitEverMemOSClient(
|
||||
commit_result(BackendType.EVERMEMOS, BackendResultStatus.SUCCESS, native_id="em_admin")
|
||||
everos_client=FakeCommitEverOSClient(
|
||||
commit_result(BackendType.EVEROS, BackendResultStatus.SUCCESS, native_id="em_admin")
|
||||
),
|
||||
)
|
||||
asyncio.run(
|
||||
@ -1907,7 +1954,7 @@ def test_v2_ingest_router_accepts_legal_request(monkeypatch):
|
||||
api_v2.v2_service = MemoryGatewayV2Service(
|
||||
repo=InMemoryRepository(),
|
||||
openviking_client_factory=fake_openviking_factory,
|
||||
evermemos_client=FakeEverMemOSClient(),
|
||||
everos_client=FakeEverOSClient(),
|
||||
)
|
||||
app = FastAPI()
|
||||
app.dependency_overrides[verify_api_key_compat] = lambda: None
|
||||
|
||||
Reference in New Issue
Block a user