Compare commits
4 Commits
827e3434b3
...
269661afff
| Author | SHA1 | Date | |
|---|---|---|---|
| 269661afff | |||
| e9e57bdb07 | |||
| 8b57159d46 | |||
| a7fe41e6a5 |
@ -67,6 +67,7 @@ WORKDIR /opt/app/backend
|
|||||||
|
|
||||||
COPY backend/pyproject.toml backend/README.md ./
|
COPY backend/pyproject.toml backend/README.md ./
|
||||||
COPY backend/beaver/ ./beaver/
|
COPY backend/beaver/ ./beaver/
|
||||||
|
COPY backend/memory/ ./memory/
|
||||||
RUN uv pip install --system --no-cache --index-url "${PYPI_INDEX_URL}" ".[channels]"
|
RUN uv pip install --system --no-cache --index-url "${PYPI_INDEX_URL}" ".[channels]"
|
||||||
|
|
||||||
WORKDIR /opt/app/frontend
|
WORKDIR /opt/app/frontend
|
||||||
|
|||||||
@ -110,6 +110,8 @@ runtime/instances/<instance-slug>/
|
|||||||
runtime/instances/<instance-slug>/
|
runtime/instances/<instance-slug>/
|
||||||
└── beaver-home
|
└── beaver-home
|
||||||
├── config.json
|
├── config.json
|
||||||
|
├── memory_gateway_users.json
|
||||||
|
├── runtime.env
|
||||||
├── web_auth_users.json
|
├── web_auth_users.json
|
||||||
└── workspace/
|
└── workspace/
|
||||||
```
|
```
|
||||||
@ -125,10 +127,21 @@ runtime/instances/<instance-slug>/
|
|||||||
```text
|
```text
|
||||||
BEAVER_CONFIG_PATH=/root/.beaver/config.json
|
BEAVER_CONFIG_PATH=/root/.beaver/config.json
|
||||||
BEAVER_WORKSPACE=/root/.beaver/workspace
|
BEAVER_WORKSPACE=/root/.beaver/workspace
|
||||||
|
BEAVER_MEMORY_GATEWAY_USERS_PATH=/root/.beaver/memory_gateway_users.json
|
||||||
```
|
```
|
||||||
|
|
||||||
所以模型 `provider/api_key/api_base/model` 配一次即可,Web / channel 请求不需要、也不应该携带 API Key。
|
所以模型 `provider/api_key/api_base/model` 配一次即可,Web / channel 请求不需要、也不应该携带 API Key。
|
||||||
|
|
||||||
|
Memory Gateway 的共享非密钥配置不放在实例目录里,而是放在仓库内的:
|
||||||
|
|
||||||
|
```text
|
||||||
|
app-instance/backend/memory/config.json
|
||||||
|
```
|
||||||
|
|
||||||
|
实例目录只保存按 Beaver 登录用户名分组的 Gateway 凭证。`create-instance.sh`
|
||||||
|
会初始化空的 `memory_gateway_users.json`,容器启动时也会兜底创建这个文件并设置
|
||||||
|
`0600` 权限。
|
||||||
|
|
||||||
`create-instance.sh` 默认会把仓库根目录的 `skills/` 非覆盖式复制到实例 workspace,并把同一个目录只读挂载到实例容器的 `/opt/app/initial-skills`。`entrypoint.sh` 每次启动都会用该目录补齐缺失的 published 初始 skills;已有 skill 目录不会被覆盖,index 只做并集追加。
|
`create-instance.sh` 默认会把仓库根目录的 `skills/` 非覆盖式复制到实例 workspace,并把同一个目录只读挂载到实例容器的 `/opt/app/initial-skills`。`entrypoint.sh` 每次启动都会用该目录补齐缺失的 published 初始 skills;已有 skill 目录不会被覆盖,index 只做并集追加。
|
||||||
|
|
||||||
## 当前状态
|
## 当前状态
|
||||||
|
|||||||
@ -35,19 +35,23 @@ Curated memory 始终启用:每轮仍会冻结并注入 `MEMORY.md` / `USER.md
|
|||||||
每轮先调用 `/memories/search`,正常完成后调用一次 `/memories/add`,成功后再调用
|
每轮先调用 `/memories/search`,正常完成后调用一次 `/memories/add`,成功后再调用
|
||||||
一次 `/memories/flush`。两套存储不会互相同步、覆盖或去重。
|
一次 `/memories/flush`。两套存储不会互相同步、覆盖或去重。
|
||||||
|
|
||||||
完整配置示例:
|
共享 Gateway 配置放在:
|
||||||
|
|
||||||
|
```text
|
||||||
|
app-instance/backend/memory/config.json
|
||||||
|
```
|
||||||
|
|
||||||
|
当前默认内容:
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{
|
{
|
||||||
"memory": {
|
"memory": {
|
||||||
"mode": "hybrid",
|
"mode": "hybrid",
|
||||||
"gateway": {
|
"gateway": {
|
||||||
"baseUrl": "http://127.0.0.1:8010",
|
"baseUrl": "http://172.19.207.37:8010",
|
||||||
"userId": "gateway_test_user",
|
|
||||||
"userKey": "uk_xxx",
|
|
||||||
"appId": "default",
|
"appId": "default",
|
||||||
"projectId": "default",
|
"projectId": "default",
|
||||||
"scope": ["current_chat", "resources"],
|
"scope": ["current_chat", "resources", "all_user_memory"],
|
||||||
"topK": 8,
|
"topK": 8,
|
||||||
"timeoutSeconds": 10
|
"timeoutSeconds": 10
|
||||||
}
|
}
|
||||||
@ -55,10 +59,28 @@ Curated memory 始终启用:每轮仍会冻结并注入 `MEMORY.md` / `USER.md
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
- `memory` 整段缺失时,默认采用隐式 `hybrid`;Gateway 凭证不完整会告警并只运行 curated memory。
|
每个实例自己的 Gateway 用户凭证放在:
|
||||||
- 显式配置 `"mode": "hybrid"` 时,`baseUrl`、`userId` 和 `userKey` 缺失会导致启动失败。
|
|
||||||
- 配置 `"mode": "curated"` 可关闭 Gateway,curated memory 行为不变。
|
```text
|
||||||
|
/root/.beaver/memory_gateway_users.json
|
||||||
|
```
|
||||||
|
|
||||||
|
格式示例:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"users": {
|
||||||
|
"tom": {
|
||||||
|
"userId": "tom",
|
||||||
|
"userKey": "uk_xxx"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
- 前端 `POST /api/auth/register` 会用 Beaver 登录用户名调用 Gateway `POST /users`,并把返回的 `userId/userKey` 写入实例凭证文件。
|
||||||
|
- REST `/api/chat` 和 WebSocket `/ws/...` 只使用登录 token 解析出的 Beaver 用户名来选择 Gateway 凭证,请求体里的 `user_id` 不参与 Gateway 身份选择。
|
||||||
|
- 某个登录用户还没有 Gateway 凭证时,这一轮只走 curated memory,不会报 chat 级错误。
|
||||||
|
- `BEAVER_MEMORY_CONFIG_PATH` 可覆盖共享 memory 配置路径,`BEAVER_MEMORY_GATEWAY_USERS_PATH` 可覆盖实例凭证路径。
|
||||||
- `userKey` 是密钥,不应写入日志、状态响应或提交到版本库。
|
- `userKey` 是密钥,不应写入日志、状态响应或提交到版本库。
|
||||||
- 容器访问宿主机 Gateway 时不能使用容器内的 `127.0.0.1`。应让 Gateway 监听
|
- 修改共享 memory 配置后需要重启 runtime,因为 Gateway 相关对象在 `EngineLoader` 启动时装配。
|
||||||
`0.0.0.0`,并把 `baseUrl` 配成该 Docker 网络的宿主机网关地址。
|
|
||||||
- 修改 memory 配置后需要重启 runtime,因为 Gateway 服务在 `EngineLoader` 启动时创建。
|
|
||||||
|
|||||||
@ -15,10 +15,16 @@ from beaver.engine.session import SessionManager
|
|||||||
from beaver.foundation.config import BeaverConfig, load_config
|
from beaver.foundation.config import BeaverConfig, load_config
|
||||||
from beaver.integrations.mcp import MCPConnectionManager
|
from beaver.integrations.mcp import MCPConnectionManager
|
||||||
from beaver.memory.curated.store import MemoryStore
|
from beaver.memory.curated.store import MemoryStore
|
||||||
|
from beaver.memory.gateway import (
|
||||||
|
MemoryGatewayConfig,
|
||||||
|
MemoryGatewayCredentialStore,
|
||||||
|
MemoryGatewayService,
|
||||||
|
MemoryGatewayUserCredential,
|
||||||
|
default_memory_gateway_users_path,
|
||||||
|
)
|
||||||
from beaver.memory.runs import RunMemoryStore
|
from beaver.memory.runs import RunMemoryStore
|
||||||
from beaver.memory.skills import SkillLearningStore
|
from beaver.memory.skills import SkillLearningStore
|
||||||
from beaver.services.memory_service import MemoryService
|
from beaver.services.memory_service import MemoryService
|
||||||
from beaver.services.memory_gateway_service import MemoryGatewayService
|
|
||||||
from beaver.skills.drafts import DraftService
|
from beaver.skills.drafts import DraftService
|
||||||
from beaver.skills.learning import EvidenceSelector, SkillDraftSynthesizer, SkillLearningPipelineService, SkillLearningService
|
from beaver.skills.learning import EvidenceSelector, SkillDraftSynthesizer, SkillLearningPipelineService, SkillLearningService
|
||||||
from beaver.skills.learning.safety import SkillDraftSafetyChecker
|
from beaver.skills.learning.safety import SkillDraftSafetyChecker
|
||||||
@ -84,7 +90,9 @@ class EngineLoadResult:
|
|||||||
session_manager: SessionManager | None = None
|
session_manager: SessionManager | None = None
|
||||||
curated_memory_store: MemoryStore | None = None
|
curated_memory_store: MemoryStore | None = None
|
||||||
memory_service: MemoryService | None = None
|
memory_service: MemoryService | None = None
|
||||||
memory_gateway_service: MemoryGatewayService | None = None
|
memory_gateway_config: MemoryGatewayConfig | None = None
|
||||||
|
memory_gateway_credentials: MemoryGatewayCredentialStore | None = None
|
||||||
|
memory_gateway_service_factory: Callable[[MemoryGatewayUserCredential], MemoryGatewayService] | None = None
|
||||||
run_memory_store: RunMemoryStore | None = None
|
run_memory_store: RunMemoryStore | None = None
|
||||||
skill_learning_store: SkillLearningStore | None = None
|
skill_learning_store: SkillLearningStore | None = None
|
||||||
tool_registry: ToolRegistry | None = None
|
tool_registry: ToolRegistry | None = None
|
||||||
@ -160,7 +168,8 @@ class EngineLoader:
|
|||||||
session_manager: SessionManager | None = None,
|
session_manager: SessionManager | None = None,
|
||||||
curated_memory_store: MemoryStore | None = None,
|
curated_memory_store: MemoryStore | None = None,
|
||||||
memory_service: MemoryService | None = None,
|
memory_service: MemoryService | None = None,
|
||||||
memory_gateway_service: MemoryGatewayService | None = None,
|
memory_gateway_credentials: MemoryGatewayCredentialStore | None = None,
|
||||||
|
memory_gateway_service_factory: Callable[[MemoryGatewayConfig, MemoryGatewayUserCredential], MemoryGatewayService] | None = None,
|
||||||
run_memory_store: RunMemoryStore | None = None,
|
run_memory_store: RunMemoryStore | None = None,
|
||||||
skill_learning_store: SkillLearningStore | None = None,
|
skill_learning_store: SkillLearningStore | None = None,
|
||||||
tool_registry: ToolRegistry | None = None,
|
tool_registry: ToolRegistry | None = None,
|
||||||
@ -186,7 +195,8 @@ class EngineLoader:
|
|||||||
self._session_manager = session_manager
|
self._session_manager = session_manager
|
||||||
self._curated_memory_store = curated_memory_store
|
self._curated_memory_store = curated_memory_store
|
||||||
self._memory_service = memory_service
|
self._memory_service = memory_service
|
||||||
self._memory_gateway_service = memory_gateway_service
|
self._memory_gateway_credentials = memory_gateway_credentials
|
||||||
|
self._memory_gateway_service_factory = memory_gateway_service_factory
|
||||||
self._run_memory_store = run_memory_store
|
self._run_memory_store = run_memory_store
|
||||||
self._skill_learning_store = skill_learning_store
|
self._skill_learning_store = skill_learning_store
|
||||||
self._tool_registry = tool_registry
|
self._tool_registry = tool_registry
|
||||||
@ -209,7 +219,11 @@ class EngineLoader:
|
|||||||
"""装配当前主链需要的最小 runtime 对象。"""
|
"""装配当前主链需要的最小 runtime 对象。"""
|
||||||
|
|
||||||
workspace = self.workspace
|
workspace = self.workspace
|
||||||
memory_gateway_service = self._resolve_memory_gateway_service()
|
(
|
||||||
|
memory_gateway_config,
|
||||||
|
memory_gateway_credentials,
|
||||||
|
memory_gateway_service_factory,
|
||||||
|
) = self._resolve_memory_gateway_components()
|
||||||
session_manager = self._session_manager or SessionManager(workspace)
|
session_manager = self._session_manager or SessionManager(workspace)
|
||||||
|
|
||||||
curated_root = workspace / "memory" / "curated"
|
curated_root = workspace / "memory" / "curated"
|
||||||
@ -306,12 +320,14 @@ class EngineLoader:
|
|||||||
config=self.config,
|
config=self.config,
|
||||||
tools=[spec.name for spec in tool_registry.list_specs()],
|
tools=[spec.name for spec in tool_registry.list_specs()],
|
||||||
skills=[record.name for record in skills_loader.list_skills(filter_unavailable=False)],
|
skills=[record.name for record in skills_loader.list_skills(filter_unavailable=False)],
|
||||||
memory_stores=["curated", *(["memory_gateway"] if memory_gateway_service is not None else [])],
|
memory_stores=["curated", *(["memory_gateway"] if memory_gateway_service_factory is not None else [])],
|
||||||
permissions=[],
|
permissions=[],
|
||||||
session_manager=session_manager,
|
session_manager=session_manager,
|
||||||
curated_memory_store=memory_service.get_store(),
|
curated_memory_store=memory_service.get_store(),
|
||||||
memory_service=memory_service,
|
memory_service=memory_service,
|
||||||
memory_gateway_service=memory_gateway_service,
|
memory_gateway_config=memory_gateway_config,
|
||||||
|
memory_gateway_credentials=memory_gateway_credentials,
|
||||||
|
memory_gateway_service_factory=memory_gateway_service_factory,
|
||||||
run_memory_store=run_memory_store,
|
run_memory_store=run_memory_store,
|
||||||
skill_learning_store=skill_learning_store,
|
skill_learning_store=skill_learning_store,
|
||||||
tool_registry=tool_registry,
|
tool_registry=tool_registry,
|
||||||
@ -337,10 +353,16 @@ class EngineLoader:
|
|||||||
result.register_closeable("mcp_manager", lambda: _close_mcp_manager(mcp_manager))
|
result.register_closeable("mcp_manager", lambda: _close_mcp_manager(mcp_manager))
|
||||||
return result
|
return result
|
||||||
|
|
||||||
def _resolve_memory_gateway_service(self) -> MemoryGatewayService | None:
|
def _resolve_memory_gateway_components(
|
||||||
|
self,
|
||||||
|
) -> tuple[
|
||||||
|
MemoryGatewayConfig | None,
|
||||||
|
MemoryGatewayCredentialStore | None,
|
||||||
|
Callable[[MemoryGatewayUserCredential], MemoryGatewayService] | None,
|
||||||
|
]:
|
||||||
memory_config = self.config.memory
|
memory_config = self.config.memory
|
||||||
if memory_config.mode == "curated":
|
if memory_config.mode == "curated":
|
||||||
return None
|
return None, None, None
|
||||||
|
|
||||||
gateway_config = memory_config.gateway
|
gateway_config = memory_config.gateway
|
||||||
if memory_config.explicit and not gateway_config.is_configured:
|
if memory_config.explicit and not gateway_config.is_configured:
|
||||||
@ -351,8 +373,18 @@ class EngineLoader:
|
|||||||
logger.warning(
|
logger.warning(
|
||||||
"Memory Gateway is not configured; continuing with curated memory only"
|
"Memory Gateway is not configured; continuing with curated memory only"
|
||||||
)
|
)
|
||||||
return None
|
return None, None, None
|
||||||
return self._memory_gateway_service or MemoryGatewayService(gateway_config)
|
|
||||||
|
credential_store = self._memory_gateway_credentials or MemoryGatewayCredentialStore(
|
||||||
|
default_memory_gateway_users_path()
|
||||||
|
)
|
||||||
|
|
||||||
|
def factory(credential: MemoryGatewayUserCredential) -> MemoryGatewayService:
|
||||||
|
if self._memory_gateway_service_factory is not None:
|
||||||
|
return self._memory_gateway_service_factory(gateway_config, credential)
|
||||||
|
return MemoryGatewayService(gateway_config, credential)
|
||||||
|
|
||||||
|
return gateway_config, credential_store, factory
|
||||||
|
|
||||||
|
|
||||||
def _close_mcp_manager(manager: MCPConnectionManager) -> None:
|
def _close_mcp_manager(manager: MCPConnectionManager) -> None:
|
||||||
|
|||||||
@ -227,6 +227,7 @@ class AgentLoop:
|
|||||||
session_id: str | None = None,
|
session_id: str | None = None,
|
||||||
source: str = "direct",
|
source: str = "direct",
|
||||||
user_id: str | None = None,
|
user_id: str | None = None,
|
||||||
|
gateway_user_id: str | None = None,
|
||||||
title: str | None = None,
|
title: str | None = None,
|
||||||
execution_context: str | None = None,
|
execution_context: str | None = None,
|
||||||
skill_selection_context: str | None = None,
|
skill_selection_context: str | None = None,
|
||||||
@ -279,6 +280,7 @@ class AgentLoop:
|
|||||||
session_id=session_id,
|
session_id=session_id,
|
||||||
source=source,
|
source=source,
|
||||||
user_id=user_id,
|
user_id=user_id,
|
||||||
|
gateway_user_id=gateway_user_id,
|
||||||
title=title,
|
title=title,
|
||||||
execution_context=execution_context,
|
execution_context=execution_context,
|
||||||
skill_selection_context=skill_selection_context,
|
skill_selection_context=skill_selection_context,
|
||||||
@ -319,6 +321,7 @@ class AgentLoop:
|
|||||||
session_id: str | None = None,
|
session_id: str | None = None,
|
||||||
source: str = "direct",
|
source: str = "direct",
|
||||||
user_id: str | None = None,
|
user_id: str | None = None,
|
||||||
|
gateway_user_id: str | None = None,
|
||||||
title: str | None = None,
|
title: str | None = None,
|
||||||
execution_context: str | None = None,
|
execution_context: str | None = None,
|
||||||
skill_selection_context: str | None = None,
|
skill_selection_context: str | None = None,
|
||||||
@ -360,6 +363,13 @@ class AgentLoop:
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
loaded = self.boot()
|
loaded = self.boot()
|
||||||
|
memory_gateway_service = None
|
||||||
|
gateway_credential_store = getattr(loaded, "memory_gateway_credentials", None)
|
||||||
|
gateway_service_factory = getattr(loaded, "memory_gateway_service_factory", None)
|
||||||
|
if gateway_user_id and gateway_credential_store is not None and gateway_service_factory is not None:
|
||||||
|
gateway_credential = gateway_credential_store.get(gateway_user_id)
|
||||||
|
if gateway_credential is not None:
|
||||||
|
memory_gateway_service = gateway_service_factory(gateway_credential)
|
||||||
session_manager = self._require_loaded("session_manager")
|
session_manager = self._require_loaded("session_manager")
|
||||||
memory_service = self._require_loaded("memory_service")
|
memory_service = self._require_loaded("memory_service")
|
||||||
context_builder = self._require_loaded("context_builder")
|
context_builder = self._require_loaded("context_builder")
|
||||||
@ -482,7 +492,6 @@ class AgentLoop:
|
|||||||
final_model: str | None = resolved_model
|
final_model: str | None = resolved_model
|
||||||
run_started_at = self._utc_now()
|
run_started_at = self._utc_now()
|
||||||
activated_receipts: list[SkillActivationReceipt] = []
|
activated_receipts: list[SkillActivationReceipt] = []
|
||||||
memory_gateway_service = getattr(loaded, "memory_gateway_service", None)
|
|
||||||
try:
|
try:
|
||||||
bundle = provider_bundle or make_provider_bundle(
|
bundle = provider_bundle or make_provider_bundle(
|
||||||
model=resolved_model,
|
model=resolved_model,
|
||||||
|
|||||||
@ -1,6 +1,6 @@
|
|||||||
"""Configuration models and loaders."""
|
"""Configuration models and loaders."""
|
||||||
|
|
||||||
from .loader import default_config_path, load_config
|
from .loader import default_config_path, default_memory_config_path, load_config
|
||||||
from .schema import (
|
from .schema import (
|
||||||
AgentDefaultsConfig,
|
AgentDefaultsConfig,
|
||||||
AuthzConfig,
|
AuthzConfig,
|
||||||
@ -26,5 +26,6 @@ __all__ = [
|
|||||||
"ProviderConfig",
|
"ProviderConfig",
|
||||||
"ToolsConfig",
|
"ToolsConfig",
|
||||||
"default_config_path",
|
"default_config_path",
|
||||||
|
"default_memory_config_path",
|
||||||
"load_config",
|
"load_config",
|
||||||
]
|
]
|
||||||
|
|||||||
@ -55,6 +55,16 @@ def default_config_path(*, workspace: str | Path | None = None) -> Path:
|
|||||||
return root / ".beaver" / "config.json"
|
return root / ".beaver" / "config.json"
|
||||||
|
|
||||||
|
|
||||||
|
def default_memory_config_path() -> Path:
|
||||||
|
"""Resolve the shared Memory Gateway config path."""
|
||||||
|
|
||||||
|
explicit = os.getenv("BEAVER_MEMORY_CONFIG_PATH")
|
||||||
|
if explicit:
|
||||||
|
return Path(explicit).expanduser()
|
||||||
|
|
||||||
|
return Path(__file__).resolve().parents[3] / "memory" / "config.json"
|
||||||
|
|
||||||
|
|
||||||
def load_config(
|
def load_config(
|
||||||
*,
|
*,
|
||||||
workspace: str | Path | None = None,
|
workspace: str | Path | None = None,
|
||||||
@ -63,24 +73,38 @@ def load_config(
|
|||||||
"""Load backend config; missing config is treated as an empty config."""
|
"""Load backend config; missing config is treated as an empty config."""
|
||||||
|
|
||||||
path = Path(config_path).expanduser() if config_path is not None else default_config_path(workspace=workspace)
|
path = Path(config_path).expanduser() if config_path is not None else default_config_path(workspace=workspace)
|
||||||
|
data: dict[str, Any] | None = None
|
||||||
|
if path.exists():
|
||||||
|
loaded = json.loads(path.read_text(encoding="utf-8"))
|
||||||
|
if not isinstance(loaded, dict):
|
||||||
|
raise ValueError(f"Beaver config must be a JSON object: {path}")
|
||||||
|
data = loaded
|
||||||
|
memory_data = _load_memory_config_data()
|
||||||
|
|
||||||
|
return BeaverConfig(
|
||||||
|
agents_defaults=_parse_agent_defaults(data or {}),
|
||||||
|
providers=_parse_providers((data or {}).get("providers")),
|
||||||
|
embedding=_parse_embedding(data or {}),
|
||||||
|
tools=_parse_tools((data or {}).get("tools")) if data is not None else ToolsConfig(),
|
||||||
|
authz=_parse_authz((data or {}).get("authz")),
|
||||||
|
channels=_parse_channels((data or {}).get("channels")),
|
||||||
|
backend_identity=_parse_backend_identity(
|
||||||
|
(data or {}).get("backend_identity") or (data or {}).get("backendIdentity")
|
||||||
|
),
|
||||||
|
memory=_parse_memory(memory_data),
|
||||||
|
config_path=path,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _load_memory_config_data() -> dict[str, Any]:
|
||||||
|
path = default_memory_config_path()
|
||||||
if not path.exists():
|
if not path.exists():
|
||||||
return BeaverConfig(config_path=path)
|
return {}
|
||||||
|
|
||||||
data = json.loads(path.read_text(encoding="utf-8"))
|
data = json.loads(path.read_text(encoding="utf-8"))
|
||||||
if not isinstance(data, dict):
|
if not isinstance(data, dict):
|
||||||
raise ValueError(f"Beaver config must be a JSON object: {path}")
|
raise ValueError(f"Beaver memory config must be a JSON object: {path}")
|
||||||
|
return data
|
||||||
return BeaverConfig(
|
|
||||||
agents_defaults=_parse_agent_defaults(data),
|
|
||||||
providers=_parse_providers(data.get("providers")),
|
|
||||||
embedding=_parse_embedding(data),
|
|
||||||
tools=_parse_tools(data.get("tools")),
|
|
||||||
authz=_parse_authz(data.get("authz")),
|
|
||||||
channels=_parse_channels(data.get("channels")),
|
|
||||||
backend_identity=_parse_backend_identity(data.get("backend_identity") or data.get("backendIdentity")),
|
|
||||||
memory=_parse_memory(data),
|
|
||||||
config_path=path,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_agent_defaults(data: dict[str, Any]) -> AgentDefaultsConfig:
|
def _parse_agent_defaults(data: dict[str, Any]) -> AgentDefaultsConfig:
|
||||||
@ -269,12 +293,10 @@ def _parse_memory(data: dict[str, Any]) -> MemoryConfig:
|
|||||||
scope = (
|
scope = (
|
||||||
_string_list(gateway_raw.get("scope"))
|
_string_list(gateway_raw.get("scope"))
|
||||||
if "scope" in gateway_raw
|
if "scope" in gateway_raw
|
||||||
else ["current_chat", "resources"]
|
else MemoryGatewayConfig().scope
|
||||||
)
|
)
|
||||||
gateway = MemoryGatewayConfig(
|
gateway = MemoryGatewayConfig(
|
||||||
base_url=_string(gateway_raw.get("baseUrl") or gateway_raw.get("base_url")) or "",
|
base_url=_string(gateway_raw.get("baseUrl") or gateway_raw.get("base_url")) or "",
|
||||||
user_id=_string(gateway_raw.get("userId") or gateway_raw.get("user_id")) or "",
|
|
||||||
user_key=_string(gateway_raw.get("userKey") or gateway_raw.get("user_key")) or "",
|
|
||||||
app_id=_string(gateway_raw.get("appId") or gateway_raw.get("app_id")) or "default",
|
app_id=_string(gateway_raw.get("appId") or gateway_raw.get("app_id")) or "default",
|
||||||
project_id=_string(gateway_raw.get("projectId") or gateway_raw.get("project_id")) or "default",
|
project_id=_string(gateway_raw.get("projectId") or gateway_raw.get("project_id")) or "default",
|
||||||
scope=scope,
|
scope=scope,
|
||||||
@ -283,15 +305,8 @@ def _parse_memory(data: dict[str, Any]) -> MemoryConfig:
|
|||||||
)
|
)
|
||||||
|
|
||||||
if mode == "hybrid" and explicit:
|
if mode == "hybrid" and explicit:
|
||||||
missing: list[str] = []
|
|
||||||
if not gateway.base_url:
|
if not gateway.base_url:
|
||||||
missing.append("baseUrl")
|
raise ValueError("Explicit hybrid memory requires gateway.baseUrl")
|
||||||
if not gateway.user_id:
|
|
||||||
missing.append("userId")
|
|
||||||
if not gateway.user_key:
|
|
||||||
missing.append("userKey")
|
|
||||||
if missing:
|
|
||||||
raise ValueError(f"Explicit hybrid memory requires gateway fields: {', '.join(missing)}")
|
|
||||||
allowed_scopes = {"current_chat", "resources", "all_user_memory"}
|
allowed_scopes = {"current_chat", "resources", "all_user_memory"}
|
||||||
if not gateway.scope or any(scope not in allowed_scopes for scope in gateway.scope):
|
if not gateway.scope or any(scope not in allowed_scopes for scope in gateway.scope):
|
||||||
raise ValueError("memory.gateway.scope contains an unsupported value")
|
raise ValueError("memory.gateway.scope contains an unsupported value")
|
||||||
|
|||||||
@ -6,6 +6,8 @@ from dataclasses import dataclass, field
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
|
from beaver.memory.gateway import MemoryConfig, MemoryGatewayConfig
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
@dataclass(slots=True)
|
||||||
class ProviderConfig:
|
class ProviderConfig:
|
||||||
@ -115,33 +117,6 @@ class BackendIdentityConfig:
|
|||||||
public_base_url: str = ""
|
public_base_url: str = ""
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
|
||||||
class MemoryGatewayConfig:
|
|
||||||
"""Fixed Memory Gateway settings for one Beaver instance."""
|
|
||||||
|
|
||||||
base_url: str = ""
|
|
||||||
user_id: str = ""
|
|
||||||
user_key: str = field(default="", repr=False)
|
|
||||||
app_id: str = "default"
|
|
||||||
project_id: str = "default"
|
|
||||||
scope: list[str] = field(default_factory=lambda: ["current_chat", "resources"])
|
|
||||||
top_k: int = 8
|
|
||||||
timeout_seconds: float = 10.0
|
|
||||||
|
|
||||||
@property
|
|
||||||
def is_configured(self) -> bool:
|
|
||||||
return bool(_clean(self.base_url) and _clean(self.user_id) and _clean(self.user_key))
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
|
||||||
class MemoryConfig:
|
|
||||||
"""Curated baseline plus optional Memory Gateway layer."""
|
|
||||||
|
|
||||||
mode: str = "hybrid"
|
|
||||||
explicit: bool = False
|
|
||||||
gateway: MemoryGatewayConfig = field(default_factory=MemoryGatewayConfig)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
@dataclass(slots=True)
|
||||||
class BeaverConfig:
|
class BeaverConfig:
|
||||||
"""Config loaded once per backend sandbox instance."""
|
"""Config loaded once per backend sandbox instance."""
|
||||||
|
|||||||
@ -1,5 +0,0 @@
|
|||||||
"""Memory Gateway HTTP integration."""
|
|
||||||
|
|
||||||
from .client import MemoryGatewayClient, MemoryGatewayClientError
|
|
||||||
|
|
||||||
__all__ = ["MemoryGatewayClient", "MemoryGatewayClientError"]
|
|
||||||
@ -5,6 +5,7 @@ from __future__ import annotations
|
|||||||
import json
|
import json
|
||||||
import asyncio
|
import asyncio
|
||||||
import io
|
import io
|
||||||
|
import logging
|
||||||
import mimetypes
|
import mimetypes
|
||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
@ -21,6 +22,13 @@ from typing import Any
|
|||||||
from beaver.engine.providers.registry import PROVIDERS, find_by_name
|
from beaver.engine.providers.registry import PROVIDERS, find_by_name
|
||||||
from beaver.foundation.config import default_config_path, load_config
|
from beaver.foundation.config import default_config_path, load_config
|
||||||
from beaver.foundation.events import ChannelIdentity, InboundMessage
|
from beaver.foundation.events import ChannelIdentity, InboundMessage
|
||||||
|
from beaver.memory.gateway import (
|
||||||
|
MemoryGatewayClient,
|
||||||
|
MemoryGatewayClientError,
|
||||||
|
MemoryGatewayCredentialStore,
|
||||||
|
MemoryGatewayUserCredential,
|
||||||
|
default_memory_gateway_users_path,
|
||||||
|
)
|
||||||
from beaver.interfaces.channels.runtime import ChannelRuntime
|
from beaver.interfaces.channels.runtime import ChannelRuntime
|
||||||
from beaver.interfaces.channels.connections import (
|
from beaver.interfaces.channels.connections import (
|
||||||
ChannelConnectionStore,
|
ChannelConnectionStore,
|
||||||
@ -97,6 +105,8 @@ from .schemas import (
|
|||||||
WebStatusResponse,
|
WebStatusResponse,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
from fastapi import FastAPI, File, Form, Header, HTTPException, Request, UploadFile, WebSocket, WebSocketDisconnect
|
from fastapi import FastAPI, File, Form, Header, HTTPException, Request, UploadFile, WebSocket, WebSocketDisconnect
|
||||||
from fastapi.middleware.cors import CORSMiddleware
|
from fastapi.middleware.cors import CORSMiddleware
|
||||||
@ -588,6 +598,10 @@ def create_app(
|
|||||||
app.state.auth_tokens = {}
|
app.state.auth_tokens = {}
|
||||||
app.state.handoff_codes = {}
|
app.state.handoff_codes = {}
|
||||||
app.state.auth_file = Path(os.getenv("BEAVER_AUTH_FILE") or "")
|
app.state.auth_file = Path(os.getenv("BEAVER_AUTH_FILE") or "")
|
||||||
|
app.state.memory_gateway_credential_store = MemoryGatewayCredentialStore(
|
||||||
|
default_memory_gateway_users_path()
|
||||||
|
)
|
||||||
|
app.state.memory_gateway_client_factory = lambda config: MemoryGatewayClient(config)
|
||||||
max_file_size = 50 * 1024 * 1024
|
max_file_size = 50 * 1024 * 1024
|
||||||
max_user_file_upload_size = _int_env("BEAVER_USER_FILES_MAX_UPLOAD_BYTES", 5 * 1024 * 1024 * 1024)
|
max_user_file_upload_size = _int_env("BEAVER_USER_FILES_MAX_UPLOAD_BYTES", 5 * 1024 * 1024 * 1024)
|
||||||
user_file_upload_part_size = _int_env("BEAVER_USER_FILES_UPLOAD_PART_SIZE", 10 * 1024 * 1024)
|
user_file_upload_part_size = _int_env("BEAVER_USER_FILES_UPLOAD_PART_SIZE", 10 * 1024 * 1024)
|
||||||
@ -1103,6 +1117,30 @@ def create_app(
|
|||||||
users[username] = password
|
users[username] = password
|
||||||
_save_auth_users(auth_file, users)
|
_save_auth_users(auth_file, users)
|
||||||
|
|
||||||
|
if config.memory.mode == "hybrid" and config.memory.gateway.is_configured:
|
||||||
|
try:
|
||||||
|
gateway_client = app.state.memory_gateway_client_factory(config.memory.gateway)
|
||||||
|
gateway_payload = await gateway_client.create_user(username)
|
||||||
|
gateway_user_id = _clean_text(gateway_payload.get("user_id"))
|
||||||
|
gateway_user_key = _clean_text(gateway_payload.get("user_key"))
|
||||||
|
if not gateway_user_id or not gateway_user_key:
|
||||||
|
raise MemoryGatewayClientError("create_user", "invalid_response")
|
||||||
|
app.state.memory_gateway_credential_store.save(
|
||||||
|
username,
|
||||||
|
MemoryGatewayUserCredential(
|
||||||
|
user_id=gateway_user_id,
|
||||||
|
user_key=gateway_user_key,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
except MemoryGatewayClientError as exc:
|
||||||
|
logger.warning(
|
||||||
|
"Memory Gateway user provisioning failed for Beaver user %s: operation=%s category=%s status_code=%s",
|
||||||
|
username,
|
||||||
|
exc.operation,
|
||||||
|
exc.category,
|
||||||
|
exc.status_code,
|
||||||
|
)
|
||||||
|
|
||||||
token = _issue_web_token(app, username)
|
token = _issue_web_token(app, username)
|
||||||
handoff_code, handoff_expires_at = _issue_handoff_code(app, username, token)
|
handoff_code, handoff_expires_at = _issue_handoff_code(app, username, token)
|
||||||
backend_connection = {
|
backend_connection = {
|
||||||
@ -2445,7 +2483,11 @@ def create_app(
|
|||||||
503: {"model": WebErrorResponse},
|
503: {"model": WebErrorResponse},
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
async def chat(request: Request, payload: WebChatRequest) -> WebChatResponse:
|
async def chat(
|
||||||
|
request: Request,
|
||||||
|
payload: WebChatRequest,
|
||||||
|
authorization: str | None = Header(default=None),
|
||||||
|
) -> WebChatResponse:
|
||||||
agent_service = get_agent_service(request)
|
agent_service = get_agent_service(request)
|
||||||
message = payload.message.strip()
|
message = payload.message.strip()
|
||||||
if not message:
|
if not message:
|
||||||
@ -2496,10 +2538,12 @@ def create_app(
|
|||||||
embedding_target = _model_dump(payload.embedding_target)
|
embedding_target = _model_dump(payload.embedding_target)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
|
gateway_user_id = _optional_web_user(app, authorization)
|
||||||
direct_kwargs = {
|
direct_kwargs = {
|
||||||
"session_id": payload.session_id,
|
"session_id": payload.session_id,
|
||||||
"source": "web",
|
"source": "web",
|
||||||
"user_id": payload.user_id,
|
"user_id": payload.user_id,
|
||||||
|
"gateway_user_id": gateway_user_id,
|
||||||
"title": payload.title,
|
"title": payload.title,
|
||||||
"execution_context": payload.execution_context,
|
"execution_context": payload.execution_context,
|
||||||
"prompt_locale": payload.prompt_locale,
|
"prompt_locale": payload.prompt_locale,
|
||||||
@ -2558,6 +2602,7 @@ def create_app(
|
|||||||
await websocket.send_json({"type": "error", "error": "AgentService is not ready"})
|
await websocket.send_json({"type": "error", "error": "AgentService is not ready"})
|
||||||
await websocket.close(code=1011)
|
await websocket.close(code=1011)
|
||||||
return
|
return
|
||||||
|
gateway_user_id = _web_user_from_token(app, websocket.query_params.get("token"))
|
||||||
|
|
||||||
while True:
|
while True:
|
||||||
try:
|
try:
|
||||||
@ -2616,6 +2661,7 @@ def create_app(
|
|||||||
"session_id": session_id,
|
"session_id": session_id,
|
||||||
"source": "websocket",
|
"source": "websocket",
|
||||||
"user_id": _clean_text(payload.get("user_id")) or None,
|
"user_id": _clean_text(payload.get("user_id")) or None,
|
||||||
|
"gateway_user_id": gateway_user_id,
|
||||||
"title": _clean_text(payload.get("title")) or None,
|
"title": _clean_text(payload.get("title")) or None,
|
||||||
"execution_context": _clean_text(payload.get("execution_context")) or None,
|
"execution_context": _clean_text(payload.get("execution_context")) or None,
|
||||||
"prompt_locale": _clean_text(payload.get("prompt_locale")) or None,
|
"prompt_locale": _clean_text(payload.get("prompt_locale")) or None,
|
||||||
@ -3680,6 +3726,22 @@ def _require_web_user(app: FastAPI, authorization: str | None) -> str:
|
|||||||
return username
|
return username
|
||||||
|
|
||||||
|
|
||||||
|
def _optional_web_user(app: FastAPI, authorization: str | None) -> str | None:
|
||||||
|
if not authorization:
|
||||||
|
return None
|
||||||
|
prefix = "bearer "
|
||||||
|
if not authorization.lower().startswith(prefix):
|
||||||
|
return None
|
||||||
|
return _web_user_from_token(app, authorization[len(prefix):].strip())
|
||||||
|
|
||||||
|
|
||||||
|
def _web_user_from_token(app: FastAPI, token: str | None) -> str | None:
|
||||||
|
cleaned = _clean_text(token)
|
||||||
|
if not cleaned:
|
||||||
|
return None
|
||||||
|
return app.state.auth_tokens.get(cleaned)
|
||||||
|
|
||||||
|
|
||||||
def _backend_connection_view(request: Request) -> dict[str, Any]:
|
def _backend_connection_view(request: Request) -> dict[str, Any]:
|
||||||
public_base_url = (
|
public_base_url = (
|
||||||
os.getenv("BEAVER_BACKEND_IDENTITY__PUBLIC_BASE_URL")
|
os.getenv("BEAVER_BACKEND_IDENTITY__PUBLIC_BASE_URL")
|
||||||
|
|||||||
23
app-instance/backend/beaver/memory/gateway/__init__.py
Normal file
23
app-instance/backend/beaver/memory/gateway/__init__.py
Normal file
@ -0,0 +1,23 @@
|
|||||||
|
"""Memory Gateway support."""
|
||||||
|
|
||||||
|
from .client import MemoryGatewayClient, MemoryGatewayClientError
|
||||||
|
from .config import MemoryConfig, MemoryGatewayConfig
|
||||||
|
from .credentials import (
|
||||||
|
MemoryGatewayCredentialStore,
|
||||||
|
MemoryGatewayUserCredential,
|
||||||
|
default_memory_gateway_users_path,
|
||||||
|
)
|
||||||
|
from .service import GatewayPersistOutcome, GatewayRecallOutcome, MemoryGatewayService
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"GatewayPersistOutcome",
|
||||||
|
"GatewayRecallOutcome",
|
||||||
|
"MemoryConfig",
|
||||||
|
"MemoryGatewayCredentialStore",
|
||||||
|
"MemoryGatewayClient",
|
||||||
|
"MemoryGatewayClientError",
|
||||||
|
"MemoryGatewayConfig",
|
||||||
|
"MemoryGatewayService",
|
||||||
|
"MemoryGatewayUserCredential",
|
||||||
|
"default_memory_gateway_users_path",
|
||||||
|
]
|
||||||
@ -6,7 +6,7 @@ from typing import Any
|
|||||||
|
|
||||||
import httpx
|
import httpx
|
||||||
|
|
||||||
from beaver.foundation.config import MemoryGatewayConfig
|
from .config import MemoryGatewayConfig
|
||||||
|
|
||||||
|
|
||||||
class MemoryGatewayClientError(RuntimeError):
|
class MemoryGatewayClientError(RuntimeError):
|
||||||
@ -21,7 +21,7 @@ class MemoryGatewayClientError(RuntimeError):
|
|||||||
|
|
||||||
|
|
||||||
class MemoryGatewayClient:
|
class MemoryGatewayClient:
|
||||||
"""HTTP transport for search, add, and flush operations."""
|
"""HTTP transport for search, add, flush, and provisioning operations."""
|
||||||
|
|
||||||
def __init__(
|
def __init__(
|
||||||
self,
|
self,
|
||||||
@ -32,6 +32,9 @@ class MemoryGatewayClient:
|
|||||||
self.config = config
|
self.config = config
|
||||||
self.transport = transport
|
self.transport = transport
|
||||||
|
|
||||||
|
async def create_user(self, user_id: str) -> dict[str, Any]:
|
||||||
|
return await self._post("create_user", "/users", {"user_id": user_id})
|
||||||
|
|
||||||
async def search(self, payload: dict[str, Any]) -> dict[str, Any]:
|
async def search(self, payload: dict[str, Any]) -> dict[str, Any]:
|
||||||
return await self._post("search", "/memories/search", payload)
|
return await self._post("search", "/memories/search", payload)
|
||||||
|
|
||||||
32
app-instance/backend/beaver/memory/gateway/config.py
Normal file
32
app-instance/backend/beaver/memory/gateway/config.py
Normal file
@ -0,0 +1,32 @@
|
|||||||
|
"""Configuration models for the Memory Gateway layer."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(slots=True)
|
||||||
|
class MemoryGatewayConfig:
|
||||||
|
"""Shared non-secret Memory Gateway settings."""
|
||||||
|
|
||||||
|
base_url: str = ""
|
||||||
|
app_id: str = "default"
|
||||||
|
project_id: str = "default"
|
||||||
|
scope: list[str] = field(
|
||||||
|
default_factory=lambda: ["current_chat", "resources", "all_user_memory"]
|
||||||
|
)
|
||||||
|
top_k: int = 8
|
||||||
|
timeout_seconds: float = 10.0
|
||||||
|
|
||||||
|
@property
|
||||||
|
def is_configured(self) -> bool:
|
||||||
|
return bool(self.base_url.strip())
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(slots=True)
|
||||||
|
class MemoryConfig:
|
||||||
|
"""Curated baseline plus optional Memory Gateway layer."""
|
||||||
|
|
||||||
|
mode: str = "hybrid"
|
||||||
|
explicit: bool = False
|
||||||
|
gateway: MemoryGatewayConfig = field(default_factory=MemoryGatewayConfig)
|
||||||
75
app-instance/backend/beaver/memory/gateway/credentials.py
Normal file
75
app-instance/backend/beaver/memory/gateway/credentials.py
Normal file
@ -0,0 +1,75 @@
|
|||||||
|
"""Per-instance credential storage for Memory Gateway users."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import tempfile
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(slots=True)
|
||||||
|
class MemoryGatewayUserCredential:
|
||||||
|
user_id: str
|
||||||
|
user_key: str = field(repr=False)
|
||||||
|
|
||||||
|
|
||||||
|
class MemoryGatewayCredentialStore:
|
||||||
|
"""Persist Beaver username -> Gateway credential mappings."""
|
||||||
|
|
||||||
|
def __init__(self, path: str | Path) -> None:
|
||||||
|
self.path = Path(path)
|
||||||
|
|
||||||
|
def get(self, username: str) -> MemoryGatewayUserCredential | None:
|
||||||
|
users = self._load_users()
|
||||||
|
payload = users.get(username)
|
||||||
|
if not isinstance(payload, dict):
|
||||||
|
return None
|
||||||
|
user_id = str(payload.get("userId") or "").strip()
|
||||||
|
user_key = str(payload.get("userKey") or "").strip()
|
||||||
|
if not user_id or not user_key:
|
||||||
|
return None
|
||||||
|
return MemoryGatewayUserCredential(user_id=user_id, user_key=user_key)
|
||||||
|
|
||||||
|
def save(self, username: str, credential: MemoryGatewayUserCredential) -> None:
|
||||||
|
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
users = self._load_users()
|
||||||
|
users[username] = {
|
||||||
|
"userId": credential.user_id,
|
||||||
|
"userKey": credential.user_key,
|
||||||
|
}
|
||||||
|
payload = {"users": dict(sorted(users.items()))}
|
||||||
|
fd, tmp_name = tempfile.mkstemp(
|
||||||
|
prefix=f".{self.path.name}.",
|
||||||
|
suffix=".tmp",
|
||||||
|
dir=str(self.path.parent),
|
||||||
|
)
|
||||||
|
tmp_path = Path(tmp_name)
|
||||||
|
try:
|
||||||
|
with os.fdopen(fd, "w", encoding="utf-8") as handle:
|
||||||
|
json.dump(payload, handle, ensure_ascii=False, indent=2)
|
||||||
|
handle.write("\n")
|
||||||
|
os.chmod(tmp_path, 0o600)
|
||||||
|
os.replace(tmp_path, self.path)
|
||||||
|
os.chmod(self.path, 0o600)
|
||||||
|
finally:
|
||||||
|
if tmp_path.exists():
|
||||||
|
tmp_path.unlink()
|
||||||
|
|
||||||
|
def _load_users(self) -> dict[str, Any]:
|
||||||
|
if not self.path.exists():
|
||||||
|
return {}
|
||||||
|
data = json.loads(self.path.read_text(encoding="utf-8"))
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
return {}
|
||||||
|
users = data.get("users")
|
||||||
|
return users if isinstance(users, dict) else {}
|
||||||
|
|
||||||
|
|
||||||
|
def default_memory_gateway_users_path() -> Path:
|
||||||
|
raw = os.getenv("BEAVER_MEMORY_GATEWAY_USERS_PATH")
|
||||||
|
if raw:
|
||||||
|
return Path(raw)
|
||||||
|
return Path.home() / ".beaver" / "memory_gateway_users.json"
|
||||||
@ -6,8 +6,9 @@ import json
|
|||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from beaver.foundation.config import MemoryGatewayConfig
|
from .client import MemoryGatewayClient, MemoryGatewayClientError
|
||||||
from beaver.integrations.memory_gateway import MemoryGatewayClient, MemoryGatewayClientError
|
from .config import MemoryGatewayConfig
|
||||||
|
from .credentials import MemoryGatewayUserCredential
|
||||||
|
|
||||||
_RECALL_FIELDS = ("id", "session_id", "text", "score", "source_scope", "resource_uri")
|
_RECALL_FIELDS = ("id", "session_id", "text", "score", "source_scope", "resource_uri")
|
||||||
|
|
||||||
@ -33,16 +34,18 @@ class MemoryGatewayService:
|
|||||||
def __init__(
|
def __init__(
|
||||||
self,
|
self,
|
||||||
config: MemoryGatewayConfig,
|
config: MemoryGatewayConfig,
|
||||||
|
credential: MemoryGatewayUserCredential,
|
||||||
*,
|
*,
|
||||||
client: MemoryGatewayClient | None = None,
|
client: MemoryGatewayClient | None = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
self.config = config
|
self.config = config
|
||||||
|
self.credential = credential
|
||||||
self.client = client or MemoryGatewayClient(config)
|
self.client = client or MemoryGatewayClient(config)
|
||||||
|
|
||||||
async def recall_before_run(self, *, session_id: str, query: str) -> GatewayRecallOutcome:
|
async def recall_before_run(self, *, session_id: str, query: str) -> GatewayRecallOutcome:
|
||||||
payload = {
|
payload = {
|
||||||
"user_id": self.config.user_id,
|
"user_id": self.credential.user_id,
|
||||||
"user_key": self.config.user_key,
|
"user_key": self.credential.user_key,
|
||||||
"conversation_id": session_id,
|
"conversation_id": session_id,
|
||||||
"query": query,
|
"query": query,
|
||||||
"scope": list(self.config.scope),
|
"scope": list(self.config.scope),
|
||||||
@ -90,8 +93,8 @@ class MemoryGatewayService:
|
|||||||
) -> GatewayPersistOutcome:
|
) -> GatewayPersistOutcome:
|
||||||
gateway_session_id = f"chat:{session_id}"
|
gateway_session_id = f"chat:{session_id}"
|
||||||
common = {
|
common = {
|
||||||
"user_id": self.config.user_id,
|
"user_id": self.credential.user_id,
|
||||||
"user_key": self.config.user_key,
|
"user_key": self.credential.user_key,
|
||||||
"session_id": gateway_session_id,
|
"session_id": gateway_session_id,
|
||||||
"app_id": self.config.app_id,
|
"app_id": self.config.app_id,
|
||||||
"project_id": self.config.project_id,
|
"project_id": self.config.project_id,
|
||||||
@ -100,7 +103,7 @@ class MemoryGatewayService:
|
|||||||
**common,
|
**common,
|
||||||
"messages": [
|
"messages": [
|
||||||
{
|
{
|
||||||
"sender_id": self.config.user_id,
|
"sender_id": self.credential.user_id,
|
||||||
"role": "user",
|
"role": "user",
|
||||||
"timestamp": user_timestamp_ms,
|
"timestamp": user_timestamp_ms,
|
||||||
"content": user_text,
|
"content": user_text,
|
||||||
@ -1,6 +1,6 @@
|
|||||||
"""Application services for Beaver."""
|
"""Application services for Beaver."""
|
||||||
|
|
||||||
__all__ = ["AgentService", "CronService", "MemoryGatewayService", "MemoryService"]
|
__all__ = ["AgentService", "CronService", "MemoryService"]
|
||||||
|
|
||||||
|
|
||||||
def __getattr__(name: str):
|
def __getattr__(name: str):
|
||||||
@ -12,10 +12,6 @@ def __getattr__(name: str):
|
|||||||
from .memory_service import MemoryService
|
from .memory_service import MemoryService
|
||||||
|
|
||||||
return MemoryService
|
return MemoryService
|
||||||
if name == "MemoryGatewayService":
|
|
||||||
from .memory_gateway_service import MemoryGatewayService
|
|
||||||
|
|
||||||
return MemoryGatewayService
|
|
||||||
if name == "CronService":
|
if name == "CronService":
|
||||||
from .cron_service import CronService
|
from .cron_service import CronService
|
||||||
|
|
||||||
|
|||||||
13
app-instance/backend/memory/config.json
Normal file
13
app-instance/backend/memory/config.json
Normal file
@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"memory": {
|
||||||
|
"mode": "hybrid",
|
||||||
|
"gateway": {
|
||||||
|
"baseUrl": "http://10.6.80.123:8010",
|
||||||
|
"appId": "default",
|
||||||
|
"projectId": "default",
|
||||||
|
"scope": ["current_chat", "resources", "all_user_memory"],
|
||||||
|
"topK": 8,
|
||||||
|
"timeoutSeconds": 10
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -12,6 +12,39 @@ from beaver.interfaces.web.app import create_app, _reload_agent_config
|
|||||||
from beaver.services.agent_service import AgentService
|
from beaver.services.agent_service import AgentService
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_config_reads_shared_memory_config(tmp_path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
config_path = tmp_path / "config.json"
|
||||||
|
config_path.write_text(json.dumps({}), encoding="utf-8")
|
||||||
|
memory_config_path = tmp_path / "memory-config.json"
|
||||||
|
memory_config_path.write_text(
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
"memory": {
|
||||||
|
"mode": "hybrid",
|
||||||
|
"gateway": {
|
||||||
|
"baseUrl": "http://172.19.207.37:8010",
|
||||||
|
"appId": "default",
|
||||||
|
"projectId": "default",
|
||||||
|
"scope": ["current_chat", "resources", "all_user_memory"],
|
||||||
|
"topK": 8,
|
||||||
|
"timeoutSeconds": 10,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
monkeypatch.setenv("BEAVER_MEMORY_CONFIG_PATH", str(memory_config_path))
|
||||||
|
|
||||||
|
config = load_config(config_path=config_path)
|
||||||
|
|
||||||
|
assert config.memory.mode == "hybrid"
|
||||||
|
assert config.memory.gateway.base_url == "http://172.19.207.37:8010"
|
||||||
|
assert config.memory.gateway.scope == ["current_chat", "resources", "all_user_memory"]
|
||||||
|
assert config.memory.gateway.top_k == 8
|
||||||
|
assert config.memory.gateway.timeout_seconds == 10
|
||||||
|
|
||||||
|
|
||||||
def test_load_config_reads_current_instance_shape(tmp_path) -> None:
|
def test_load_config_reads_current_instance_shape(tmp_path) -> None:
|
||||||
config_path = tmp_path / "config.json"
|
config_path = tmp_path / "config.json"
|
||||||
config_path.write_text(
|
config_path.write_text(
|
||||||
@ -477,17 +510,25 @@ def test_load_config_adds_managed_local_mcp_servers(tmp_path) -> None:
|
|||||||
assert "beaver.interfaces.mcp.tools_server" in local.args
|
assert "beaver.interfaces.mcp.tools_server" in local.args
|
||||||
|
|
||||||
|
|
||||||
def test_missing_memory_config_defaults_to_implicit_hybrid(tmp_path) -> None:
|
def test_missing_memory_config_defaults_to_implicit_hybrid(
|
||||||
|
tmp_path, monkeypatch: pytest.MonkeyPatch
|
||||||
|
) -> None:
|
||||||
|
monkeypatch.setenv("BEAVER_MEMORY_CONFIG_PATH", str(tmp_path / "missing-memory.json"))
|
||||||
config = load_config(config_path=tmp_path / "missing.json")
|
config = load_config(config_path=tmp_path / "missing.json")
|
||||||
|
|
||||||
assert config.memory.mode == "hybrid"
|
assert config.memory.mode == "hybrid"
|
||||||
assert config.memory.explicit is False
|
assert config.memory.explicit is False
|
||||||
assert config.memory.gateway.scope == ["current_chat", "resources"]
|
assert config.memory.gateway.scope == ["current_chat", "resources", "all_user_memory"]
|
||||||
|
|
||||||
|
|
||||||
def test_load_config_reads_explicit_curated_memory_mode(tmp_path) -> None:
|
def test_load_config_reads_explicit_curated_memory_mode(
|
||||||
|
tmp_path, monkeypatch: pytest.MonkeyPatch
|
||||||
|
) -> None:
|
||||||
config_path = tmp_path / "config.json"
|
config_path = tmp_path / "config.json"
|
||||||
config_path.write_text(json.dumps({"memory": {"mode": "curated"}}), encoding="utf-8")
|
config_path.write_text(json.dumps({}), encoding="utf-8")
|
||||||
|
memory_config_path = tmp_path / "memory-config.json"
|
||||||
|
memory_config_path.write_text(json.dumps({"memory": {"mode": "curated"}}), encoding="utf-8")
|
||||||
|
monkeypatch.setenv("BEAVER_MEMORY_CONFIG_PATH", str(memory_config_path))
|
||||||
|
|
||||||
config = load_config(config_path=config_path)
|
config = load_config(config_path=config_path)
|
||||||
|
|
||||||
@ -495,17 +536,19 @@ def test_load_config_reads_explicit_curated_memory_mode(tmp_path) -> None:
|
|||||||
assert config.memory.explicit is True
|
assert config.memory.explicit is True
|
||||||
|
|
||||||
|
|
||||||
def test_load_config_reads_explicit_hybrid_gateway_settings(tmp_path) -> None:
|
def test_load_config_reads_explicit_hybrid_gateway_settings(
|
||||||
|
tmp_path, monkeypatch: pytest.MonkeyPatch
|
||||||
|
) -> None:
|
||||||
config_path = tmp_path / "config.json"
|
config_path = tmp_path / "config.json"
|
||||||
config_path.write_text(
|
config_path.write_text(json.dumps({}), encoding="utf-8")
|
||||||
|
memory_config_path = tmp_path / "memory-config.json"
|
||||||
|
memory_config_path.write_text(
|
||||||
json.dumps(
|
json.dumps(
|
||||||
{
|
{
|
||||||
"memory": {
|
"memory": {
|
||||||
"mode": "hybrid",
|
"mode": "hybrid",
|
||||||
"gateway": {
|
"gateway": {
|
||||||
"baseUrl": "http://127.0.0.1:8010",
|
"baseUrl": "http://127.0.0.1:8010",
|
||||||
"userId": "gateway-user",
|
|
||||||
"userKey": "uk_secret",
|
|
||||||
"appId": "beaver",
|
"appId": "beaver",
|
||||||
"projectId": "sandbox",
|
"projectId": "sandbox",
|
||||||
"scope": ["current_chat", "resources"],
|
"scope": ["current_chat", "resources"],
|
||||||
@ -517,14 +560,13 @@ def test_load_config_reads_explicit_hybrid_gateway_settings(tmp_path) -> None:
|
|||||||
),
|
),
|
||||||
encoding="utf-8",
|
encoding="utf-8",
|
||||||
)
|
)
|
||||||
|
monkeypatch.setenv("BEAVER_MEMORY_CONFIG_PATH", str(memory_config_path))
|
||||||
|
|
||||||
config = load_config(config_path=config_path)
|
config = load_config(config_path=config_path)
|
||||||
|
|
||||||
assert config.memory.mode == "hybrid"
|
assert config.memory.mode == "hybrid"
|
||||||
assert config.memory.explicit is True
|
assert config.memory.explicit is True
|
||||||
assert config.memory.gateway.base_url == "http://127.0.0.1:8010"
|
assert config.memory.gateway.base_url == "http://127.0.0.1:8010"
|
||||||
assert config.memory.gateway.user_id == "gateway-user"
|
|
||||||
assert config.memory.gateway.user_key == "uk_secret"
|
|
||||||
assert config.memory.gateway.app_id == "beaver"
|
assert config.memory.gateway.app_id == "beaver"
|
||||||
assert config.memory.gateway.project_id == "sandbox"
|
assert config.memory.gateway.project_id == "sandbox"
|
||||||
assert config.memory.gateway.scope == ["current_chat", "resources"]
|
assert config.memory.gateway.scope == ["current_chat", "resources"]
|
||||||
@ -532,41 +574,33 @@ def test_load_config_reads_explicit_hybrid_gateway_settings(tmp_path) -> None:
|
|||||||
assert config.memory.gateway.timeout_seconds == 12.5
|
assert config.memory.gateway.timeout_seconds == 12.5
|
||||||
|
|
||||||
|
|
||||||
def test_explicit_hybrid_requires_gateway_credentials_without_leaking_secret(tmp_path) -> None:
|
def test_explicit_hybrid_requires_gateway_base_url(tmp_path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
config_path = tmp_path / "config.json"
|
config_path = tmp_path / "config.json"
|
||||||
config_path.write_text(
|
config_path.write_text(json.dumps({}), encoding="utf-8")
|
||||||
json.dumps(
|
memory_config_path = tmp_path / "memory-config.json"
|
||||||
{
|
memory_config_path.write_text(
|
||||||
"memory": {
|
json.dumps({"memory": {"mode": "hybrid", "gateway": {"appId": "beaver"}}}),
|
||||||
"mode": "hybrid",
|
|
||||||
"gateway": {
|
|
||||||
"baseUrl": "http://127.0.0.1:8010",
|
|
||||||
"userKey": "uk_super_secret",
|
|
||||||
},
|
|
||||||
}
|
|
||||||
}
|
|
||||||
),
|
|
||||||
encoding="utf-8",
|
encoding="utf-8",
|
||||||
)
|
)
|
||||||
|
monkeypatch.setenv("BEAVER_MEMORY_CONFIG_PATH", str(memory_config_path))
|
||||||
|
|
||||||
with pytest.raises(ValueError) as exc_info:
|
with pytest.raises(ValueError) as exc_info:
|
||||||
load_config(config_path=config_path)
|
load_config(config_path=config_path)
|
||||||
|
|
||||||
assert "userId" in str(exc_info.value)
|
assert "baseUrl" in str(exc_info.value)
|
||||||
assert "uk_super_secret" not in str(exc_info.value)
|
|
||||||
|
|
||||||
|
|
||||||
def test_hybrid_memory_rejects_unknown_scope(tmp_path) -> None:
|
def test_hybrid_memory_rejects_unknown_scope(tmp_path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
config_path = tmp_path / "config.json"
|
config_path = tmp_path / "config.json"
|
||||||
config_path.write_text(
|
config_path.write_text(json.dumps({}), encoding="utf-8")
|
||||||
|
memory_config_path = tmp_path / "memory-config.json"
|
||||||
|
memory_config_path.write_text(
|
||||||
json.dumps(
|
json.dumps(
|
||||||
{
|
{
|
||||||
"memory": {
|
"memory": {
|
||||||
"mode": "hybrid",
|
"mode": "hybrid",
|
||||||
"gateway": {
|
"gateway": {
|
||||||
"baseUrl": "http://127.0.0.1:8010",
|
"baseUrl": "http://127.0.0.1:8010",
|
||||||
"userId": "gateway-user",
|
|
||||||
"userKey": "uk_secret",
|
|
||||||
"scope": ["current_chat", "unknown"],
|
"scope": ["current_chat", "unknown"],
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
@ -574,22 +608,23 @@ def test_hybrid_memory_rejects_unknown_scope(tmp_path) -> None:
|
|||||||
),
|
),
|
||||||
encoding="utf-8",
|
encoding="utf-8",
|
||||||
)
|
)
|
||||||
|
monkeypatch.setenv("BEAVER_MEMORY_CONFIG_PATH", str(memory_config_path))
|
||||||
|
|
||||||
with pytest.raises(ValueError, match="scope"):
|
with pytest.raises(ValueError, match="scope"):
|
||||||
load_config(config_path=config_path)
|
load_config(config_path=config_path)
|
||||||
|
|
||||||
|
|
||||||
def test_hybrid_memory_rejects_empty_scope(tmp_path) -> None:
|
def test_hybrid_memory_rejects_empty_scope(tmp_path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
config_path = tmp_path / "config.json"
|
config_path = tmp_path / "config.json"
|
||||||
config_path.write_text(
|
config_path.write_text(json.dumps({}), encoding="utf-8")
|
||||||
|
memory_config_path = tmp_path / "memory-config.json"
|
||||||
|
memory_config_path.write_text(
|
||||||
json.dumps(
|
json.dumps(
|
||||||
{
|
{
|
||||||
"memory": {
|
"memory": {
|
||||||
"mode": "hybrid",
|
"mode": "hybrid",
|
||||||
"gateway": {
|
"gateway": {
|
||||||
"baseUrl": "http://127.0.0.1:8010",
|
"baseUrl": "http://127.0.0.1:8010",
|
||||||
"userId": "gateway-user",
|
|
||||||
"userKey": "uk_secret",
|
|
||||||
"scope": [],
|
"scope": [],
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
@ -597,6 +632,7 @@ def test_hybrid_memory_rejects_empty_scope(tmp_path) -> None:
|
|||||||
),
|
),
|
||||||
encoding="utf-8",
|
encoding="utf-8",
|
||||||
)
|
)
|
||||||
|
monkeypatch.setenv("BEAVER_MEMORY_CONFIG_PATH", str(memory_config_path))
|
||||||
|
|
||||||
with pytest.raises(ValueError, match="scope"):
|
with pytest.raises(ValueError, match="scope"):
|
||||||
load_config(config_path=config_path)
|
load_config(config_path=config_path)
|
||||||
@ -610,18 +646,21 @@ def test_hybrid_memory_rejects_empty_scope(tmp_path) -> None:
|
|||||||
({"timeoutSeconds": 0}, "timeoutSeconds"),
|
({"timeoutSeconds": 0}, "timeoutSeconds"),
|
||||||
],
|
],
|
||||||
)
|
)
|
||||||
def test_hybrid_memory_rejects_invalid_limits(tmp_path, gateway_override, expected_error) -> None:
|
def test_hybrid_memory_rejects_invalid_limits(
|
||||||
|
tmp_path, gateway_override, expected_error, monkeypatch: pytest.MonkeyPatch
|
||||||
|
) -> None:
|
||||||
config_path = tmp_path / "config.json"
|
config_path = tmp_path / "config.json"
|
||||||
|
config_path.write_text(json.dumps({}), encoding="utf-8")
|
||||||
gateway = {
|
gateway = {
|
||||||
"baseUrl": "http://127.0.0.1:8010",
|
"baseUrl": "http://127.0.0.1:8010",
|
||||||
"userId": "gateway-user",
|
|
||||||
"userKey": "uk_secret",
|
|
||||||
**gateway_override,
|
**gateway_override,
|
||||||
}
|
}
|
||||||
config_path.write_text(
|
memory_config_path = tmp_path / "memory-config.json"
|
||||||
|
memory_config_path.write_text(
|
||||||
json.dumps({"memory": {"mode": "hybrid", "gateway": gateway}}),
|
json.dumps({"memory": {"mode": "hybrid", "gateway": gateway}}),
|
||||||
encoding="utf-8",
|
encoding="utf-8",
|
||||||
)
|
)
|
||||||
|
monkeypatch.setenv("BEAVER_MEMORY_CONFIG_PATH", str(memory_config_path))
|
||||||
|
|
||||||
with pytest.raises(ValueError, match=expected_error):
|
with pytest.raises(ValueError, match=expected_error):
|
||||||
load_config(config_path=config_path)
|
load_config(config_path=config_path)
|
||||||
|
|||||||
@ -8,8 +8,13 @@ from beaver.engine import AgentLoop, EngineLoader
|
|||||||
from beaver.engine.providers.base import LLMProvider, LLMResponse
|
from beaver.engine.providers.base import LLMProvider, LLMResponse
|
||||||
from beaver.engine.providers.factory import ProviderBundle
|
from beaver.engine.providers.factory import ProviderBundle
|
||||||
from beaver.foundation.config import BeaverConfig, MemoryConfig, MemoryGatewayConfig
|
from beaver.foundation.config import BeaverConfig, MemoryConfig, MemoryGatewayConfig
|
||||||
from beaver.integrations.memory_gateway import MemoryGatewayClientError
|
from beaver.memory.gateway import (
|
||||||
from beaver.services.memory_gateway_service import GatewayPersistOutcome, GatewayRecallOutcome
|
GatewayPersistOutcome,
|
||||||
|
GatewayRecallOutcome,
|
||||||
|
MemoryGatewayClientError,
|
||||||
|
MemoryGatewayCredentialStore,
|
||||||
|
MemoryGatewayUserCredential,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class RecordingProvider(LLMProvider):
|
class RecordingProvider(LLMProvider):
|
||||||
@ -74,8 +79,6 @@ def _hybrid_config() -> BeaverConfig:
|
|||||||
explicit=True,
|
explicit=True,
|
||||||
gateway=MemoryGatewayConfig(
|
gateway=MemoryGatewayConfig(
|
||||||
base_url="http://gateway.test",
|
base_url="http://gateway.test",
|
||||||
user_id="gateway-user",
|
|
||||||
user_key="uk_secret",
|
|
||||||
scope=["current_chat", "resources"],
|
scope=["current_chat", "resources"],
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
@ -93,11 +96,24 @@ def _write_curated_user_memory(workspace: Path) -> None:
|
|||||||
(root / "USER.md").write_text("The user prefers concise answers.", encoding="utf-8")
|
(root / "USER.md").write_text("The user prefers concise answers.", encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
def _run(loop: AgentLoop, provider: LLMProvider, *, session_id: str = "web:gateway-test"):
|
def _gateway_store(tmp_path: Path) -> MemoryGatewayCredentialStore:
|
||||||
|
store = MemoryGatewayCredentialStore(tmp_path / "memory_gateway_users.json")
|
||||||
|
store.save("tom", MemoryGatewayUserCredential(user_id="gateway-user", user_key="uk_secret"))
|
||||||
|
return store
|
||||||
|
|
||||||
|
|
||||||
|
def _run(
|
||||||
|
loop: AgentLoop,
|
||||||
|
provider: LLMProvider,
|
||||||
|
*,
|
||||||
|
session_id: str = "web:gateway-test",
|
||||||
|
gateway_user_id: str | None = "tom",
|
||||||
|
):
|
||||||
return asyncio.run(
|
return asyncio.run(
|
||||||
loop.process_direct(
|
loop.process_direct(
|
||||||
"What should I remember?",
|
"What should I remember?",
|
||||||
session_id=session_id,
|
session_id=session_id,
|
||||||
|
gateway_user_id=gateway_user_id,
|
||||||
provider_bundle=_bundle(provider),
|
provider_bundle=_bundle(provider),
|
||||||
include_skill_assembly=False,
|
include_skill_assembly=False,
|
||||||
include_tools=False,
|
include_tools=False,
|
||||||
@ -134,7 +150,8 @@ def test_hybrid_run_keeps_curated_context_and_persists_gateway_turn(tmp_path: Pa
|
|||||||
loader=EngineLoader(
|
loader=EngineLoader(
|
||||||
workspace=tmp_path,
|
workspace=tmp_path,
|
||||||
config=_hybrid_config(),
|
config=_hybrid_config(),
|
||||||
memory_gateway_service=gateway,
|
memory_gateway_credentials=_gateway_store(tmp_path),
|
||||||
|
memory_gateway_service_factory=lambda _config, _credential: gateway,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
@ -182,7 +199,8 @@ def test_gateway_recall_failure_is_audited_without_changing_result(tmp_path: Pat
|
|||||||
loader=EngineLoader(
|
loader=EngineLoader(
|
||||||
workspace=tmp_path,
|
workspace=tmp_path,
|
||||||
config=_hybrid_config(),
|
config=_hybrid_config(),
|
||||||
memory_gateway_service=gateway,
|
memory_gateway_credentials=_gateway_store(tmp_path),
|
||||||
|
memory_gateway_service_factory=lambda _config, _credential: gateway,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
@ -210,7 +228,8 @@ def test_gateway_add_failure_skips_flush_audit_and_preserves_result(tmp_path: Pa
|
|||||||
loader=EngineLoader(
|
loader=EngineLoader(
|
||||||
workspace=tmp_path,
|
workspace=tmp_path,
|
||||||
config=_hybrid_config(),
|
config=_hybrid_config(),
|
||||||
memory_gateway_service=gateway,
|
memory_gateway_credentials=_gateway_store(tmp_path),
|
||||||
|
memory_gateway_service_factory=lambda _config, _credential: gateway,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
@ -235,7 +254,8 @@ def test_gateway_flush_failure_records_add_success_and_flush_failure(tmp_path: P
|
|||||||
loader=EngineLoader(
|
loader=EngineLoader(
|
||||||
workspace=tmp_path,
|
workspace=tmp_path,
|
||||||
config=_hybrid_config(),
|
config=_hybrid_config(),
|
||||||
memory_gateway_service=gateway,
|
memory_gateway_credentials=_gateway_store(tmp_path),
|
||||||
|
memory_gateway_service_factory=lambda _config, _credential: gateway,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
@ -276,7 +296,8 @@ def test_failed_run_is_not_persisted_to_gateway(tmp_path: Path) -> None:
|
|||||||
loader=EngineLoader(
|
loader=EngineLoader(
|
||||||
workspace=tmp_path,
|
workspace=tmp_path,
|
||||||
config=_hybrid_config(),
|
config=_hybrid_config(),
|
||||||
memory_gateway_service=gateway,
|
memory_gateway_credentials=_gateway_store(tmp_path),
|
||||||
|
memory_gateway_service_factory=lambda _config, _credential: gateway,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
@ -286,3 +307,23 @@ def test_failed_run_is_not_persisted_to_gateway(tmp_path: Path) -> None:
|
|||||||
assert gateway.recall_calls
|
assert gateway.recall_calls
|
||||||
assert gateway.persist_calls == []
|
assert gateway.persist_calls == []
|
||||||
loop.close()
|
loop.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_missing_gateway_identity_skips_gateway_calls(tmp_path: Path) -> None:
|
||||||
|
gateway = FakeGatewayService()
|
||||||
|
provider = RecordingProvider(LLMResponse(content="Curated only.", finish_reason="stop"))
|
||||||
|
loop = AgentLoop(
|
||||||
|
loader=EngineLoader(
|
||||||
|
workspace=tmp_path,
|
||||||
|
config=_hybrid_config(),
|
||||||
|
memory_gateway_credentials=_gateway_store(tmp_path),
|
||||||
|
memory_gateway_service_factory=lambda _config, _credential: gateway,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
result = _run(loop, provider, session_id="web:no-gateway-user", gateway_user_id=None)
|
||||||
|
|
||||||
|
assert result.output_text == "Curated only."
|
||||||
|
assert gateway.recall_calls == []
|
||||||
|
assert gateway.persist_calls == []
|
||||||
|
loop.close()
|
||||||
|
|||||||
@ -0,0 +1,58 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import stat
|
||||||
|
|
||||||
|
from beaver.memory.gateway import (
|
||||||
|
MemoryGatewayCredentialStore,
|
||||||
|
MemoryGatewayUserCredential,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_credential_store_returns_none_for_missing_user(tmp_path) -> None:
|
||||||
|
store = MemoryGatewayCredentialStore(tmp_path / "memory_gateway_users.json")
|
||||||
|
|
||||||
|
assert store.get("tom") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_credential_store_round_trips_multiple_users(tmp_path) -> None:
|
||||||
|
path = tmp_path / "memory_gateway_users.json"
|
||||||
|
store = MemoryGatewayCredentialStore(path)
|
||||||
|
|
||||||
|
store.save("tom", MemoryGatewayUserCredential(user_id="tom", user_key="uk_tom"))
|
||||||
|
store.save("alice", MemoryGatewayUserCredential(user_id="alice", user_key="uk_alice"))
|
||||||
|
|
||||||
|
assert store.get("tom") == MemoryGatewayUserCredential(user_id="tom", user_key="uk_tom")
|
||||||
|
assert store.get("alice") == MemoryGatewayUserCredential(user_id="alice", user_key="uk_alice")
|
||||||
|
|
||||||
|
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||||
|
assert payload == {
|
||||||
|
"users": {
|
||||||
|
"alice": {"userId": "alice", "userKey": "uk_alice"},
|
||||||
|
"tom": {"userId": "tom", "userKey": "uk_tom"},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_credential_store_update_preserves_other_users(tmp_path) -> None:
|
||||||
|
path = tmp_path / "memory_gateway_users.json"
|
||||||
|
store = MemoryGatewayCredentialStore(path)
|
||||||
|
store.save("tom", MemoryGatewayUserCredential(user_id="tom", user_key="uk_old"))
|
||||||
|
store.save("alice", MemoryGatewayUserCredential(user_id="alice", user_key="uk_alice"))
|
||||||
|
|
||||||
|
store.save("tom", MemoryGatewayUserCredential(user_id="tom", user_key="uk_new"))
|
||||||
|
|
||||||
|
assert store.get("tom") == MemoryGatewayUserCredential(user_id="tom", user_key="uk_new")
|
||||||
|
assert store.get("alice") == MemoryGatewayUserCredential(user_id="alice", user_key="uk_alice")
|
||||||
|
|
||||||
|
|
||||||
|
def test_credential_store_masks_secret_in_repr_and_uses_private_mode(tmp_path) -> None:
|
||||||
|
path = tmp_path / "memory_gateway_users.json"
|
||||||
|
credential = MemoryGatewayUserCredential(user_id="tom", user_key="uk_super_secret")
|
||||||
|
store = MemoryGatewayCredentialStore(path)
|
||||||
|
|
||||||
|
store.save("tom", credential)
|
||||||
|
|
||||||
|
assert "uk_super_secret" not in repr(credential)
|
||||||
|
assert stat.S_IMODE(path.stat().st_mode) == 0o600
|
||||||
|
assert not any(child.suffix == ".tmp" for child in tmp_path.iterdir())
|
||||||
@ -6,6 +6,7 @@ import pytest
|
|||||||
|
|
||||||
from beaver.engine import EngineLoader
|
from beaver.engine import EngineLoader
|
||||||
from beaver.foundation.config import BeaverConfig, MemoryConfig, MemoryGatewayConfig
|
from beaver.foundation.config import BeaverConfig, MemoryConfig, MemoryGatewayConfig
|
||||||
|
from beaver.memory.gateway import MemoryGatewayCredentialStore, MemoryGatewayUserCredential
|
||||||
|
|
||||||
|
|
||||||
def test_loader_keeps_curated_memory_in_explicit_curated_mode(tmp_path) -> None:
|
def test_loader_keeps_curated_memory_in_explicit_curated_mode(tmp_path) -> None:
|
||||||
@ -14,7 +15,9 @@ def test_loader_keeps_curated_memory_in_explicit_curated_mode(tmp_path) -> None:
|
|||||||
loaded = EngineLoader(workspace=tmp_path, config=config).load()
|
loaded = EngineLoader(workspace=tmp_path, config=config).load()
|
||||||
|
|
||||||
try:
|
try:
|
||||||
assert loaded.memory_gateway_service is None
|
assert loaded.memory_gateway_config is None
|
||||||
|
assert loaded.memory_gateway_credentials is None
|
||||||
|
assert loaded.memory_gateway_service_factory is None
|
||||||
assert loaded.curated_memory_store is not None
|
assert loaded.curated_memory_store is not None
|
||||||
assert loaded.memory_service is not None
|
assert loaded.memory_service is not None
|
||||||
assert "memory" in loaded.tools
|
assert "memory" in loaded.tools
|
||||||
@ -26,22 +29,30 @@ def test_loader_keeps_curated_memory_in_explicit_curated_mode(tmp_path) -> None:
|
|||||||
def test_loader_adds_gateway_service_without_disabling_curated_memory(tmp_path) -> None:
|
def test_loader_adds_gateway_service_without_disabling_curated_memory(tmp_path) -> None:
|
||||||
gateway_config = MemoryGatewayConfig(
|
gateway_config = MemoryGatewayConfig(
|
||||||
base_url="http://gateway.test",
|
base_url="http://gateway.test",
|
||||||
user_id="gateway-user",
|
|
||||||
user_key="uk_secret",
|
|
||||||
)
|
)
|
||||||
config = BeaverConfig(
|
config = BeaverConfig(
|
||||||
memory=MemoryConfig(mode="hybrid", explicit=True, gateway=gateway_config)
|
memory=MemoryConfig(mode="hybrid", explicit=True, gateway=gateway_config)
|
||||||
)
|
)
|
||||||
|
credential_store = MemoryGatewayCredentialStore(tmp_path / "memory_gateway_users.json")
|
||||||
fake_gateway_service = object()
|
fake_gateway_service = object()
|
||||||
|
|
||||||
loaded = EngineLoader(
|
loaded = EngineLoader(
|
||||||
workspace=tmp_path,
|
workspace=tmp_path,
|
||||||
config=config,
|
config=config,
|
||||||
memory_gateway_service=fake_gateway_service,
|
memory_gateway_credentials=credential_store,
|
||||||
|
memory_gateway_service_factory=lambda cfg, credential: fake_gateway_service,
|
||||||
).load()
|
).load()
|
||||||
|
|
||||||
try:
|
try:
|
||||||
assert loaded.memory_gateway_service is fake_gateway_service
|
assert loaded.memory_gateway_config == gateway_config
|
||||||
|
assert loaded.memory_gateway_credentials is credential_store
|
||||||
|
assert loaded.memory_gateway_service_factory is not None
|
||||||
|
assert (
|
||||||
|
loaded.memory_gateway_service_factory(
|
||||||
|
MemoryGatewayUserCredential(user_id="gateway-user", user_key="uk_secret")
|
||||||
|
)
|
||||||
|
is fake_gateway_service
|
||||||
|
)
|
||||||
assert loaded.curated_memory_store is not None
|
assert loaded.curated_memory_store is not None
|
||||||
assert loaded.memory_service is not None
|
assert loaded.memory_service is not None
|
||||||
assert "memory" in loaded.tools
|
assert "memory" in loaded.tools
|
||||||
@ -60,7 +71,7 @@ def test_loader_implicit_hybrid_without_credentials_warns_and_degrades(
|
|||||||
loaded = EngineLoader(workspace=tmp_path, config=config).load()
|
loaded = EngineLoader(workspace=tmp_path, config=config).load()
|
||||||
|
|
||||||
try:
|
try:
|
||||||
assert loaded.memory_gateway_service is None
|
assert loaded.memory_gateway_config is None
|
||||||
assert loaded.curated_memory_store is not None
|
assert loaded.curated_memory_store is not None
|
||||||
assert "memory" in loaded.tools
|
assert "memory" in loaded.tools
|
||||||
assert "continuing with curated memory only" in caplog.text
|
assert "continuing with curated memory only" in caplog.text
|
||||||
@ -76,7 +87,7 @@ def test_loader_explicit_hybrid_without_credentials_fails_before_opening_session
|
|||||||
memory=MemoryConfig(
|
memory=MemoryConfig(
|
||||||
mode="hybrid",
|
mode="hybrid",
|
||||||
explicit=True,
|
explicit=True,
|
||||||
gateway=MemoryGatewayConfig(user_key="uk_super_secret"),
|
gateway=MemoryGatewayConfig(),
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
@ -89,4 +100,3 @@ def test_loader_explicit_hybrid_without_credentials_fails_before_opening_session
|
|||||||
EngineLoader(workspace=tmp_path, config=config).load()
|
EngineLoader(workspace=tmp_path, config=config).load()
|
||||||
|
|
||||||
assert "Memory Gateway" in str(exc_info.value)
|
assert "Memory Gateway" in str(exc_info.value)
|
||||||
assert "uk_super_secret" not in str(exc_info.value)
|
|
||||||
|
|||||||
@ -0,0 +1,123 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from beaver.interfaces.web.app import create_app
|
||||||
|
from beaver.memory.gateway import (
|
||||||
|
MemoryGatewayClientError,
|
||||||
|
MemoryGatewayCredentialStore,
|
||||||
|
)
|
||||||
|
from beaver.services.agent_service import AgentService
|
||||||
|
|
||||||
|
|
||||||
|
class FakeGatewayClient:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
response: dict[str, str] | None = None,
|
||||||
|
error: MemoryGatewayClientError | None = None,
|
||||||
|
) -> None:
|
||||||
|
self.response = response or {"user_id": "tom", "user_key": "uk_tom"}
|
||||||
|
self.error = error
|
||||||
|
self.calls: list[str] = []
|
||||||
|
|
||||||
|
async def create_user(self, user_id: str) -> dict[str, str]:
|
||||||
|
self.calls.append(user_id)
|
||||||
|
if self.error is not None:
|
||||||
|
raise self.error
|
||||||
|
return dict(self.response)
|
||||||
|
|
||||||
|
|
||||||
|
def _service(tmp_path) -> AgentService:
|
||||||
|
config_path = tmp_path / "config.json"
|
||||||
|
config_path.write_text(json.dumps({}), encoding="utf-8")
|
||||||
|
return AgentService(config_path=config_path)
|
||||||
|
|
||||||
|
|
||||||
|
def _write_memory_config(tmp_path) -> None:
|
||||||
|
memory_config_path = tmp_path / "memory-config.json"
|
||||||
|
memory_config_path.write_text(
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
"memory": {
|
||||||
|
"mode": "hybrid",
|
||||||
|
"gateway": {
|
||||||
|
"baseUrl": "http://172.19.207.37:8010",
|
||||||
|
"appId": "default",
|
||||||
|
"projectId": "default",
|
||||||
|
"scope": ["current_chat", "resources", "all_user_memory"],
|
||||||
|
"topK": 8,
|
||||||
|
"timeoutSeconds": 10,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_register_provisions_gateway_user_and_hides_key(
|
||||||
|
tmp_path, monkeypatch
|
||||||
|
) -> None:
|
||||||
|
auth_path = tmp_path / "web_auth_users.json"
|
||||||
|
users_path = tmp_path / "memory_gateway_users.json"
|
||||||
|
monkeypatch.setenv("BEAVER_AUTH_FILE", str(auth_path))
|
||||||
|
monkeypatch.setenv("BEAVER_MEMORY_GATEWAY_USERS_PATH", str(users_path))
|
||||||
|
monkeypatch.setenv("BEAVER_MEMORY_CONFIG_PATH", str(tmp_path / "memory-config.json"))
|
||||||
|
_write_memory_config(tmp_path)
|
||||||
|
|
||||||
|
service = _service(tmp_path)
|
||||||
|
app = create_app(service=service, manage_service_lifecycle=False)
|
||||||
|
fake_client = FakeGatewayClient(response={"user_id": "tom", "user_key": "uk_tom"})
|
||||||
|
app.state.memory_gateway_client_factory = lambda _config: fake_client
|
||||||
|
|
||||||
|
with TestClient(app) as client:
|
||||||
|
response = client.post(
|
||||||
|
"/api/auth/register",
|
||||||
|
json={"username": "tom", "password": "pw"},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 200
|
||||||
|
assert fake_client.calls == ["tom"]
|
||||||
|
body = response.json()
|
||||||
|
assert "user_key" not in json.dumps(body)
|
||||||
|
assert MemoryGatewayCredentialStore(users_path).get("tom") is not None
|
||||||
|
assert MemoryGatewayCredentialStore(users_path).get("tom").user_key == "uk_tom"
|
||||||
|
service.close()
|
||||||
|
|
||||||
|
|
||||||
|
def test_register_keeps_local_user_and_logs_when_gateway_provisioning_fails(
|
||||||
|
tmp_path, monkeypatch, caplog
|
||||||
|
) -> None:
|
||||||
|
auth_path = tmp_path / "web_auth_users.json"
|
||||||
|
users_path = tmp_path / "memory_gateway_users.json"
|
||||||
|
monkeypatch.setenv("BEAVER_AUTH_FILE", str(auth_path))
|
||||||
|
monkeypatch.setenv("BEAVER_MEMORY_GATEWAY_USERS_PATH", str(users_path))
|
||||||
|
monkeypatch.setenv("BEAVER_MEMORY_CONFIG_PATH", str(tmp_path / "memory-config.json"))
|
||||||
|
_write_memory_config(tmp_path)
|
||||||
|
|
||||||
|
service = _service(tmp_path)
|
||||||
|
app = create_app(service=service, manage_service_lifecycle=False)
|
||||||
|
app.state.memory_gateway_client_factory = lambda _config: FakeGatewayClient(
|
||||||
|
error=MemoryGatewayClientError("create_user", "network")
|
||||||
|
)
|
||||||
|
|
||||||
|
with caplog.at_level(logging.WARNING, logger="beaver.interfaces.web.app"):
|
||||||
|
with TestClient(app) as client:
|
||||||
|
response = client.post(
|
||||||
|
"/api/auth/register",
|
||||||
|
json={"username": "tom", "password": "pw"},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 200
|
||||||
|
auth_payload = json.loads(auth_path.read_text(encoding="utf-8"))
|
||||||
|
assert auth_payload == {"users": [{"username": "tom", "password": "pw"}]}
|
||||||
|
assert MemoryGatewayCredentialStore(users_path).get("tom") is None
|
||||||
|
assert "Memory Gateway user provisioning failed" in caplog.text
|
||||||
|
assert "operation=create_user" in caplog.text
|
||||||
|
assert "category=network" in caplog.text
|
||||||
|
assert "user_key" not in caplog.text
|
||||||
|
service.close()
|
||||||
@ -5,16 +5,18 @@ import json
|
|||||||
import httpx
|
import httpx
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from beaver.foundation.config import MemoryGatewayConfig
|
from beaver.memory.gateway import (
|
||||||
from beaver.integrations.memory_gateway import MemoryGatewayClient, MemoryGatewayClientError
|
MemoryGatewayClient,
|
||||||
from beaver.services.memory_gateway_service import MemoryGatewayService
|
MemoryGatewayClientError,
|
||||||
|
MemoryGatewayConfig,
|
||||||
|
MemoryGatewayService,
|
||||||
|
MemoryGatewayUserCredential,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _config() -> MemoryGatewayConfig:
|
def _config() -> MemoryGatewayConfig:
|
||||||
return MemoryGatewayConfig(
|
return MemoryGatewayConfig(
|
||||||
base_url="http://gateway.test",
|
base_url="http://gateway.test",
|
||||||
user_id="gateway-user",
|
|
||||||
user_key="uk_super_secret",
|
|
||||||
app_id="beaver",
|
app_id="beaver",
|
||||||
project_id="sandbox",
|
project_id="sandbox",
|
||||||
scope=["current_chat", "resources"],
|
scope=["current_chat", "resources"],
|
||||||
@ -23,6 +25,10 @@ def _config() -> MemoryGatewayConfig:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _credential() -> MemoryGatewayUserCredential:
|
||||||
|
return MemoryGatewayUserCredential(user_id="gateway-user", user_key="uk_super_secret")
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_client_uses_exact_gateway_paths_and_payloads() -> None:
|
async def test_client_uses_exact_gateway_paths_and_payloads() -> None:
|
||||||
requests: list[httpx.Request] = []
|
requests: list[httpx.Request] = []
|
||||||
@ -113,7 +119,7 @@ async def test_recall_sanitizes_results_and_builds_reference_message() -> None:
|
|||||||
]
|
]
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
service = MemoryGatewayService(_config(), client=client)
|
service = MemoryGatewayService(_config(), _credential(), client=client)
|
||||||
|
|
||||||
outcome = await service.recall_before_run(session_id="web:alpha", query="contract")
|
outcome = await service.recall_before_run(session_id="web:alpha", query="contract")
|
||||||
|
|
||||||
@ -146,6 +152,7 @@ async def test_recall_sanitizes_results_and_builds_reference_message() -> None:
|
|||||||
async def test_recall_rejects_malformed_results_shape() -> None:
|
async def test_recall_rejects_malformed_results_shape() -> None:
|
||||||
service = MemoryGatewayService(
|
service = MemoryGatewayService(
|
||||||
_config(),
|
_config(),
|
||||||
|
_credential(),
|
||||||
client=FakeGatewayClient(search_response={"results": {"not": "a list"}}),
|
client=FakeGatewayClient(search_response={"results": {"not": "a list"}}),
|
||||||
)
|
)
|
||||||
|
|
||||||
@ -160,7 +167,7 @@ async def test_recall_rejects_malformed_results_shape() -> None:
|
|||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_persist_after_run_adds_two_messages_then_flushes() -> None:
|
async def test_persist_after_run_adds_two_messages_then_flushes() -> None:
|
||||||
client = FakeGatewayClient()
|
client = FakeGatewayClient()
|
||||||
service = MemoryGatewayService(_config(), client=client)
|
service = MemoryGatewayService(_config(), _credential(), client=client)
|
||||||
|
|
||||||
outcome = await service.persist_after_run(
|
outcome = await service.persist_after_run(
|
||||||
session_id="web:alpha",
|
session_id="web:alpha",
|
||||||
@ -206,7 +213,7 @@ async def test_persist_after_run_adds_two_messages_then_flushes() -> None:
|
|||||||
async def test_add_failure_skips_flush() -> None:
|
async def test_add_failure_skips_flush() -> None:
|
||||||
add_error = MemoryGatewayClientError("add", "http_status", status_code=503)
|
add_error = MemoryGatewayClientError("add", "http_status", status_code=503)
|
||||||
client = FakeGatewayClient(add_error=add_error)
|
client = FakeGatewayClient(add_error=add_error)
|
||||||
service = MemoryGatewayService(_config(), client=client)
|
service = MemoryGatewayService(_config(), _credential(), client=client)
|
||||||
|
|
||||||
outcome = await service.persist_after_run(
|
outcome = await service.persist_after_run(
|
||||||
session_id="web:alpha",
|
session_id="web:alpha",
|
||||||
@ -226,7 +233,7 @@ async def test_add_failure_skips_flush() -> None:
|
|||||||
async def test_flush_failure_preserves_successful_add() -> None:
|
async def test_flush_failure_preserves_successful_add() -> None:
|
||||||
flush_error = MemoryGatewayClientError("flush", "network")
|
flush_error = MemoryGatewayClientError("flush", "network")
|
||||||
client = FakeGatewayClient(flush_error=flush_error)
|
client = FakeGatewayClient(flush_error=flush_error)
|
||||||
service = MemoryGatewayService(_config(), client=client)
|
service = MemoryGatewayService(_config(), _credential(), client=client)
|
||||||
|
|
||||||
outcome = await service.persist_after_run(
|
outcome = await service.persist_after_run(
|
||||||
session_id="web:alpha",
|
session_id="web:alpha",
|
||||||
|
|||||||
@ -88,6 +88,7 @@ def test_websocket_message_returns_chat_metadata_and_session_updated() -> None:
|
|||||||
"session_id": "web:alpha",
|
"session_id": "web:alpha",
|
||||||
"source": "websocket",
|
"source": "websocket",
|
||||||
"user_id": None,
|
"user_id": None,
|
||||||
|
"gateway_user_id": None,
|
||||||
"title": None,
|
"title": None,
|
||||||
"execution_context": None,
|
"execution_context": None,
|
||||||
"prompt_locale": "zh-Hant",
|
"prompt_locale": "zh-Hant",
|
||||||
@ -134,6 +135,7 @@ def test_websocket_message_uses_direct_processing_when_loop_is_not_running() ->
|
|||||||
"session_id": "web:alpha",
|
"session_id": "web:alpha",
|
||||||
"source": "websocket",
|
"source": "websocket",
|
||||||
"user_id": None,
|
"user_id": None,
|
||||||
|
"gateway_user_id": None,
|
||||||
"title": None,
|
"title": None,
|
||||||
"execution_context": None,
|
"execution_context": None,
|
||||||
"prompt_locale": None,
|
"prompt_locale": None,
|
||||||
@ -164,6 +166,7 @@ def test_rest_chat_uses_direct_processing_when_loop_is_not_running() -> None:
|
|||||||
"session_id": "web:alpha",
|
"session_id": "web:alpha",
|
||||||
"source": "web",
|
"source": "web",
|
||||||
"user_id": None,
|
"user_id": None,
|
||||||
|
"gateway_user_id": None,
|
||||||
"title": None,
|
"title": None,
|
||||||
"execution_context": None,
|
"execution_context": None,
|
||||||
"prompt_locale": "en",
|
"prompt_locale": "en",
|
||||||
@ -181,6 +184,72 @@ def test_rest_chat_uses_direct_processing_when_loop_is_not_running() -> None:
|
|||||||
assert response.json()["output_text"] == "echo:hello"
|
assert response.json()["output_text"] == "echo:hello"
|
||||||
|
|
||||||
|
|
||||||
|
def test_rest_chat_uses_authenticated_user_for_gateway_identity() -> None:
|
||||||
|
service = DirectModeOnlyAgentService()
|
||||||
|
app = create_app(service=service, manage_service_lifecycle=False)
|
||||||
|
app.state.auth_tokens["token-1"] = "tom"
|
||||||
|
|
||||||
|
with TestClient(app) as client:
|
||||||
|
response = client.post(
|
||||||
|
"/api/chat",
|
||||||
|
headers={"Authorization": "Bearer token-1"},
|
||||||
|
json={"session_id": "web:alpha", "message": "hello", "user_id": "other"},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 200
|
||||||
|
assert service.calls == [
|
||||||
|
{
|
||||||
|
"message": "hello",
|
||||||
|
"session_id": "web:alpha",
|
||||||
|
"source": "web",
|
||||||
|
"user_id": "other",
|
||||||
|
"gateway_user_id": "tom",
|
||||||
|
"title": None,
|
||||||
|
"execution_context": None,
|
||||||
|
"prompt_locale": None,
|
||||||
|
"model": None,
|
||||||
|
"provider_name": None,
|
||||||
|
"embedding_model": None,
|
||||||
|
"temperature": None,
|
||||||
|
"max_tokens": None,
|
||||||
|
"max_tool_iterations": None,
|
||||||
|
"fallback_target": None,
|
||||||
|
"auxiliary_target": None,
|
||||||
|
"embedding_target": None,
|
||||||
|
}
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def test_websocket_uses_authenticated_user_for_gateway_identity() -> None:
|
||||||
|
service = StubAgentService()
|
||||||
|
app = create_app(service=service, manage_service_lifecycle=False)
|
||||||
|
app.state.auth_tokens["token-1"] = "tom"
|
||||||
|
|
||||||
|
with TestClient(app) as client:
|
||||||
|
with client.websocket_connect("/ws/web:alpha?token=token-1") as websocket:
|
||||||
|
websocket.send_json({"type": "message", "content": "hello", "user_id": "other"})
|
||||||
|
assert websocket.receive_json() == {"type": "status", "status": "thinking"}
|
||||||
|
websocket.receive_json()
|
||||||
|
websocket.receive_json()
|
||||||
|
|
||||||
|
assert service.calls == [
|
||||||
|
{
|
||||||
|
"message": "hello",
|
||||||
|
"session_id": "web:alpha",
|
||||||
|
"source": "websocket",
|
||||||
|
"user_id": "other",
|
||||||
|
"gateway_user_id": "tom",
|
||||||
|
"title": None,
|
||||||
|
"execution_context": None,
|
||||||
|
"prompt_locale": None,
|
||||||
|
"model": None,
|
||||||
|
"provider_name": None,
|
||||||
|
"embedding_model": None,
|
||||||
|
"max_tool_iterations": None,
|
||||||
|
}
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
def test_websocket_empty_content_returns_error_without_runtime_call() -> None:
|
def test_websocket_empty_content_returns_error_without_runtime_call() -> None:
|
||||||
service = StubAgentService()
|
service = StubAgentService()
|
||||||
app = create_app(service=service, manage_service_lifecycle=False)
|
app = create_app(service=service, manage_service_lifecycle=False)
|
||||||
|
|||||||
@ -737,6 +737,7 @@ INSTANCE_ROOT="${INSTANCES_ROOT}/${INSTANCE_SLUG}"
|
|||||||
BEAVER_HOME="${INSTANCE_ROOT}/beaver-home"
|
BEAVER_HOME="${INSTANCE_ROOT}/beaver-home"
|
||||||
CONFIG_PATH="${BEAVER_HOME}/config.json"
|
CONFIG_PATH="${BEAVER_HOME}/config.json"
|
||||||
AUTH_USERS_PATH="${BEAVER_HOME}/web_auth_users.json"
|
AUTH_USERS_PATH="${BEAVER_HOME}/web_auth_users.json"
|
||||||
|
MEMORY_GATEWAY_USERS_PATH="${BEAVER_HOME}/memory_gateway_users.json"
|
||||||
RUNTIME_ENV_PATH="${BEAVER_HOME}/runtime.env"
|
RUNTIME_ENV_PATH="${BEAVER_HOME}/runtime.env"
|
||||||
WORKSPACE_PATH="${BEAVER_HOME}/workspace"
|
WORKSPACE_PATH="${BEAVER_HOME}/workspace"
|
||||||
|
|
||||||
@ -745,6 +746,8 @@ mkdir -p "$BEAVER_HOME" "$WORKSPACE_PATH"
|
|||||||
render_config_json "$CONFIG_PATH"
|
render_config_json "$CONFIG_PATH"
|
||||||
render_auth_users_json "$AUTH_USERS_PATH"
|
render_auth_users_json "$AUTH_USERS_PATH"
|
||||||
render_runtime_env_file "$RUNTIME_ENV_PATH"
|
render_runtime_env_file "$RUNTIME_ENV_PATH"
|
||||||
|
printf '{\n "users": {}\n}\n' >"$MEMORY_GATEWAY_USERS_PATH"
|
||||||
|
chmod 600 "$MEMORY_GATEWAY_USERS_PATH"
|
||||||
seed_initial_skills "$WORKSPACE_PATH" "$INITIAL_SKILLS_DIR"
|
seed_initial_skills "$WORKSPACE_PATH" "$INITIAL_SKILLS_DIR"
|
||||||
|
|
||||||
if [[ "$FORCE_BUILD" -eq 1 ]] || ! image_exists; then
|
if [[ "$FORCE_BUILD" -eq 1 ]] || ! image_exists; then
|
||||||
@ -775,6 +778,7 @@ RUN_ARGS=(
|
|||||||
-e "BEAVER_CONFIG_PATH=/root/.beaver/config.json"
|
-e "BEAVER_CONFIG_PATH=/root/.beaver/config.json"
|
||||||
-e "BEAVER_WORKSPACE=/root/.beaver/workspace"
|
-e "BEAVER_WORKSPACE=/root/.beaver/workspace"
|
||||||
-e "BEAVER_AUTH_FILE=/root/.beaver/web_auth_users.json"
|
-e "BEAVER_AUTH_FILE=/root/.beaver/web_auth_users.json"
|
||||||
|
-e "BEAVER_MEMORY_GATEWAY_USERS_PATH=/root/.beaver/memory_gateway_users.json"
|
||||||
-e "BEAVER_FRONTEND_PUBLIC_BASE_URL=${PUBLIC_URL}"
|
-e "BEAVER_FRONTEND_PUBLIC_BASE_URL=${PUBLIC_URL}"
|
||||||
-e "APP_PUBLIC_PORT=8080"
|
-e "APP_PUBLIC_PORT=8080"
|
||||||
-e "APP_FRONTEND_PORT=3000"
|
-e "APP_FRONTEND_PORT=3000"
|
||||||
|
|||||||
@ -11,6 +11,7 @@ BEAVER_HOME="${BEAVER_HOME:-/root/.beaver}"
|
|||||||
BEAVER_CONFIG_PATH="${BEAVER_CONFIG_PATH:-$BEAVER_HOME/config.json}"
|
BEAVER_CONFIG_PATH="${BEAVER_CONFIG_PATH:-$BEAVER_HOME/config.json}"
|
||||||
BEAVER_WORKSPACE="${BEAVER_WORKSPACE:-$BEAVER_HOME/workspace}"
|
BEAVER_WORKSPACE="${BEAVER_WORKSPACE:-$BEAVER_HOME/workspace}"
|
||||||
BEAVER_AUTH_FILE="${BEAVER_AUTH_FILE:-$BEAVER_HOME/web_auth_users.json}"
|
BEAVER_AUTH_FILE="${BEAVER_AUTH_FILE:-$BEAVER_HOME/web_auth_users.json}"
|
||||||
|
BEAVER_MEMORY_GATEWAY_USERS_PATH="${BEAVER_MEMORY_GATEWAY_USERS_PATH:-$BEAVER_HOME/memory_gateway_users.json}"
|
||||||
BEAVER_RUNTIME_ENV_FILE="${BEAVER_RUNTIME_ENV_FILE:-$BEAVER_HOME/runtime.env}"
|
BEAVER_RUNTIME_ENV_FILE="${BEAVER_RUNTIME_ENV_FILE:-$BEAVER_HOME/runtime.env}"
|
||||||
BEAVER_INITIAL_SKILLS_DIR="${BEAVER_INITIAL_SKILLS_DIR:-/opt/app/initial-skills}"
|
BEAVER_INITIAL_SKILLS_DIR="${BEAVER_INITIAL_SKILLS_DIR:-/opt/app/initial-skills}"
|
||||||
BEAVER_INITIAL_SKILLS_EXCLUDE="${BEAVER_INITIAL_SKILLS_EXCLUDE:-officebench-mcp}"
|
BEAVER_INITIAL_SKILLS_EXCLUDE="${BEAVER_INITIAL_SKILLS_EXCLUDE:-officebench-mcp}"
|
||||||
@ -111,6 +112,11 @@ trap cleanup EXIT INT TERM
|
|||||||
|
|
||||||
mkdir -p "$BEAVER_HOME" "$BEAVER_WORKSPACE"
|
mkdir -p "$BEAVER_HOME" "$BEAVER_WORKSPACE"
|
||||||
|
|
||||||
|
if [[ ! -f "$BEAVER_MEMORY_GATEWAY_USERS_PATH" ]]; then
|
||||||
|
printf '{\n "users": {}\n}\n' >"$BEAVER_MEMORY_GATEWAY_USERS_PATH"
|
||||||
|
chmod 600 "$BEAVER_MEMORY_GATEWAY_USERS_PATH"
|
||||||
|
fi
|
||||||
|
|
||||||
if [[ -f "$BEAVER_RUNTIME_ENV_FILE" ]]; then
|
if [[ -f "$BEAVER_RUNTIME_ENV_FILE" ]]; then
|
||||||
set -a
|
set -a
|
||||||
. "$BEAVER_RUNTIME_ENV_FILE"
|
. "$BEAVER_RUNTIME_ENV_FILE"
|
||||||
@ -121,6 +127,7 @@ require_file "$BEAVER_CONFIG_PATH" "Missing Beaver config"
|
|||||||
seed_initial_skills "$BEAVER_INITIAL_SKILLS_DIR" "$BEAVER_WORKSPACE/skills"
|
seed_initial_skills "$BEAVER_INITIAL_SKILLS_DIR" "$BEAVER_WORKSPACE/skills"
|
||||||
|
|
||||||
export BEAVER_AUTH_FILE
|
export BEAVER_AUTH_FILE
|
||||||
|
export BEAVER_MEMORY_GATEWAY_USERS_PATH
|
||||||
export BEAVER_RUNTIME_ENV_FILE
|
export BEAVER_RUNTIME_ENV_FILE
|
||||||
export BEAVER_HOME
|
export BEAVER_HOME
|
||||||
export BEAVER_CONFIG_PATH
|
export BEAVER_CONFIG_PATH
|
||||||
|
|||||||
@ -0,0 +1,266 @@
|
|||||||
|
# Memory Gateway User Provisioning Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Move Beaver's Gateway code into `beaver.memory.gateway`, load one shared non-secret Gateway configuration, provision Gateway users during Beaver registration, and resolve per-user credentials for each authenticated chat run.
|
||||||
|
|
||||||
|
**Architecture:** `EngineLoader` loads curated memory, a shared Gateway config, and an instance-local credential store. Registration calls Gateway `/users` and atomically stores credentials by Beaver username. REST/WebSocket chat derive a trusted username from the access token and `AgentLoop` creates a run-local Gateway service for that user, leaving unauthenticated or unprovisioned runs on curated memory only.
|
||||||
|
|
||||||
|
**Tech Stack:** Python 3.14, dataclasses, FastAPI, httpx, pytest, shell-based Docker instance creation.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Task 1: Move Gateway source and load shared configuration
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `app-instance/backend/beaver/memory/gateway/__init__.py`
|
||||||
|
- Create: `app-instance/backend/beaver/memory/gateway/config.py`
|
||||||
|
- Create: `app-instance/backend/beaver/memory/gateway/client.py`
|
||||||
|
- Create: `app-instance/backend/beaver/memory/gateway/service.py`
|
||||||
|
- Create: `app-instance/backend/memory/config.json`
|
||||||
|
- Modify: `app-instance/backend/beaver/foundation/config/schema.py`
|
||||||
|
- Modify: `app-instance/backend/beaver/foundation/config/loader.py`
|
||||||
|
- Modify: `app-instance/backend/beaver/foundation/config/__init__.py`
|
||||||
|
- Delete: `app-instance/backend/beaver/integrations/memory_gateway/`
|
||||||
|
- Delete: `app-instance/backend/beaver/services/memory_gateway_service.py`
|
||||||
|
- Modify: `app-instance/backend/beaver/services/__init__.py`
|
||||||
|
- Test: `app-instance/backend/tests/unit/test_config_loader.py`
|
||||||
|
- Test: `app-instance/backend/tests/unit/test_memory_gateway_service.py`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Write failing shared-config and import tests**
|
||||||
|
|
||||||
|
Set `BEAVER_MEMORY_CONFIG_PATH` to a temporary JSON file and assert `load_config()` obtains `memory.mode`, URL, and all three scopes from that file. Change all Gateway tests to import from `beaver.memory.gateway`.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run tests and verify RED**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd app-instance/backend
|
||||||
|
.venv/bin/pytest -q tests/unit/test_config_loader.py tests/unit/test_memory_gateway_service.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: failures because the new package and shared config loading do not exist.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Implement package migration and shared config parsing**
|
||||||
|
|
||||||
|
Move existing client/service behavior without changing payloads. Define `MemoryConfig` and `MemoryGatewayConfig` in `beaver.memory.gateway.config`, without `userId/userKey`. Add `default_memory_config_path()` using `BEAVER_MEMORY_CONFIG_PATH` then `<backend-root>/memory/config.json`. Instance config parsing remains responsible for non-memory settings; shared config supplies `BeaverConfig.memory`.
|
||||||
|
|
||||||
|
Create tracked `memory/config.json` with `http://172.19.207.37:8010`, scopes `current_chat`, `resources`, `all_user_memory`, top K 8, and timeout 10.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Run targeted tests and verify GREEN**
|
||||||
|
|
||||||
|
Run the command from Step 2. Expected: selected tests pass.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add app-instance/backend/beaver/memory/gateway app-instance/backend/memory/config.json app-instance/backend/beaver/foundation/config app-instance/backend/beaver/services app-instance/backend/tests/unit/test_config_loader.py app-instance/backend/tests/unit/test_memory_gateway_service.py
|
||||||
|
git commit -m "refactor(memory): move gateway into memory domain"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Task 2: Add per-instance Gateway credential storage
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create: `app-instance/backend/beaver/memory/gateway/credentials.py`
|
||||||
|
- Modify: `app-instance/backend/beaver/memory/gateway/__init__.py`
|
||||||
|
- Create: `app-instance/backend/tests/unit/test_memory_gateway_credentials.py`
|
||||||
|
- Modify: `app-instance/create-instance.sh`
|
||||||
|
- Modify: `app-instance/entrypoint.sh`
|
||||||
|
- Modify: `app-instance/README.md`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Write failing credential-store tests**
|
||||||
|
|
||||||
|
Cover missing files, multi-user round trips, updates preserving other users, secret-free repr, atomic replace, and mode `0600`.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run test and verify RED**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd app-instance/backend
|
||||||
|
.venv/bin/pytest -q tests/unit/test_memory_gateway_credentials.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: import failure because the credential store does not exist.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Implement atomic credential persistence**
|
||||||
|
|
||||||
|
Implement `MemoryGatewayUserCredential(user_id, user_key)` and `MemoryGatewayCredentialStore.get/save`. Use JSON shape `{"users": {username: {"userId": ..., "userKey": ...}}}`, sibling temporary file, `os.replace`, and `chmod(0o600)`.
|
||||||
|
|
||||||
|
Update `create-instance.sh` to create `$BEAVER_HOME/memory_gateway_users.json` as `{"users": {}}`, chmod it `0600`, and pass `BEAVER_MEMORY_GATEWAY_USERS_PATH=/root/.beaver/memory_gateway_users.json`. `entrypoint.sh` exports the same default.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Run credential and shell syntax tests**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd app-instance/backend
|
||||||
|
.venv/bin/pytest -q tests/unit/test_memory_gateway_credentials.py
|
||||||
|
cd ../..
|
||||||
|
bash -n app-instance/create-instance.sh app-instance/entrypoint.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: tests pass and shell syntax exits zero.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add app-instance/backend/beaver/memory/gateway app-instance/backend/tests/unit/test_memory_gateway_credentials.py app-instance/create-instance.sh app-instance/entrypoint.sh app-instance/README.md
|
||||||
|
git commit -m "feat(memory): persist gateway user credentials"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Task 3: Provision Gateway identities during frontend registration
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `app-instance/backend/beaver/memory/gateway/client.py`
|
||||||
|
- Modify: `app-instance/backend/beaver/interfaces/web/app.py`
|
||||||
|
- Create: `app-instance/backend/tests/unit/test_memory_gateway_registration.py`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Write failing registration tests**
|
||||||
|
|
||||||
|
Use a temporary auth file and fake Gateway client. Assert registration sends `{"user_id": "tom"}`, stores the returned key, never returns the key to the browser, and still registers the Beaver user without a partial credential when Gateway provisioning fails.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run tests and verify RED**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd app-instance/backend
|
||||||
|
.venv/bin/pytest -q tests/unit/test_memory_gateway_registration.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: failures because `/users` provisioning is not connected.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Implement provisioning**
|
||||||
|
|
||||||
|
Add `MemoryGatewayClient.create_user(user_id)`, validating non-empty response `user_id/user_key`. During `/api/auth/register`, after local/AuthZ registration succeeds, call it with the Beaver username and save through the credential store. Catch sanitized Gateway failures without retrying or rolling back Beaver registration. Never include the Gateway credential in the response.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Run registration tests and verify GREEN**
|
||||||
|
|
||||||
|
Run the command from Step 2. Expected: all registration tests pass.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add app-instance/backend/beaver/memory/gateway/client.py app-instance/backend/beaver/interfaces/web/app.py app-instance/backend/tests/unit/test_memory_gateway_registration.py
|
||||||
|
git commit -m "feat(memory): provision gateway users on registration"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Task 4: Pass trusted authenticated identity into chat runs
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `app-instance/backend/beaver/interfaces/web/app.py`
|
||||||
|
- Modify: `app-instance/backend/beaver/engine/loop.py`
|
||||||
|
- Modify: `app-instance/backend/beaver/services/agent_service.py`
|
||||||
|
- Modify: `app-instance/backend/tests/unit/test_websocket_chat.py`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Write failing REST/WebSocket identity tests**
|
||||||
|
|
||||||
|
Issue a web token for `tom`. Assert REST and WebSocket calls pass `gateway_user_id="tom"`. Send a conflicting client `user_id="other"` and assert the trusted identity remains `tom`. Unauthenticated calls pass `gateway_user_id=None`.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run tests and verify RED**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd app-instance/backend
|
||||||
|
.venv/bin/pytest -q tests/unit/test_websocket_chat.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: identity assertions fail because chat does not pass a trusted Gateway principal.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Implement optional trusted identity resolution**
|
||||||
|
|
||||||
|
Add `gateway_user_id: str | None` to AgentLoop direct-run kwargs. REST reads the optional bearer token from `Authorization`; WebSocket reads the existing `?token=` parameter. Both resolve only through `app.state.auth_tokens`. Request `user_id` remains session metadata and never selects Gateway credentials.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Run identity tests and verify GREEN**
|
||||||
|
|
||||||
|
Run the command from Step 2. Expected: tests pass.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add app-instance/backend/beaver/interfaces/web/app.py app-instance/backend/beaver/engine/loop.py app-instance/backend/beaver/services/agent_service.py app-instance/backend/tests/unit/test_websocket_chat.py
|
||||||
|
git commit -m "feat(memory): bind gateway runs to authenticated users"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Task 5: Resolve a run-local Gateway service per user
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `app-instance/backend/beaver/engine/loader.py`
|
||||||
|
- Modify: `app-instance/backend/beaver/engine/loop.py`
|
||||||
|
- Modify: `app-instance/backend/beaver/memory/gateway/service.py`
|
||||||
|
- Modify: `app-instance/backend/tests/unit/test_memory_gateway_loader.py`
|
||||||
|
- Modify: `app-instance/backend/tests/unit/test_memory_gateway_agent_loop.py`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Write failing loader and AgentLoop tests**
|
||||||
|
|
||||||
|
Assert loader exposes shared config, credential store, and service factory instead of a fixed-user service. Add two users with different keys and verify each run constructs a service from only the selected credential. Missing identity or credential performs no Gateway calls while curated memory remains present.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Run tests and verify RED**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd app-instance/backend
|
||||||
|
.venv/bin/pytest -q tests/unit/test_memory_gateway_loader.py tests/unit/test_memory_gateway_agent_loop.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: failures because loader still creates one fixed-user service.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Implement per-run service resolution**
|
||||||
|
|
||||||
|
Expose `memory_gateway_config`, `memory_gateway_credentials`, and a service factory on `EngineLoadResult`. At run start, resolve the credential by `gateway_user_id`; construct a fresh service only in hybrid mode when a credential exists. Pass shared config and credential separately to the service and preserve current recall/add/flush/audit behavior.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Run Gateway runtime tests and verify GREEN**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd app-instance/backend
|
||||||
|
.venv/bin/pytest -q tests/unit/test_memory_gateway_loader.py tests/unit/test_memory_gateway_agent_loop.py tests/unit/test_memory_gateway_service.py tests/unit/test_context_builder.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: all selected tests pass.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add app-instance/backend/beaver/engine app-instance/backend/beaver/memory/gateway app-instance/backend/tests/unit/test_memory_gateway_loader.py app-instance/backend/tests/unit/test_memory_gateway_agent_loop.py
|
||||||
|
git commit -m "feat(memory): resolve gateway service per user"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Task 6: Update documentation and perform final verification
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Modify: `app-instance/backend/README.md`
|
||||||
|
- Modify: `app-instance/README.md`
|
||||||
|
- Modify: `docs/superpowers/plans/2026-06-15-hybrid-memory-gateway.md`
|
||||||
|
|
||||||
|
- [ ] **Step 1: Update operational documentation**
|
||||||
|
|
||||||
|
Document the shared config path, instance credential path, registration provisioning, token-based identity, secret handling, and rebuild/restart requirements. Remove examples that place `userId/userKey` in instance `config.json`.
|
||||||
|
|
||||||
|
- [ ] **Step 2: Verify removed source imports**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
rg -n "beaver\.integrations\.memory_gateway|beaver\.services\.memory_gateway_service" app-instance/backend/beaver app-instance/backend/tests
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: no matches.
|
||||||
|
|
||||||
|
- [ ] **Step 3: Run full verification**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd app-instance/backend
|
||||||
|
.venv/bin/python -m compileall -q beaver
|
||||||
|
.venv/bin/pytest -q
|
||||||
|
cd ../..
|
||||||
|
bash -n app-instance/create-instance.sh app-instance/entrypoint.sh
|
||||||
|
git diff --check
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: compile and shell checks exit zero, all tests pass, and diff check is clean.
|
||||||
|
|
||||||
|
- [ ] **Step 4: Scan tracked content for credentials**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git grep -nE 'uk_[A-Za-z0-9]{8,}' -- ':!docs/superpowers/specs/*' ':!docs/superpowers/plans/*'
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: no real Gateway key in tracked source or runtime files; obvious test placeholders are reviewed manually.
|
||||||
|
|
||||||
|
- [ ] **Step 5: Commit**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git add app-instance/backend/README.md app-instance/README.md docs/superpowers/plans/2026-06-15-hybrid-memory-gateway.md
|
||||||
|
git commit -m "docs(memory): document gateway user provisioning"
|
||||||
|
```
|
||||||
|
|
||||||
@ -0,0 +1,282 @@
|
|||||||
|
# Memory Gateway Package and User Provisioning Design
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Reorganize Beaver's Memory Gateway code under the `beaver.memory` domain and
|
||||||
|
replace the single fixed Gateway identity with per-Beaver-user credentials.
|
||||||
|
|
||||||
|
The final model has two independent configuration layers:
|
||||||
|
|
||||||
|
- One shared, non-secret Memory Gateway configuration used by every Beaver
|
||||||
|
instance.
|
||||||
|
- One per-instance credential file containing the Gateway identities created
|
||||||
|
for Beaver frontend users.
|
||||||
|
|
||||||
|
Curated memory remains enabled and isolated. Gateway failures or missing user
|
||||||
|
credentials must not modify `MEMORY.md`, `USER.md`, or the `memory` tool.
|
||||||
|
|
||||||
|
## Source Package
|
||||||
|
|
||||||
|
All Beaver-side Gateway source moves to:
|
||||||
|
|
||||||
|
```text
|
||||||
|
app-instance/backend/beaver/memory/gateway/
|
||||||
|
├── __init__.py
|
||||||
|
├── config.py
|
||||||
|
├── client.py
|
||||||
|
├── credentials.py
|
||||||
|
└── service.py
|
||||||
|
```
|
||||||
|
|
||||||
|
- `config.py` owns the shared typed Gateway configuration.
|
||||||
|
- `client.py` owns `MemoryGatewayClient` and sanitized client exceptions.
|
||||||
|
- `credentials.py` owns typed user credentials and atomic credential-file
|
||||||
|
persistence.
|
||||||
|
- `service.py` owns search/add/flush orchestration and result types.
|
||||||
|
- `__init__.py` exposes the supported public Gateway API.
|
||||||
|
|
||||||
|
Remove the old source locations:
|
||||||
|
|
||||||
|
- `beaver/integrations/memory_gateway/`
|
||||||
|
- `beaver/services/memory_gateway_service.py`
|
||||||
|
- Gateway configuration dataclasses in `beaver.foundation.config.schema`
|
||||||
|
- The lazy `MemoryGatewayService` export from `beaver.services`
|
||||||
|
|
||||||
|
No compatibility forwarding modules are retained. After migration,
|
||||||
|
`beaver.memory.gateway` is the only supported source entry point.
|
||||||
|
|
||||||
|
## Shared Configuration
|
||||||
|
|
||||||
|
All Beaver instances read the same public Gateway configuration from:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/home/tom/beaver_project/app-instance/backend/memory/config.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Inside the app-instance image this is available as:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/opt/app/backend/memory/config.json
|
||||||
|
```
|
||||||
|
|
||||||
|
The file contains no user credentials:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"memory": {
|
||||||
|
"mode": "hybrid",
|
||||||
|
"gateway": {
|
||||||
|
"baseUrl": "http://172.19.207.37:8010",
|
||||||
|
"appId": "default",
|
||||||
|
"projectId": "default",
|
||||||
|
"scope": ["current_chat", "resources", "all_user_memory"],
|
||||||
|
"topK": 8,
|
||||||
|
"timeoutSeconds": 10
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
- Valid modes remain `curated` and `hybrid`.
|
||||||
|
- Curated memory is always initialized.
|
||||||
|
- `hybrid` enables Gateway only for runs with a resolved user credential.
|
||||||
|
- `baseUrl` is fixed to `http://172.19.207.37:8010` in the initial shared
|
||||||
|
configuration.
|
||||||
|
- Scope includes `current_chat`, `resources`, and `all_user_memory`.
|
||||||
|
- The shared file is the authoritative Memory Gateway configuration. Instance
|
||||||
|
`config.json` files continue to own providers, tools, channels, AuthZ, and
|
||||||
|
backend identity, but no longer carry Gateway user credentials.
|
||||||
|
- An optional `BEAVER_MEMORY_CONFIG_PATH` may override the shared file path for
|
||||||
|
tests or non-image development runs.
|
||||||
|
|
||||||
|
## Per-Instance User Credentials
|
||||||
|
|
||||||
|
Each Beaver instance stores Gateway user credentials alongside its existing
|
||||||
|
`config.json`, `runtime.env`, and `web_auth_users.json`:
|
||||||
|
|
||||||
|
```text
|
||||||
|
app-instance/runtime/instances/<instance-slug>/beaver-home/
|
||||||
|
├── config.json
|
||||||
|
├── runtime.env
|
||||||
|
├── web_auth_users.json
|
||||||
|
└── memory_gateway_users.json
|
||||||
|
```
|
||||||
|
|
||||||
|
The existing `beaver-home` mount exposes the file inside the container as:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/root/.beaver/memory_gateway_users.json
|
||||||
|
```
|
||||||
|
|
||||||
|
The JSON format is:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"users": {
|
||||||
|
"tom": {
|
||||||
|
"userId": "tom",
|
||||||
|
"userKey": "uk_xxx"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
- The map key is the authenticated Beaver login username.
|
||||||
|
- Gateway `userId` is exactly the Beaver login username, with no prefix.
|
||||||
|
- `userKey` is secret and must never appear in API responses, logs, audit
|
||||||
|
events, exceptions, or tracked configuration.
|
||||||
|
- Writes use a sibling temporary file followed by atomic replace.
|
||||||
|
- The credential file is created with mode `0600`.
|
||||||
|
- `BEAVER_MEMORY_GATEWAY_USERS_PATH` may override the default path for tests.
|
||||||
|
|
||||||
|
## Frontend User Provisioning
|
||||||
|
|
||||||
|
The frontend continues to call Beaver's existing `POST /api/auth/register`
|
||||||
|
endpoint. The browser never calls Memory Gateway directly and never receives
|
||||||
|
the Gateway `userKey`.
|
||||||
|
|
||||||
|
For a registration request with username `tom`, Beaver performs:
|
||||||
|
|
||||||
|
```http
|
||||||
|
POST http://172.19.207.37:8010/users
|
||||||
|
Content-Type: application/json
|
||||||
|
|
||||||
|
{"user_id":"tom"}
|
||||||
|
```
|
||||||
|
|
||||||
|
Beaver validates that the response contains non-empty `user_id` and
|
||||||
|
`user_key`, requires the returned `user_id` to equal `tom`, and stores the
|
||||||
|
credential under the `tom` entry in `memory_gateway_users.json`.
|
||||||
|
|
||||||
|
The Gateway `/users` API is treated as idempotent. Registering an existing
|
||||||
|
Beaver username may refresh the same credential entry without creating a
|
||||||
|
second local identity.
|
||||||
|
|
||||||
|
For this first version:
|
||||||
|
|
||||||
|
- Gateway provisioning has no Beaver-side retries.
|
||||||
|
- A Gateway provisioning failure does not roll back an otherwise valid Beaver
|
||||||
|
registration.
|
||||||
|
- A user without stored Gateway credentials continues with curated memory only.
|
||||||
|
- No separate repair UI or background credential provisioning job is added.
|
||||||
|
|
||||||
|
## Authenticated Chat Identity
|
||||||
|
|
||||||
|
Gateway credential selection must use a trusted server-side principal.
|
||||||
|
|
||||||
|
- REST and WebSocket frontend chat paths resolve the Beaver username from the
|
||||||
|
issued access token.
|
||||||
|
- The resolved username is passed separately into the agent runtime as the
|
||||||
|
Gateway identity key.
|
||||||
|
- Client-provided `user_id` fields do not select Gateway credentials and cannot
|
||||||
|
impersonate another Gateway user.
|
||||||
|
- Runs without an authenticated frontend username, including channel or
|
||||||
|
scheduled runs without a trusted mapped identity, continue with curated
|
||||||
|
memory only.
|
||||||
|
|
||||||
|
This identity key is runtime-only. It is not included in provider prompts or
|
||||||
|
Gateway persisted message content.
|
||||||
|
|
||||||
|
## Runtime Architecture
|
||||||
|
|
||||||
|
`EngineLoader` loads:
|
||||||
|
|
||||||
|
1. Curated `MemoryService`, unconditionally.
|
||||||
|
2. Shared `MemoryGatewayConfig` from `memory/config.json`.
|
||||||
|
3. A `MemoryGatewayCredentialStore` for the instance credential file.
|
||||||
|
|
||||||
|
It does not construct one fixed-user `MemoryGatewayService` at startup.
|
||||||
|
|
||||||
|
For each authenticated run in hybrid mode:
|
||||||
|
|
||||||
|
1. `AgentLoop` receives the trusted Beaver username.
|
||||||
|
2. It reads that username's credential from the credential store.
|
||||||
|
3. If a credential exists, it constructs a run-local Gateway service/client
|
||||||
|
from the shared config and that credential.
|
||||||
|
4. It performs Gateway recall before context construction.
|
||||||
|
5. It performs Gateway add and flush after normal completion.
|
||||||
|
|
||||||
|
The run-local service has no shared mutable credential state, so concurrent
|
||||||
|
runs for different users cannot exchange identities. No service cache is added
|
||||||
|
in this version.
|
||||||
|
|
||||||
|
## Recall and Persistence
|
||||||
|
|
||||||
|
The existing hybrid behavior remains unchanged once a user credential has
|
||||||
|
been resolved:
|
||||||
|
|
||||||
|
- Search uses the current Beaver session id, current prompt, configured top K,
|
||||||
|
and all three configured scopes.
|
||||||
|
- Sanitized Gateway results are injected as one ephemeral untrusted-reference
|
||||||
|
message outside the system prompt.
|
||||||
|
- Normal completion persists exactly the original current user prompt and final
|
||||||
|
assistant text.
|
||||||
|
- Add is called once, followed by flush once only after add succeeds.
|
||||||
|
- Tool calls, tool results, system prompts, curated memory, recalled Gateway
|
||||||
|
text, reasoning, and skills are not persisted to Gateway.
|
||||||
|
- Gateway and curated memory remain isolated and do not synchronize, merge,
|
||||||
|
overwrite, or deduplicate each other.
|
||||||
|
|
||||||
|
## Security
|
||||||
|
|
||||||
|
- The shared configuration is safe to track because it contains no `userKey`.
|
||||||
|
- Per-user credentials live only under ignored instance runtime data.
|
||||||
|
- Credential-file permissions are `0600`.
|
||||||
|
- Credential objects suppress secrets from `repr`.
|
||||||
|
- Gateway client exceptions contain only operation, category, path, and status
|
||||||
|
metadata.
|
||||||
|
- Registration responses expose Beaver authentication data only; Gateway
|
||||||
|
credentials remain server-side.
|
||||||
|
- Hidden Gateway audit events may include the Beaver/Gateway user id but never
|
||||||
|
the user key or complete request/response body.
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
### Package migration
|
||||||
|
|
||||||
|
- All imports use `beaver.memory.gateway`.
|
||||||
|
- No references remain to the removed integration/service modules.
|
||||||
|
- Gateway config, client, service, and credential-store tests remain isolated
|
||||||
|
from curated memory.
|
||||||
|
|
||||||
|
### Shared configuration
|
||||||
|
|
||||||
|
- The shared file parses the fixed URL and three scopes.
|
||||||
|
- Invalid mode, URL, scope, top K, or timeout fails with sanitized errors.
|
||||||
|
- Instance config loading remains unchanged for non-memory settings.
|
||||||
|
- Test overrides can select a temporary shared config file.
|
||||||
|
|
||||||
|
### Credential persistence
|
||||||
|
|
||||||
|
- Missing files produce an empty credential map.
|
||||||
|
- Credentials round-trip by Beaver username.
|
||||||
|
- Updating one user preserves all other users.
|
||||||
|
- Files are atomically replaced and have mode `0600`.
|
||||||
|
- No exception or representation contains `userKey`.
|
||||||
|
|
||||||
|
### Registration
|
||||||
|
|
||||||
|
- New frontend registration calls `/users` with the Beaver username.
|
||||||
|
- Valid Gateway responses are stored without returning the key to the browser.
|
||||||
|
- Existing usernames refresh the same credential entry.
|
||||||
|
- Provisioning failure does not roll back Beaver registration and stores no
|
||||||
|
partial credential.
|
||||||
|
|
||||||
|
### Agent runtime
|
||||||
|
|
||||||
|
- Authenticated username selects only its own Gateway credential.
|
||||||
|
- Client-provided `user_id` cannot select another user's credential.
|
||||||
|
- Concurrent users construct independent run-local Gateway services.
|
||||||
|
- Missing credentials perform no Gateway calls and preserve curated behavior.
|
||||||
|
- Existing recall/add/flush ordering, payload, audit, and failure tests remain
|
||||||
|
valid.
|
||||||
|
|
||||||
|
### Verification
|
||||||
|
|
||||||
|
- Run targeted Gateway/config/auth/chat tests.
|
||||||
|
- Run Python compile checks and the complete backend test suite.
|
||||||
|
- Scan tracked files and diffs for real `userKey` values.
|
||||||
Reference in New Issue
Block a user