feat: extend memory search and attachment mapping

This commit is contained in:
2026-06-15 17:25:44 +08:00
parent 15462a95cb
commit e5cd87789f
9 changed files with 1194 additions and 54 deletions

View File

@ -0,0 +1,57 @@
# Memory Attachment Path Mapping Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Persist attachment-to-session mappings for resource and direct memory ingestion, then return filename-matched real URIs from memory search results.
**Architecture:** Add one SQLite attachment table and repository methods. Register resource files directly, materialize base64 memory attachments under Gateway storage, and enrich normalized search results by matching attachment names against recursive raw string values.
**Tech Stack:** Python 3.10+, FastAPI, SQLite, Pydantic, pytest, httpx.
---
### Task 1: Attachment persistence
**Files:**
- Modify: `core/db.py`
- Modify: `core/repository.py`
- Modify: `tests/test_gateway.py`
- [x] Write failing tests proving attachment records can be created, listed by user/session, deduplicated, and soft-deleted with resources.
- [x] Run focused tests and verify failure because the table and methods do not exist.
- [x] Add `memory_attachments`, indexes, resource backfill SQL, and focused repository methods.
- [x] Run focused tests and verify they pass.
### Task 2: Register attachments during ingestion
**Files:**
- Modify: `core/api.py`
- Modify: `core/service.py`
- Modify: `tests/test_gateway.py`
- [x] Write failing tests for `/resources`, `/memories/add` URI items, and `/memories/add` base64 items.
- [x] Run focused tests and verify missing mappings and files.
- [x] Register resource mappings, pass authenticated `user_id` into add service, materialize base64 files, and persist successful add mappings.
- [x] Run focused tests and verify they pass.
### Task 3: Enrich search results
**Files:**
- Modify: `core/service.py`
- Modify: `tests/test_gateway.py`
- [x] Write failing tests for filename match, no match, base64-key exclusion, and cross-user isolation.
- [x] Run focused tests and verify `attachments` is absent.
- [x] Recursively collect raw strings excluding base64 and return deduplicated matching attachments.
- [x] Run focused tests and verify they pass.
### Task 4: Documentation and regression
**Files:**
- Modify: `README.md`
- Modify: `tests/test_command.md`
- [x] Document attachment persistence, historical backfill limits, matching behavior, and response shape.
- [x] Update the search response example with `attachments`.
- [x] Run `git diff --check`, compile checks, and the complete pytest suite.
- [x] Review the final diff for user isolation and unintended URI exposure outside search.

View File

@ -0,0 +1,118 @@
# Memory Search Upstream Options Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Extend `POST /memories/search` with all upstream search options while preserving Gateway authentication, scopes, resource isolation, tombstones, and overrides.
**Architecture:** Extend the existing Pydantic request model and pass the validated values through `MemoryGatewayService`. Keep scope orchestration intact, combine caller filters with scope-generated session filters using `AND`, and tag normalized results according to their upstream response array.
**Tech Stack:** Python 3.10+, FastAPI, Pydantic v2, pytest, pytest-asyncio, httpx ASGI transport.
---
### Task 1: Search request options and defaults
**Files:**
- Modify: `tests/test_gateway.py`
- Modify: `core/api.py`
- Modify: `core/service.py`
- [x] **Step 1: Write failing tests for defaults, custom options, and validation**
Add API tests that assert a default search sends `method="hybrid"`, `include_profile=true`, and `enable_llm_rerank=true`; a custom request forwards `agent_id`, `keyword`, `radius`, `top_k=-1`, and both false flags; and invalid `method`, `radius`, and `top_k=0` return HTTP 422.
- [x] **Step 2: Run tests and verify expected failures**
Run:
```bash
uv run pytest tests/test_gateway.py -k 'search_forwards_default_upstream_options or search_forwards_all_upstream_options or search_rejects_invalid_upstream_options' -q
```
Expected: assertions fail because the request model and service do not yet accept or forward the new fields.
- [x] **Step 3: Implement request fields and payload forwarding**
Extend `SearchMemoriesRequest` with:
```python
agent_id: str | None = Field(default=None, min_length=1)
method: Literal["keyword", "vector", "hybrid", "agentic"] = "hybrid"
radius: float | None = Field(default=None, ge=0, le=1)
include_profile: bool = True
enable_llm_rerank: bool = True
filters: dict[str, Any] | None = None
```
Validate `top_k` as `-1` or `1..100`, pass all values to the service, and make `_search_payload` select exactly one upstream owner key (`agent_id` when present, otherwise `user_id`).
- [x] **Step 4: Run focused tests and verify they pass**
Run the command from Step 2. Expected: all selected tests pass.
### Task 2: Filter composition and result memory types
**Files:**
- Modify: `tests/test_gateway.py`
- Modify: `core/service.py`
- [x] **Step 1: Write failing tests for filter composition and result types**
Add a resource-scope test asserting caller filters and `session_id in [...]` are combined as:
```python
{"AND": [caller_filters, {"session_id": {"in": [session_id]}}]}
```
Extend the fake backend to return all response arrays and assert normalized results have `memory_type` values `episode`, `profile`, `agent_case`, `agent_skill`, and `unprocessed_message`.
- [x] **Step 2: Run tests and verify expected failures**
Run:
```bash
uv run pytest tests/test_gateway.py -k 'search_combines_custom_and_scope_filters or search_labels_all_memory_types' -q
```
Expected: failures because caller filters are not composed and normalized results have no `memory_type`.
- [x] **Step 3: Implement composition and typed normalization**
Add a small `_combine_filters` helper that returns either condition directly, returns `None` when both are absent, or returns `{"AND": [custom, scope]}` when both exist. Iterate an explicit mapping from response array name to memory type in `_extract_results` and include the mapped value in every normalized result.
- [x] **Step 4: Run focused tests and verify they pass**
Run the command from Step 2. Expected: both tests pass.
### Task 3: Documentation and regression verification
**Files:**
- Modify: `README.md`
- Verify: `tests/test_gateway.py`
- Verify: `tests/test_memory_gateway_skill.py`
- [x] **Step 1: Update the Chinese API documentation**
Document `agent_id`, `method`, `radius`, `include_profile`, `enable_llm_rerank`, `filters`, the `top_k=-1` rule, filter composition, and the `memory_type` response field. Update the curl and JSON examples with the new defaults.
- [x] **Step 2: Run formatting and full tests**
Run:
```bash
git diff --check
uv run pytest -q
```
Expected: no whitespace errors and all tests pass.
- [x] **Step 3: Review the final diff**
Run:
```bash
git diff --stat
git diff -- core/api.py core/service.py tests/test_gateway.py README.md
```
Expected: changes are limited to the approved search compatibility scope and documentation.