20 KiB
20 KiB
Memory Gateway multimodal API test
This file records a real end-to-end test through Memory Gateway, not direct upstream memory service calls.
Gateway URL used by curl:
http://127.0.0.1:8010
Gateway upstream memory service:
http://10.6.80.123:1995
Test assets:
/home/tom/memory-gateway/tests/simple-multimodal-image.png
/home/tom/memory-gateway/tests/simple-tone.wav
Asset check:
tests/simple-multimodal-image.png: PNG image data, 96 x 64, 8-bit/color RGB, non-interlaced
tests/simple-tone.wav: RIFF (little-endian) data, WAVE audio, Microsoft PCM, 16 bit, mono 8000 Hz
Start Gateway
Command:
cd /home/tom/memory-gateway
MEMORY_GATEWAY_BACKEND_BASE_URL=http://10.6.80.123:1995 \
MEMORY_GATEWAY_DB_PATH=/tmp/memory_gateway_curl.sqlite3 \
MEMORY_GATEWAY_STORAGE_DIR=/tmp/memory_gateway_curl_storage \
MEMORY_GATEWAY_HOST=127.0.0.1 \
MEMORY_GATEWAY_PORT=8010 \
.venv/bin/python main.py
Observed startup:
INFO: Started server process [771099]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://127.0.0.1:8010 (Press CTRL+C to quit)
1. Create Gateway user
Request:
USER_ID="gateway_user_20260611180257"
curl -sS --location 'http://127.0.0.1:8010/users' \
--header 'Content-Type: application/json' \
--data "{
\"user_id\": \"${USER_ID}\"
}"
Response:
{
"user_id": "gateway_user_20260611180257",
"user_key": "uk_REDACTED",
"created_at": "2026-06-11T10:02:57.435437+00:00"
}
HTTP metadata:
HTTP_STATUS:200
TOTAL_TIME:0.022431
2. Add text + audio(base64) + image(file) through Gateway
Request:
cd /home/tom/memory-gateway
USER_KEY="uk_REDACTED"
CONVERSATION_ID="gateway-multimodal-20260611180257"
SESSION_ID="chat:${CONVERSATION_ID}"
TIMESTAMP_MS="1781172177000"
AUDIO_BASE64="$(base64 -w0 tests/simple-tone.wav)"
curl -sS --location 'http://127.0.0.1:8010/memories/add' \
--header 'Content-Type: application/json' \
--data "{
\"user_id\": \"${USER_ID}\",
\"user_key\": \"${USER_KEY}\",
\"session_id\": \"${SESSION_ID}\",
\"app_id\": \"default\",
\"project_id\": \"default\",
\"messages\": [
{
\"sender_id\": \"${USER_ID}\",
\"role\": \"user\",
\"timestamp\": ${TIMESTAMP_MS},
\"content\": [
{
\"type\": \"text\",
\"text\": \"请通过 Memory Gateway 同时记住这段文字、音频和图片：图片里有左侧红色方块、右侧蓝色圆形、底部绿色横条；音频是一段短促的测试音。以后可能会问图片中各个物体的位置和颜色。\"
},
{
\"type\": \"audio\",
\"base64\": \"${AUDIO_BASE64}\",
\"ext\": \"wav\",
\"name\": \"simple-tone.wav\"
},
{
\"type\": \"image\",
\"uri\": \"file:///home/tom/memory-gateway/tests/simple-multimodal-image.png\",
\"ext\": \"png\",
\"name\": \"simple-multimodal-image.png\"
}
]
}
]
}"
Response:
{
"session_id": "chat:gateway-multimodal-20260611180257",
"backend": {
"request_id": "c9e24b8d27ee4ad08a8df70273336637",
"data": {
"message_count": 1,
"status": "accumulated"
}
}
}
HTTP metadata:
HTTP_STATUS:200
TOTAL_TIME:1.552665
3. Flush through Gateway
Request:
curl -sS --location 'http://127.0.0.1:8010/memories/flush' \
--header 'Content-Type: application/json' \
--data "{
\"user_id\": \"${USER_ID}\",
\"user_key\": \"${USER_KEY}\",
\"session_id\": \"${SESSION_ID}\",
\"app_id\": \"default\",
\"project_id\": \"default\"
}"
Response:
{
"session_id": "chat:gateway-multimodal-20260611180257",
"backend": {
"request_id": "8eb7d5db2d3b43f4999f445aabb813b1",
"data": {
"status": "extracted"
}
}
}
HTTP metadata:
HTTP_STATUS:200
TOTAL_TIME:2.135721
4. Search through Gateway
upstream memory service indexing can lag briefly after flush, so this test waited about 2 seconds before searching.
Request:
sleep 2
curl -sS --location 'http://127.0.0.1:8010/memories/search' \
--header 'Content-Type: application/json' \
--data "{
\"user_id\": \"${USER_ID}\",
\"user_key\": \"${USER_KEY}\",
\"conversation_id\": \"${CONVERSATION_ID}\",
\"query\": \"图片里的蓝色圆形在哪里？音频是什么？\",
\"scope\": [\"current_chat\"],
\"top_k\": 5,
\"app_id\": \"default\",
\"project_id\": \"default\"
}"
Response:
{
"results": [
{
"id": "gateway_user_20260611180257_ep_20260611_00000001",
"session_id": "chat:gateway-multimodal-20260611180257",
"text": "On June 11, 2026 at 10:02 AM UTC, user gateway_user_20260611180257 uploaded a multimodal memory package via Memory Gateway. The package included an image file named simple-multimodal-image.png and a short test audio clip. The image displayed three geometric shapes on a light gray background: a solid red square in the upper-left, a solid blue circle in the upper-right (horizontally aligned with the square), and a long, thin green horizontal rectangle spanning the bottom below both shapes. The user instructed the system to retain these details, anticipating future queries regarding the objects' positions and colors.",
"score": 0.6069304347038269,
"source_scope": "current_chat",
"resource_id": null,
"resource_uri": null,
"raw": {
"id": "gateway_user_20260611180257_ep_20260611_00000001",
"user_id": "gateway_user_20260611180257",
"app_id": "default",
"project_id": "default",
"session_id": "chat:gateway-multimodal-20260611180257",
"timestamp": "2026-06-11T10:02:57Z",
"sender_ids": [
"gateway_user_20260611180257"
],
"summary": "On June 11, 2026 at 10:02 AM UTC, user gateway_user_20260611180257 uploaded a multimodal memory package via Memory Gateway. The package included an image file named simple-multimodal-image.png and a s",
"subject": "gateway_user_20260611180257 Multimodal Memory Upload June 11, 2026",
"episode": "On June 11, 2026 at 10:02 AM UTC, user gateway_user_20260611180257 uploaded a multimodal memory package via Memory Gateway. The package included an image file named simple-multimodal-image.png and a short test audio clip. The image displayed three geometric shapes on a light gray background: a solid red square in the upper-left, a solid blue circle in the upper-right (horizontally aligned with the square), and a long, thin green horizontal rectangle spanning the bottom below both shapes. The user instructed the system to retain these details, anticipating future queries regarding the objects' positions and colors.",
"type": "Conversation",
"score": 0.6069304347038269,
"atomic_facts": [
{
"id": "gateway_user_20260611180257_af_20260611_00000004",
"content": "gateway_user_20260611180257 stated that questions about the positions and colors of the objects in the image might be asked in the future.",
"score": 0.6069304347038269
}
]
}
}
]
}
HTTP metadata:
HTTP_STATUS:200
TOTAL_TIME:0.064128
Other Memory Gateway API tests
The following calls used a temporary Gateway database and storage directory. All requests target Memory Gateway at http://127.0.0.1:8010.
5. Health
Request:
curl -sS --location 'http://127.0.0.1:8010/health'
Response:
{
"status": "ok",
"api": {"status": "ok"},
"backend": {
"status": "ok",
"base_url": "http://10.6.80.123:1995",
"data": {"status": "ok"}
}
}
HTTP_STATUS:200
TOTAL_TIME:0.034914
6. Invalid credentials
Request:
curl -sS --location \
'http://127.0.0.1:8010/resources?user_id=other_api_20260612095541&user_key=wrong-key'
Response:
{"detail":"invalid user credentials"}
HTTP_STATUS:401
TOTAL_TIME:0.001447
7. Upload resource
The temporary test user was created with:
curl -sS --location 'http://127.0.0.1:8010/users' \
--header 'Content-Type: application/json' \
--data '{"user_id":"other_api_20260612095541"}'
User response:
{
"user_id": "other_api_20260612095541",
"user_key": "uk_REDACTED",
"created_at": "2026-06-12T01:55:41.448076+00:00"
}
Upload request:
cd /home/tom/memory-gateway
curl -sS --location 'http://127.0.0.1:8010/resources' \
--form 'user_id=other_api_20260612095541' \
--form 'user_key=uk_REDACTED' \
--form 'app_id=default' \
--form 'project_id=default' \
--form 'title=Gateway API image resource' \
--form 'description=Resource lifecycle test through Memory Gateway' \
--form 'file=@tests/simple-multimodal-image.png;type=image/png'
Response:
{
"resource_id": "r_2700e435f72a49e6a7f736d17f8c7ac7",
"session_id": "resource:other_api_20260612095541:r_2700e435f72a49e6a7f736d17f8c7ac7",
"uri": "resource://other_api_20260612095541/r_2700e435f72a49e6a7f736d17f8c7ac7",
"status": "extracted"
}
HTTP_STATUS:200
TOTAL_TIME:4.700296
8. List resources
Request:
curl -sS --location \
'http://127.0.0.1:8010/resources?user_id=other_api_20260612095541&user_key=uk_REDACTED'
Response:
{
"resources": [
{
"resource_id": "r_2700e435f72a49e6a7f736d17f8c7ac7",
"user_id": "other_api_20260612095541",
"filename": "simple-multimodal-image.png",
"content_type": "image",
"mime_type": "image/png",
"uri": "resource://other_api_20260612095541/r_2700e435f72a49e6a7f736d17f8c7ac7",
"session_id": "resource:other_api_20260612095541:r_2700e435f72a49e6a7f736d17f8c7ac7",
"status": "extracted",
"title": "Gateway API image resource",
"description": "Resource lifecycle test through Memory Gateway",
"created_at": "2026-06-12T01:55:41.527716+00:00",
"updated_at": "2026-06-12T01:55:46.204082+00:00"
}
]
}
HTTP_STATUS:200
TOTAL_TIME:0.001785
9. Resource detail
Request:
curl -sS --location \
'http://127.0.0.1:8010/resources/r_2700e435f72a49e6a7f736d17f8c7ac7?user_id=other_api_20260612095541&user_key=uk_REDACTED'
Response:
{
"resources": [
{
"resource_id": "r_2700e435f72a49e6a7f736d17f8c7ac7",
"user_id": "other_api_20260612095541",
"filename": "simple-multimodal-image.png",
"content_type": "image",
"mime_type": "image/png",
"uri": "resource://other_api_20260612095541/r_2700e435f72a49e6a7f736d17f8c7ac7",
"session_id": "resource:other_api_20260612095541:r_2700e435f72a49e6a7f736d17f8c7ac7",
"status": "extracted",
"title": "Gateway API image resource",
"description": "Resource lifecycle test through Memory Gateway",
"created_at": "2026-06-12T01:55:41.527716+00:00",
"updated_at": "2026-06-12T01:55:46.204082+00:00"
}
]
}
HTTP_STATUS:200
TOTAL_TIME:0.001634
10. Search resource memory
Request:
curl -sS --location 'http://127.0.0.1:8010/memories/search' \
--header 'Content-Type: application/json' \
--data '{
"user_id": "other_api_20260612095541",
"user_key": "uk_REDACTED",
"query": "图片中有哪些颜色和形状？",
"scope": ["resources"],
"top_k": 5,
"app_id": "default",
"project_id": "default"
}'
Response:
{
"results": [
{
"id": "other_api_20260612095541_ep_20260612_00000001",
"session_id": "resource:other_api_20260612095541:r_2700e435f72a49e6a7f736d17f8c7ac7",
"text": "On June 12, 2026 at 01:55 AM UTC, the user other_api_20260612095541 uploaded an image titled 'simple-multimodal-image.png' for visual analysis. The image displayed three distinct geometric shapes on a plain, light gray background. The composition included a solid red square in the upper-left portion, a solid blue circle in the upper-right portion, and a long, thin, horizontal green rectangle situated below both shapes. The red square and blue circle were roughly aligned horizontally, while the green rectangle spanned a width greater than either of the upper shapes.",
"score": 0.6418947577476501,
"source_scope": "resources",
"resource_id": "r_2700e435f72a49e6a7f736d17f8c7ac7",
"resource_uri": "resource://other_api_20260612095541/r_2700e435f72a49e6a7f736d17f8c7ac7",
"raw": {
"id": "other_api_20260612095541_ep_20260612_00000001",
"user_id": "other_api_20260612095541",
"app_id": "default",
"project_id": "default",
"session_id": "resource:other_api_20260612095541:r_2700e435f72a49e6a7f736d17f8c7ac7",
"timestamp": "2026-06-12T01:55:41.541000Z",
"sender_ids": ["other_api_20260612095541"],
"summary": "On June 12, 2026 at 01:55 AM UTC, the user other_api_20260612095541 uploaded an image titled 'simple-multimodal-image.png' for visual analysis. The image displayed three distinct geometric shapes on a",
"subject": "Visual Analysis of Geometric Shapes Uploaded by other_api_20260612095541 on June 12, 2026",
"episode": "On June 12, 2026 at 01:55 AM UTC, the user other_api_20260612095541 uploaded an image titled 'simple-multimodal-image.png' for visual analysis. The image displayed three distinct geometric shapes on a plain, light gray background. The composition included a solid red square in the upper-left portion, a solid blue circle in the upper-right portion, and a long, thin, horizontal green rectangle situated below both shapes. The red square and blue circle were roughly aligned horizontally, while the green rectangle spanned a width greater than either of the upper shapes.",
"type": "Conversation",
"score": 0.6418947577476501,
"atomic_facts": [
{
"id": "other_api_20260612095541_af_20260612_00000001",
"content": "The image displays three distinct geometric shapes on a plain, light gray background.",
"score": 0.6418947577476501
}
]
}
}
]
}
HTTP_STATUS:200
TOTAL_TIME:0.176981
11. Override memory
Request:
curl -sS --location --request PATCH \
'http://127.0.0.1:8010/memories/other_api_20260612095541_ep_20260612_00000001' \
--header 'Content-Type: application/json' \
--data '{
"user_id": "other_api_20260612095541",
"user_key": "uk_REDACTED",
"session_id": "resource:other_api_20260612095541:r_2700e435f72a49e6a7f736d17f8c7ac7",
"override_text": "OVERRIDE: 图片左侧是红色方块，右侧是蓝色圆形，底部是绿色横条。"
}'
Response:
{
"memory_id": "other_api_20260612095541_ep_20260612_00000001",
"override_id": "o_328f03b40b164c4896640fd2567042cb",
"status": "active"
}
HTTP_STATUS:200
TOTAL_TIME:0.007037
The next search returned the overridden text:
Request:
curl -sS --location 'http://127.0.0.1:8010/memories/search' \
--header 'Content-Type: application/json' \
--data '{
"user_id": "other_api_20260612095541",
"user_key": "uk_REDACTED",
"query": "图片中有哪些颜色和形状？",
"scope": ["resources"],
"top_k": 5,
"app_id": "default",
"project_id": "default"
}'
{
"results": [
{
"id": "other_api_20260612095541_ep_20260612_00000001",
"session_id": "resource:other_api_20260612095541:r_2700e435f72a49e6a7f736d17f8c7ac7",
"text": "OVERRIDE: 图片左侧是红色方块，右侧是蓝色圆形，底部是绿色横条。",
"score": 0.6418947577476501,
"source_scope": "resources",
"resource_id": "r_2700e435f72a49e6a7f736d17f8c7ac7",
"resource_uri": "resource://other_api_20260612095541/r_2700e435f72a49e6a7f736d17f8c7ac7",
"raw": {
"id": "other_api_20260612095541_ep_20260612_00000001",
"user_id": "other_api_20260612095541",
"app_id": "default",
"project_id": "default",
"session_id": "resource:other_api_20260612095541:r_2700e435f72a49e6a7f736d17f8c7ac7",
"timestamp": "2026-06-12T01:55:41.541000Z",
"sender_ids": ["other_api_20260612095541"],
"summary": "On June 12, 2026 at 01:55 AM UTC, the user other_api_20260612095541 uploaded an image titled 'simple-multimodal-image.png' for visual analysis. The image displayed three distinct geometric shapes on a",
"subject": "Visual Analysis of Geometric Shapes Uploaded by other_api_20260612095541 on June 12, 2026",
"episode": "On June 12, 2026 at 01:55 AM UTC, the user other_api_20260612095541 uploaded an image titled 'simple-multimodal-image.png' for visual analysis. The image displayed three distinct geometric shapes on a plain, light gray background. The composition included a solid red square in the upper-left portion, a solid blue circle in the upper-right portion, and a long, thin, horizontal green rectangle situated below both shapes. The red square and blue circle were roughly aligned horizontally, while the green rectangle spanned a width greater than either of the upper shapes.",
"type": "Conversation",
"score": 0.6418947577476501,
"atomic_facts": [
{
"id": "other_api_20260612095541_af_20260612_00000001",
"content": "The image displays three distinct geometric shapes on a plain, light gray background.",
"score": 0.6418947577476501
}
]
},
"override_id": "o_328f03b40b164c4896640fd2567042cb"
}
]
}
HTTP_STATUS:200
TOTAL_TIME:0.055485
12. Delete memory with tombstone
Request:
curl -sS --location --request DELETE \
'http://127.0.0.1:8010/memories/other_api_20260612095541_ep_20260612_00000001' \
--header 'Content-Type: application/json' \
--data '{
"user_id": "other_api_20260612095541",
"user_key": "uk_REDACTED",
"session_id": "resource:other_api_20260612095541:r_2700e435f72a49e6a7f736d17f8c7ac7",
"reason": "Gateway API tombstone test"
}'
Response:
{
"memory_id": "other_api_20260612095541_ep_20260612_00000001",
"tombstone_id": "t_2cba49bf3b6641ea96865612deebc036",
"status": "deleted"
}
HTTP_STATUS:200
TOTAL_TIME:0.006502
Repeating the resource search after creating the tombstone:
curl -sS --location 'http://127.0.0.1:8010/memories/search' \
--header 'Content-Type: application/json' \
--data '{
"user_id": "other_api_20260612095541",
"user_key": "uk_REDACTED",
"query": "图片中有哪些颜色和形状？",
"scope": ["resources"],
"top_k": 5,
"app_id": "default",
"project_id": "default"
}'
{"results":[]}
HTTP_STATUS:200
TOTAL_TIME:0.067841
13. Delete resource
Request:
curl -sS --location --request DELETE \
'http://127.0.0.1:8010/resources/r_2700e435f72a49e6a7f736d17f8c7ac7?user_id=other_api_20260612095541&user_key=uk_REDACTED'
Response:
{
"resource_id": "r_2700e435f72a49e6a7f736d17f8c7ac7",
"session_id": "resource:other_api_20260612095541:r_2700e435f72a49e6a7f736d17f8c7ac7",
"uri": "resource://other_api_20260612095541/r_2700e435f72a49e6a7f736d17f8c7ac7",
"status": "deleted"
}
HTTP_STATUS:200
TOTAL_TIME:0.014089
List after deletion:
curl -sS --location \
'http://127.0.0.1:8010/resources?user_id=other_api_20260612095541&user_key=uk_REDACTED'
{"resources":[]}
HTTP_STATUS:200
TOTAL_TIME:0.001226
Detail after deletion:
curl -sS --location \
'http://127.0.0.1:8010/resources/r_2700e435f72a49e6a7f736d17f8c7ac7?user_id=other_api_20260612095541&user_key=uk_REDACTED'
{"resources":[]}
HTTP_STATUS:200
TOTAL_TIME:0.001223