tomtan/EverOS
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: initialize EverOS 1.0.0

Browse Source 
md-first memory extraction framework for AI agents.

Markdown is the single source of truth; SQLite holds state and LanceDB
provides the rebuildable vector + BM25 + scalar index. The codebase follows
a single-direction DDD layering (entrypoints -> service -> memory -> infra,
with component / core / config cross-cutting) enforced by import-linter.

Engineering surface:
- Coding conventions in .claude/rules/ (path-scoped) and workflows in
  .claude/skills/ (/commit, /new-branch, /pr).
- GitHub Actions CI runs make lint + test + integration; pre-commit mirrors
  the gates locally (ruff, hygiene hooks, gitlint commit-msg).
- Commit messages follow Conventional Commits, enforced by gitlint.
- make lint also enforces datetime two-zone discipline and OpenAPI drift.
This commit is contained in:
Elliot Chen
2026-06-05 22:35:51 +08:00
commit 518b8eca85
636 changed files with 160553 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

32
.claude/rules/architecture.md Normal file
View File

@ -0,0 +1,32 @@
# Architecture rule (always loaded)
EverOS is a DDD-layered framework. The dependency direction is **single, downward only**:
```
entrypoints → service → memory → infra
↓
component / core / config
```
- `entrypoints/` — CLI + HTTP API (presentation). No business logic.
- `service/` — use-case orchestration (memorize / retrieve / evolve / manage).
- `memory/` — domain (extract / search / cascade / prompt_slots / models).
- `infra/` — storage adapters (markdown + sqlite + lancedb) and the OME subsystem.
- `component/` — injectable providers (llm / embedding / config / utils).
- `core/` — runtime base (observability / lifespan / context / persistence primitives).
- `config/` — configuration data (Settings + default TOML).
## Hard constraints (enforced by `import-linter`, run in `make lint`)
1. **Layering**: an outer layer may import an inner layer, never the reverse.
`entrypoints → service → memory → infra`.
2. **Private internals**: `service`, `memory`, and `entrypoints` must not import
`infra.persistence.{markdown,lancedb,sqlite}.**` internals — go through the
package facade (`from everos.infra.persistence.markdown import ...`).
3. **OME isolation**: `infra.ome` must not import `persistence`, `memory`,
`service`, or `entrypoints`. It is a low-level scheduler with no domain knowledge.
If a change needs to cross a boundary the wrong way, the design is wrong — refactor,
don't add an exception.
Full rationale: [docs/architecture.md](../../docs/architecture.md).

21
.claude/rules/async-programming.md Normal file
View File

@ -0,0 +1,21 @@
---
paths:
- "src/**/*.py"
- "tests/**/*.py"
---
# Async programming rule
The write/read paths are async end-to-end. Keep them non-blocking.
- **No blocking calls in async functions** — no synchronous file I/O, no `time.sleep`,
no blocking DB/network calls inside `async def`. Ruff `ASYNC` flags the common cases.
- **Offload CPU/blocking work** with `anyio.to_thread.run_sync` (or the established
helper) rather than blocking the event loop.
- **Concurrency** via `asyncio.gather` / `asyncio.TaskGroup` for independent awaits;
don't `await` in a loop when the calls are independent.
- **Tests**: `pytest-asyncio` is in `auto` mode — an `async def test_*` just works,
no `@pytest.mark.asyncio` needed.
- **Don't fire-and-forget** without holding a reference (`asyncio.create_task` results
must be tracked, or you lose exceptions). The OME subsystem owns the long-running
background loops — application code shouldn't spawn its own.

17
.claude/rules/code-style.md Normal file
View File

@ -0,0 +1,17 @@
# Code style rule (always loaded)
- **Formatter & linter**: `ruff` is the single tool (replaces black / isort / flake8).
Line length 88, target `py312`. Run `make format` to auto-fix; `make lint` checks.
- **Active ruff rule sets**: `E F I N UP B SIM ASYNC`. Don't disable a rule inline
unless there's a genuine reason — prefer fixing the code.
- **Type hints**: annotate every public function signature (params + return). The
codebase is ~100% typed; keep it that way.
- **`from __future__ import annotations`** at the top of every module — annotations
are strings, so forward refs and `X | None` unions are free.
- **Prefer `collections.abc`** (`Sequence`, `Mapping`) over concrete `list`/`dict`
in signatures; use `Protocol` for structural interfaces.
- **No dead code**: no commented-out blocks, no unused imports, no speculative
abstractions. Delete rather than comment out.
- **Naming**: `*Manager` (orchestrators), `*Provider` (injectable services),
`*Reader`/`*Writer` (persistence), `*Recaller` (search routes). Follow the
established suffix when adding a sibling.

33
.claude/rules/datetime-handling.md Normal file
View File

@ -0,0 +1,33 @@
---
paths:
- "src/**/*.py"
- "tests/**/*.py"
---
# Datetime handling rule (two-zone discipline)
**Never** construct or read "now" directly. All datetime flows through
`everos.component.utils.datetime`. This is a **hard CI gate**
(`make check-datetime`, wired into `make lint`).
## Banned (the checker fails the build on these)
- `datetime.now()`, `datetime.utcnow()`, `datetime.today()`
- `time.time()`, `time.time_ns()`
- `datetime(YYYY, ...)` without `tzinfo=`
- `.astimezone(...)` / `.replace(tzinfo=...)` outside the helper module
## Use instead
| Need | Helper |
|---|---|
| "now" for **storage** (UTC) | `get_utc_now()` |
| "now" for **display** (configured TZ) | `get_now_with_timezone()` |
| today's date, display TZ | `today_with_timezone()` |
| normalize a value to UTC | `ensure_utc(d)` |
| render to display TZ | `to_display_tz(d)` |
| parse ISO / epoch / str | `from_iso_format(v)`, `from_timestamp(ts)` |
| serialize | `to_iso_format(d)`, `to_date_str(d)`, `to_timestamp_ms(d)` |
**Two zones**: persist in UTC, present in the configured display TZ. Crossing them
goes through the helpers — never ad-hoc. See [docs/datetime.md](../../docs/datetime.md).

22
.claude/rules/imports.md Normal file
View File

@ -0,0 +1,22 @@
---
paths:
- "src/**/*.py"
- "tests/**/*.py"
---
# Imports rule
- **`from __future__ import annotations`** is the first import in every module.
- **Import order** (ruff `I` enforces, `make format` fixes): stdlib → third-party
→ first-party (`everalgo`, then `everos`). One group per blank-line-separated block.
- **Absolute imports** for cross-package references (`from everos.memory import ...`).
Relative imports (`from .models import ...`) only **within** a package, typically
in its `__init__.py`.
- **`TYPE_CHECKING` guard** for import cycles and type-only imports:
```python
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from everos.config import Settings
```
- Never import a private internal across a package boundary — respect the
`import-linter` contracts (see [architecture.md](architecture.md)).

37
.claude/rules/init-py-and-reexport.md Normal file
View File

@ -0,0 +1,37 @@
---
paths:
- "src/**/__init__.py"
- "src/**/*.py"
---
# `__init__.py` and re-export rule
A package's `__init__.py` is its **public facade**. Consumers import from the
package, never from its internal modules.
## Pattern
```python
"""One-paragraph module docstring: what this package is and how to use it."""
from .models import Episode as Episode
from .models import MemCell as MemCell
__all__ = [
"Episode",
"MemCell",
]
```
- **Explicit `X as X` redundant-alias form** on each re-export. This is intentional:
it marks the name as a deliberate public re-export (ruff `F401` / `PLC0414` aware)
rather than an accidental unused import.
- **`__all__`** lists every public name, alphabetically sorted, matching the
re-exports. It is the contract; keep it in sync.
- **Internal modules stay private** — don't re-export helpers that aren't part of
the public API.
- New subpackage? Add an `__init__.py` with a docstring + `__all__` even if it
starts small. Empty-but-documented beats missing.
This facade discipline is what lets `import-linter` forbid deep imports across
package boundaries (see [architecture.md](architecture.md)).

12
.claude/rules/language-policy.md Normal file
View File

@ -0,0 +1,12 @@
# Language policy rule (always loaded)
The project targets a global audience and is **English-first**.
- **Code, comments, docstrings, docs, commit messages, identifiers**: English only.
- **CJK characters are allowed only in**:
- test fixtures under `tests/` and `data/` (multilingual input is the point), and
- locale-suffixed mirror files (e.g. `*_zh.json`).
- Do **not** introduce CJK into `src/`, `docs/`, or config.
Enforcement: `make check-cjk` scans for stray CJK outside the allowlist (advisory).
Keep user-facing strings and error messages in English.

23
.claude/rules/logging-observability.md Normal file
View File

@ -0,0 +1,23 @@
---
paths:
- "src/**/*.py"
---
# Logging & observability rule
- **Use the project logger**, never `print` or the stdlib `logging` directly:
```python
from everos.core.observability.logging import get_logger
logger = get_logger(__name__)
```
- **Structured logging** (`structlog`): pass context as keyword fields, not f-strings.
```python
logger.info("memory.search.completed", owner_type=owner, n_results=len(items))
```
Event name first (dotted, stable), structured kwargs after. This keeps logs
queryable and avoids leaking interpolated PII into the message string.
- **Levels**: `debug` for developer detail, `info` for lifecycle milestones,
`warning` for recoverable anomalies, `error` for failures with a stack/context.
- **Metrics** go through `core.observability.metrics` (Prometheus); don't invent
ad-hoc counters. Histograms/counters/gauges have registry helpers.
- Don't log secrets, API keys, or full memory content at `info`/above.

35
.claude/rules/module-docstring.md Normal file
View File

@ -0,0 +1,35 @@
---
paths:
- "src/everos/infra/**/*.py"
- "src/everos/memory/**/*.py"
- "src/everos/service/**/*.py"
- "src/everos/component/**/*.py"
- "src/everos/core/**/*.py"
---
# Module docstring rule
Every non-trivial module in the domain/infra layers opens with a docstring that
explains **intent and contract**, not just a one-line label.
A good module docstring states:
- **What** the module is responsible for (one sentence).
- **The load-bearing invariants** — the rules a reader must know to change it
safely (partition keys, what is/isn't written, defaults, ignored flags).
- **External usage** when the module is a package facade (a short import example).
Example (abbreviated, from `memory/search/manager.py`):
```python
"""SearchManager — top-level orchestrator for POST /api/v1/memory/search.
Hard partition by owner_type: user → episodes (+ profiles), agent →
agent_cases + agent_skills. The manager never writes to storage; it only
reads LanceDB + markdown.
"""
```
Prefer prose that would save the next engineer a debugging session over
boilerplate. If a module is genuinely trivial (a 3-line constant), a one-liner
is fine — but most modules here are not.

27
.claude/rules/testing.md Normal file
View File

@ -0,0 +1,27 @@
---
paths:
- "tests/**/*.py"
---
# Testing rule
Tests mirror the source layout: `tests/unit/test_<layer>/...`,
`tests/integration/...`, `tests/e2e/...`.
- **Structure**: `tests/unit/` mirrors `src/everos/` package-for-package. Put a test
next to where its subject lives in the mirror.
- **Async**: `pytest-asyncio` is in `auto` mode — write `async def test_*` directly,
no marker needed.
- **Markers** (default run excludes both — `-m "not slow and not live_llm"`):
- `@pytest.mark.slow` — tests ≥ ~10s.
- `@pytest.mark.live_llm` — needs real LLM/embedder credentials.
Keep unit tests fast and credential-free; push anything needing real services
behind a marker or into `integration`/`e2e`.
- **Fixtures**: shared fixtures live in the nearest `conftest.py`. The root conftest
resets module caches (settings/logging/datetime) per test — rely on that for
isolation rather than mutating globals.
- **Module docstring** on each test file stating what contract it pins (see existing
tests for the style).
- **Coverage gate**: `make cov` enforces 80% (`--cov-fail-under=80`). New code should
not drop coverage below the gate.
- Run `make test` (unit) and `make integration` before pushing; both run in CI.

19
.claude/settings.json Normal file
View File

@ -0,0 +1,19 @@
{
"permissions": {
"allow": [
"Bash(make:*)",
"Bash(uv sync:*)",
"Bash(uv run:*)",
"Bash(uv pip:*)",
"Bash(ruff:*)",
"Bash(pytest:*)",
"Bash(git status:*)",
"Bash(git diff:*)",
"Bash(git log:*)",
"Bash(git branch:*)",
"Bash(git show:*)",
"Bash(gh pr view:*)",
"Bash(gh pr list:*)"
]
}
}

54
.claude/skills/commit/SKILL.md Normal file
View File

@ -0,0 +1,54 @@
---
name: commit
description: Stage and create a Conventional Commits message for the current change
---
# /commit
Create a well-formed commit following the [Conventional Commits](https://www.conventionalcommits.org)
standard. The format is enforced by `gitlint` in the `commit-msg` hook.
## Steps
1. Run `git status` and `git diff` (and `git diff --staged`) to see what changed.
2. Review recent history for style: `git log --oneline -10`.
3. Group the change into a single focused commit. If the working tree mixes
unrelated changes, stage selectively (`git add -p` / specific paths) rather
than committing everything at once.
4. Write the message in **Conventional Commits** form:
```
<type>[(scope)][!]: <imperative summary, ≤72 chars>
<optional body: what & why, wrapped at 72>
<optional footer: BREAKING CHANGE: …, Refs: #123>
```
5. Never use `--no-verify`. If pre-commit hooks fail, fix the cause and re-commit.
6. Do not commit secrets, generated artifacts, or work-in-progress to a
protected branch (`main` / `dev` / `master`).
## Types
| Type | Use for |
|---|---|
| `feat` | new feature |
| `fix` | bug fix |
| `refactor` | behavior-preserving restructure |
| `test` | add / update tests |
| `docs` | documentation |
| `style` | formatting only |
| `perf` | performance |
| `chore` | config / build / tooling |
| `build` | build system or dependencies |
| `ci` | CI configuration |
| `revert` | revert a previous commit |
## Notes
- No emoji — the title must start with the `type`.
- One logical change per commit; keep the history bisectable.
- The summary is imperative mood: "add", not "added" / "adds".
- `scope` is optional: `fix(search): …`. A `!` before the colon (or a
`BREAKING CHANGE:` footer) marks a breaking change.

43
.claude/skills/new-branch/SKILL.md Normal file
View File

@ -0,0 +1,43 @@
---
name: new-branch
description: Create a branch following the project's GitFlow Lite model
---
# /new-branch
Cut a new branch under the GitFlow Lite model.
## Branch model
```
master = released / stable (tagged on release; protected)
dev = integration branch (protected)
feat/* = cut from dev → PR → merge into dev
fix/* = cut from dev → PR → merge into dev
hotfix/* = cut from master → merge into master AND synced into dev (double merge)
release = dev → master + tag on master (no separate release branch)
```
## Steps
1. Ask (or infer) the change type: `feat`, `fix`, or `hotfix`.
2. Pick the parent:
- `feat/*`, `fix/*` → branch from **`dev`**.
- `hotfix/*` → branch from **`master`**.
3. Update the parent first:
```bash
git checkout <parent>
git pull --ff-only
```
4. Create the branch with a kebab-case slug:
```bash
git checkout -b feat/<short-slug>
```
5. For a `hotfix`, remember it must later merge into **both** `master` and `dev`.
## Naming
- `feat/add-agentic-rerank`, `fix/empty-profile-crash`, `hotfix/lancedb-conn-leak`.
- Lowercase, hyphen-separated, no spaces, concise.
Never commit directly to `master` or `dev` — always via a branch + PR.

41
.claude/skills/pr/SKILL.md Normal file
View File

@ -0,0 +1,41 @@
---
name: pr
description: Open a GitHub PR targeting the correct branch with the project template
---
# /pr
Open a pull request on GitHub using the `gh` CLI and the repo's PR template.
## Steps
1. Confirm the branch and target:
- `feat/*`, `fix/*` → base **`dev`**.
- `hotfix/*` → base **`master`** (then a follow-up PR/sync into `dev`).
2. Ensure local checks pass first:
```bash
make ci
```
Do not open a PR with failing lint/tests.
3. Push the branch:
```bash
git push -u origin HEAD
```
4. Create the PR, filling the template
([.github/PULL_REQUEST_TEMPLATE.md](../../../.github/PULL_REQUEST_TEMPLATE.md)):
```bash
gh pr create --base dev --fill-first
```
Then edit the body to complete each section:
- **Summary** — what changed and why.
- **Area** — tick the relevant box (architecture / benchmark / use case /
docs / DX / CI-build-release).
- **Verification** — paste the commands you ran (`make ci`, manual checks).
- **Checklist** — tick honestly; don't tick boxes you didn't satisfy.
- **Notes for Reviewers** — anything subtle.
## Notes
- Keep the PR scoped to one area. Split unrelated changes.
- If `make ci` was not fully run, say so in Verification rather than implying it passed.
- A `hotfix` is not done until it has landed on **both** `master` and `dev`.

54
.github/ISSUE_TEMPLATE/bug_report.yml vendored Normal file
View File

@ -0,0 +1,54 @@
name: Bug report
description: Report a reproducible problem in EverOS
title: "[Bug]: "
labels: ["bug"]
body:
- type: markdown
attributes:
value: Thanks for helping improve EverOS. Clear reproduction steps make bugs much faster to fix.
- type: dropdown
id: area
attributes:
label: Area
options:
- methods/EverCore
- methods/HyperMem
- benchmarks/EverMemBench
- benchmarks/EvoAgentBench
- use-cases
- documentation
- other
validations:
required: true
- type: textarea
id: problem
attributes:
label: What happened?
description: Describe the bug and the behavior you expected.
validations:
required: true
- type: textarea
id: reproduce
attributes:
label: Steps to reproduce
placeholder: |
1. Go to ...
2. Run ...
3. See ...
validations:
required: true
- type: textarea
id: environment
attributes:
label: Environment
placeholder: |
OS:
Python:
Node:
Docker:
Commit:
- type: textarea
id: logs
attributes:
label: Logs or screenshots
render: shell

5
.github/ISSUE_TEMPLATE/config.yml vendored Normal file
View File

@ -0,0 +1,5 @@
blank_issues_enabled: true
contact_links:
- name: EverOS Discussions
url: https://github.com/EverMind-AI/EverOS/discussions
about: Ask questions, share ideas, and discuss roadmap topics.

22
.github/ISSUE_TEMPLATE/docs.yml vendored Normal file
View File

@ -0,0 +1,22 @@
name: Documentation issue
description: Report unclear, missing, or outdated documentation
title: "[Docs]: "
labels: ["documentation"]
body:
- type: input
id: page
attributes:
label: Page or file
placeholder: README.md, methods/EverCore/docs/...
validations:
required: true
- type: textarea
id: issue
attributes:
label: What should be improved?
validations:
required: true
- type: textarea
id: suggestion
attributes:
label: Suggested wording or structure

40
.github/ISSUE_TEMPLATE/feature_request.yml vendored Normal file
View File

@ -0,0 +1,40 @@
name: Feature request
description: Suggest an improvement or new capability
title: "[Feature]: "
labels: ["enhancement"]
body:
- type: dropdown
id: area
attributes:
label: Area
options:
- architecture methods
- benchmarks
- use cases
- developer experience
- documentation
- other
validations:
required: true
- type: textarea
id: problem
attributes:
label: Problem or opportunity
description: What user need, research gap, or workflow pain does this address?
validations:
required: true
- type: textarea
id: proposal
attributes:
label: Proposed solution
description: Describe the change you would like to see.
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: Alternatives considered
- type: textarea
id: context
attributes:
label: Additional context

37
.github/ISSUE_TEMPLATE/use_case.yml vendored Normal file
View File

@ -0,0 +1,37 @@
name: Use-case proposal
description: Propose an app, demo, or integration for use-cases/
title: "[Use Case]: "
labels: ["use case"]
body:
- type: textarea
id: summary
attributes:
label: Summary
description: What does the use case demonstrate?
validations:
required: true
- type: textarea
id: memory
attributes:
label: Memory behavior
description: What should the agent remember, retrieve, or evolve over time?
validations:
required: true
- type: textarea
id: stack
attributes:
label: Stack and dependencies
description: List frameworks, services, models, or external APIs.
- type: textarea
id: run
attributes:
label: Run path
description: How should a new developer run or inspect it?
- type: checkboxes
id: checklist
attributes:
label: Contribution checklist
options:
- label: I can include a README with setup instructions.
- label: I can avoid committing secrets, generated output, dependency folders, and image files.
- label: I can include `.env.example` if configuration is needed.

37
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View File

@ -0,0 +1,37 @@
## Summary
<!-- What changed, and why? -->
## Area
<!-- Mark the relevant option with an x. -->
- [ ] Architecture method
- [ ] Benchmark
- [ ] Use case
- [ ] Documentation
- [ ] Developer experience
- [ ] CI, build, or release
## Verification
<!-- List commands run, manual checks, screenshots, or reasons verification was not needed. -->
```text
```
## Checklist
- [ ] I kept the change scoped to the relevant area.
- [ ] I updated docs, examples, or setup notes when behavior changed.
- [ ] I added or updated tests when the change affects behavior.
- [ ] I did not commit secrets, `.env` files, dependency folders, or generated output.
- [ ] Active relative links in Markdown files resolve.
## Notes for Reviewers
<!-- Anything reviewers should pay special attention to? -->
By submitting this pull request, I agree that my contribution is licensed under
the Apache License 2.0.

20
.github/dependabot.yml vendored Normal file
View File

@ -0,0 +1,20 @@
version: 2
updates:
# GitHub Actions
- package-ecosystem: github-actions
directory: /
schedule:
interval: weekly
groups:
actions:
patterns: ["*"]
# Python dependencies (pip/uv) at the project root
- package-ecosystem: pip
directory: /
schedule:
interval: weekly
groups:
python-deps:
patterns: ["*"]

42
.github/workflows/ci.yml vendored Normal file
View File

@ -0,0 +1,42 @@
name: CI
on:
push:
branches: [main, dev, master]
pull_request:
# Cancel superseded runs on the same ref to save CI minutes.
concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
ci:
name: lint + test + integration
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v4
with:
enable-cache: true
cache-dependency-glob: uv.lock
- name: Set up Python
run: uv python install 3.12
- name: Install dependencies (frozen)
run: make install-deps
- name: Lint (ruff + import-linter + datetime + openapi drift)
run: make lint
- name: Unit tests
run: make test
- name: Integration tests
run: make integration

137
.github/workflows/docs.yml vendored Normal file
View File

@ -0,0 +1,137 @@
name: Docs
on:
pull_request:
paths:
- "**/*.md"
- ".github/ISSUE_TEMPLATE/**"
- ".github/PULL_REQUEST_TEMPLATE.md"
- ".github/workflows/docs.yml"
push:
branches: [main]
paths:
- "**/*.md"
- ".github/ISSUE_TEMPLATE/**"
- ".github/PULL_REQUEST_TEMPLATE.md"
- ".github/workflows/docs.yml"
permissions:
contents: read
concurrency:
group: docs-${{ github.ref }}
cancel-in-progress: true
jobs:
links:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Validate active relative Markdown links
run: |
python3 - <<'PY'
from pathlib import Path
import re
import sys
# Validate every Markdown file in the documentation surface. Globbing
# (rather than a hand-maintained list) means new docs are covered
# automatically and phantom links cannot slip in. Skip vendored trees.
skip_dirs = {".git", "node_modules", ".venv", ".uv-cache"}
files = sorted(
p
for p in Path(".").rglob("*.md")
if not any(part in skip_dirs for part in p.parts)
)
missing = []
for path in files:
if not path.exists():
continue
text = path.read_text()
active = re.sub(r"<!--.*?-->", "", text, flags=re.S)
for raw in re.findall(r"\[[^\]]*\]\(([^)]+)\)", active):
link = raw.split("#", 1)[0]
if not link or link.startswith(("http://", "https://", "mailto:")):
continue
target = (path.parent / link).resolve()
try:
target.relative_to(Path.cwd().resolve())
except ValueError:
missing.append((path, raw, "outside repository"))
continue
if not target.exists():
missing.append((path, raw, "missing"))
if missing:
for path, raw, reason in missing:
print(f"{path}: {raw} -> {reason}")
sys.exit(1)
print("Active relative Markdown links resolve.")
PY
- name: Validate use-case banner links
run: |
python3 - <<'PY'
from pathlib import Path
import re
import sys
files = {
Path("README.md"): "## Use Cases",
Path("use-cases/README.md"): "## Use Case Catalogue",
}
failures = []
warnings = []
primary_link_pattern = re.compile(
r"^\[(?:Code|Plugin|Live Demo|Learn more)\]\(([^)]+)\)",
flags=re.M,
)
for path, heading in files.items():
text = path.read_text()
start = text.find(heading)
if start == -1:
failures.append(f"{path}: missing {heading}")
continue
table_start = text.find("<table>", start)
table_end = text.find("</table>", table_start)
if table_start == -1 or table_end == -1:
failures.append(f"{path}: missing use-case table")
continue
table = text[table_start:table_end]
cells = re.findall(r"<td[^>]*>(.*?)</td>", table, flags=re.S)
for index, cell in enumerate(cells, start=1):
title_match = re.search(r"####\s+(.+)", cell)
title = title_match.group(1).strip() if title_match else f"use case {index}"
banner_match = re.search(r"\[!\[[^\]]*\]\([^)]+\)\]\(([^)]+)\)", cell)
primary_match = primary_link_pattern.search(cell)
if not banner_match and primary_match:
warnings.append(f"{path}: {title}: primary link has no linked banner")
elif banner_match and not primary_match:
failures.append(f"{path}: {title}: missing primary link")
elif banner_match and primary_match and banner_match.group(1) != primary_match.group(1):
failures.append(
f"{path}: {title}: banner link {banner_match.group(1)} "
f"does not match primary link {primary_match.group(1)}"
)
if warnings:
print("\n".join(f"warning: {warning}" for warning in warnings))
if failures:
print("\n".join(failures))
sys.exit(1)
print("Use-case banner links match primary links.")
PY
- name: Validate issue template YAML
run: |
ruby -e 'require "yaml"; Dir[".github/ISSUE_TEMPLATE/*.yml"].sort.each { |p| YAML.load_file(p); puts "YAML ok: #{p}" }'

163
.gitignore vendored Executable file
View File

@ -0,0 +1,163 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
pip-wheel-metadata/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
.python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyderworkspace
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# ───────────────────────────────────────────────────────────────────────────
# Project specific
# ───────────────────────────────────────────────────────────────────────────
# macOS
.DS_Store
# Editor / IDE
.trae/
.cursor/
# Claude Code: track team-shared rules/skills/settings; ignore personal + runtime
.claude/*
!.claude/rules/
!.claude/skills/
!.claude/settings.json
.claude/settings.local.json
.claude/worktrees/
# Runtime data (the default memory root + local databases)
.everos/
*.duckdb
# Large / generated artefacts that should never be committed
*.tar
*.zip
tmp/
outputs/
logs/
nohup.out
# Use-cases: exclude lock files to keep the repo lean
use-cases/**/package-lock.json

30
.gitlint Normal file
View File

@ -0,0 +1,30 @@
# gitlint — commit-message format gate (commit-msg pre-commit stage).
# Enforces Conventional Commits (https://www.conventionalcommits.org):
# <type>[(scope)][!]: <description>
# e.g. feat: add agentic rerank | fix(search): guard empty profile
# See .claude/skills/commit/SKILL.md for the type list and guidance.
[general]
# Enable gitlint's built-in Conventional Commits title check.
contrib=contrib-title-conventional-commits
# Autosquash / generated commits are exempt from the format check.
ignore-merge-commits=true
ignore-revert-commits=true
ignore-fixup-commits=true
ignore-squash-commits=true
# Allowed Conventional Commit types (extends the spec defaults with the ones
# this project uses). A commit title's type must be one of these.
[contrib-title-conventional-commits]
types=feat,fix,refactor,test,docs,style,perf,chore,build,ci,revert
[title-max-length]
line-length=72
[body-max-line-length]
line-length=80
# A single-line commit (no body) is allowed.
[body-min-length]
min-length=0

36
.pre-commit-config.yaml Normal file
View File

@ -0,0 +1,36 @@
# Local quality gate, run before each commit. Mirrors the checks CI enforces so
# failures surface locally first. Install with `make install` (sets up both the
# pre-commit and commit-msg hook stages).
#
# Run manually across the repo: uv run pre-commit run --all-files
default_install_hook_types: [pre-commit, commit-msg]
repos:
# ruff version is kept in sync with the `ruff` dev dependency in uv.lock.
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.15.12
hooks:
- id: ruff
name: ruff (lint + autofix)
args: [--fix]
- id: ruff-format
name: ruff (format)
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-toml
- id: check-added-large-files
args: [--maxkb=1024]
- id: detect-private-key
- id: check-merge-conflict
- repo: https://github.com/jorisroovers/gitlint
rev: v0.19.1
hooks:
- id: gitlint
name: gitlint (commit message format)
stages: [commit-msg]

130
ACKNOWLEDGMENTS.md Normal file
View File

@ -0,0 +1,130 @@
# Acknowledgments
[Home](README.md) > [Docs](docs/index.md) > Acknowledgments
Thanks to the following projects, communities, and individuals for their
inspiration and support.
---
## Inspiration & references
### [memsearch](https://github.com/zilliztech/memsearch)
Inspired our markdown-as-source-of-truth design and the SHA-256 +
file-watcher incremental sync model. memsearch is the closest project in
spirit to EverOS.
### [mem0](https://github.com/mem0ai/mem0)
Inspired the "one provider per file" flat adapter layout that EverOS uses
for `component/llm/` and `component/embedding/`.
### [Letta (MemGPT)](https://github.com/letta-ai/letta)
Inspired the multi-tier memory mapping (Core / Recall / Archival) that maps
naturally onto our MemCell / Episode / Archival pipeline.
### [MemOS](https://github.com/MemTensor/MemOS)
Provided a reference for memory taxonomy decisions (textual / parametric /
activation) and helped sharpen our scope choice to focus on textual memory.
### [Memos](https://github.com/usememos/memos)
A comprehensive open-source note-taking service whose plain-text-first
design philosophy reinforced our decision to keep markdown files as the
single source of truth.
### [Nemori](https://github.com/nemori-ai/nemori)
A self-organising long-term memory substrate for agentic LLM workflows that
provided valuable inspiration for our extraction pipeline.
---
## Open-source libraries
EverOS is built on top of excellent open-source libraries and frameworks:
### Core
- **[Python](https://www.python.org/)** — Programming language (3.12+)
- **[uv](https://github.com/astral-sh/uv)** — Fast Python package manager
- **[FastAPI](https://fastapi.tiangolo.com/)** — Modern async web framework (HTTP API)
- **[Pydantic](https://docs.pydantic.dev/)** — Data validation and settings
### Storage
- **[LanceDB](https://lancedb.com/)** — Embedded vector + BM25 + scalar database
- **[SQLite](https://sqlite.org/)** — Embedded relational database (state + audit log)
### Tooling
- **[Ruff](https://docs.astral.sh/ruff/)** — Lint + format
- **[import-linter](https://import-linter.readthedocs.io/)** — Layered architecture enforcement
- **[Hatchling](https://hatch.pypa.io/)** — Wheel build backend
- **[pytest](https://pytest.org/)** — Testing framework
- **[pre-commit](https://pre-commit.com/)** — Git hooks framework
### LLM & embedding providers
EverOS is provider-agnostic by design. Tested provider integrations include
OpenAI, Anthropic, Ollama, and SBERT. See [`component/llm/`](src/everos/component/llm/)
and [`component/embedding/`](src/everos/component/embedding/) for the
adapter layouts.
---
## Contributors
Thanks to all the developers who have contributed to this project.
See the full list of contributors on
[GitHub](https://github.com/EverMind-AI/EverOS/graphs/contributors).
<!-- Future: contributor image grid
<a href="https://github.com/EverMind-AI/EverOS/graphs/contributors">
<img src="https://contrib.rocks/image?repo=EverMind-AI/EverOS" />
</a>
-->
---
## Community
Thanks to our community for valuable feedback, bug reports, and feature
suggestions:
- **GitHub Issues & Discussions** — bug reports and feature requests
- **Discord** — [Join our Discord server](https://discord.gg/pfwwskxp)
- **X / Twitter** — [@EverMindAI](https://x.com/EverMindAI)
---
## Supporting organizations
- **Shanda Group** — for supporting the development of EverOS
---
## Special thanks
- To everyone who starred the repository
- To those who shared EverOS with others
- To researchers and developers using EverOS in their work
---
## Want to contribute?
Contributions are welcome! See the [Contributing Guide](CONTRIBUTING.md)
to get started.
---
## See also
- [Citation](CITATION.md)
- [Changelog](CHANGELOG.md)
- [Contributing Guide](CONTRIBUTING.md)

40
CHANGELOG.md Normal file
View File

@ -0,0 +1,40 @@
# Changelog
All notable changes to **EverOS** are documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
_Unreleased changes on `dev` will be listed here._
## [1.0.0] - 2026-06-03
First public release of EverOS — a Markdown-first memory extraction framework
for AI agents.
### Added
- **Markdown as source of truth** — all memory persists as plain `.md` files you
can open, edit, grep, and version with Git.
- **Lightweight three-piece storage** — Markdown (truth) + SQLite (state / queue
/ audit) + LanceDB (vector + BM25 + scalar index). No external services
required.
- **Hybrid retrieval** — BM25, vector, and scalar filtering in a single LanceDB
query.
- **Cascade index sync** — editing a `.md` file triggers a file watcher →
entry-level diff → sub-second LanceDB sync.
- **Dual-track memory** — user-track (Episodes / Profiles) and agent-track
(Cases / Skills).
- **Multi-source extraction** — conversations, workflows, agent traces, and file
knowledge.
- **CLI + HTTP API** — the `everos` command-line tool and a FastAPI server,
async-first throughout.
- **Pluggable providers** — LLM / embedding / rerank via the OpenAI-compatible
protocol (works with OpenAI, OpenRouter, vLLM, Ollama, …).
- **Decoupled algorithms** — memory extraction algorithms live in the standalone
`everalgo-*` libraries published on PyPI.
[Unreleased]: https://github.com/EverMind-AI/EverOS/compare/v1.0.0...HEAD
[1.0.0]: https://github.com/EverMind-AI/EverOS/releases/tag/v1.0.0

65
CITATION.md Normal file
View File

@ -0,0 +1,65 @@
# Citation
[Home](README.md) > [Docs](docs/index.md) > Citation
If you use **EverOS** in research or a published project, the suggested
citation forms are below.
---
## Paper
📄 **Paper coming soon** — a technical paper is in preparation.
```
Coming soon
```
BibTeX citation information will be added here once the paper is published.
---
## BibTeX
```bibtex
% Will be added when the paper is published
```
---
## Software citation
To cite the software itself:
```
EverOS: md-first memory extraction framework for AI agents
Version: 1.0.0
URL: https://github.com/EverMind-AI/EverOS
License: Apache 2.0
```
---
## Acknowledgments in your work
If you use EverOS, we appreciate:
- Citing the paper (when available)
- Starring the GitHub repository
- Mentioning EverOS in your acknowledgments section
---
## Stay updated
- Watch the [GitHub repository](https://github.com/EverMind-AI/EverOS) for paper announcements
- Follow [@EverMindAI](https://x.com/EverMindAI) on X / Twitter
- Join [GitHub Discussions](https://github.com/EverMind-AI/EverOS/discussions)
---
## See also
- [Overview](docs/overview.md)
- [Acknowledgments](ACKNOWLEDGMENTS.md)
- [Changelog](CHANGELOG.md)

82
CLAUDE.md Normal file
View File

@ -0,0 +1,82 @@
# EverOS — md-first Memory Extraction Framework
This is a Python framework for md-first memory extraction (lightweight; single-user or small-team).
## Quick commands
```bash
uv sync # install deps
make lint # ruff (check + format-check) + import-linter
make format # auto-fix formatting
make test # pytest tests/unit
make integration # pytest tests/integration
make ci # full CI: lint + test + integration
```
## Architecture
DDD 5 layers + cross-cutting:
```
entrypoints → service → memory → infra
↓
component / core / config
```
- `entrypoints/` — cli + api (Presentation)
- `service/` — use case orchestration (memorize / retrieve / evolve / manage)
- `memory/` — domain (extract + search + cascade + prompt_slots + models)
- `infra/` — storage adapters (markdown + sqlite + lancedb)
- `component/` — injectable providers (llm / embedding / config / utils)
- `core/` — runtime base (observability / lifespan / context)
- `config/` — configuration data (Settings + default.toml)
**Dependency rule**: `entrypoints → service → memory → infra`. Single-direction, enforced by `import-linter`.
Detailed: [docs/architecture.md](docs/architecture.md).
## Engineering practices
- **Coding rules** auto-loaded from [.claude/rules/](.claude/rules/) (10 rules; the three always-loaded ones cover architecture / code-style / language-policy, the rest are path-scoped and load when Claude Code opens a matching file)
- **Workflows** as slash commands in [.claude/skills/](.claude/skills/) — `/commit`, `/new-branch`, `/pr`
- **Project-level decisions** in [docs/](docs/) (low-frequency, human-judgment-required)
- **Language policy**: the project targets a global audience — docs and code are English; CJK appears only in test fixtures and locale-suffixed mirrors. Scanned by `make check-cjk`.
- **Datetime discipline**: never call `datetime.now()` / `time.time()` directly — use `everos.component.utils.datetime`. Enforced by `make check-datetime`.
Engineering infrastructure overview: [docs/engineering.md](docs/engineering.md).
## Branch strategy
`master` = released stable (hidden); `dev` = integration; `feat/* fix/*` → dev; `hotfix/*` → master + dev (sync).
See [docs/engineering.md](docs/engineering.md) for the full GitFlow Lite rationale.
## Storage three-piece set
```
Markdown (truth) + SQLite (state) + LanceDB (vector + BM25 + scalar)
```
- Memory root: `~/.everos/{agents,users,knowledge}/` (md files = single source of truth)
- System DB: `~/.everos/.index/sqlite/system.db` (state + audit + queue + metadata)
- Index: `~/.everos/.index/lancedb/` (rebuildable from md)
Selection rationale: [docs/architecture.md](docs/architecture.md).
## Source layout
**src layout** (`src/everos/<...>`): standard PyPA project structure — code lives under `src/` so the working tree is not on the import path until installed, preventing accidental imports of in-development modules.
Algorithm assets (prompts, extractors) live in the separate [`everalgo`](https://github.com/EverMind-AI/EverAlgo) library, consumed here as the `everalgo-*` PyPI packages.
## Where things go
| Want to... | Look at |
|---|---|
| Understand architecture | [docs/architecture.md](docs/architecture.md) |
| Understand storage choice | [docs/architecture.md](docs/architecture.md) (storage section) |
| Engineering tooling overview | [docs/engineering.md](docs/engineering.md) |
| Add a new module | [.claude/rules/init-py-and-reexport.md](.claude/rules/init-py-and-reexport.md) |
| Make a commit | use `/commit` |
| Open a branch / PR | use `/new-branch` / `/pr` |
| Run checks before pushing | `make ci` |

83
CODE_OF_CONDUCT.md Normal file
View File

@ -0,0 +1,83 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our community include:
* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
* Focusing on what is best not just for us as individuals, but for the overall community
Examples of unacceptable behavior include:
* The use of sexualized language or imagery, and sexual attention or advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful.
Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate.
## Scope
This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at **evermind@shanda.com**. All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series of actions.
**Consequence**: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within the community.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 2.1, available at [https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
Community Impact Guidelines were inspired by [Mozilla's code of conduct enforcement ladder][Mozilla CoC].
For answers to common questions about this code of conduct, see the FAQ at [https://www.contributor-covenant.org/faq][FAQ]. Translations are available at [https://www.contributor-covenant.org/translations][translations].
[homepage]: https://www.contributor-covenant.org
[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
[Mozilla CoC]: https://github.com/mozilla/diversity
[FAQ]: https://www.contributor-covenant.org/faq
[translations]: https://www.contributor-covenant.org/translations

157
CONTRIBUTING.md Normal file
View File

@ -0,0 +1,157 @@
# Contributing to EverOS
Thanks for your interest in EverOS! This page explains how contribution works
on this project.
## How EverOS accepts contributions
EverOS follows an **"open source, not open contribution"** model (similar to
SQLite). The codebase is developed and maintained by the EverMind core team, and
we **do not merge external pull requests**. This keeps copyright provenance
clean and the architecture coherent.
What we actively welcome from the community:
| Type | Where |
|---|---|
| 🐛 Bug reports | [Open a bug issue](https://github.com/EverMind-AI/EverOS/issues/new?template=bug_report.yml) |
| 💡 Feature ideas / use cases | [Open a feature issue](https://github.com/EverMind-AI/EverOS/issues/new?template=feature_request.yml) |
| 🔧 Suggested fixes | An issue with a code snippet / patch attached (see below) |
| ❓ Questions & discussion | [GitHub Discussions](https://github.com/EverMind-AI/EverOS/discussions) / [Discord](https://discord.gg/pfwwskxp) |
> **Pull requests opened against this repository will be closed** with a pointer
> to this policy. Please open an issue instead — it is the fastest path to
> getting a change in.
## Reporting a bug
Use the [bug report template](https://github.com/EverMind-AI/EverOS/issues/new?template=bug_report.yml). Include:
- Clear reproduction steps
- Expected vs. actual behavior
- Environment (OS, Python version, everos version)
- Relevant logs (**with secrets redacted**)
## Suggesting a feature
Use the [feature request template](https://github.com/EverMind-AI/EverOS/issues/new?template=feature_request.yml). Provide:
- The use case / problem being solved
- Proposed API or behavior
- Backward-compatibility considerations
## Suggesting a fix (code welcome)
Found the bug *and* the fix? Great — paste a minimal patch or code snippet
**in the issue**. Treat it as a proposal: the core team will review it, adapt it
to the project's conventions, and land the actual commit (crediting you in the
commit message / changelog).
> By posting a code suggestion in an issue, you agree it may be incorporated into
> EverOS under the project's [Apache-2.0](LICENSE) license.
## Reporting security issues
**Do not** open a public issue for security vulnerabilities. Follow the private
process in [SECURITY.md](SECURITY.md).
## Code of Conduct
This project and everyone participating in it is governed by the
[Contributor Covenant Code of Conduct](CODE_OF_CONDUCT.md). By participating, you
are expected to uphold it. Report unacceptable behavior to evermind@shanda.com.
## Questions
- [GitHub Discussions](https://github.com/EverMind-AI/EverOS/discussions) — general Q&A
- [Discord](https://discord.gg/pfwwskxp) — community chat
- Email: evermind@shanda.com
---
## For maintainers (core team)
The workflow below is for core-team members with write access. **You do not need
any of this to file an issue** — it documents how the team develops EverOS
internally.
### Prerequisites
- **Python 3.12+**
- [`uv`](https://docs.astral.sh/uv/) package manager
- Git
> No Docker / database services required — EverOS is lightweight (Markdown +
> SQLite + LanceDB embedded).
### Setup
```bash
git clone https://github.com/EverMind-AI/EverOS.git
cd EverOS
make install # deps + pre-commit hooks (one-stop dev setup)
everos init # write ./.env, then fill in the API key slots
make ci # verify: lint + unit + integration
```
### Code style
Conventions are auto-loaded by Claude Code from [.claude/rules/](.claude/rules/).
Highlights:
- **Python 3.12+**, Ruff formatting (88-char line)
- **Absolute imports** only
- **English only** in code / comments / docstrings (no CJK — see
[.claude/rules/language-policy.md](.claude/rules/language-policy.md))
- **Type hints** required on signatures; Pydantic v2 for data models
- **`__init__.py`** in every package; subpackages re-export public API via
`from .x import Y as Y` + `__all__`
- **DDD layered**: `entrypoints → service → memory → infra`, single direction,
enforced by `import-linter`
```bash
make format # ruff fix + format
make lint # ruff check + import-linter
```
### Branch strategy (GitFlow Lite)
| Branch | Role |
|---|---|
| `master` | Released stable |
| `dev` | Default integration branch |
| `feat/<scope>-<desc>` | New features (from dev → dev) |
| `fix/<scope>-<desc>` | Bug fixes (from dev → dev) |
| `hotfix/<scope>-<desc>` | Emergency fixes (from master → master + dev) |
Full rationale: [.claude/skills/new-branch/SKILL.md](.claude/skills/new-branch/SKILL.md).
### Commit messages
**[Conventional Commits](https://www.conventionalcommits.org)**: `<type>[(scope)]: <description>`
(e.g. `feat: add agentic rerank`, `fix(search): guard empty profile`). Enforced
by `gitlint` in the `commit-msg` hook. Use `/commit` for guided generation; full
type list: [.claude/skills/commit/SKILL.md](.claude/skills/commit/SKILL.md).
### Testing
```bash
make test # tests/unit
make integration # tests/integration
make cov # coverage report
```
- Add unit tests for new functions (`tests/unit/test_<module>/test_<action>_<expected>.py`)
- Add integration or e2e coverage for behavior changes (`tests/integration/`, `tests/e2e/`)
Full conventions: [.claude/rules/testing.md](.claude/rules/testing.md).
### Slash commands (Claude Code)
- `/new-branch` — create branch with proper naming
- `/commit` — generate a Conventional Commits message
- `/pr` — internal merge request with correct target branch
---
Thank you for helping make EverOS better! 🎉

201
LICENSE Normal file
View File

@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2025 EverMind AI
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

84
Makefile Normal file
View File

@ -0,0 +1,84 @@
.PHONY: help install install-deps lint check-cjk check-datetime openapi check-openapi format test integration cov ci clean
help:
@echo "Targets:"
@echo " install Install deps + pre-commit hooks (full dev setup)"
@echo " install-deps Install deps only (uv sync --frozen, used by CI)"
@echo " lint ruff (check + format-check) + import-linter + datetime discipline + openapi drift"
@echo " check-cjk Scan for CJK outside the language-policy allowlist (advisory)"
@echo " check-datetime Scan for code that bypasses component/utils/datetime (HARD gate, run via lint)"
@echo " openapi Regenerate docs/openapi.json from the FastAPI app"
@echo " check-openapi Verify docs/openapi.json matches app.openapi() (HARD gate, run via lint)"
@echo " format Format src/tests with ruff"
@echo " test pytest tests/unit"
@echo " integration pytest tests/integration"
@echo " cov pytest tests/unit + tests/integration with coverage (fail under 80%)"
@echo " ci full CI: lint + test + integration"
@echo " clean Remove caches"
# Sync deps from uv.lock; CI calls this directly. --frozen means "lock is the
# source of truth — fail rather than update it".
install-deps:
uv sync --frozen
# One-stop dev setup: deps + pre-commit hooks (both pre-commit and commit-msg
# stages — gitlint runs on commit-msg).
install: install-deps
uv run pre-commit install
uv run pre-commit install --hook-type commit-msg
lint:
uv run ruff check src tests
uv run ruff format --check src tests
uv run lint-imports
uv run python scripts/check_datetime_discipline.py
uv run python scripts/dump_openapi.py --check
# Advisory CJK scan (see .claude/rules/language-policy.md). Deliberately NOT
# wired into `lint` / `ci`: the policy is enforced by review and the rules
# doc, not a hard gate. Run on demand when touching potentially-CJK files.
check-cjk:
uv run python scripts/check_cjk.py
# Datetime two-zone discipline scanner (see .claude/rules/datetime-handling.md).
# Wired into `lint` (and therefore `ci`) as a HARD gate — any code that
# bypasses ``component/utils/datetime`` (raw ``datetime.now()``,
# ``time.time()``, naked ``datetime(...)`` constructor, etc.) fails the build.
check-datetime:
uv run python scripts/check_datetime_discipline.py
# OpenAPI schema export — produce docs/openapi.json from the FastAPI app.
# Run this after touching any HTTP route / DTO; commit the result.
openapi:
uv run python scripts/dump_openapi.py
# OpenAPI drift gate (wired into `lint`). Re-renders the schema in memory
# and diffs it against the committed ``docs/openapi.json``; any drift
# fails the build with a unified diff. Forces the contract doc to track
# the code on every PR that touches the API surface.
check-openapi:
uv run python scripts/dump_openapi.py --check
format:
uv run ruff check --fix src tests
uv run ruff format src tests
test:
uv run pytest tests/unit -v
integration:
uv run pytest tests/integration -v
# Coverage runs unit + integration so the number matches what CI's `test` and
# `integration` jobs actually exercise. Threshold starts at 80% (unit-only is
# currently 87%, unit+integration 91% — 80% leaves ~10pp headroom for normal
# churn). Bump as the suite stabilises.
cov:
uv run pytest tests/unit tests/integration --cov=src/everos --cov-report=term-missing --cov-branch --cov-fail-under=80
ci: lint test integration
clean:
rm -rf .pytest_cache .ruff_cache .uv-cache .mypy_cache .coverage htmlcov
find . -type d -name __pycache__ -exec rm -rf {} +
find . -type f -name '*.pyc' -delete

56
NOTICE Normal file
View File

@ -0,0 +1,56 @@
EverOS
Copyright 2025 EverMind AI
This product includes software developed by EverMind AI.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================================================
Third-Party Licenses
================================================================================
The Apache License (above) governs the EverOS source code as a whole.
The items listed below are bundled in this distribution but governed by
their own licenses; redistribute them only under those terms.
--------------------------------------------------------------------------------
LoCoMo conversation fixture (CC BY-NC 4.0 — NOT covered by Apache-2.0)
--------------------------------------------------------------------------------
File: tests/fixtures/long_conversation_locomo_caroline_melanie.json
Source: Derived from the LoCoMo dataset (snap-research/locomo).
License: Creative Commons Attribution-NonCommercial 4.0 International
https://creativecommons.org/licenses/by-nc/4.0/
This single file is licensed under CC BY-NC 4.0 and is therefore NOT
covered by the Apache-2.0 license that applies to the rest of EverOS.
Downstream use is restricted to non-commercial purposes per the upstream
license. Treat this file as a carve-out from the project-wide Apache
license when redistributing.
--------------------------------------------------------------------------------
CairoSVG (LGPL-3.0 — optional dependency, not installed by default)
--------------------------------------------------------------------------------
Trigger: ``pip install everos[multimodal]``
Path: everos[multimodal] -> everalgo-parser[svg] -> cairosvg
License: GNU Lesser General Public License, version 3
https://www.gnu.org/licenses/lgpl-3.0.html
Project: https://cairosvg.org/
CairoSVG is dynamically linked through ``everalgo-parser`` and is only
pulled in when the user explicitly opts into the ``multimodal`` extra.
LGPL-3.0 is compatible with Apache-2.0 redistribution in this dynamic-
linking arrangement; if you statically link or modify CairoSVG itself,
the LGPL-3.0 terms apply to that derivative work.

306
QUICKSTART.md Normal file
View File

@ -0,0 +1,306 @@
# Quickstart
> Five minutes from zero to "I added a conversation, queried it back, and
> can read it as plain Markdown."
EverOS runs as a **service** — start the server, then call the HTTP API.
There is no in-process library mode; an `everos` server is always in
front of your agent.
## Prerequisites
- **Python 3.12+**
- **An OpenRouter API key** — covers the chat LLM (memory extraction)
*and* the multimodal LLM (parsing image / pdf / audio content items)
with a single key.
- **A DeepInfra API key** — for the embedding + rerank models that
OpenRouter doesn't ship.
Two keys total. Any OpenAI-compatible endpoint plugs in via the
matching `*__BASE_URL` env var if you'd rather use OpenAI directly,
self-host vLLM, route to Ollama, etc.
## 1. Install
```bash
pip install everos
# or: uv pip install everos
```
## 2. Configure
Generate a starter `.env` and drop in your two keys:
```bash
everos init # writes ./.env (use --xdg for ~/.config/everos/.env)
# Edit .env and fill four API key slots (only two distinct keys needed):
# EVEROS_LLM__API_KEY (OpenRouter — chat LLM)
# EVEROS_MULTIMODAL__API_KEY (OpenRouter — same key works)
# EVEROS_EMBEDDING__API_KEY (DeepInfra)
# EVEROS_RERANK__API_KEY (DeepInfra — same key works)
```
`everos init` reads the template bundled inside the wheel and writes it
with `0600` permissions (only your user can read the API keys).
The shipped template already points LLM + multimodal → OpenRouter
(`openai/gpt-4.1-mini` and `google/gemini-3-flash-preview`) and
embedding + rerank → DeepInfra (`Qwen/Qwen3-Embedding-4B` and
`Qwen/Qwen3-Reranker-4B`). To use a different OpenAI-compatible
endpoint, override the matching `*__BASE_URL` env var.
> **Where to store `.env`** — `everos server start` searches in order:
> `--env-file <path>` → `./.env` (cwd) → `${XDG_CONFIG_HOME:-~/.config}/everos/.env` →
> `~/.everos/.env`. The first existing file wins. Use `everos init --xdg` to write
> the XDG location so the same config works from any cwd.
## 3. Start the server
```bash
everos server start
```
You should see (port and host are configurable):
```
starting everos on 127.0.0.1:8000
INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
```
- Default bind is `127.0.0.1` (loopback only). To expose the API
elsewhere, put your own auth/gateway in front first
([see SECURITY.md](SECURITY.md)).
- The cascade index daemon runs **in the same process** as a FastAPI
lifespan coroutine — you don't need a separate worker.
- The server runs in the foreground; **open a second terminal** for the
steps below, and use `Ctrl+C` to stop the server when you're done.
In the second terminal, verify the server is up:
```bash
$ curl http://127.0.0.1:8000/health
{"status":"ok"}
```
## 4. Add a conversation
EverOS ingests memory at the **conversation level**, not as standalone
sentences: you POST a batch of `messages` tied to a `session_id`, and
the server accumulates them until the boundary detector trips (you can
also force a flush — see step 5).
```bash
TS=$(($(date +%s)*1000)) # Unix epoch in **milliseconds** (v1 contract)
curl -X POST http://127.0.0.1:8000/api/v1/memory/add \
-H 'Content-Type: application/json' \
-d "{
\"session_id\": \"demo-001\",
\"app_id\": \"default\",
\"project_id\": \"default\",
\"messages\": [
{\"sender_id\": \"alice\", \"role\": \"user\", \"timestamp\": $TS, \"content\": \"I love climbing in Yosemite every spring.\"},
{\"sender_id\": \"alice\", \"role\": \"user\", \"timestamp\": $((TS+10000)), \"content\": \"My favorite coffee shop is Blue Bottle in SOMA.\"},
{\"sender_id\": \"alice\", \"role\": \"user\", \"timestamp\": $((TS+20000)), \"content\": \"I bike to work most days.\"}
]
}"
```
Response:
```json
{
"request_id": "bf86e4e857834eba804841f8bff29106",
"data": {
"message_count": 3,
"status": "accumulated"
}
}
```
`status: "accumulated"` means the three messages are in the session
buffer, but the boundary detector hasn't decided to extract a memory
cell yet. For a quick demo we'll force it.
## 5. Force boundary extraction
```bash
curl -X POST http://127.0.0.1:8000/api/v1/memory/flush \
-H 'Content-Type: application/json' \
-d '{"session_id":"demo-001","app_id":"default","project_id":"default"}'
```
Response (this takes a few seconds — one LLM call for extraction):
```json
{
"request_id": "ec0e7a00c3bd4b00bb21212a411b7763",
"data": {
"status": "extracted"
}
}
```
`status: "extracted"` means at least one memory cell was carved out and
written to disk + indexed.
> `/flush` is **OSS-only**. The cloud edition decides boundary timing
> server-side and does not expose this endpoint.
## 6. Search the memory you just added
```bash
curl -X POST http://127.0.0.1:8000/api/v1/memory/search \
-H 'Content-Type: application/json' \
-d '{
"user_id": "alice",
"app_id": "default",
"project_id": "default",
"query": "Where do I like to climb?",
"top_k": 5
}'
```
Response (trimmed):
```json
{
"request_id": "b53a3a94a080472d97692c503c88afdf",
"data": {
"episodes": [
{
"id": "alice_ep_20260528_00000002",
"user_id": "alice",
"session_id": "demo-001",
"summary": "On May 28, 2026 ... Alice shared that she loves climbing in Yosemite every spring ...",
"score": 0.6284722685813904,
"atomic_facts": [
{
"id": "alice_af_20260528_00000016",
"content": "Alice said she loves climbing in Yosemite every spring.",
"score": 0.6284722685813904
}
]
}
],
"profiles": [],
"agent_cases": [],
"agent_skills": []
}
}
```
The hybrid retrieval (BM25 + vector + scalar) returns the episode
that contains the climbing fact, with the matching atomic fact nested
under it. Other response arrays (`profiles` / `agent_cases` /
`agent_skills`) are always present for client-side symmetry, populated
only when the requested kind matches.
## 7. Your memory is just Markdown
This is what makes EverOS different — your memory persists as plain
Markdown files on disk:
```
$ tree ~/.everos -L 5 -a
~/.everos
├── default_app/ ← app_id ("default" → "default_app")
│ └── default_project/ ← project_id ("default" → "default_project")
│ └── users/
│ └── alice/ ← user_id (mirror dir: agents/<agent_id>/)
│ ├── episodes/
│ │ └── episode-2026-05-28.md
│ ├── .atomic_facts/ ← hidden (dot-prefix)
│ │ └── atomic_fact-2026-05-28.md
│ ├── .foresights/
│ │ └── foresight-2026-05-28.md
│ └── user.md ← profile
├── .index/ ← derived indexes (rebuildable from md)
│ ├── sqlite/system.db
│ └── lancedb/*.lance/
└── .tmp/
```
The `default` scope id materialises as `default_app` / `default_project`
on disk (with the `_app` / `_project` suffix) so the default space is
visually distinct from any user-named space. Any other id maps to itself
(e.g. `app_id: "my-app"` → `my-app/`).
Top-level `.index/` holds SQLite + LanceDB **derived** indexes — wipe it
and the cascade daemon rebuilds everything from the Markdown alone.
Read the episode we just created:
```
$ cat ~/.everos/default_app/default_project/users/alice/episodes/episode-2026-05-28.md
---
id: episode_log_alice_2026-05-28
type: episode_daily
file_type: episode_daily
schema_version: 1
user_id: alice
track: user
date: '2026-05-28'
entry_count: 1
last_appended_at: '2026-05-28T08:32:24.966944+00:00'
---
<!-- entry:ep_20260528_00000002 -->
## ep_20260528_00000002
**owner_id**: alice
**session_id**: demo-001
**timestamp**: 2026-05-28T08:32:13+00:00
**parent_type**: memcell
**parent_id**: mc_3779c20f1c53
**sender_ids**: [alice]
### Subject
Alice's Outdoor Activities and Daily Routine on May 28, 2026 Morning
### Content
On May 28, 2026 at 8:32 AM UTC, Alice shared that she loves climbing in
Yosemite every spring, highlighting a recurring seasonal outdoor activity.
She also mentioned that her favorite coffee shop is Blue Bottle located in
SOMA, indicating a preferred local spot. Additionally, Alice stated that
she bikes to work most days, revealing a habitual commuting practice.
<!-- /entry:ep_20260528_00000002 -->
```
Every memory entry is a plain Markdown file you can:
- `cat` / `grep` / `vim` directly — no driver, no service to query
- Version with Git (or rsync to backup)
- Open the `~/.everos/default_app/default_project/users/alice/` folder
in Obsidian (the dotfile directories stay hidden by default)
## Stopping the server
`Ctrl+C` in the server terminal. Uvicorn catches `SIGINT` and shuts each
lifespan provider down in reverse order (cascade → LanceDB → SQLite →
LLM → metrics) before exiting.
## Next steps
- **Integrate into your agent** — wrap the three endpoints (`/add`,
`/flush`, `/search`) in a thin Python client (`httpx.AsyncClient`) and
call them from your agent loop.
- **App + project scope** — set `app_id` / `project_id` to anything
other than `"default"` to partition memory spaces inside one server.
- **Multi-modal messages** — `messages[].content` accepts a list of
typed `ContentItem`s (`text` / `image` / `audio` / `doc` / `pdf` /
`html` / `email`) for non-text input. Install the optional extra
to enable parsing:
`uv pip install 'everos[multimodal]'`. Office documents
(`doc` / `docx` / `xls` / `ppt` / `…`) additionally need
**LibreOffice** on the host (`brew install --cask libreoffice` /
`apt-get install libreoffice`) — without it those uploads return
HTTP 415; PDF / image / audio / HTML still work.
- **Filter DSL and search modes** — `/search` supports a filter DSL
(`AND` / `OR` / scalar predicates) and four methods (`HYBRID` /
`KEYWORD` / `VECTOR` / `AGENTIC`). See the OpenAPI schema served at
`/docs`.
- **Architecture** — see [docs/architecture.md](docs/architecture.md)
for the DDD layering and cascade design, and
[docs/storage_layout.md](docs/storage_layout.md) for the on-disk
layout.
- **Found a bug?** — open an issue (see [CONTRIBUTING.md](CONTRIBUTING.md);
external pull requests are not merged).

569
README.md Normal file
View File

@ -0,0 +1,569 @@
<div align="center" id="readme-top">
![banner-gif](https://github.com/user-attachments/assets/0bf97efd-580f-4a53-a2a2-58d6daea7290)
<p align="center">
<a href="https://x.com/evermind"><img src="https://img.shields.io/badge/EverMind-000000?labelColor=gray&style=for-the-badge&logo=x&logoColor=white" alt="X"></a>
<a href="https://huggingface.co/EverMind-AI"><img src="https://img.shields.io/badge/🤗_HuggingFace-EverMind-F5C842?labelColor=gray&style=for-the-badge" alt="HuggingFace"></a>
<a href="https://discord.gg/gYep5nQRZJ"><img src="https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fdiscord.com%2Fapi%2Fv10%2Finvites%2FgYep5nQRZJ%3Fwith_counts%3Dtrue&query=%24.approximate_presence_count&suffix=%20online&label=Discord&color=404EED&labelColor=gray&style=for-the-badge&logo=discord&logoColor=white" alt="Discord"></a>
<a href="https://github.com/EverMind-AI/EverOS/discussions/67"><img src="https://img.shields.io/badge/WeCom-EverMind_社区-07C160?labelColor=gray&style=for-the-badge&logo=wechat&logoColor=white" alt="WeChat"></a>
</p>
[Website](https://evermind.ai) · [Documentation](https://docs.evermind.ai) · [Blog](https://evermind.ai/blogs)
</div>
<br>
<details open>
<summary><kbd>Table of Contents</kbd></summary>
<br>
- [What is EverOS](#what-is-everos)
- [Architecture at a glance](#architecture-at-a-glance)
- [Quick start](#quick-start)
- [Storage layout](#storage-layout)
- [Features](#features)
- [Project structure](#project-structure)
- [Documentation](#documentation)
- [Use Cases](#use-cases)
- [Stay Tuned](#stay-tuned)
- [Contributing](#contributing)
<br>
</details>
## What is EverOS
EverOS is an open-source Python framework that turns conversations, agent trajectories, and files into **structured, retrievable, evolving long-term memory** for AI agents and user chats. Designed for **lightweight local deployments** (small teams, individual developers), with three core principles:
1. **Markdown as Source of Truth** — All memory persists as plain `.md` files. Open, edit, grep, version with Git, view in Obsidian. No black-box database lock-in.
2. **Lightweight three-piece storage** — `Markdown` files (truth) + `SQLite` (state/queue) + `LanceDB` (vector + BM25 + scalar). No MongoDB / Elasticsearch / Milvus / Redis / Kafka required.
3. **[EverAlgo](https://github.com/EverMind-AI/EverAlgo) as pure algorithm library** — Memory extraction algorithms are decoupled into a separate library; this project orchestrates and persists.
<br>
## Architecture at a glance
```
┌───────────────────────────────────────────────┐
│ entrypoints/ (CLI + HTTP API) │ presentation
├───────────────────────────────────────────────┤
│ service/ (use cases: memorize/retrieve) │ application
├───────────────────────────────────────────────┤
│ memory/ (extract + search + cascade) │ domain
├───────────────────────────────────────────────┤
│ infra/ (markdown / sqlite / lancedb) │ infrastructure
└───────────────────────────────────────────────┘
↑ ↑
component/ core/
(LLM/Embedding) (observability/lifespan)
```
DDD 5 layers, single-direction dependency. See [docs/architecture.md](docs/architecture.md).
<br>
## Quick start
### Install as a package
```bash
uv pip install everos # or: pip install everos
# Generate a starter .env (OpenRouter + DeepInfra defaults; bundled inside the wheel)
everos init # writes ./.env (use --xdg for ~/.config/everos/.env)
# Edit .env and fill the API key fields (see comments inside).
everos --help
everos server start
```
`everos server start` searches for `.env` in this order: `--env-file <path>` →
`./.env` (cwd) → `${XDG_CONFIG_HOME:-~/.config}/everos/.env` → `~/.everos/.env`.
The endpoint stack is OpenAI-protocol compatible (OpenAI / OpenRouter / vLLM /
Ollama / DeepInfra …) — override `*__BASE_URL` in the generated `.env` to point
at any of them.
#### Multi-modal (optional)
To ingest non-text content (image / pdf / audio / office documents)
through `/api/v1/memory/add` `content` items, install the optional
extra:
```bash
uv pip install 'everos[multimodal]' # or: pip install 'everos[multimodal]'
```
This pulls in `everalgo-parser` (with the `[svg]` bundle for SVG
support via cairosvg) and wires up the multimodal LLM client
(`EVEROS_MULTIMODAL__*` fields in `.env`, defaults to
`google/gemini-3-flash-preview` via OpenRouter).
**Office document support requires LibreOffice as a system dependency.**
The parser shells out to `soffice` (LibreOffice's headless renderer) to
convert `.doc` / `.docx` / `.ppt` / `.pptx` / `.xls` / `.xlsx` to PDF
before feeding the result into the multimodal LLM. Without LibreOffice,
office uploads return HTTP 415 with a clear error message; PDF / image
/ audio / HTML / email parsing is unaffected.
Install on the host before serving office documents:
```bash
brew install --cask libreoffice # macOS
sudo apt-get install -y libreoffice # Debian / Ubuntu
```
For a step-by-step walkthrough (add a conversation → flush → search →
read the markdown), see [QUICKSTART.md](QUICKSTART.md).
### Develop locally
```bash
git clone <repo>
cd everos
uv sync # creates ./.venv and installs deps
source .venv/bin/activate # — or skip activation and prefix every command with `uv run`
everos init # fill in EVEROS_LLM__API_KEY in the generated .env
everos --help
make test
```
<br>
## Storage layout
```
~/.everos/
├── default_app/ # app_id ("default" → "default_app" on disk)
│ └── default_project/ # project_id ("default" → "default_project")
│ ├── users/<user_id>/
│ │ ├── user.md # profile
│ │ ├── episodes/ # daily-log episodes (visible)
│ │ ├── .atomic_facts/ # nested facts (dotfile-hidden)
│ │ └── .foresights/ # predictive memory (dotfile-hidden)
│ └── agents/<agent_id>/
│ ├── agent.md
│ ├── .cases/ # one task case per entry
│ └── skills/ # named procedural memories
├── .index/ # derived indexes (rebuildable from md)
│ ├── sqlite/system.db # state + queue + audit
│ └── lancedb/*.lance/ # vector + BM25 + scalar
└── .tmp/ # transient working files
```
Open any `<app>/<project>/users/<user_id>/` folder in Obsidian — your
agent's brain is just files. The dotfile directories (`.atomic_facts/`,
`.foresights/`, `.cases/`) stay hidden by default so the visible folder
is the user-facing memory surface, while extracted derivatives sit
quietly alongside.
<br>
## Features
- **Hybrid retrieval**: BM25 + vector (HNSW/IVF-PQ) + scalar filter, single-query in LanceDB
- **Cascade index sync**: edit a `.md` → file watcher → entry-level diff → LanceDB sync, sub-second
- **Multi-source extraction**: conversations / agent trajectories / file knowledge
- **Dual-track memory**: user-track (Episodes / Profiles) + agent-track (Cases / Skills)
- **Async-first**: full asyncio, single event loop
- **Multi-modal**: text + small image / audio inline; large media via S3/OSS reference
<br>
## Project structure
```
everos/ # repo root
├── src/everos/ # main package (src layout)
│ ├── entrypoints/ # cli + api
│ ├── service/ # use case orchestration
│ ├── memory/ # domain: extract + search + cascade + prompt_slots
│ ├── infra/ # storage: markdown + lancedb + sqlite
│ ├── component/ # cross-cutting: llm / embedding / config / utils
│ ├── core/ # runtime: observability / lifespan / context
│ └── config/ # configuration data + Settings schema
├── tests/ # unit / integration / golden / fixtures
├── docs/ # design docs
└── .claude/ # team-shared rules + skills (auto-loaded by Claude Code)
```
<br>
## Documentation
- [docs/overview.md](docs/overview.md) — Project overview & vision
- [docs/architecture.md](docs/architecture.md) — DDD layered architecture & dependency rules
- [docs/engineering.md](docs/engineering.md) — Engineering & dev-efficiency infrastructure (CI / tooling / Claude Code)
- [CHANGELOG.md](CHANGELOG.md) — Release notes
- [CONTRIBUTING.md](CONTRIBUTING.md) — How to contribute
- [.claude/rules/](.claude/rules/) — Detailed coding conventions (auto-loaded by Claude Code)
<br>
## Use Cases
Use cases show what persistent memory makes possible in real products and workflows. Some examples are packaged in this repository; others point to external demos or integrations you can study and adapt.
<table>
<tr>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/840470d7-a838-4c05-8685-dd797d4e9cdf)](https://evermind.ai/usecase_reunite)
#### Reunite - Find with EverOS
Parents describe what they remember. Children describe what they recall. Reunite uses semantic memory to surface the connections.
[Learn more](https://evermind.ai/usecase_reunite)
</td>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/7282b38b-56bf-4356-aa7b-06a845e7683d)](https://github.com/tt-a1i/hive)
#### Hive Orchestrator
Browser-native hive-mind for CLI coding agents — Claude Code, Codex, Gemini, and OpenCode collaborate as real PTY processes via a team protocol.
[Code](https://github.com/tt-a1i/hive)
</td>
</tr>
<tr>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/867d9329-ce9a-496f-ab1e-15c77974e5fa)](https://github.com/tt-a1i/evermemos-mcp)
#### AI Coding Assistants with EverOS
Universal long-term memory layer for AI coding assistants, powered by EverOS.
[Code](https://github.com/tt-a1i/evermemos-mcp)
</td>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/a4f0fd86-1c81-4445-bebc-e51eb5e33b30)](https://github.com/yuansui123/AI-Data-Technician-EverMemOS)
#### AI Data Techician
An agentic AI system that learns from scientist interaction to inspect, analyze, and classify high-dimensional time series data — with persistent memory that improves across sessions.
[Code](https://github.com/yuansui123/AI-Data-Technician-EverMemOS)
</td>
</tr>
<tr>
<td width="50%" valign="top">
![banner-gif](https://github.com/user-attachments/assets/650b901b-c9ba-4001-bac7-626b009df830)
#### Rokid AI Assistant with EverOS
Connect to EverOS within Rokid Glasses enabling long-term memory for all of your smart activities.
Coming soon
</td>
<td width="50%" valign="top">
![banner-gif](https://github.com/user-attachments/assets/85b338b2-e48e-4a65-9f30-0bc6998df872)
#### Creative Assistant with Memory
Creative assistant with long-term memory, never forget your crativites anymore.
Coming soon
</td>
</tr>
<tr>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/f30617a1-adc0-4271-bc0e-c3a0b28cb903)](https://github.com/xunyud/Earth-Online)
#### Earth Online Memory Game
Earth Online is a memory-aware productivity game that turns everyday planning into a living quest log.
[Code](https://github.com/xunyud/Earth-Online)
</td>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/57d8cda7-35a5-4561-b794-5520dffc917b)](https://github.com/golutra/golutra)
#### Multi-Agent Orchestration Platform
Golutra presents a multi-agent workforce for engineering teams, extending the IDE model from a single assistant to coordinated agents.
[Code](https://github.com/golutra/golutra)
</td>
</tr>
<tr>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/75f19db5-30f6-4eed-9b1e-c9c6a0e6b7de)](https://github.com/Yangtze-Seventh/taste-verse)
#### Your Personal Tasting Universe
Record, visualize, and explore your tasting journey through an immersive 3D star map.
[Code](https://github.com/Yangtze-Seventh/taste-verse)
</td>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/93ac2a68-4f18-4fcb-8d87-80aeb00a9d7c)](https://github.com/kellyvv/OpenHer)
#### EverOS Open Her
Build AI that feels. Open-source persona engine — personality emerges from neural drives, not prompts. Inspired by Her.
[Code](https://github.com/kellyvv/OpenHer)
</td>
</tr>
<tr>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/550071c1-dc39-4964-9f67-ffdfad792345)](https://chromewebstore.google.com/detail/ruminer-browser-agent/lbccjohfpdpimbhpckljimgolndfmfif)
#### Browser Agent for Personal Memory
Ruminer brings persistent memory to a browser agent so it can carry personal context across web tasks.
[Plugin](https://chromewebstore.google.com/detail/ruminer-browser-agent/lbccjohfpdpimbhpckljimgolndfmfif)
</td>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/c258a6c4-fe70-497a-98d1-3dade4a932f6)](https://github.com/nanxingw/EverMem)
#### EverMem Sync with EverOS
One command to connect any AI coding CLI to EverMemOS long-term memory.
[Code](https://github.com/nanxingw/EverMem)
</td>
</tr>
<tr>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/39274473-ceb3-48fb-a031-e22230decbe2)](https://github.com/mco-org/mco)
#### MCO - Orchestrate AI Coding Agents
MCO equips your primary agent with an agent team that can work together to solve complex tasks.
[Code](https://github.com/mco-org/mco)
</td>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/314c9126-8e08-4688-bbbb-8555ad58cf67)](https://github.com/onenewborn/StudyBuddy-public)
#### Study Buddy with Self-Evolving Memory
Study proactively with an agent that has self-evolving memory.
[Code](https://github.com/onenewborn/StudyBuddy-public)
</td>
</tr>
<tr>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/21da76aa-9a8b-48e0-9134-42429d7390e7)](https://github.com/TonyLiangDesign/MemoCare)
#### Alzheimer’s Memory Assistant
Empowering individuals with advanced memory support and daily assistance.
[Code](https://github.com/TonyLiangDesign/MemoCare)
</td>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/e2428df3-ea11-4e88-8f9c-dad437dd8998)](https://github.com/AlexL1024/NeuralConnect)
#### Memory-Driven Multi-Agent NPC Experience
An iOS sci-fi mystery game where players explore and uncover the truth.
[Code](https://github.com/AlexL1024/NeuralConnect)
</td>
</tr>
<tr>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/e6eaf308-a874-483f-8874-6934bf95a78f)](https://github.com/elontusk5219-prog/Mobi)
#### Mobi Companion
An iOS app where users create, nurture, and live with a personalized AI companion called Mobi.
[Code](https://github.com/elontusk5219-prog/Mobi)
</td>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/9aabcaa9-f97a-49d2-9109-0b5bb696ed41)](https://github.com/JaMesLiMers/EvermemCompetition-Spiro)
#### AI Wearable with Memory
A context-native AI wearable that listens to everyday life and converts conversations into memory.
[Code](https://github.com/JaMesLiMers/EvermemCompetition-Spiro)
</td>
</tr>
<tr>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/df9677ec-386f-4c56-a428-08bca25c54dc)](https://github.com/EverMind-AI/EverOS/tree/0f49826ba0f9a94e1974c97614a46a68e0a08b52/evermemos-openclaw-plugin)
#### OpenClaw Agent Memory
A 24/7 agent workflow with continuous learning memory across sessions.
[Plugin](https://github.com/EverMind-AI/EverOS/tree/0f49826ba0f9a94e1974c97614a46a68e0a08b52/evermemos-openclaw-plugin)
</td>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/3a2357a1-c0c3-464a-8979-0d1cdfc9b0d4)](https://github.com/TEN-framework/ten-framework/tree/04cb80601374fa9e35b4e544b2dbd23286ca7763/ai_agents/agents/examples/voice-assistant-with-EverMemOS)
#### Live2D Character with Memory
Add long-term memory to a real-time Live2D character, powered by [TEN Framework](https://github.com/TEN-framework/ten-framework).
[Code](https://github.com/TEN-framework/ten-framework/tree/04cb80601374fa9e35b4e544b2dbd23286ca7763/ai_agents/agents/examples/voice-assistant-with-EverMemOS)
</td>
</tr>
<tr>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/c36bdc04-97d3-4fe9-97d9-4b93b475595a)](https://screenshot-analysis-vercel.vercel.app/)
#### Computer-Use with Memory
Run screenshot-based analysis with computer-use and store the results in memory.
[Live Demo](https://screenshot-analysis-vercel.vercel.app/)
</td>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/54a7cf8f-62c4-4fbc-9d50-b214d034e051)](use-cases/game-of-throne-demo)
#### Game of Thrones Memories
A demonstration of AI memory infrastructure through an interactive Q&A experience with *A Game of Thrones*.
[Code](use-cases/game-of-throne-demo)
</td>
</tr>
<tr>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/af37c1f6-7ba5-430c-b99d-2a7e7eac618f)](use-cases/claude-code-plugin)
#### Claude Code Plugin
Persistent memory for Claude Code. Automatically saves and recalls context from past coding sessions.
[Code](use-cases/claude-code-plugin)
</td>
<td width="50%" valign="top">
[![banner-gif](https://github.com/user-attachments/assets/d521d28c-0ccd-44ff-aecc-828245e2f973)](https://main.d2j21qxnymu6wl.amplifyapp.com/graph.html)
#### Memory Graph Visualization
Explore stored entities and relationships in a graph interface. Frontend demo; backend integration is in progress.
[Live Demo](https://main.d2j21qxnymu6wl.amplifyapp.com/graph.html)
</td>
</tr>
</table>
<br>
<div align="right">
[![](https://img.shields.io/badge/-Back_to_top-gray?style=flat-square)](#readme-top)
</div>
## Stay Tuned
Star the repo or join the community links above to follow new architecture methods, benchmark releases, and memory-enabled use cases.
![star us gif](https://github.com/user-attachments/assets/0c512570-945a-483a-9f47-8e067bd34484)
<br>
<div align="right">
[![](https://img.shields.io/badge/-Back_to_top-gray?style=flat-square)](#readme-top)
</div>
## Contributing
Contributions are welcome across the whole repository: architecture methods, benchmark coverage, use-case examples, documentation, and bug fixes. Browse [Issues](https://github.com/EverMind-AI/EverOS/issues) to find a good entry point, then open a PR when you are ready.
<br>
> [!TIP]
>
> **Welcome all kinds of contributions** 🎉
>
> Help make EverOS better. Code, documentation, benchmark reports, use-case write-ups, and integration examples are all valuable. Share your projects on social media to inspire others.
>
> Connect with one of the EverOS maintainers [@elliotchen200](https://x.com/elliotchen200) on 𝕏 or [@cyfyifanchen](https://github.com/cyfyifanchen) on GitHub for project updates, discussions, and collaboration opportunities.
![divider](https://github.com/user-attachments/assets/2e2bbcc6-e6d8-4227-83c6-0620fc96f761#gh-light-mode-only)
![divider](https://github.com/user-attachments/assets/d57fad08-4f49-4a1c-bdfc-f659a5d86150#gh-dark-mode-only)
### Code Contributors
[![EverOS Contributors](https://contrib.rocks/image?repo=EverMind-AI/EverOS)](https://github.com/EverMind-AI/EverOS/graphs/contributors)
![divider](https://github.com/user-attachments/assets/2e2bbcc6-e6d8-4227-83c6-0620fc96f761#gh-light-mode-only)
![divider](https://github.com/user-attachments/assets/d57fad08-4f49-4a1c-bdfc-f659a5d86150#gh-dark-mode-only)
### License
[Apache License 2.0](LICENSE) — see [NOTICE](NOTICE) for third-party attributions.
### Citation
If you use EverOS in research, see [CITATION.md](CITATION.md).
<br>
<div align="right">
[![](https://img.shields.io/badge/-Back_to_top-gray?style=flat-square)](#readme-top)
</div>

46
SECURITY.md Normal file
View File

@ -0,0 +1,46 @@
# Security Policy
## Supported Versions
EverOS is in active alpha development. Security fixes are applied to the latest
release line only.
| Version | Supported |
|---------|-----------|
| 0.1.x | ✅ |
| < 0.1 | ❌ |
## Reporting a Vulnerability
**Please do not report security vulnerabilities through public GitHub issues,
discussions, or pull requests.**
Instead, email **evermind@shanda.com** with:
- A description of the vulnerability and its potential impact
- Steps to reproduce, or a proof-of-concept
- The affected version / commit
- Any suggested mitigation, if you have one
We will acknowledge your report within **5 business days**, keep you informed of
progress, and aim to ship a fix or mitigation before any public disclosure.
Reporters are credited in the release notes unless you prefer to remain
anonymous.
## Scope & Threat Model
EverOS runs as a **local-first service** for single users or small teams
(Markdown + SQLite + LanceDB on the local filesystem). Please keep the
following in mind:
- Exposing the HTTP API (`everos server`) to an untrusted network is **outside
the supported threat model** — it assumes a trusted local caller. The server
binds to `127.0.0.1` by default (env `EVEROS_API__HOST`) so a fresh install
is loopback-only. Only set the bind to `0.0.0.0` (or any routable interface)
after you have placed your own gateway / auth layer in front;
`everos server start` will log a warning when you bind to `0.0.0.0`.
- Secrets (LLM / embedding API keys) live in your local `.env`; protect that
file as you would any credential. EverOS never transmits them anywhere except
the providers you configure.
- Memory content is stored as plaintext `.md` files; apply OS-level file
permissions or disk encryption if your data is sensitive.

58
config.example.toml Normal file
View File

@ -0,0 +1,58 @@
# Example user-level config — copy to ~/.everos/config.toml.
#
# cp config.example.toml ~/.everos/config.toml
# $EDITOR ~/.everos/config.toml # fill in api_key / base_url
#
# Override priority (low → high):
#
# src/everos/config/default.toml
# < ~/.everos/config.toml (THIS FILE)
# < .env in the working directory
# < EVEROS_<SECTION>__<KEY> process envs
# < programmatic init args
#
# The path is configurable: set EVEROS_CONFIG_FILE=/path/to/your.toml to
# point at any location. Missing file = silently skipped (no error).
#
# Comment out anything you do not need — only the fields you set here
# override the shipped defaults; everything else falls back to
# src/everos/config/default.toml.
# ── LLM ───────────────────────────────────────────────
# OpenAI-protocol chat-completions endpoint used by the algo extractors.
[llm]
model = "gpt-4o-mini"
api_key = "sk-..."
base_url = "https://api.openai.com/v1"
# ── Embedding ─────────────────────────────────────────
[embedding]
model = "Qwen/Qwen3-Embedding-4B"
api_key = "..."
base_url = "https://api.deepinfra.com/v1/openai"
# Runtime knobs (defaults shown):
timeout_seconds = 30.0
max_retries = 3
batch_size = 10
max_concurrent = 5
# ── Rerank ────────────────────────────────────────────
[rerank]
model = "Qwen/Qwen3-Reranker-4B"
api_key = "..."
base_url = "https://api.deepinfra.com/v1/inference"
timeout_seconds = 30.0
max_retries = 3
batch_size = 10
max_concurrent = 5
# ── Other tunables you may want to override here ──────
# [memory]
# root = "~/.everos"
# timezone = "Asia/Shanghai"
#
# [sqlite]
# busy_timeout_ms = 10000
#
# [lancedb]
# read_consistency_seconds = 5.0

982
data/solo_chat_en.json Normal file
View File

@ -0,0 +1,982 @@
{
"version": "1.0.0",
"session_meta": {
"scene": "solo",
"scene_desc": {
"description": "Project discussion group chat"
},
"name": "User health consultation dialogue",
"description": "Conversation records between users and AI assistants on topics such as tourism in Beijing, health management, and sports rehabilitation.",
"group_id": "chat_user_001_assistant",
"created_at": "2025-06-26T00:00:00Z",
"default_timezone": "UTC",
"user_details": {
"user_001": {
"full_name": "User",
"role": "user",
"extra": {
"height": 170,
"weight": 86,
"bmi": 29.8,
"waist_circumference": 104,
"origin": "Sichuan",
"preferences": {
"food": "hot pot",
"activities": "Group activities"
}
}
},
"robot_001": {
"full_name": "AI Assistant",
"role": "assistant",
"extra": {
"type": "assistant"
}
}
},
"tags": [
"Health Consultation",
"Travel Planning",
"Sports Rehabilitation",
"Dietary Recommendations"
]
},
"conversation_list": [
{
"message_id": "msg_001",
"create_time": "2025-06-26T00:00:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I will travel to Beijing next week. Could you give me some suggestions?",
"refer_list": []
},
{
"message_id": "msg_002",
"create_time": "2025-06-26T00:00:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Of course! Beijing attractions: Forbidden City, Great Wall, Summer Palace, etc. (book in advance). Hutong experiences: Nanluoguxiang, Shichahai, Yangmeizhu Xiejie. Food: Peking Duck, Zhajiangmian, Luzhu Huoshao, Douzhi'er with Jiaoqiu. Transportation: Subway is well-developed, download \"Yitongxing\" or \"Beijing Yikatong\" app, avoid rush hours. Weather: Late October in autumn, 10-20°C during daytime, cooler mornings and evenings, bring a jacket. Remember to bring ID, popular attractions require advance booking.",
"refer_list": []
},
{
"message_id": "msg_003",
"create_time": "2025-06-26T00:03:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I am very interested in the variety of foods you mentioned. Please recommend some delicious dishes based on my preferences. I am from Sichuan, and my favorite food is hot pot.",
"refer_list": []
},
{
"message_id": "msg_004",
"create_time": "2025-06-26T00:03:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Based on your Sichuan flavor preferences, recommendations: 1. Sichuan Hot Pot: Xiaolongkan, Da Long Yi, Shu Daxia, Hou Tang Old Hot Pot (get a number in advance). 2. Fusion Style: Old Beijing hotpot with mala dipping sauce, or Mala Tang/Mao Cai. 3. Sichuan Restaurants: Chen Mapo Tofu, Rong Li Ji, Ba Guo Bu Yi, Meizhou Dongpo. 4. Food Districts: Guijie (late-night snacks), Shuangjing/Wangjing (Sichuan-Chongqing flavors), Xidan Joy City/Sanlitun (new-style Sichuan cuisine).",
"refer_list": []
},
{
"message_id": "msg_005",
"create_time": "2025-06-26T00:04:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "Actually, I'm just telling you my preference as a reference, but I need you to recommend some Beijing specialty foods based on my taste.",
"refer_list": []
},
{
"message_id": "msg_006",
"create_time": "2025-06-26T00:04:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Based on your strong-flavor preferences, recommended Beijing specialties: 1. Baodu: Dipped in sesame paste, chili oil, minced garlic, crispy and chewy texture. Recommended: Menkuang Hutong Century-old Lu Zhu, Baodu Feng. 2. Luzhu Huoshao: Rich broth with offal, can add chili oil. Recommended: Xiao Chang Chen, Menkuang Hutong Luzhu. 3. Zhajiangmian: Can add chili oil, pair with mustard-stuffed vegetables. Recommended: Haiwanju, Fangzhuanchang No. 69. 4. Peking Duck: Salt-and-pepper duck frame is spicy and crispy. Recommended: Sijiminfu, Da Dong. 5. Mending Roubing: Pair with spicy lamb offal soup. Recommended: Baorui Mending Roubing Shop. 6. Douzhi'er with Jiaoqiu: Pair with spicy salted vegetable shreds. Recommended: Yin San Douzhi, Huguo Si Snacks. 7. Chao Gan: Can add garlic and chili oil. Recommended: Yao Ji Chao Gan, Qingfeng Baozi Shop.",
"refer_list": []
},
{
"message_id": "msg_007",
"create_time": "2025-06-26T00:16:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I will go with five or six of my friends. I really enjoy activities that we can all do together. Please recommend some entertainment options for me.",
"refer_list": []
},
{
"message_id": "msg_008",
"create_time": "2025-06-26T00:16:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Entertainment suitable for groups of 6-8 people: 1. Hutong Immersive Script Murder/City Exploration: \"Beiping Stories\" live-action script murder, Mystery Hutong puzzle game. 2. Shichahai/Hutong Cycling: Rent shared bikes, set check-in tasks, great for photos. 3. Escape Room/Immersive Theater: \"Last Bus\" \"Yayoi\" escape rooms, \"Heart Stealing Dinner\" interactive drama. 4. Beijing Cultural Experiences: Deyun Society crosstalk, Lao She Teahouse experience, DIY candied fruits. 5. Nightlife: Guijie late-night snacks + board game bars, Livehouse concerts, KTV singing. 6. Outdoor Activities: Olympic Forest Park BBQ + frisbee, Liangma River night cruise. Tips: Popular activities need 1-3 day advance booking, budget around 100-200 RMB per person.",
"refer_list": []
},
{
"message_id": "msg_009",
"create_time": "2025-06-26T00:17:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "We will try immersive script murder games in the hutongs and cycling around Shichahai/hutongs. Are there any precautions we should be aware of for these activities?",
"refer_list": []
},
{
"message_id": "msg_010",
"create_time": "2025-06-26T00:17:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Precautions: I. Hutong Script Murder: 1. Book 2-3 days in advance, 6-8 people can book private session (800-1500 RMB/session). 2. Wear flat shoes, bring jacket. 3. Allow 4 hours (including costume changes, debriefing). 4. Bring power bank, can take photos after the game ends. 5. Beginners should choose fun or mechanism-based scripts. II. Shichahai/Hutong Cycling: 1. Rent shared bikes or tandem/triple bikes (30-50 RMB/hour), pay attention to safety. 2. Best times: 9-11 AM or 3-5 PM. 3. Recommended route: Shichahai Station → Yinding Bridge → Yandai Xiejie → North shore of Houhai → North entrance of Nanluoguxiang → Mao'er Hutong → Gulou East Street (2-3 hours). 4. Designate lead riders and sweepers, use WeChat location sharing. 5. Watch for no-parking zones, restrooms near Yinding Bridge and Gulou. 6. Can rent pedal boats, watch sunset in the evening. General: Bring ID, pack snacks and water, check weather.",
"refer_list": []
},
{
"message_id": "msg_011",
"create_time": "2025-07-05T07:00:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I'm back from Beijing. The travel experience was really hard to describe in words, even somewhat terrible. I was full of anticipation to climb the Great Wall, but when I got there it was absolutely packed with people. Forget about enjoying the scenery - I was basically pushed along by the crowd. It was hard to even stop and catch my breath, and I couldn't take a single decent photo - just heads everywhere. What drove me crazy the most was that I didn't book Forbidden City tickets in advance, so when I arrived at the gate I was told tickets had long sold out. I could only stand there staring, watching others go in, feeling really regretful. And Shichahai was way too commercialized - vendors hawking everywhere, nothing like the peaceful old Beijing atmosphere I had imagined. After this trip, not only is my body exhausted, but my heart is even more tired. It completely failed to achieve the purpose of relaxation.",
"refer_list": []
},
{
"message_id": "msg_012",
"create_time": "2025-07-05T07:00:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "I can tell you really went through a lot on this trip. Coming with high expectations only to encounter such overwhelming crowds - anyone would feel extremely disappointed. The Great Wall during peak season is indeed a challenge for any \"hero\" - being swept along by the crowd completely ruins the mood for sightseeing. As for Forbidden City tickets, they're truly hard to get nowadays. Many tourists who don't plan ahead fall into this trap - that feeling of being so close yet unable to enter is really frustrating. The commercialization of Shichahai is also something many seasoned travelers complain about. Although this trip left many regrets, it's still a memorable experience (albeit not a pleasant one). Don't dwell on these unhappy things for now - treat them as lessons for traveling during off-peak times next time.",
"refer_list": []
},
{
"message_id": "msg_013",
"create_time": "2025-07-05T07:04:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "These past few days, to check in at all the attractions, I walked over 20,000 steps every day. Meals were rushed - just a few quick bites. Now my legs feel like they're about to fall off, and my body feels completely drained. I still have to work tomorrow, and I'm feeling very anxious right now. Please give me some suggestions for quick recovery.",
"refer_list": []
},
{
"message_id": "msg_014",
"create_time": "2025-07-05T07:04:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "For your current physical and mental exhaustion and work anxiety, here's a quick recovery plan: Tonight: 1. Be in bed by 10:30 PM, get 7-8 hours of sleep. Stay away from phone before sleep, soak feet in warm water, try 4-7-8 breathing method. 2. Light dinner: millet porridge + stir-fried greens + steamed fish/tofu, drink light honey water. 3. Apply warm compress to calves/lower back for 10 minutes, or do wall stretches. Tomorrow morning: 1. Breakfast: whole wheat bread + boiled egg + nuts + warm soy milk. 2. Drink 300ml warm water after waking, splash cold water on wrists/back of neck, listen to upbeat music. 3. Rest eyes during commute or recall happy travel moments. Next 2-3 days: Eat light, plenty of vegetables, whole grains, quality protein; light exercise 15 minutes; accept lower productivity on day one, prioritize urgent tasks.",
"refer_list": []
},
{
"message_id": "msg_015",
"create_time": "2025-07-07T06:00:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I feel like I've gained some weight. Are there any self-assessment methods?",
"refer_list": []
},
{
"message_id": "msg_016",
"create_time": "2025-07-07T06:00:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Self-assessment methods: 1. BMI Calculation: weight(kg) ÷ height(m)². Standards: <18.5 underweight, 18.5-23.9 normal, 24.0-27.9 overweight, ≥28.0 obese. Note: Not suitable for those with very high muscle mass or elderly. 2. Waist Circumference: Measure horizontally at midpoint between lower rib margin and pelvic crest, read at end of exhalation. Standards: Men ≥90cm, Women ≥85cm indicates abdominal obesity. 3. Waist-to-Hip Ratio: Waist ÷ Hip circumference. Standards: Men <0.90, Women <0.85. 4. Body Signals: Clothes getting tighter, fatigue and shortness of breath, obvious abdominal protrusion after prolonged sitting, dark skin folds. Suggestion: If overweight or obese, combine dietary adjustments with exercise, consult a doctor if necessary.",
"refer_list": []
},
{
"message_id": "msg_017",
"create_time": "2025-07-07T06:03:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I just measured my waist circumference, and it is 104 cm. Can you give me some advice?",
"refer_list": []
},
{
"message_id": "msg_018",
"create_time": "2025-07-07T06:03:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Health Assessment: BMI ≈29.8 (obese), waist circumference 104cm (abdominal obesity), increased risk of type 2 diabetes, hypertension, hyperlipidemia, cardiovascular disease, fatty liver. Phase 1 Goals (1-3 months): Reduce waist ≥5cm, lose 4-5kg. Action Plan: Diet: Control refined carbs, replace with brown rice/oats/sweet potato/whole wheat bread, staple food one fist-size per meal; increase protein (eggs/fish/shrimp/chicken breast/tofu), palm-sized portion; eat plenty of high-fiber vegetables, half the plate; quit sugary drinks, alcohol, fried foods. Exercise: Aerobic 5 days/week, 30-45 minutes each; strength training 2 times/week, 15 minutes each. Monitoring: Measure waist and weight weekly at fixed time, record feelings. Warning Signs: Fasting blood glucose ≥6.1, blood pressure ≥140/90, abnormal liver function - seek medical attention. Mindset: 2-3kg/month is safe pace, every 1cm waist reduction lowers diabetes risk by 10%.",
"refer_list": []
},
{
"message_id": "msg_019",
"create_time": "2025-07-13T08:00:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I ate a lot of greasy food for lunch today and now I feel a bit tired, have lost my appetite, and feel bloated. What should I do?",
"refer_list": []
},
{
"message_id": "msg_020",
"create_time": "2025-07-13T08:00:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Causes: Excessive fat intake, delayed gastric emptying, blood sugar fluctuations. Short-term relief: 1. Drink warm water or light tea. 2. Take a slow walk for 10-15 minutes about 30 minutes after eating. 3. Avoid eating more greasy or sweet foods, choose light foods tonight (porridge/vegetable soup/steamed eggs/small amount of fruit). 4. Apply warm compress to abdomen. Tomorrow care: Breakfast choose porridge/oatmeal/whole wheat bread + protein; lunch and dinner focus on steaming, boiling, stewing, eat more vegetables; supplement probiotics. If discomfort persists: Eat smaller, more frequent meals, avoid raw, cold, and spicy foods, can take digestive aid tablets; if symptoms last more than two days or accompanied by abdominal pain/vomiting, seek medical attention.",
"refer_list": []
},
{
"message_id": "msg_021",
"create_time": "2025-07-13T08:02:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "Okay, I will take your advice, drink some warm water tonight, and move around a bit. Are there any other precautions I should be aware of?",
"refer_list": []
},
{
"message_id": "msg_022",
"create_time": "2025-07-13T08:02:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Tonight's Precautions: Things you can do: 1. Drink warm water in multiple small sips. 2. Take a slow walk or light stretch 30-60 minutes after eating, then keep upper body slightly elevated. 3. Keep abdomen warm, apply warm compress for 10-15 minutes. 4. Gently massage abdomen clockwise. Things to avoid: 1. Fried foods, desserts, barbecue, etc. 2. Cold drinks, alcohol. 3. Lying flat within 2 hours after eating. 4. Tight pants or waist-binding clothing.",
"refer_list": []
},
{
"message_id": "msg_023",
"create_time": "2025-07-14T04:00:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I followed your advice, but I still felt a bit unwell after waking up today. So I went to the hospital for a check-up, and the results showed that I am overweight and have moderate fatty liver disease. Please provide me with some health advice.",
"refer_list": []
},
{
"message_id": "msg_024",
"create_time": "2025-07-14T04:00:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Moderate fatty liver can be significantly improved or even reversed through lifestyle changes. Causes: Excessive calorie intake, sedentary lifestyle, weight gain, insulin resistance. Losing 5-10% of body weight can significantly improve the condition. Dietary Recommendations: 1. Control oil, control sugar, increase protein: Daily cooking oil ≤25g, limit sugar, protein 1-1.2g/kg body weight. 2. Mediterranean-style diet: Breakfast oatmeal + eggs + tomatoes/fruit; lunch brown rice/whole wheat rice + steamed fish/chicken breast + vegetables; dinner vegetable soup + tofu/eggs + small amount of whole grains; low-sugar fruits 1-2 servings. 3. Reduce hidden fats: Eat fewer nuts, sauces, hot pot bases; when eating out, choose steamed/cold dishes/low-oil options. Exercise: At least 150 minutes of moderate intensity per week, brisk walking/jogging 30-45 minutes × 5 days, strength training × 2 times, walk for 10 minutes after meals. Lifestyle habits: Sleep before 11 PM, quit alcohol, manage stress. Goals: Month 1 lose 2-3kg, Month 3 lose 5-7%, Month 6 reach healthy BMI or mild fatty liver. Recommend recheck every 3-6 months.",
"refer_list": []
},
{
"message_id": "msg_025",
"create_time": "2025-07-14T04:02:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "Why do I have moderate fatty liver disease?",
"refer_list": []
},
{
"message_id": "msg_026",
"create_time": "2025-07-14T04:02:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Causes of moderate fatty liver: 1. Excessive calorie intake: Greasy foods and fruit juices contain high fat and sugar, excess energy converts to fat stored in liver. 2. Overweight/Obesity: Visceral fat is transported to liver through bloodstream. 3. Blood sugar and insulin metabolism issues: High-sugar, high-fat diet causes insulin resistance, liver stores fat more easily. 4. Lack of exercise: Low basal metabolism, reduced liver ability to utilize fat. 5. Irregular lifestyle and sleep: Staying up late and stress affect liver metabolism. 6. Others: Genetic predisposition, medications, or diseases. Summary: Mainly due to excessive calorie intake + fat accumulation + metabolic imbalance. Can be significantly improved or even reversed through diet control, exercise, and regular routines.",
"refer_list": []
},
{
"message_id": "msg_027",
"create_time": "2025-07-14T04:04:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I often eat hot pot and have an ice cream every day. Is this lifestyle habit not very good?",
"refer_list": []
},
{
"message_id": "msg_028",
"create_time": "2025-07-14T04:04:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "This habit is not friendly to fatty liver. Hot Pot: High in oil and salt, easy to overeat. Suggestions: Choose clear broth/tomato soup base, less fatty meat, more vegetables/tofu/fish/shrimp, eat slowly. Ice Cream: High in sugar and fat, eating daily promotes liver fat accumulation. Suggestions: Limit to ≤1-2 times per week, small portions, or choose low-sugar yogurt/frozen fruit puree/sugar-free popsicles. Summary: High oil, high sugar, high calories are high-risk factors for fatty liver. Improvements: Switch hot pot to clear broth with more vegetables, reduce ice cream frequency, combine with regular diet and aerobic exercise.",
"refer_list": []
},
{
"message_id": "msg_029",
"create_time": "2025-07-14T04:10:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I would like you to give me some dietary advice.",
"refer_list": []
},
{
"message_id": "msg_030",
"create_time": "2025-07-14T04:10:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Dietary Principles: 1. Control oil: Daily ≤25g, avoid fried/braised/hot pot. 2. Control sugar: Reject sugary drinks/milk tea/desserts, staples mainly whole grains. 3. Quality protein: Chicken breast/fish/tofu/eggs/milk. 4. Plenty of vegetables: Daily ≥500g, half dark green. 5. Small frequent meals: Three regular meals, avoid binge eating. Dietary Structure: Carbs 40-45% (brown rice/whole wheat noodles/oats/corn/sweet potato), protein 25-30%, fats 20-25% (olive oil/flaxseed oil/small amount of nuts), vegetables and fruits ≥500g. Daily Suggestions: Breakfast: Oatmeal porridge + boiled egg + tomato/cucumber, or whole wheat bread + unsweetened soy milk + fruit. Lunch: Brown rice/whole wheat rice + steamed fish/chicken breast/tofu + vegetables. Dinner: Vegetable soup + tofu + whole grains, or steamed chicken breast + cold broccoli salad. Snacks: Low-fat yogurt/small apple/nuts (within 10g), no eating after 9 PM. Others: Drink 1500-2000ml water daily, moderate black coffee (no sugar/creamer), completely quit alcohol, salt ≤5g/day. One light-eating day per week, weigh once weekly.",
"refer_list": []
},
{
"message_id": "msg_031",
"create_time": "2025-07-14T04:12:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "Give me some more advice on exercise.",
"refer_list": []
},
{
"message_id": "msg_032",
"create_time": "2025-07-14T04:12:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Exercise Principles: Gradual progression, combine aerobic and strength training, emphasize light activity after meals, at least 5 days/week, at least 30 minutes/day. Aerobic Exercise: At least 150 minutes of moderate intensity weekly. Beginner (first 2 weeks): Brisk walking/light yoga/stretching, 30-40 minutes. Progression (weeks 3-6): Brisk walking + slow jogging/swimming/cycling, 40-60 minutes. Maintenance: Slow jogging + interval training, 5 times/week. Intensity gauge: Can talk but can't sing. Strength Training: 2-3 times/week, bodyweight squats 10-15 reps × 3 sets, push-ups 10 reps × 3 sets, plank 20-30 seconds × 3 sets. Post-meal Activity: Walk for 10-15 minutes 20-30 minutes after each meal. Daily Schedule: Morning brisk walk/stretching 10-15 minutes, afternoon light walk 10 minutes, evening brisk walk/slow jog 40-60 minutes, strength training 2 days/week. Safety: Warm up and stretch before and after exercise, wait at least 30 minutes after meals before exercising, avoid high-impact activities, ensure sleep ≥7 hours. Goals: 1 month - mild decrease in ALT/AST, 3 months - lose 5-7% weight, 6 months - establish regular habits.",
"refer_list": []
},
{
"message_id": "msg_033",
"create_time": "2025-08-08T01:30:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I have gradually adjusted my lifestyle habits according to your advice, and over the past month, I have noticeably felt that my mental state has improved. After measuring my weight, I found that I have lost 3kg compared to last month. I feel healthier. However, I haven't eaten my favorite hot pot for about a month, which makes me feel a bit uncomfortable. Do you have any good suggestions?",
"refer_list": []
},
{
"message_id": "msg_034",
"create_time": "2025-08-08T01:30:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "You've done very well! About hot pot: 1. Control frequency: Once every two weeks or once a month. For portions, eat more vegetables/tofu/mushrooms, less fatty meat and processed meats. 2. Healthy approach: Choose bone broth/tomato soup/mushroom soup base, use more chicken breast/fish slices/shrimp, dipping sauce use soy sauce/minced garlic/cilantro, reduce oil dip/sesame paste. 3. Pairing: Start with vegetables and mushrooms, control staple food portions. 4. After meal: Avoid high-calorie snacks that day, moderate exercise. 5. Psychological: Occasional indulgence doesn't affect overall health, can serve as a reward. Summary: Use strategies of small amounts, healthy ingredients, light broth, and reasonable frequency.",
"refer_list": []
},
{
"message_id": "msg_035",
"create_time": "2025-08-08T01:31:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "It seems I can still continue eating hot pot, right? I just need to eat more healthily; I used to eat it too greasy.",
"refer_list": []
},
{
"message_id": "msg_036",
"create_time": "2025-08-08T01:31:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "You can continue eating hot pot, just adjust the approach: 1. Light broth: Tomato soup/mushroom soup/clear bone broth. 2. Eat more vegetables and protein: Mushrooms/tofu/leafy greens/fish slices/chicken breast. 3. Less oil and sauce: Use soy sauce/minced garlic/cilantro, avoid large amounts of sesame paste/oil dip. 4. Control portions of staples and meat. 5. Eat slowly, start with vegetables. In short, eat smarter, not stop eating entirely.",
"refer_list": []
},
{
"message_id": "msg_037",
"create_time": "2025-08-12T01:17:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I have been jogging consistently for over a month, but I'm about to give up because I find it very boring. I want to do some sports with my friends. Do you have any recommendations?",
"refer_list": []
},
{
"message_id": "msg_038",
"create_time": "2025-08-12T01:17:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Exercising with friends can boost fun and motivation. Recommendations: 1. Ball Sports: Badminton (doubles)/basketball (half-court 3v3)/table tennis/football, high fat-burning efficiency, 2-3 times/week, 60-90 minutes each. 2. Cycling/Hiking: Weekend greenway cycling or light suburban hiking, adjustable intensity, chat while moving. 3. Group Fitness Classes: Body Combat/Zumba/Spinning/TRX, high-efficiency fat burning, sign up with 1-2 friends. 4. Light Combo: After-dinner brisk walk with friends for 40 minutes + stretching. Action suggestions: Send a group message this week to invite friends, prioritize sports you liked as a child, focus on sustainability and enjoyment first.",
"refer_list": []
},
{
"message_id": "msg_039",
"create_time": "2025-08-12T01:20:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "Badminton and basketball are indeed great choices, they make me feel really good. But I've never tried them before, do you have any suggestions?",
"refer_list": []
},
{
"message_id": "msg_040",
"create_time": "2025-08-12T01:20:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Zero experience is completely fine. Badminton (recommended first): Moderate intensity, less knee impact, less pressure in doubles. Gear: Racket 100-200 RMB entry-level, must wear badminton shoes, breathable quick-dry clothes. Basic moves: Clear shots, net drops/net play, blocking smashes. Find beginner sessions or learn with friends, stretch after playing. Basketball: Start with half-court 3v3 or shooting practice, must wear basketball shoes (mid/high-top), tell teammates you're a beginner, initial sessions 30-45 minutes, do wall sits after playing. General Safety: Warm up 5 minutes, avoid consecutive jumps and sudden stops, stop immediately if uncomfortable, hydrate and get protein after exercise. First step: This weekend invite 1-2 friends to rent a badminton court (20-40 RMB/hour), watch a 5-minute tutorial video beforehand, goal is not to play well but to have fun.",
"refer_list": []
},
{
"message_id": "msg_041",
"create_time": "2025-08-12T01:22:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "Your advice reminded me. Please tell me what I should pay attention to when choosing the appropriate sports equipment?",
"refer_list": []
},
{
"message_id": "msg_042",
"create_time": "2025-08-12T01:22:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Badminton Equipment: 1. Badminton Shoes (most important): Must choose specialized shoes, non-black rubber sole, mid/low-cut with anti-twist design, entry-level brands 200-400 RMB (Li-Ning Rush series/Yonex Power Cushion/Victor A series). 2. Racket: 100-250 RMB (Li-Ning Steel Cannon/Victor Little Hammer/Yonex Nanoray 10F), weight 3U, head-light or balanced type. 3. Clothing: Quick-dry T-shirt + sports shorts, mid-calf sports socks, wristbands optional. Basketball Equipment: 1. Basketball Shoes (most crucial): Mid/high-top, with cushioning technology, herringbone/zigzag pattern sole, 200-500 RMB. 2. Basketball: Size 7, rubber/synthetic leather, 50-100 RMB. 3. Clothing: Loose quick-dry + basketball shorts, long basketball socks, knee pads optional. General Tips: Break in new shoes at home for 1-2 days first, change socks daily, clean insoles after playing. Budget: Prioritize shoes - badminton 200-400 RMB, basketball 250-500 RMB; main gear - badminton 100-200 RMB, basketball 50-100 RMB; clothing 100-200 RMB.",
"refer_list": []
},
{
"message_id": "msg_043",
"create_time": "2025-08-12T01:23:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I can't wait. Please customize a one-week workout plan for me.",
"refer_list": []
},
{
"message_id": "msg_044",
"create_time": "2025-08-12T01:23:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "One-Week Workout Plan: Monday: Brisk walk + stretching 40 minutes (pace 6-7 km/h). Tuesday: Badminton (doubles) 60 minutes, warm up 5 minutes beforehand. Wednesday: Rest or light activity 30 minutes walk/10 minutes stretching. Thursday: Brisk walk + core activation 45 minutes (brisk walk 30 minutes + wall sits 3 sets × 30 seconds + plank 2 sets × 20 seconds). Friday: Recreational basketball (half-court 3v3) 45-60 minutes. Saturday: Free choice 50 minutes (cycling/badminton/group class). Sunday: Active recovery 20-30 minutes (foam rolling + stretching/walking). Execution Tips: Warm up 5 minutes before exercise, wear proper shoes, keep heart rate where you can talk but can't sing, hydrate every 15-20 minutes, stretch 5 minutes after exercise. Pairing: Reduce dinner carbs by 1/3 on workout days, increase vegetables and protein, avoid overeating. Weekly Goals: Complete ≥4 workouts, exercise with friends ≥2 times, measure waist on Sunday.",
"refer_list": []
},
{
"message_id": "msg_045",
"create_time": "2025-08-12T01:24:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "What is the approximate amount of water intake needed for each workout? I also want to choose a suitable water bottle.",
"refer_list": []
},
{
"message_id": "msg_046",
"create_time": "2025-08-12T01:24:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Hydration Recommendations: General principles: 200-300ml 2 hours before exercise, 100-200ml every 15-20 minutes during exercise, 300-500ml after exercise. By activity type: Brisk walking (40-45 min) 400-600ml, badminton (60 min) 600-800ml, basketball (45-60 min) 700-1000ml. Water Bottle Selection: Brisk walking/light exercise: Direct-drink sport bottle 500ml. Badminton/Basketball: Large-capacity sport bottle 750ml-1L. Material: Tritan recommended, stainless steel for winter, avoid ordinary plastic. Brands: Decathlon 20-30 RMB, Lock&Lock/Thermos/Tiger 50-100 RMB, Keep/Li-Ning/Anta 30-60 RMB. Suggestion: Get two bottles - main large bottle 750ml-1L for high-intensity workouts, portable small bottle 500ml for brisk walking. How to know if you've drunk enough: Urine is light yellow, or weight change before/after exercise (every 1kg lost needs 1-1.5L replenishment).",
"refer_list": []
},
{
"message_id": "msg_047",
"create_time": "2025-08-12T01:25:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "What should I pay attention to when playing badminton for the first time?",
"refer_list": []
},
{
"message_id": "msg_048",
"create_time": "2025-08-12T01:25:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "First-time badminton considerations: 1. Exercise Preparation: Beginner basics, warm-up movements, weekly frequency. 2. Technique: Clear shots, grip, smashing, positioning, footwork. 3. Equipment: Racket price range, badminton shoes, string tension, shuttlecock choice. 4. Health: Preventing knee/ankle injuries, post-sprain care, calf soreness relief, considerations for heavier players, avoiding tennis elbow. 5. Recovery: Sprain treatment, rehabilitation training, returning to court. 6. Health Management: Fat loss effects, suitable intensity for BMI 29.5, pre/post-game diet. 7. Social: Finding partners, competition rules, doubles coordination. 8. Advanced: Systematic training, tactical understanding, playing styles.",
"refer_list": []
},
{
"message_id": "msg_049",
"create_time": "2025-08-12T01:26:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "How to eat before and after playing ball to help with fat loss without causing low blood sugar?",
"refer_list": []
},
{
"message_id": "msg_050",
"create_time": "2025-08-12T01:26:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Before Exercise (1-2 hours before playing): Recommended (choose 1): 1 banana + 150ml unsweetened yogurt, or 1 slice whole wheat bread + boiled egg, or 30g oatmeal porridge, or mixed grain porridge + 5-6 nuts. Avoid: Empty stomach, hot pot, fried foods, sweet drinks, large meals. Timing: Play 1.5-2 hours after main meal, 30-60 minutes after small snack. During Exercise: ≤60 minutes only hydrate (100-150ml every 15-20 minutes), >60 minutes can supplement small amount of quick carbs (half banana/2-3 soda crackers). After Exercise (within 30-60 minutes): Ideal combinations (carbs + protein = 3:1 or 4:1): Unsweetened soy milk + small apple, or egg + corn + blueberries, or 80g chicken breast/tofu + brown rice + vegetables, or Greek yogurt + cherry tomatoes + whole wheat crackers. Avoid: High oil, high salt hot pot - if eating, choose clear broth base, more lean meat/tofu/leafy greens, dipping sauce use vinegar + garlic + small chili + little soy sauce. All-day pairing: Control total calories, protein 1.2-1.6g/kg body weight, carbs choose low GI foods, don't overeat at dinner. Special reminder: Pair with strength training to enhance lower limb stability, control single exercise sessions ≤60 minutes.",
"refer_list": []
},
{
"message_id": "msg_051",
"create_time": "2025-08-20T12:03:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I have tried playing badminton a few times, and this sport is really full of fun. But I have to tell you some unfortunate news: I sprained my leg, and I just came back from the hospital. Please give me some recovery advice.",
"refer_list": []
},
{
"message_id": "msg_052",
"create_time": "2025-08-20T12:03:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Ankle sprains can fully recover with proper rehabilitation. Confirm injury: Rule out fractures, complete ligament tears, severe swelling. If diagnosed as Grade I-II sprain, the following plan applies. Phase 1: Acute Phase (0-3 days) - POLICE Principle: Protection (avoid weight-bearing, use crutches/ankle brace), Optimal Loading (move toes within pain-free range, 10 towel curls per hour), Ice (15 minutes every 2-3 hours), Compression (elastic bandage/ankle brace), Elevation (foot above heart level). Avoid: Heat, massage, alcohol, vigorous activity. Phase 2: Recovery Phase (4 days-2 weeks): Ankle pumps 20 reps × 4-6 sets/day, alphabet writing (A-Z) 2 times/day, resistance band exercises (inversion/eversion/dorsiflexion/plantarflexion) 10 reps × 3 sets, single-leg stance (healthy side) 30 seconds × 3 sets. Criteria to stop using crutches: Pain-free walking, no obvious swelling, stable ankle. Phase 3: Functional Reconstruction (after 2 weeks): Balance training (single-leg stance eyes open to closed), agility training (T-shaped shuttle runs), jump training (double to single leg). Criteria before returning to badminton: 20 pain-free single-leg calf raises, 10 single-leg jumps without discomfort, running and changing directions without ankle giving way. Protection: Wear ankle brace for at least 1-2 months, thorough warm-ups, proper shoes, avoid exercising when fatigued. Recovery time: Grade I 1-2 weeks, Grade II 3-6 weeks, Grade III 6-12 weeks. Key: Don't return to playing just because pain subsides - ligaments need time to heal.",
"refer_list": []
},
{
"message_id": "msg_053",
"create_time": "2025-08-20T12:04:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "The doctor's specific diagnosis is a grade II ankle sprain on the right foot, and the injured area has already been given preliminary treatment.",
"refer_list": []
},
{
"message_id": "msg_054",
"create_time": "2025-08-20T12:04:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Grade II ankle sprain: Partial tear of lateral ligament, with moderate pain/swelling/bruising/joint instability, no fracture or complete rupture. Rehabilitation core: Restore stability/proprioception/strength, prevent re-sprains, total timeline 4-6 weeks for light activities, 8-10 weeks to return to badminton. Week 1 (Acute Phase): Days 1-3 use crutches no weight-bearing, days 4-7 partial weight-bearing (pain ≤2/10), ice every 2-3 hours for 15 minutes, compression (elastic bandage/rigid ankle brace), elevation, ankle pumps 20 reps × 6 sets/day, towel curls 10 reps × 4 sets/day. Weeks 2-3 (Recovery Phase): Week 2 full weight-bearing, week 3 slow walk 10-15 minutes, continue using ankle brace, alphabet writing 2 times/day, resistance band exercises (focus on eversion) 15 reps × 3 sets, calf stretches 30 seconds × 3 sets, single-leg stance (healthy side) 30 seconds × 3 sets. Weeks 4-6 (Functional Rebuilding): Single-leg stance (injured side) eyes open to closed 30 seconds × 3 sets, heel raises double to single leg 15 reps × 3 sets, mini squats 10 reps × 3 sets, lateral band walks 10 steps × 3 sets. Week 6 assessment: Single-leg heel raise ≥15 reps pain-free, single-leg stance eyes closed ≥20 seconds, jogging without discomfort. Weeks 7-10 (Return to Sport): Movement tests (straight jog/lateral shuffles/figure-8 run/sudden stops) all pain-free and stable, multi-directional small jumps 2 sets × 10 reps, simulated footwork, wear ankle brace for half-court starting 15 minutes. Protection: Wear ankle brace for at least 3 months when playing, thorough warm-ups, proper shoes, avoid exercising when fatigued. Support: Medications as prescribed, physical therapy can accelerate healing, nutrition increase protein/vitamin C/zinc. Key: Don't skip balance training just because pain is gone - 80% of re-sprains occur because proprioception hasn't fully recovered.",
"refer_list": []
},
{
"message_id": "msg_055",
"create_time": "2025-08-20T12:06:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "Give me some dietary advice.",
"refer_list": []
},
{
"message_id": "msg_056",
"create_time": "2025-08-20T12:06:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Recovery diet core principles: Promote ligament repair (protein/vitamin C/zinc/copper), reduce inflammation (Omega-3/antioxidants), prevent muscle loss (quality protein + moderate calories), avoid weight gain (control refined carbs and total calories). Specific Recommendations: 1. Quality protein each meal (palm-sized): Eggs 1-2/day, chicken breast/fish-shrimp (salmon/mackerel contain Omega-3)/tofu/soy milk/edamame/low-fat milk/unsweetened yogurt. 2. Anti-inflammatory vegetables and fruits: Dark vegetables ≥300g/day (broccoli/spinach/purple cabbage/carrots), low-sugar fruits 1 fist/day split into 2 servings (blueberries/strawberries/kiwi/oranges), spices (turmeric/ginger/garlic), avoid high-sugar fruits and juices. 3. Slow-digesting carbs: Brown rice/oats/sweet potato/quinoa/whole wheat bread, each meal ≤1 fist (50-70g raw weight). 4. Healthy fats: 10 nuts/day, olive oil/camellia oil, deep-sea fish 2 times/week or fish oil capsules (EPA+DHA ≥500mg/day), avoid fried foods/fatty meat/butter. 5. Hydration: 1500-2000ml/day, plain water/light tea/lemon water, avoid sweet drinks/alcohol. Sample Daily Meals (1600-1800 kcal): Breakfast: Boiled egg + unsweetened soy milk + oatmeal porridge (with blueberries). Snack: 10g nuts or kiwi. Lunch: Half bowl brown rice + steamed sea bass + garlic broccoli + cold spinach salad. Snack: 100g unsweetened yogurt + strawberries. Dinner: Tomato tofu soup + chicken breast stir-fried with bell peppers (little oil) + half steamed sweet potato. Total calories reduced 200-300 kcal from usual, but not below 1500 kcal. Special Reminders: Don't crash diet, beware of eating too much while sedentary, supplement vitamin D (800-1000 IU/day).",
"refer_list": []
},
{
"message_id": "msg_057",
"create_time": "2025-09-15T09:00:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "My ankle has almost recovered and I can walk normally now. The doctor said I can try light exercise in two more weeks. During this time, because I couldn't exercise, I feel like my weight has rebounded a bit.",
"refer_list": []
},
{
"message_id": "msg_058",
"create_time": "2025-09-15T09:00:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "A slight weight rebound during recovery is normal, don't worry. Suggestions for current phase: 1. Continue diet control: Maintain previous healthy eating habits, slightly reduce staple food portions (due to reduced activity). 2. Upper body training: You can do seated dumbbell exercises, resistance band upper body training, 15-20 minutes daily. 3. Core training: Crunches, planks, dead bugs and other movements that don't involve the ankle. 4. Resume exercise in two weeks: Start with brisk walking, 20-30 minutes each time, gradually increasing intensity. Temporarily avoid sudden stop and direction change sports (like badminton), suggest swimming or cycling first. A 1-2kg weight rebound is normal fluctuation, you'll lose it quickly once you resume exercise.",
"refer_list": []
},
{
"message_id": "msg_059",
"create_time": "2025-09-15T09:02:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "Besides the weight rebound, I noticed that when sitting and working, my injured ankle tends to swell easily, and by evening my shoes feel tight. Is this normal?",
"refer_list": []
},
{
"message_id": "msg_060",
"create_time": "2025-09-15T09:02:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "This is a very common phenomenon during recovery. Cause: Long-term hanging down leads to poor blood return, plus the circulatory system of the injured area hasn't fully recovered. Relief suggestions: 1. Elevate the affected limb: Find a small stool to prop up your foot while working. 2. Ankle pump exercises: Do toe pointing up and down movements while sitting, using muscle contractions to promote blood return. 3. Avoid prolonged sitting: Get up every 45-60 minutes to stand on one leg (healthy side) or walk around with crutches. 4. Foot soak: Soak feet in warm water (not hot) for 10-15 minutes in the evening. If the swelling goes down in the morning, it's not a big problem; if it persists, suggest a follow-up checkup.",
"refer_list": []
},
{
"message_id": "msg_061",
"create_time": "2025-09-15T09:04:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I understand, I'll pay attention to moving around more. Also, seeing my previous exercise progress seemingly wasted because of this injury, I've been feeling a lot of psychological pressure and anxiety lately.",
"refer_list": []
},
{
"message_id": "msg_062",
"create_time": "2025-09-15T09:04:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "I completely understand your feelings. This kind of 'exercise withdrawal reaction' and anxiety about body shape is very common among sports enthusiasts after injury. Psychological adjustment suggestions: 1. Accept the current situation: Injury is part of an athletic journey, treat this period as a 'mandatory vacation' for your body. 2. Focus on the present: Don't fixate on functions that haven't recovered yet, instead record small daily progress (like walking more steadily today). 3. Substitute satisfaction: Temporarily transfer your passion for exercise to learning sports theory, watching matches, or doing upper body training. 4. Trust muscle memory: Once you resume exercise, the recovery speed of fitness and muscle mass is much faster than starting from zero. Be patient, you haven't regressed, you're just building up strength.",
"refer_list": []
},
{
"message_id": "msg_063",
"create_time": "2025-09-15T09:06:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "Okay, I'll adjust according to your suggestions. Looking forward to resuming exercise in two weeks!",
"refer_list": []
},
{
"message_id": "msg_064",
"create_time": "2025-09-15T09:06:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Keep it up! These two weeks you can rest well while preparing to resume exercise. Suggestions to prepare in advance: 1. Check if your sports equipment is in good condition. 2. Buy an ankle brace (recommend LP or McDavid, 50-100 RMB). 3. Plan your post-recovery exercise schedule. Start with brisk walking in two weeks, progress gradually, don't rush. Feel free to ask me any questions!",
"refer_list": []
},
{
"message_id": "msg_065",
"create_time": "2025-10-05T14:30:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I've fully recovered! I went for a follow-up last week and the doctor said I've recovered very well. I can't wait to start playing badminton again.",
"refer_list": []
},
{
"message_id": "msg_066",
"create_time": "2025-10-05T14:30:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Great news! Precautions for returning to the court: 1. Progress gradually: First week only play 15-20 minutes, focus on practicing basic movements, avoid intense competition. 2. Must wear ankle brace: Wear ankle brace while playing for at least the first 3 months. 3. Warm up thoroughly: 10 minutes warm-up before playing, including ankle circles, calf stretches, light jumping. 4. Avoid fatigue: Rest when you feel tired, don't push through. 5. Proper technique: Pay attention to footwork, avoid awkward sudden stop movements. First week goal: Enjoy the fun of sports, regain ball sense. Starting from the second week, you can gradually increase time to 30-40 minutes. Stop immediately if you feel any discomfort.",
"refer_list": []
},
{
"message_id": "msg_067",
"create_time": "2025-10-05T14:32:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I've already bought the ankle brace, but I'm a bit worried about getting injured again. Are there any preventive measures?",
"refer_list": []
},
{
"message_id": "msg_068",
"create_time": "2025-10-05T14:32:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Key measures to prevent re-spraining: 1. Strengthen proprioception training: Single-leg stance (injured side) 1 minute × 3 sets daily, can close eyes to increase difficulty. This is the most important training to prevent re-spraining. 2. Strengthen muscles around the ankle: Resistance band eversion training 15 reps × 3 sets/day, calf raises 20 reps × 3 sets/day. 3. Must do before playing: 5-minute ankle warm-up (circles, dorsiflexion/plantarflexion), calf stretches, light jumps to get the feel. 4. Correct footwork: Learn proper badminton push-off and landing posture, avoid inversion landing. 5. Fatigue management: Rest when tired, 80% of sports injuries occur in fatigued states. 6. Court selection: Avoid slippery or uneven courts. Keep doing proprioception training, and the probability of re-spraining will greatly decrease after 3 months.",
"refer_list": []
},
{
"message_id": "msg_069",
"create_time": "2025-10-05T14:34:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "Got it. By the way, has my weight changed during this period without exercise?",
"refer_list": []
},
{
"message_id": "msg_070",
"create_time": "2025-10-05T14:34:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "You can weigh yourself now to see the changes during the recovery period. Based on your previous diet control habits, if you've maintained them well, weight should fluctuate minimally (±1-2kg). Even if there's a slight rebound, don't worry, you'll lose it quickly once you resume exercise. The current priority is to safely recover your exercise ability; weight management can be focused on after complete recovery. What's your current weight?",
"refer_list": []
},
{
"message_id": "msg_071",
"create_time": "2025-10-05T14:36:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I just weighed myself, it's now 82kg, about 1kg heavier than before the injury. At least I kept it under control.",
"refer_list": []
},
{
"message_id": "msg_072",
"create_time": "2025-10-05T14:36:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Very good! Only gaining 1kg during recovery shows your diet control has been excellent. This 1kg is likely just mild water retention due to reduced activity, and will come off within 1-2 weeks of resuming exercise. Next plan: Weeks 1-2 resume light exercise (brisk walking, swimming), week 3 gradually resume badminton, while maintaining healthy diet. Goal: Get back below 81kg within 1 month, then continue toward 80kg. You've done great, keep it up!",
"refer_list": []
},
{
"message_id": "msg_073",
"create_time": "2025-10-20T19:00:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I've played badminton three times in the past two weeks, and I'm feeling better and better. I also weighed myself - now I'm 80kg, even lighter than before the injury!",
"refer_list": []
},
{
"message_id": "msg_074",
"create_time": "2025-10-20T19:00:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Excellent! From 86kg to 80kg, losing 6kg, BMI dropped from 29.8 to about 27.7, you've moved from obese to overweight range - this is significant progress! And your fatty liver should have noticeably improved too. Suggestions: 1. Keep maintaining: Your current exercise and diet habits are effective, stick with them. 2. Next phase goal: Lose another 3-5kg, get BMI below 25 (normal range). 3. Recommend follow-up: Schedule a liver ultrasound to check the improvement of fatty liver. 4. Measure waist circumference: See how much it's decreased compared to the initial 104cm. You've proven you can do it, keep going!",
"refer_list": []
},
{
"message_id": "msg_075",
"create_time": "2025-10-20T19:02:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I just measured my waist, it's now 96cm, down 8cm! My pants feel loose.",
"refer_list": []
},
{
"message_id": "msg_076",
"create_time": "2025-10-20T19:02:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Waist circumference dropped from 104cm to 96cm, a decrease of 8cm - this is a very significant change! Reduced waist circumference means significant reduction in visceral fat, which is very beneficial for fatty liver improvement. Scientific research shows that for every 1cm reduction in waist circumference, the risk of type 2 diabetes decreases by about 10%. At 96cm now, you're only 6cm away from the healthy standard for men (<90cm). Loose pants are the most direct feedback - consider buying a couple new pairs to reward yourself! Keep it up, next goal: Get waist circumference below 90cm.",
"refer_list": []
},
{
"message_id": "msg_077",
"create_time": "2025-10-20T19:04:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "How does your ankle feel when playing badminton? I notice it still feels a bit unstable sometimes.",
"refer_list": []
},
{
"message_id": "msg_078",
"create_time": "2025-10-20T19:04:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Occasional ankle instability is normal, indicating proprioception is still recovering. This may persist for 3-6 months. Suggestions: 1. Continue wearing ankle brace while playing, for at least 2 more months. 2. Daily proprioception training: Single-leg stance 1 minute × 3 sets (can close eyes to increase difficulty), balance pad training (if available). 3. While playing: Avoid excessive fatigue, rest when tired; avoid playing on slippery courts; focus on footwork, don't make overly aggressive save movements. 4. Strengthen ankle joint: Continue resistance band eversion training and calf raises. With consistent training, the ankle will become more and more stable. If pain or obvious giving way occurs, stop immediately and rest.",
"refer_list": []
},
{
"message_id": "msg_079",
"create_time": "2025-10-20T19:06:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "Okay, I'll keep paying attention. Recently my friends all say I've lost weight and my energy seems much better.",
"refer_list": []
},
{
"message_id": "msg_080",
"create_time": "2025-10-20T19:06:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Feedback from friends is the best motivation! Losing 6kg and 8cm off the waist - the external changes are indeed very noticeable. More importantly, your improved energy indicates overall health improvement: better sleep quality, more active metabolism, enhanced cardiopulmonary function. These are all rewards from maintaining a healthy lifestyle. Keep it up, enjoy the happiness and health that exercise brings! Remember to schedule a liver ultrasound follow-up to check the improvement of fatty liver.",
"refer_list": []
},
{
"message_id": "msg_081",
"create_time": "2025-11-03T10:15:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I went for a follow-up checkup, and the doctor said my fatty liver has gone from moderate to mild! Waist is now 95cm, 9cm less than before. I'm really happy!",
"refer_list": []
},
{
"message_id": "msg_082",
"create_time": "2025-11-03T10:15:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "This is a remarkable achievement! Fatty liver reversed from moderate to mild, waist circumference reduced by 9cm - this shows your visceral fat has significantly decreased and liver health has noticeably improved. Reviewing your progress: Weight 86kg→80kg (-6kg), waist circumference 104cm→95cm (-9cm), BMI 29.8→27.7, fatty liver moderate→mild. All achieved through your own efforts! Next goals: Continue maintaining healthy lifestyle, aim to completely eliminate fatty liver in the next 3-6 months, weight down to around 75kg (BMI about 26), waist circumference below 90cm. You're already on the right path!",
"refer_list": []
},
{
"message_id": "msg_083",
"create_time": "2025-11-03T10:17:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "The doctor also said my blood lipid levels have improved - triglycerides dropped from 2.8 to 1.9. Is this good news?",
"refer_list": []
},
{
"message_id": "msg_084",
"create_time": "2025-11-03T10:17:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "This is very good news! Triglycerides dropped from 2.8mmol/L to 1.9mmol/L, a decrease of about 32%. Normal range is <1.7mmol/L, and your 1.9 is already close to normal. High triglycerides are closely related to fatty liver, and this decrease indicates: 1. Your liver's fat metabolism ability is recovering. 2. Diet control and exercise are indeed effective. 3. Cardiovascular disease risk is decreasing. Continue maintaining your current lifestyle, and triglycerides will likely drop to normal range by your next checkup. Recommend another blood lipid and liver ultrasound follow-up in 3 months.",
"refer_list": []
},
{
"message_id": "msg_085",
"create_time": "2025-11-03T10:19:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "Great! Do I need to adjust my current exercise and diet?",
"refer_list": []
},
{
"message_id": "msg_086",
"create_time": "2025-11-03T10:19:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Your current exercise and diet plan is very effective, recommend maintaining with minor adjustments: Exercise: Continue 3-4 times per week of badminton or other sports, 45-60 minutes each time. Can appropriately add strength training (twice weekly) to help boost basal metabolism. Diet: Continue controlling oil and sugar, maintain high protein high fiber diet structure. Can relax a little - occasional hot pot is fine, just choose clear soup base. No major adjustments needed, stability is key. The goal is to make healthy eating and regular exercise become life habits, not short-term dieting. You're in great shape now, keep it up!",
"refer_list": []
},
{
"message_id": "msg_087",
"create_time": "2025-11-03T10:21:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "Okay, I'll keep persisting. Thank you for your guidance all along!",
"refer_list": []
},
{
"message_id": "msg_088",
"create_time": "2025-11-03T10:21:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "You're welcome! I'm happy to see your progress too. Since you started changing your lifestyle in July, it's been 4 months now, and you've achieved remarkable results. The hardest part you've already accomplished - building new habits. From now on, just maintain them, and health will become your norm. Feel free to ask me anytime if you have questions, keep up the good work!",
"refer_list": []
},
{
"message_id": "msg_089",
"create_time": "2025-11-15T20:00:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "The weather has gotten cold, and I find my exercise frequency has decreased. Sometimes I really don't want to go outside. Any suggestions?",
"refer_list": []
},
{
"message_id": "msg_090",
"create_time": "2025-11-15T20:00:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Winter exercise does require more motivation. Suggestions: 1. Indoor exercise alternatives: Do HIIT training at home (15-20 minutes), jump rope, yoga, fitness ring games. 2. Indoor badminton courts: Most badminton courts have heating; booking with friends in advance provides more motivation. 3. Adjust timing: Choose midday or afternoon when temperatures are higher, avoid cold mornings and evenings. 4. Warm gear: Sports jacket, moisture-wicking base layer, gloves; warm up before removing outer layer. 5. Lower the threshold: When you don't feel like exercising, first put on workout clothes and step outside - often you'll find motivation once you do. 6. Winter goal: Maintaining weight without rebound is enough, don't push for continued weight loss. Keep at least 2-3 exercise sessions per week; even lower intensity is better than no movement at all.",
"refer_list": []
},
{
"message_id": "msg_091",
"create_time": "2025-11-15T20:02:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "How exactly do I do indoor HIIT training? I want to maintain exercise at home.",
"refer_list": []
},
{
"message_id": "msg_092",
"create_time": "2025-11-15T20:02:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Here's a simple 20-minute indoor HIIT routine: Warm-up (3 minutes): March in place, arm circles, light jumps. Main training (15 minutes, each exercise 30 seconds, rest 15 seconds, 3 rounds): 1. Jumping jacks (or low-impact version: toe-tap side steps). 2. Squats. 3. High knees (or fast marching). 4. Push-ups (can do kneeling version). 5. Mountain climbers (or standing knee raises). Cool-down (2 minutes): Stretching. Notes: Since your ankle is still recovering, avoid too many jumping movements; use low-impact versions instead. Wear athletic shoes while exercising to protect joints. 2-3 times per week is sufficient. Search 'HIIT no jumping' on YouTube for follow-along videos - Pamela Reif or similar channels are recommended.",
"refer_list": []
},
{
"message_id": "msg_093",
"create_time": "2025-11-15T20:04:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "In winter I get cravings, especially for hot pot and warm foods. Are there any healthy alternatives?",
"refer_list": []
},
{
"message_id": "msg_094",
"create_time": "2025-11-15T20:04:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "It's natural to crave warm foods in winter. Healthy alternatives: 1. Clear soup hot pot: Make it yourself at home with tomato or mushroom soup base, cook vegetables, tofu, fish slices, shrimp; use soy sauce, minced garlic, cilantro as dip, avoid oil dip. 2. Various soups: Tomato egg drop soup, seaweed and dried shrimp soup, winter melon pork rib soup (degreased), chicken soup (skim the fat). 3. Warming porridge: Multigrain porridge, pumpkin porridge, sweet potato porridge, add a few red dates and goji berries. 4. Steamed dishes: Steamed fish, steamed eggs, steamed chicken breast with vegetables. 5. Oden: Homemade low-oil version with radish, konjac, tofu, eggs. Hot pot is fine to eat - once every two weeks is no problem, just choose light soup base and healthy ingredients.",
"refer_list": []
},
{
"message_id": "msg_095",
"create_time": "2025-11-15T20:06:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I weighed myself today, still 80kg, no rebound. Looks like I can keep it under control even in winter.",
"refer_list": []
},
{
"message_id": "msg_096",
"create_time": "2025-11-15T20:06:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Great! Winter is the season when weight is most likely to rebound, and your ability to maintain 80kg shows your lifestyle habits are stable. Maintaining weight without rebound in winter is itself a victory. For the rest of winter, the goal is to maintain: Keep weight fluctuation within ±1kg, exercise at least 2-3 times per week, eat until 70-80% full. When spring comes and the weather warms up, continue toward the 75kg goal. You've established a healthy lifestyle - that's the most important thing!",
"refer_list": []
},
{
"message_id": "msg_097",
"create_time": "2025-11-28T18:30:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "Recently I've had hot pot with friends a few times. I followed your earlier advice and chose clear soup base, ate more vegetables and tofu. It felt satisfying without too much burden.",
"refer_list": []
},
{
"message_id": "msg_098",
"create_time": "2025-11-28T18:30:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "This is the essence of a healthy lifestyle - not complete prohibition, but learning to make choices. The clear soup base + vegetables and tofu approach has 60-70% fewer calories than traditional hot pot, allowing you to enjoy delicious food without burdening your health. You've learned how to make healthy choices in social situations - this is a very important ability.",
"refer_list": []
},
{
"message_id": "msg_099",
"create_time": "2025-11-28T18:32:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "By the way, I want to ask - my badminton skills have improved a bit now, and I want to buy a better racket. Any recommendations?",
"refer_list": []
},
{
"message_id": "msg_100",
"create_time": "2025-11-28T18:32:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Wanting to upgrade your racket is great - it shows you're getting more interested in badminton. Intermediate racket recommendations (300-600 RMB): 1. Li-Ning Windstorm 500: Balanced offense and defense, suitable for advancing players, about 400 RMB. 2. Victor Jetspeed 12: Good feel, easy swing, about 350 RMB. 3. Yonex Nanoflare 700: Good shaft flexibility, suitable for doubles, about 500 RMB. Buying tips: Try them at a physical store first to feel different rackets; choose based on your playing style (head-heavy for offense, head-light for defense); recommended string tension 22-24 lbs, not too high. Upgrading your racket can be a reward for sticking with exercise!",
"refer_list": []
},
{
"message_id": "msg_101",
"create_time": "2025-11-28T18:34:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "Okay, I'll check it out at the store this weekend. Thank you for all your help these past few months - from diet to exercise to recovery, you've given me so much useful advice!",
"refer_list": []
},
{
"message_id": "msg_102",
"create_time": "2025-11-28T18:34:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "You've done extremely well! Being able to find a balance between enjoying food and staying healthy - that's what a sustainable lifestyle is about. Looking back over these months, you've experienced: learning healthy eating, starting regular exercise, overcoming an ankle sprain, successfully losing weight and fat. Most importantly, you've built new life habits rather than suffering through a painful diet. Keep this mindset, occasional hot pot is totally fine. Wishing you good health and enjoyable exercise! Feel free to ask me anytime you have questions.",
"refer_list": []
},
{
"message_id": "msg_103",
"create_time": "2025-11-28T18:36:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "I'll keep persisting! Next year's goal is to get my weight down to 75kg and completely eliminate fatty liver.",
"refer_list": []
},
{
"message_id": "msg_104",
"create_time": "2025-11-28T18:36:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "Those are excellent goals! Based on your current progress, they're completely achievable. From 86kg to 75kg, that's 11kg total weight loss, and you've already completed more than half. Keep pushing next spring when the weather warms up - I believe you can definitely reach your goals. Remember: Health is a lifelong matter, no need to rush. Maintain your pace, enjoy the process, and results will come naturally. Looking forward to your good news!",
"refer_list": []
}
]
}

982
data/solo_chat_zh.json Normal file
View File

@ -0,0 +1,982 @@
{
"version": "1.0.0",
"session_meta": {
"scene": "solo",
"scene_desc": {
"description": "项目讨论群聊"
},
"name": "用户健康咨询对话",
"description": "用户与AI助手关于北京旅游、健康管理、运动康复等主题的对话记录",
"group_id": "chat_user_001_assistant",
"created_at": "2025-06-26T00:00:00Z",
"default_timezone": "UTC",
"user_details": {
"user_001": {
"full_name": "用户",
"role": "user",
"extra": {
"height": 170,
"weight": 86,
"bmi": 29.8,
"waist_circumference": 104,
"origin": "四川",
"preferences": {
"food": "火锅",
"activities": "团体活动"
}
}
},
"robot_001": {
"full_name": "AI助手",
"role": "assistant",
"extra": {
"type": "assistant"
}
}
},
"tags": [
"健康咨询",
"旅游规划",
"运动康复",
"饮食建议"
]
},
"conversation_list": [
{
"message_id": "msg_001",
"create_time": "2025-06-26T00:00:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "下周我会去北京旅游，可以给我一些建议吗？",
"refer_list": []
},
{
"message_id": "msg_002",
"create_time": "2025-06-26T00:00:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "当然可以！北京景点推荐：故宫、长城、颐和园等。胡同体验：南锣鼓巷、什刹海、杨梅竹斜街。美食：烤鸭、炸酱面、卤煮火烧、豆汁儿焦圈。交通：地铁发达，下载“亿通行”或“北京一卡通”App，避开早晚高峰。天气：10月下旬入秋，白天10-20°C，早晚较凉，带外套。记得带身份证，热门景点需提前预约。",
"refer_list": []
},
{
"message_id": "msg_003",
"create_time": "2025-06-26T00:03:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "我对你说的一系列吃的东西非常感兴趣，请根据我的喜好推荐一些美食。我是四川人，我最喜欢吃的就是火锅。",
"refer_list": []
},
{
"message_id": "msg_004",
"create_time": "2025-06-26T00:03:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "根据你的川味偏好，推荐：1. 川味火锅：小龙坎、大龙燚、蜀大侠、吼堂老火锅，建议提前取号。2. 融合吃法：老北京涮肉配麻辣蘸料，或麻辣烫/冒菜。3. 川菜馆：陈麻婆豆腐、蓉李记、巴国布衣、眉州东坡。4. 美食聚集区：簋街（夜宵）、双井/望京（川渝风味）、西单大悦城/三里屯（新派川菜）。",
"refer_list": []
},
{
"message_id": "msg_005",
"create_time": "2025-06-26T00:04:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "实际上，我只是告诉你我的喜好参考，但我需要你根据我的口味推荐一些北京特色的美食。",
"refer_list": []
},
{
"message_id": "msg_006",
"create_time": "2025-06-26T00:04:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "基于你的重口味偏好，推荐北京特色美食：1. 爆肚：蘸麻酱、辣椒油、蒜泥，口感脆韧。推荐：门框胡同百年卤煮、爆肚冯。2. 卤煮火烧：浓汤配内脏，可加辣椒油。推荐：小肠陈、门框胡同卤煮。3. 炸酱面：可加辣椒油，配芥末墩儿。推荐：海碗居、方砖厂69号。4. 烤鸭：椒盐鸭架香辣酥脆。推荐：四季民福、大董。5. 门钉肉饼：配辣味羊杂汤。推荐：宝瑞门钉肉饼店。6. 豆汁儿加焦圈：配辣咸菜丝。推荐：尹三豆汁、护国寺小吃。7. 炒肝：可加蒜和辣椒油。推荐：姚记炒肝、庆丰包子铺。",
"refer_list": []
},
{
"message_id": "msg_007",
"create_time": "2025-06-26T00:16:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "我会和我五六个朋友一起去，我非常喜欢大家一起玩的项目，请为我推荐一些娱乐项目。",
"refer_list": []
},
{
"message_id": "msg_008",
"create_time": "2025-06-26T00:16:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "适合6-8人小团体的娱乐项目：1. 胡同沉浸式剧本杀/城市探秘：《北平往事》实景剧本杀、谜踪胡同解谜游戏。2. 什刹海/胡同骑行：租共享单车，设定打卡任务，适合拍照。3. 密室/沉浸剧场：《末班车》《弥生》密室、《偷心晚宴》互动戏剧。4. 京味文化体验：德云社听相声、老舍茶馆体验、DIY糖葫芦。5. 夜生活：簋街夜宵+桌游吧、Livehouse听乐队、KTV欢唱。6. 户外活动：奥林匹克森林公园烧烤+飞盘、亮马河游船夜游。提示：热门项目需提前1-3天预约，人均预算100-200元。",
"refer_list": []
},
{
"message_id": "msg_009",
"create_time": "2025-06-26T00:17:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "胡同沉浸式剧本杀和什刹海/胡同骑行我们都会尝试，这些项目有什么需要注意的吗？",
"refer_list": []
},
{
"message_id": "msg_010",
"create_time": "2025-06-26T00:17:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "注意事项：一、胡同剧本杀：1. 提前2-3天预约，6-8人可包场（800-1500元/场）。2. 穿平底鞋，带外套。3. 预留4小时（含换装、复盘）。4. 带充电宝，剧情结束后可拍照。5. 新手选欢乐本或机制本。二、什刹海/胡同骑行：1. 租共享单车或双人/三人协力车（30-50元/小时），注意安全。2. 最佳时段：上午9-11点或下午3-5点。3. 推荐路线：什刹海站→银锭桥→烟袋斜街→后海北沿→南锣鼓巷北口→帽儿胡同→鼓楼东大街（2-3小时）。4. 指定领骑和压队，用微信位置共享。5. 注意禁停区，卫生间在银锭桥、鼓楼附近。6. 可租脚踏船，傍晚看日落。通用：带身份证，备零食和水，关注天气。",
"refer_list": []
},
{
"message_id": "msg_011",
"create_time": "2025-07-05T07:00:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "我从北京回来了，这次旅行的体验真的一言难尽，甚至可以说有点糟糕。本来满怀期待去爬长城，结果到了那里简直是人山人海，别说欣赏风景了，根本就是被人流推着走，想停下来喘口气都难，完全拍不到一张像样的照片，全是人头。而且最让我崩溃的是，我因为没提前预约故宫的票，到了门口被告知票早就卖光了，只能在门口干瞪眼，看着别人进去，感觉特别遗憾。还有那个什刹海，商业化气息太重了，到处都是叫卖声，根本没有想象中那种老北京的宁静。这一趟下来，感觉不仅身体累，心更累，完全没有达到放松的目的。",
"refer_list": []
},
{
"message_id": "msg_012",
"create_time": "2025-07-05T07:00:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "听得出来你这次旅行确实受了不少委屈，满怀期待却遇到这种“人从众”的局面，换做是谁都会觉得特别扫兴。旺季的长城确实是“好汉难当”，这种被人流裹挟的体验完全破坏了游览的心情。至于故宫的票，现在确实是一票难求，很多不做足攻略的游客都会踩这个坑，这种近在咫尺却进不去的遗憾真的太搞心态了。什刹海现在的商业化也是很多老游客诟病的地方。虽然这次留下了不少遗憾，但也算是一次难忘的经历吧（虽然是不太愉快的）。先别想这些不开心的事情了，把它们当做下次错峰出游的经验吧。",
"refer_list": []
},
{
"message_id": "msg_013",
"create_time": "2025-07-05T07:04:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "这几天为了打卡各个景点，我每天步行都在2万步以上，吃饭也是匆匆忙忙对付几口，现在感觉腿都要断了，身体被掏空。明天还要上班，我现在特别焦虑，请给我一些快速恢复状态的建议。",
"refer_list": []
},
{
"message_id": "msg_014",
"create_time": "2025-07-05T07:04:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "针对你现在的身心俱疲和上班焦虑，这里有一个快速回血方案：今晚：1. 22:30前上床，睡足7-8小时。睡前远离手机，温水泡脚，4-7-8呼吸法助眠。2. 晚餐清淡：小米粥+清炒青菜+蒸鱼/豆腐，喝淡蜂蜜水。3. 热敷小腿/腰背10分钟，或靠墙拉伸。明早：1. 早餐：全麦面包+水煮蛋+坚果+温豆浆。2. 起床喝300ml温水，冷水拍手腕/后颈，听节奏明快的歌。3. 通勤时闭目养神或回想旅行美好瞬间。未来2-3天：饮食清淡，多吃蔬菜、粗粮、优质蛋白；轻度运动15分钟；接受第一天效率低，优先处理紧急任务。",
"refer_list": []
},
{
"message_id": "msg_015",
"create_time": "2025-07-07T06:00:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "我觉得我有些胖了，有什么自测方法吗？",
"refer_list": []
},
{
"message_id": "msg_016",
"create_time": "2025-07-07T06:00:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "自测方法：1. BMI计算：体重(kg)÷身高(m)²。标准：<18.5过轻，18.5-23.9正常，24.0-27.9超重，≥28.0肥胖。注意：不适用于肌肉量极高者或老年人。2. 腰围测量：肋骨下缘与骨盆上缘中点水平测量，呼气末读数。标准：男性≥90cm、女性≥85cm提示腹型肥胖。3. 腰臀比：腰围÷臀围。标准：男性<0.90、女性<0.85。4. 身体信号：衣服变紧、易疲劳气喘、久坐腹部突出、皮肤深色褶皱。建议：超重或肥胖需结合饮食和运动调整，必要时咨询医生。",
"refer_list": []
},
{
"message_id": "msg_017",
"create_time": "2025-07-07T06:03:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "我刚刚测量了我的腰围，是104厘米。你可以给我提些建议吗？",
"refer_list": []
},
{
"message_id": "msg_018",
"create_time": "2025-07-07T06:03:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "健康评估：BMI≈29.8（肥胖），腰围104cm（腹型肥胖），增加2型糖尿病、高血压、高血脂、心血管疾病、脂肪肝风险。第一阶段目标（1-3个月）：腰围缩小≥5cm，体重减轻4-5kg。行动方案：饮食：控制精制碳水，替换为糙米/燕麦/红薯/全麦面包，每餐主食一拳头；增加蛋白质（鸡蛋/鱼虾/鸡胸肉/豆腐），掌心大小；多吃高纤维蔬菜，占盘子一半；戒含糖饮料、酒精、油炸食品。运动：有氧每周5天，每次30-45分钟；力量训练每周2次，每次15分钟。监测：每周固定时间测腰围和体重，记录感受。警惕信号：空腹血糖≥6.1、血压≥140/90、肝功能异常需就医。心态：每月减2-3kg是安全速度，腰围缩小1cm可降低10%糖尿病风险。",
"refer_list": []
},
{
"message_id": "msg_019",
"create_time": "2025-07-13T08:00:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "我今天午餐吃了很多油腻的食物，感觉有些疲倦、食欲下降、腹胀。我应该怎么办？",
"refer_list": []
},
{
"message_id": "msg_020",
"create_time": "2025-07-13T08:00:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "原因：脂肪摄入过多、胃排空延迟、血糖波动。短期缓解：1. 喝温水或淡茶。2. 饭后30分钟慢走10-15分钟。3. 避免继续进食油腻或甜食，今晚选择清淡食物（粥/蔬菜汤/蒸蛋/少量水果）。4. 温敷腹部。明天调理：早餐选择粥/燕麦/全麦面包+蛋白质；午餐晚餐以蒸煮炖为主，多吃蔬菜；补充益生菌。若持续不适：少食多餐，避免生冷辛辣，可服用健胃消食片；症状持续超过两天或伴随腹痛呕吐需就医。",
"refer_list": []
},
{
"message_id": "msg_021",
"create_time": "2025-07-13T08:02:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "好的，我会参考你的建议，今晚先喝点温水并稍微活动一下。还有什么注意事项吗？",
"refer_list": []
},
{
"message_id": "msg_022",
"create_time": "2025-07-13T08:02:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "今晚注意事项：可以做：1. 分多次小口喝温水。2. 饭后30-60分钟慢走或轻微伸展，之后上半身略微抬高。3. 腹部保暖，热敷10-15分钟。4. 顺时针轻揉腹部。避免：1. 油炸、甜点、烧烤等。2. 冰饮、酒精。3. 餐后2小时内躺平。4. 紧身裤或束腰衣物。",
"refer_list": []
},
{
"message_id": "msg_023",
"create_time": "2025-07-14T04:00:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "我听从了你的建议，可是今天起床后我还是感觉有些不舒服。所以我去了医院进行体检，体检结果表明我过于肥胖，而且患有中度脂肪肝。请给我提供一些健康建议吧。",
"refer_list": []
},
{
"message_id": "msg_024",
"create_time": "2025-07-14T04:00:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "中度脂肪肝可通过生活方式改善甚至逆转。原因：热量摄入超标、久坐少动、体重上升、胰岛素抵抗。减重5-10%可明显改善。饮食建议：1. 控油控糖提蛋白：每日食用油≤25克，限制糖分，蛋白质1-1.2g/kg体重。2. 地中海式饮食：早餐燕麦+鸡蛋+番茄/水果；午餐糙米/全麦饭+清蒸鱼/鸡胸肉+蔬菜；晚餐蔬菜汤+豆腐/鸡蛋+少量粗粮；低糖水果1-2份。3. 减少隐形脂肪：少吃坚果、酱料、火锅底料；外食选清蒸/凉拌/少油菜式。运动：每周≥150分钟中等强度，快走/慢跑30-45分钟×5天，力量训练×2次，饭后走动10分钟。生活习惯：23:00前睡，戒酒，控制压力。目标：第1月减重2-3kg，第3月减重5-7%，第6月达到健康BMI或轻度脂肪肝。建议3-6个月复查。",
"refer_list": []
},
{
"message_id": "msg_025",
"create_time": "2025-07-14T04:02:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "为什么我会得中度脂肪肝？",
"refer_list": []
},
{
"message_id": "msg_026",
"create_time": "2025-07-14T04:02:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "中度脂肪肝形成原因：1. 热量摄入过多：油腻食物和果汁含高脂肪高糖，多余能量转化为脂肪存储在肝脏。2. 体重超标/肥胖：内脏脂肪通过血液运送到肝脏。3. 血糖与胰岛素代谢问题：高糖高油饮食导致胰岛素抵抗，肝脏更易储存脂肪。4. 运动不足：基础代谢低，肝脏利用脂肪能力下降。5. 生活作息不规律：熬夜、压力大影响肝脏代谢。6. 其他：家族遗传、药物或疾病。总结：主要是热量摄入过多+脂肪累积+代谢不平衡。通过饮食控制、运动、规律作息可显著改善甚至逆转。",
"refer_list": []
},
{
"message_id": "msg_027",
"create_time": "2025-07-14T04:04:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "我经常吃火锅，而且每天吃一个冰淇淋，这种生活习惯是否不太好？",
"refer_list": []
},
{
"message_id": "msg_028",
"create_time": "2025-07-14T04:04:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "这种习惯对脂肪肝不友好。火锅：油脂和盐分高，容易吃多。建议选择清汤/番茄汤底，少涮肥肉，多涮蔬菜/豆腐/鱼虾，慢慢吃。冰淇淋：糖分和脂肪高，每天吃会促进肝脏脂肪堆积。建议每周≤1-2次，小份，或选择低糖酸奶/冻水果泥/无糖冰棒。总结：高油高糖高热量是脂肪肝高危因素。改善：火锅改清汤多蔬菜，冰淇淋减少频率，结合规律饮食和有氧运动。",
"refer_list": []
},
{
"message_id": "msg_029",
"create_time": "2025-07-14T04:10:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "我想让你给我一些饮食方面的建议。",
"refer_list": []
},
{
"message_id": "msg_030",
"create_time": "2025-07-14T04:10:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "饮食原则：1. 控油：每日≤25克，避免油炸/红烧/火锅。2. 控糖：拒绝含糖饮料/奶茶/甜点，主食以粗粮为主。3. 优质蛋白：鸡胸肉/鱼/豆腐/鸡蛋/牛奶。4. 多蔬菜：每天≥500克，一半深绿色。5. 少量多餐：三餐规律，避免暴饮暴食。饮食结构：主食40-45%（糙米/全麦面/燕麦/玉米/红薯），蛋白质25-30%，脂肪20-25%（橄榄油/亚麻籽油/少量坚果），蔬菜水果≥500克。每日建议：早餐：燕麦粥+煮蛋+番茄/黄瓜，或全麦面包+无糖豆浆+水果。午餐：糙米/全麦饭+清蒸鱼/鸡胸肉/豆腐+蔬菜。晚餐：蔬菜汤+豆腐+粗粮，或蒸鸡胸肉+凉拌西兰花。加餐：低脂酸奶/小苹果/坚果（10克以内），21点后不进食。其他：每天饮水1500-2000ml，适量黑咖啡（无糖无奶精），完全戒酒，盐≤5克/天。每周一次清淡日，每周称重一次。",
"refer_list": []
},
{
"message_id": "msg_031",
"create_time": "2025-07-14T04:12:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "再给我一些运动方面的建议。",
"refer_list": []
},
{
"message_id": "msg_032",
"create_time": "2025-07-14T04:12:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "运动原则：循序渐进，有氧+力量结合，重视饭后轻运动，每周≥5天，每天≥30分钟。有氧运动：每周≥150分钟中等强度。入门（前2周）：快走/轻瑜伽/拉伸，30-40分钟。提升（3-6周）：快走+慢跑/游泳/骑车，40-60分钟。稳定期：慢跑+间歇训练，每周5次。强度判断：能说话但不能唱歌。力量训练：每周2-3次，自重深蹲10-15次×3组，俯卧撑10次×3组，平板支撑20-30秒×3次。饭后活动：餐后20-30分钟散步10-15分钟。每日时间表：早晨快走/拉伸10-15分钟，午后轻走10分钟，晚上快走/慢跑40-60分钟，每周2天力量训练。安全：运动前后热身拉伸，饭后运动间隔≥30分钟，避免高冲击运动，保证睡眠≥7小时。目标：1个月ALT/AST轻度下降，3个月减重5-7%，6个月建立规律习惯。",
"refer_list": []
},
{
"message_id": "msg_033",
"create_time": "2025-08-08T01:30:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "我已经遵照你的建议慢慢调整了我的生活习惯，最近一个月我明显感受到精神状态更好了。经过体重测量，我发现相比上个月我瘦了3kg。我感觉自己更加健康了。但是近一个月左右没有吃我最爱的火锅，让我感觉不太舒服，你有什么好的建议吗？",
"refer_list": []
},
{
"message_id": "msg_034",
"create_time": "2025-08-08T01:30:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "你已经做得很好！关于火锅：1. 控制频率：每两周或一个月一次。份量上多吃蔬菜/豆腐/菌菇，少吃肥肉和加工肉类。2. 健康方式：选择骨汤/番茄汤/菌汤底，多用鸡胸肉/鱼片/虾，蘸料用酱油/蒜泥/香菜，减少油碟/芝麻酱。3. 搭配：先吃蔬菜和菌菇，控制主食。4. 餐后：当天晚餐避免高热量零食，适度运动。5. 心理：偶尔放纵不影响整体健康，可作为奖励。总结：采用少量、健康食材、清淡汤底、合理频率的策略。",
"refer_list": []
},
{
"message_id": "msg_035",
"create_time": "2025-08-08T01:31:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "看来我还是可以继续吃火锅，对吗？只是我需要吃得更健康，以前吃得太油腻了。",
"refer_list": []
},
{
"message_id": "msg_036",
"create_time": "2025-08-08T01:31:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "可以继续吃火锅，只需调整方式：1. 汤底清淡：番茄汤/菌汤/清骨汤。2. 多吃蔬菜和蛋白质：菌菇/豆腐/绿叶菜/鱼片/鸡胸肉。3. 少油少酱：用酱油/蒜泥/香菜，避免大量芝麻酱/油碟。4. 控制主食和肉类量。5. 慢慢吃，先吃蔬菜。简单说，就是吃得聪明一点，而不是完全不吃。",
"refer_list": []
},
{
"message_id": "msg_037",
"create_time": "2025-08-12T01:17:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "我已经坚持了超过一个月的慢跑，可是我快坚持不下去了，因为我觉得很无聊。我希望和朋友们一起做一些运动，你有什么推荐吗？",
"refer_list": []
},
{
"message_id": "msg_038",
"create_time": "2025-08-12T01:17:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "和朋友一起运动能提升乐趣和动力。推荐：1. 球类运动：羽毛球（双打）/篮球（半场3v3）/乒乓球/足球，燃脂效率高，每周2-3次，每次60-90分钟。2. 骑行/徒步：周末环城绿道骑行或郊区轻徒步，强度可调，边走边聊。3. 团体健身课程：搏击操/尊巴/动感单车/TRX，高效燃脂，约1-2个朋友一起报课。4. 轻量组合：晚饭后和朋友快走40分钟+拉伸。行动建议：本周发群消息约朋友，优先选小时候喜欢的运动，先追求可持续和开心。",
"refer_list": []
},
{
"message_id": "msg_039",
"create_time": "2025-08-12T01:20:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "羽毛球和篮球确实是不错的选择，让我感觉很好。但是我此前从没有尝试过，你有什么建议吗？",
"refer_list": []
},
{
"message_id": "msg_040",
"create_time": "2025-08-12T01:20:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "零基础完全没问题。羽毛球（推荐优先）：强度适中，对膝盖冲击小，双打压力小。装备：球拍100-200元入门款，必须穿羽毛球鞋，衣服透气速干。基础动作：高远球、网前搓球/放网、接杀挡网。找新手局或约朋友一起学，运动后拉伸。篮球：从半场3v3或投篮练习开始，必须穿篮球鞋（高帮/中帮），告诉队友你是新手，初期每次30-45分钟，打完做靠墙静蹲。通用安全：热身5分钟，避免连续跳跃和急停，不适立即停止，运动后补充水分和蛋白质。第一步：周末约1-2个朋友租羽毛球场（20-40元/小时），提前看5分钟教学视频，目标不是打好而是玩得开心。",
"refer_list": []
},
{
"message_id": "msg_041",
"create_time": "2025-08-12T01:22:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "你的建议提醒了我，请告诉我选择相应运动装备时应该注意什么？",
"refer_list": []
},
{
"message_id": "msg_042",
"create_time": "2025-08-12T01:22:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "羽毛球装备：1. 羽毛球鞋（最重要）：必须选专用鞋，非黑色橡胶底，中帮/低帮带抗扭设计，品牌入门款200-400元（李宁突袭系列/Yonex Power Cushion/胜利A系列）。2. 球拍：100-250元（李宁小钢炮/胜利小铁锤/Yonex Nanoray 10F），重量3U，头轻或平衡型。3. 服装：速干T恤+运动短裤，中筒运动袜，护腕可选。篮球装备：1. 篮球鞋（重中之重）：中帮/高帮，有缓震科技，人字纹/鱼骨纹鞋底，200-500元。2. 篮球：7号球，橡胶/合成皮，50-100元。3. 服装：宽松速干+篮球裤，长筒篮球袜，护膝可选。通用：新鞋先在家穿1-2天适应，袜子每天换新，运动后清洁鞋垫。预算：鞋子优先，羽毛球200-400元，篮球250-500元；主装备羽毛球100-200元，篮球50-100元；服装100-200元。",
"refer_list": []
},
{
"message_id": "msg_043",
"create_time": "2025-08-12T01:23:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "我已经迫不及待了，请为我定制一周的运动计划。",
"refer_list": []
},
{
"message_id": "msg_044",
"create_time": "2025-08-12T01:23:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "一周运动计划：周一：快走+拉伸40分钟（配速6-7km/h）。周二：羽毛球（双打）60分钟，热身5分钟。周三：休息或散步30分钟/拉伸10分钟。周四：快走+核心激活45分钟（快走30分钟+靠墙静蹲3组×30秒+平板支撑2组×20秒）。周五：篮球（半场3v3）45-60分钟。周六：自由选择50分钟（骑行/羽毛球/团体课）。周日：主动恢复20-30分钟（泡沫轴放松+拉伸/散步）。执行提示：运动前热身5分钟，穿对鞋，心率控制在能说话但不能唱歌，每15-20分钟补水，运动后拉伸5分钟。配合：运动当天晚餐主食减1/3，增加蔬菜和蛋白质，避免暴饮暴食。本周目标：完成≥4次运动，和朋友一起运动≥2次，周日测腰围。",
"refer_list": []
},
{
"message_id": "msg_045",
"create_time": "2025-08-12T01:24:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "每场运动大概的补水量应该是多少？我还想选购一个合适的水杯。",
"refer_list": []
},
{
"message_id": "msg_046",
"create_time": "2025-08-12T01:24:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "补水量建议：通用原则：运动前2小时200-300ml，运动中每15-20分钟100-200ml，运动后300-500ml。按类型：快走（40-45分钟）400-600ml，羽毛球（60分钟）600-800ml，篮球（45-60分钟）700-1000ml。水杯选购：快走/轻度运动：直饮运动水壶500ml。羽毛球/篮球：大容量运动水壶750ml-1L。材质：Tritan推荐，不锈钢适合冬天，避免普通塑料。品牌：迪卡侬20-30元，乐扣乐扣/膳魔师/虎牌50-100元，Keep/李宁/安踏30-60元。建议买两个：主力大水壶750ml-1L用于高强度运动，便携小水壶500ml用于快走。判断喝够：尿液淡黄色，或运动前后体重变化（每减1kg需补充1-1.5L水）。",
"refer_list": []
},
{
"message_id": "msg_047",
"create_time": "2025-08-12T01:25:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "第一次打羽毛球要注意什么？",
"refer_list": []
},
{
"message_id": "msg_048",
"create_time": "2025-08-12T01:25:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "第一次打羽毛球注意事项：1. 运动准备：新手基础准备、热身动作、每周频率。2. 技术：高远球、握拍、杀球、站位、步伐。3. 装备：球拍价位、羽毛球鞋、球线磅数、球的选择。4. 健康：预防膝盖/脚踝损伤、扭伤后注意事项、小腿酸痛缓解、大体重人群注意事项、避免网球肘。5. 恢复：扭伤处理、康复训练、重返球场。6. 健康管理：减脂效果、BMI29.5适合强度、打球前后饮食。7. 社交：找搭子、比赛规则、双打配合。8. 进阶：系统训练、战术理解、技术流派。",
"refer_list": []
},
{
"message_id": "msg_049",
"create_time": "2025-08-12T01:26:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "打球前后怎么吃才有助于减脂又不低血糖？",
"refer_list": []
},
{
"message_id": "msg_050",
"create_time": "2025-08-12T01:26:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "运动前（打球前1-2小时）：推荐（选1种）：1根香蕉+无糖酸奶150ml，或1片全麦面包+水煮蛋，或燕麦粥30g，或杂粮粥+坚果5-6颗。避免：空腹、火锅、油炸、甜饮料、大量进食。时间：正餐后1.5-2小时，小加餐后30-60分钟。运动中：≤60分钟只补水（每15-20分钟100-150ml），>60分钟可补充少量快碳（半根香蕉/2-3片苏打饼干）。运动后（30-60分钟内）：理想搭配（碳水+蛋白质=3:1或4:1）：无糖豆浆+小苹果，或鸡蛋+玉米+蓝莓，或鸡胸肉/豆腐80g+糙米饭+蔬菜，或希腊酸奶+小番茄+全麦饼干。避免：高油高盐火锅，如要吃选清汤锅底，多吃瘦肉/豆腐/绿叶菜，蘸料用醋+蒜+小米辣+少量酱油。全天配合：总热量控制，蛋白质1.2-1.6g/kg体重，碳水选低GI食物，晚餐不过量。特别提醒：搭配力量训练增强下肢稳定性，控制单次运动时间≤60分钟。",
"refer_list": []
},
{
"message_id": "msg_051",
"create_time": "2025-08-20T12:03:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "我尝试了几次羽毛球，这项运动真是充满趣味。但是我得告诉你一个不幸的消息，我的腿扭伤了，我刚刚从医院回来。请给我一些康复建议。",
"refer_list": []
},
{
"message_id": "msg_052",
"create_time": "2025-08-20T12:03:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "踝关节扭伤可通过科学康复完全恢复。确认伤情：排除骨折、韧带完全断裂、严重肿胀。如诊断为Ⅰ-Ⅱ度扭伤，以下方案适用。阶段一：急性期（0-3天）- POLICE原则：保护（避免负重，使用拐杖/护踝），适度负荷（无痛范围内活动脚趾，每小时10次脚趾抓毛巾），冰敷（每2-3小时15分钟），加压（弹性绷带/护踝），抬高（脚高于心脏）。禁止：热敷、按摩、饮酒、剧烈活动。阶段二：恢复期（4天-2周）：踝泵练习20次×4-6组/天，字母写字法（A-Z）2次/天，弹力带抗阻训练（内翻/外翻/背屈/跖屈）10次×3组，单腿站立（健侧）30秒×3组。可脱拐标准：无痛行走、无明显肿胀、脚踝不打软。阶段三：功能重建期（2周后）：平衡训练（单腿站立睁眼到闭眼）、敏捷训练（T字变向跑）、跳跃训练（从双脚到单脚）。重返球场前必须：单腿提踵20次无痛，单腿跳10次无不适，跑步/变向无打软腿感。防护：至少前1-2个月佩戴护踝，强化热身，选择合适球鞋，避免疲劳运动。康复时间：Ⅰ度1-2周，Ⅱ度3-6周，Ⅲ度6-12周。关键：不要因不痛就立刻打球，韧带愈合需要时间。",
"refer_list": []
},
{
"message_id": "msg_053",
"create_time": "2025-08-20T12:04:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "医生的具体诊断是右脚二级踝关节扭伤，已经对我的伤处进行了初步处理。",
"refer_list": []
},
{
"message_id": "msg_054",
"create_time": "2025-08-20T12:04:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "二级（Ⅱ度）踝关节扭伤：外侧韧带部分撕裂，伴随中度疼痛/肿胀/淤青/关节不稳，无骨折或完全断裂。康复核心：恢复稳定性/本体感觉/力量，防止反复扭伤，总周期4-6周恢复轻度运动，8-10周重返羽毛球。第1周（急性期）：前3天拄拐不承重，第4-7天部分负重（疼痛≤2/10），冰敷每2-3小时15分钟，加压（弹性绷带/硬质护踝），抬高，踝泵20次×6组/天，脚趾抓毛巾10次×4组/天。第2-3周（恢复期）：第2周完全负重，第3周慢走10-15分钟，继续使用护踝，字母写字法2遍/天，弹力带抗阻（重点外翻）15次×3组，小腿拉伸30秒×3组，单腿站立（健侧）30秒×3组。第4-6周（功能重建期）：单腿站立（患侧）睁眼到闭眼30秒×3组，提踵训练双脚→单脚15次×3组，迷你蹲10次×3组，弹力带侧向行走10步×3组。第6周评估：单腿提踵≥15次无痛，单腿站立闭眼≥20秒，慢跑无不适。第7-10周（重返运动期）：运动测试（直线慢跑/侧向滑步/8字跑/急停变向）全部无痛无打软腿，多方向小跳2组×10次，模拟接球步法，戴护踝打半场从15分钟开始。防护：至少3个月内打球必戴护踝，强化热身，选择合适球鞋，避免疲劳运动。辅助：药物遵医嘱，理疗可加速修复，营养增加蛋白质/维生素C/锌。关键：不要因不痛就跳过平衡训练，80%再扭伤因本体感觉未恢复。",
"refer_list": []
},
{
"message_id": "msg_055",
"create_time": "2025-08-20T12:06:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "给我一些饮食方面的建议",
"refer_list": []
},
{
"message_id": "msg_056",
"create_time": "2025-08-20T12:06:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "康复期饮食核心原则：促进韧带修复（蛋白质/维生素C/锌/铜），减轻炎症（Omega-3/抗氧化物），防止肌肉流失（优质蛋白+适度热量），避免体重增加（控制精制碳水和总热量）。具体建议：1. 每餐优质蛋白质（掌心大小）：鸡蛋1-2个/天，鸡胸肉/鱼虾（三文鱼/鲭鱼含Omega-3）/豆腐/豆浆/毛豆/低脂牛奶/无糖酸奶。2. 抗炎蔬菜水果：深色蔬菜≥300克/天（西兰花/菠菜/紫甘蓝/胡萝卜），低糖水果1拳头/天分两次（蓝莓/草莓/猕猴桃/橙子），香料（姜黄/生姜/大蒜），避免高糖水果和果汁。3. 慢碳主食：糙米/燕麦/红薯/藜麦/全麦面包，每餐≤1拳头（50-70克生重）。4. 健康脂肪：坚果10颗/天，橄榄油/山茶油，鱼油每周2次或补充胶囊（EPA+DHA≥500mg/天），避免油炸/肥肉/奶油。5. 饮水：1500-2000ml/天，白开水/淡茶/柠檬水，避免甜饮料/酒精。一日三餐示例（1600-1800千卡）：早餐：水煮蛋+无糖豆浆+燕麦粥（加蓝莓）。加餐：坚果10克或猕猴桃。午餐：糙米饭半碗+清蒸鲈鱼1掌心+蒜蓉西兰花+凉拌菠菜。加餐：无糖酸奶100克+草莓。晚餐：番茄豆腐汤+鸡胸肉炒彩椒（少油）+蒸红薯半根。总热量比平时减少200-300千卡，但不低于1500千卡。特别提醒：不要节食减肥，警惕坐着不动还吃多，补充维生素D（800-1000IU/天）。",
"refer_list": []
},
{
"message_id": "msg_057",
"create_time": "2025-09-15T09:00:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "我的脚踝恢复得差不多了，已经可以正常走路了。医生说再过两周可以尝试轻度运动。这段时间因为不能运动，我感觉体重又有点反弹了。",
"refer_list": []
},
{
"message_id": "msg_058",
"create_time": "2025-09-15T09:00:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "恢复期体重小幅反弹很正常，不必焦虑。当前阶段建议：1. 饮食继续控制：保持之前的健康饮食习惯，适当减少主食量（因为运动量减少）。2. 上肢训练：可以做坐姿哑铃训练、弹力带上肢训练，每天15-20分钟。3. 核心训练：仰卧起坐、平板支撑、死虫式等不涉及脚踝的动作。4. 两周后恢复运动：从快走开始，每次20-30分钟，逐渐增加强度。暂时避免急停变向类运动（如羽毛球），建议先游泳或骑单车。体重反弹1-2kg属于正常波动，恢复运动后很快能减回来。",
"refer_list": []
},
{
"message_id": "msg_059",
"create_time": "2025-09-15T09:02:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "除了体重反弹，我发现最近坐着工作时，患侧的脚踝总是很容易肿，到了晚上鞋子都觉得紧。这是正常的吗？",
"refer_list": []
},
{
"message_id": "msg_060",
"create_time": "2025-09-15T09:02:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "这是康复期非常常见的现象。原因：长期下垂导致血液回流不畅，加上受伤部位的循环系统尚未完全恢复。缓解建议：1. 抬高患肢：工作时找个小凳子把脚垫高。2. 踝泵练习：坐着时多做勾脚尖、绷脚尖的动作，利用肌肉收缩促进血液回流。3. 避免久坐：每45-60分钟起来单腿（健侧）站立或拄拐走动一下。4. 泡脚：晚上用温水（不是热水）泡脚10-15分钟。如果早上起床消肿，说明问题不大；如果持续肿胀不退，建议复查。",
"refer_list": []
},
{
"message_id": "msg_061",
"create_time": "2025-09-15T09:04:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "明白了，我会注意多活动。另外，看着之前的运动成果因为这次受伤好像都白费了，我最近心理压力挺大的，总是很焦虑。",
"refer_list": []
},
{
"message_id": "msg_062",
"create_time": "2025-09-15T09:04:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "非常理解你的感受，这种“运动戒断反应”和对身材的焦虑在运动爱好者受伤后很常见。心理调节建议：1. 接纳现状：受伤是运动生涯的一部分，把这段时间当作身体的“强制休假”。2. 关注当下：不要盯着还没恢复的功能，而是记录每天的微小进步（比如今天走路更稳了）。3. 替代满足：把对运动的热情暂时转移到学习运动理论、观看比赛或做上肢训练上。4. 相信肌肉记忆：一旦恢复运动，体能和肌肉量的恢复速度远比从零开始要快得多。保持耐心，你没有退步，只是在蓄力。",
"refer_list": []
},
{
"message_id": "msg_063",
"create_time": "2025-09-15T09:06:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "好的，我会按照你的建议调整。期待两周后能恢复运动！",
"refer_list": []
},
{
"message_id": "msg_064",
"create_time": "2025-09-15T09:06:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "加油！这两周可以好好休息，同时做好恢复运动的准备。建议提前：1. 检查运动装备是否完好。2. 买一个护踝（推荐LP或迈克达威，50-100元）。3. 规划恢复后的运动计划。两周后从快走开始，循序渐进，不要急于求成。有任何问题随时问我！",
"refer_list": []
},
{
"message_id": "msg_065",
"create_time": "2025-10-05T14:30:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "我已经完全康复了！上周去复查，医生说恢复得很好。我迫不及待想重新开始打羽毛球了。",
"refer_list": []
},
{
"message_id": "msg_066",
"create_time": "2025-10-05T14:30:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "太好了！重返球场注意事项：1. 循序渐进：第一周只打15-20分钟，以练习基本动作为主，避免激烈对抗。2. 必戴护踝：至少前3个月打球时都要戴护踝。3. 充分热身：打球前10分钟热身，包括踝关节绕环、小腿拉伸、轻跳。4. 避免疲劳：感觉累了就休息，不要硬撑。5. 动作规范：注意步法，避免别扭的急停动作。第一周目标：享受运动乐趣，找回球感。第二周开始可以逐渐增加时间到30-40分钟。有任何不适立即停止。",
"refer_list": []
},
{
"message_id": "msg_067",
"create_time": "2025-10-05T14:32:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "护踝我已经买了，但是有点担心再次受伤。有什么预防措施吗？",
"refer_list": []
},
{
"message_id": "msg_068",
"create_time": "2025-10-05T14:32:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "预防再次扭伤的关键措施：1. 强化本体感觉训练：每天单腿站立（患侧）1分钟×3组，可以闭眼增加难度。这是预防再扭伤最重要的训练。2. 加强踝关节周围肌肉：弹力带外翻训练15次×3组/天，提踵训练20次×3组/天。3. 打球前必做：5分钟踝关节热身（绕环、背屈跖屈），小腿拉伸，轻跳找感觉。4. 正确步法：学习羽毛球正确的蹬地和落地姿势，避免内翻落地。5. 疲劳管理：累了就休息，80%的运动损伤发生在疲劳状态。6. 场地选择：避免湿滑或不平的场地。坚持做本体感觉训练，3个月后再扭伤的概率会大大降低。",
"refer_list": []
},
{
"message_id": "msg_069",
"create_time": "2025-10-05T14:34:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "明白了。对了，这段时间不运动，我的体重有什么变化吗？",
"refer_list": []
},
{
"message_id": "msg_070",
"create_time": "2025-10-05T14:34:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "你可以现在称一下体重，看看康复期间的变化。根据你之前的饮食控制习惯，如果坚持得好，体重应该波动不大（±1-2kg）。即使有小幅反弹也不用担心，恢复运动后很快就能减回来。现在的首要任务是安全恢复运动能力，体重管理可以在完全恢复后再重点关注。你现在体重是多少？",
"refer_list": []
},
{
"message_id": "msg_071",
"create_time": "2025-10-05T14:36:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "我刚称了一下，现在是82kg，比受伤前重了1kg左右。还好控制住了。",
"refer_list": []
},
{
"message_id": "msg_072",
"create_time": "2025-10-05T14:36:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "非常好！康复期只增加1kg，说明你的饮食控制做得很到位。这1kg很可能是因为活动量减少导致的轻微水肿，恢复运动后1-2周就会减下来。接下来的计划：第1-2周恢复轻度运动（快走、游泳），第3周开始逐步恢复羽毛球，同时保持健康饮食。目标：1个月内回到81kg以下，然后继续向80kg迈进。你做得很棒，继续保持！",
"refer_list": []
},
{
"message_id": "msg_073",
"create_time": "2025-10-20T19:00:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "这两周打了三次羽毛球，感觉状态越来越好了。而且我称了一下体重，现在是80kg，比受伤前还轻了！",
"refer_list": []
},
{
"message_id": "msg_074",
"create_time": "2025-10-20T19:00:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "太棒了！从86kg到80kg，减重6kg，BMI从29.8降到约27.7，已经从肥胖进入超重区间，这是非常显著的进步！而且你的脂肪肝情况应该也有明显改善。建议：1. 继续保持：当前的运动和饮食习惯很有效，坚持下去。2. 下一阶段目标：再减3-5kg，BMI降到25以下（正常范围）。3. 建议复查：可以预约一次肝脏B超，看看脂肪肝的改善情况。4. 测量腰围：看看相比最初的104cm减少了多少。你已经证明了自己可以做到，继续加油！",
"refer_list": []
},
{
"message_id": "msg_075",
"create_time": "2025-10-20T19:02:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "我刚量了一下腰围，现在是96cm，少了8cm！感觉裤子都松了。",
"refer_list": []
},
{
"message_id": "msg_076",
"create_time": "2025-10-20T19:02:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "腰围从104cm减到96cm，减少8cm，这是非常显著的变化！腰围减少意味着内脏脂肪大幅减少，这对脂肪肝的改善非常有利。科学研究表明，腰围每减少1cm，2型糖尿病风险降低约10%。你现在96cm，离男性健康标准（<90cm）只差6cm了。裤子松了是最直观的反馈，可以考虑买一两条新裤子奖励自己！继续保持，下一个目标：腰围降到90cm以下。",
"refer_list": []
},
{
"message_id": "msg_077",
"create_time": "2025-10-20T19:04:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "打羽毛球的时候脚踝感觉怎么样？我注意到有时候还是有点不太稳。",
"refer_list": []
},
{
"message_id": "msg_078",
"create_time": "2025-10-20T19:04:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "脚踝偶尔感觉不稳是正常的，说明本体感觉还在恢复中。这种情况可能持续3-6个月。建议：1. 继续戴护踝打球，至少再戴2个月。2. 每天做本体感觉训练：单腿站立1分钟×3组（可以闭眼增加难度），平衡垫训练（如果有的话）。3. 打球时注意：避免过度疲劳，累了就休息；避免在湿滑场地打球；专注步法，不要做过于激进的救球动作。4. 加强踝关节力量：继续做弹力带外翻训练和提踵训练。只要坚持训练，脚踝会越来越稳定。如果出现疼痛或明显打软腿，要立即停止并休息。",
"refer_list": []
},
{
"message_id": "msg_079",
"create_time": "2025-10-20T19:06:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "好的，我会继续注意的。最近朋友们都说我瘦了，精神状态也好了很多。",
"refer_list": []
},
{
"message_id": "msg_080",
"create_time": "2025-10-20T19:06:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "朋友的反馈是最好的激励！减重6kg、腰围减8cm，外在变化确实很明显。更重要的是，你的精神状态改善说明身体整体健康水平在提升：睡眠质量更好、新陈代谢更活跃、心肺功能增强。这些都是坚持健康生活方式带来的回报。继续保持，享受运动带来的快乐和健康！记得预约肝脏B超复查，看看脂肪肝的改善情况。",
"refer_list": []
},
{
"message_id": "msg_081",
"create_time": "2025-11-03T10:15:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "我去复查了，医生说我的脂肪肝从中度变成轻度了！腰围现在是95cm，比之前少了9cm。我真的很开心！",
"refer_list": []
},
{
"message_id": "msg_082",
"create_time": "2025-11-03T10:15:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "这是非常了不起的成就！脂肪肝从中度逆转到轻度，腰围减少9cm，这说明你的内脏脂肪大幅减少，肝脏健康明显改善。回顾你的进步：体重86kg→80kg（-6kg），腰围104cm→95cm（-9cm），BMI 29.8→27.7，脂肪肝中度→轻度。这些都是通过你自己的努力实现的！接下来的目标：继续保持健康生活方式，争取在未来3-6个月内让脂肪肝完全消失，体重降到75kg左右（BMI约26），腰围降到90cm以下。你已经走在正确的道路上了！",
"refer_list": []
},
{
"message_id": "msg_083",
"create_time": "2025-11-03T10:17:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "医生还说我的血脂指标也有改善，甘油三酯从2.8降到了1.9。这是好消息吗？",
"refer_list": []
},
{
"message_id": "msg_084",
"create_time": "2025-11-03T10:17:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "这是非常好的消息！甘油三酯从2.8mmol/L降到1.9mmol/L，下降了约32%。正常范围是<1.7mmol/L，你现在1.9已经接近正常了。甘油三酯偏高与脂肪肝密切相关，它的下降说明：1. 你的肝脏脂肪代谢能力在恢复。2. 饮食控制和运动确实有效。3. 心血管疾病风险在降低。继续保持当前的生活方式，甘油三酯很可能在下次复查时降到正常范围。建议3个月后再复查一次血脂和肝脏B超。",
"refer_list": []
},
{
"message_id": "msg_085",
"create_time": "2025-11-03T10:19:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "太好了！那我现在的运动量和饮食还需要调整吗？",
"refer_list": []
},
{
"message_id": "msg_086",
"create_time": "2025-11-03T10:19:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "当前的运动和饮食方案很有效，建议保持为主，微调为辅：运动方面：继续每周3-4次羽毛球或其他运动，每次45-60分钟。可以适当增加力量训练（每周2次），帮助提高基础代谢。饮食方面：继续控油控糖，保持高蛋白高纤维的饮食结构。可以稍微放松一点，偶尔吃顿火锅没问题，但要选清汤锅底。不需要大幅调整，稳定才是关键。目标是让健康饮食和规律运动成为生活习惯，而不是短期节食。你现在的状态很好，继续保持！",
"refer_list": []
},
{
"message_id": "msg_087",
"create_time": "2025-11-03T10:21:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "好的，我会继续坚持的。谢谢你一直以来的指导！",
"refer_list": []
},
{
"message_id": "msg_088",
"create_time": "2025-11-03T10:21:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "不客气！看到你的进步我也很开心。从7月份开始改变生活方式，到现在4个月，你已经取得了显著的成效。最难的部分你已经完成了——建立新习惯。接下来只需要保持，健康会成为你的常态。有任何问题随时问我，继续加油！",
"refer_list": []
},
{
"message_id": "msg_089",
"create_time": "2025-11-15T20:00:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "天气变冷了，我发现自己运动的频率下降了，有时候实在不想出门。有什么建议吗？",
"refer_list": []
},
{
"message_id": "msg_090",
"create_time": "2025-11-15T20:00:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "冬季运动确实需要更多动力。建议：1. 室内运动替代：在家做HIIT训练（15-20分钟）、跳绳、瑜伽、健身环游戏。2. 室内羽毛球馆：大多数羽毛球馆有暖气，提前约好朋友更有动力。3. 调整时间：选择中午或下午温度较高时运动，避开早晚寒冷时段。4. 保暖装备：运动外套、速干内衣、手套、暖身后再脱外套。5. 降低门槛：不想运动时，先穿好运动服，走出家门，往往就有动力了。6. 冬季目标：维持体重不反弹即可，不必追求继续减重。每周至少保持2-3次运动，哪怕强度低一点也比完全不动好。",
"refer_list": []
},
{
"message_id": "msg_091",
"create_time": "2025-11-15T20:02:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "室内HIIT训练具体怎么做？我想在家里也能保持运动。",
"refer_list": []
},
{
"message_id": "msg_092",
"create_time": "2025-11-15T20:02:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "推荐一个简单的20分钟室内HIIT方案：热身（3分钟）：原地踏步、手臂画圈、轻跳。主训练（15分钟，每个动作30秒，休息15秒，循环3轮）：1. 开合跳（或低冲击版：踮脚侧移）。2. 深蹲。3. 高抬腿（或快速踏步）。4. 俯卧撑（可以跪姿）。5. 登山跑（或站姿提膝）。放松（2分钟）：拉伸。注意事项：脚踝还在恢复期，避免过多跳跃动作，可以用低冲击版本替代。运动时穿运动鞋，保护关节。每周2-3次即可。B站/抖音搜索「HIIT无跳跃」有很多跟练视频，推荐帕梅拉或周六野。",
"refer_list": []
},
{
"message_id": "msg_093",
"create_time": "2025-11-15T20:04:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "冬天容易嘴馋，特别想吃火锅和热乎乎的东西。有什么健康的替代方案吗？",
"refer_list": []
},
{
"message_id": "msg_094",
"create_time": "2025-11-15T20:04:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "冬天想吃热乎乎的食物很正常。健康替代方案：1. 清汤火锅：自己在家做，番茄汤底或菌菇汤底，涮蔬菜、豆腐、鱼片、虾，蘸料用酱油蒜泥香菜，避免油碟。2. 各种汤类：番茄蛋花汤、紫菜虾皮汤、冬瓜排骨汤（去油）、鸡汤（撇去浮油）。3. 暖身粥：杂粮粥、南瓜粥、红薯粥，加少量红枣枸杞。4. 蒸菜：蒸鱼、蒸蛋、蒸鸡胸肉配蔬菜。5. 关东煮：自制低油版，萝卜、魔芋、豆腐、鸡蛋。火锅可以吃，每两周一次没问题，选择清淡汤底和健康食材即可。",
"refer_list": []
},
{
"message_id": "msg_095",
"create_time": "2025-11-15T20:06:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "今天称了一下体重，还是80kg，没有反弹。看来冬天也能控制住。",
"refer_list": []
},
{
"message_id": "msg_096",
"create_time": "2025-11-15T20:06:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "太好了！冬天是体重最容易反弹的季节，你能保持住80kg说明你的生活习惯已经稳定了。冬季保持体重不反弹本身就是胜利。接下来的冬天，目标就是维持：体重波动在±1kg内，每周至少运动2-3次，饮食保持七八分饱。等明年春天天气转暖，再继续向75kg的目标迈进。你已经建立了健康的生活方式，这是最重要的！",
"refer_list": []
},
{
"message_id": "msg_097",
"create_time": "2025-11-28T18:30:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "最近和朋友们约了几次火锅，我按照你之前的建议选择了清汤锅底，多吃蔬菜和豆腐，感觉既满足了口腹之欲，又没有太大负担。",
"refer_list": []
},
{
"message_id": "msg_098",
"create_time": "2025-11-28T18:30:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "这就是健康生活方式的精髓——不是完全禁止，而是学会选择。清汤锅底+蔬菜豆腐的吃法，热量比传统火锅低60-70%，既享受了美食，又不会对健康造成负担。你已经学会了如何在社交场合做出健康选择，这是很重要的能力。",
"refer_list": []
},
{
"message_id": "msg_099",
"create_time": "2025-11-28T18:32:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "对了，我想问一下，现在我的羽毛球水平有点进步了，想买一个好一点的球拍，有推荐吗？",
"refer_list": []
},
{
"message_id": "msg_100",
"create_time": "2025-11-28T18:32:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "想升级球拍是好事，说明你对羽毛球越来越有兴趣了。进阶球拍推荐（300-600元）：1. 李宁风刃500：攻守兼备，适合进阶选手，约400元。2. 胜利极速12：手感好，挥拍轻松，约350元。3. 尤尼克斯NF700：中杆弹性好，适合双打，约500元。选购建议：先去实体店试打，感受不同拍子的手感；根据你的打法选择（进攻型选头重拍，防守型选头轻拍）；磅数建议22-24磅，不要太高。升级球拍可以作为对自己坚持运动的奖励！",
"refer_list": []
},
{
"message_id": "msg_101",
"create_time": "2025-11-28T18:34:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "好的，我周末去店里看看。谢谢你这几个月的帮助，从饮食到运动到康复，你给了我很多有用的建议！",
"refer_list": []
},
{
"message_id": "msg_102",
"create_time": "2025-11-28T18:34:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "你做得非常好！能够找到享受美食和保持健康之间的平衡，这才是可持续的生活方式。回顾这几个月，你经历了：学会健康饮食、开始规律运动、克服脚踝扭伤、成功减重减脂。最重要的是，你建立了新的生活习惯，而不是痛苦的节食。继续保持这种心态，偶尔享受火锅完全没问题。祝你身体健康，运动愉快！有任何问题随时来问我。",
"refer_list": []
},
{
"message_id": "msg_103",
"create_time": "2025-11-28T18:36:00Z",
"sender": "user_001",
"sender_name": "user",
"type": "text",
"content": "我会继续坚持的！明年的目标是体重降到75kg，脂肪肝完全消失。",
"refer_list": []
},
{
"message_id": "msg_104",
"create_time": "2025-11-28T18:36:30Z",
"sender": "robot_001",
"sender_name": "assistant",
"type": "text",
"content": "这是非常好的目标！按照你目前的进度，完全可以实现。从86kg到75kg，总共减11kg，你已经完成了一半以上。明年春天继续加油，相信你一定可以达成目标。记住：健康是一辈子的事，不急于一时。保持节奏，享受过程，结果自然会来。期待你的好消息！",
"refer_list": []
}
]
}

4676
data/team_chat_en.json Normal file
View File

File diff suppressed because it is too large Load Diff

4685
data/team_chat_zh.json Normal file
View File

File diff suppressed because it is too large Load Diff

1049
docs/api.md Normal file
View File

File diff suppressed because it is too large Load Diff

213
docs/architecture.md Normal file
View File

@ -0,0 +1,213 @@
# Architecture
> Companion: [.claude/rules/architecture.md](../.claude/rules/architecture.md) (auto-loaded coding rules)
## DDD layered architecture
```
┌──────────────────────────────────────────────────────┐
│ entrypoints/ (Presentation) │
│ cli + api │
├──────────────────────────────────────────────────────┤
│ service/ (Application — Use Case orchestration) │
│ memorize / retrieve / evolve / manage │
├──────────────────────────────────────────────────────┤
│ memory/ (Domain — Business core) │
│ models + extract + search + cascade + prompt_slots │
├──────────────────────────────────────────────────────┤
│ infra/persistence (Storage adapters; infra/ may host other adapter types) │
│ markdown + sqlite + lancedb │
└──────────────────────────────────────────────────────┘
Cross-cutting (used by all layers, depends on none):
component/ ← Injectable providers (LLM / Embedding / config / utils)
core/ ← Runtime base (observability / lifespan / context)
config/ ← Configuration data (Settings schema + default.toml)
```
## Dependency direction (single-direction, enforced)
```
entrypoints → service → memory → infra
```
| from → to | Allowed? |
|---|---|
| entrypoints → service | ✅ |
| entrypoints → memory / infra | ❌ (must go through service) |
| service → memory | ✅ |
| memory → infra | ✅ |
| memory → service | ❌ |
| infra → memory | ❌ |
| infra cross-subpackage (e.g. lancedb → markdown within persistence/) | ❌ (use service to orchestrate) |
| any → component / core / config | ✅ (cross-cutting) |
Enforced via `import-linter` in CI:
```toml
[tool.importlinter]
root_packages = ["everos"]
[[tool.importlinter.contracts]]
name = "Layered architecture"
type = "layers"
layers = [
"everos.entrypoints",
"everos.service",
"everos.memory",
"everos.infra",
]
```
## Storage three-piece set
```
┌────────────────────────────────────────────────────────────────┐
│ md-first storage stack │
└────────────────────────────────────────────────────────────────┘
┌──────────────┐ ┌──────────────┐ ┌─────────────────┐
│ Markdown │ │ SQLite │ │ LanceDB │
│ (truth) │ │ (state) │ │ (index) │
├──────────────┤ ├──────────────┤ ├─────────────────┤
│ entries + │ │ change queue │ │ vector ANN │
│ frontmatter │ │ + state/LSN │ │ BM25 (Tantivy) │
│ Git friendly │ │ buffer / │ │ scalar filter │
│ Obsidian OK │ │ audit │ │ multi-modal │
└──────────────┘ └──────────────┘ └─────────────────┘
│ │ │
▼ ▼ ▼
memory-root/ .index/sqlite/ .index/lancedb/
(truth source) (system data) (rebuildable)
```
## Write path
```
External message
│
▼
1. service.memorize (entrypoint of write path)
│
▼
2. memory.extract.pipeline (calls everalgo)
│
▼
3. infra.persistence.markdown.write (atomic: tmp + fsync + rename)
│ ✅ md write success → return immediately
│
┌───┴────┐
│ │
▼ ▼
4a. SQLite 4b. memory.cascade (async daemon)
audit watches md → diff entries → LanceDB sync
```
**Key guarantee**: md write is strongly consistent (fsync). LanceDB is eventually consistent. LanceDB unavailability does not block response — changes buffer in the SQLite `md_change_state` queue, replayed on recovery.
## Read path
```
User query
│
▼
1. service.retrieve
│
▼
2. memory.search.hybrid single LanceDB query =
BM25 + vector ANN + scalar filter
│
▼
3. (optional) read md original markdown for context
│
▼
Return
```
## Key components
### `memory/extract/`
```
extract/
├── ingest/ Standardized message intake + multi-modal parser dispatch
├── pipeline/ Main extraction pipeline (calls everalgo + dual-track split + writes store)
└── evolution/ Async memory evolution (event/counter/cron triggers)
```
### `memory/cascade/`
Daemon that watches markdown changes and syncs to LanceDB:
- inotify / FSEvents file watcher (cross-platform via `watchdog`)
- 500ms debounce
- Entry-level diff (added / changed / removed)
- LanceDB single-transaction update (text + vector columns atomic)
- LSN-based crash recovery via the SQLite `md_change_state` queue
### `memory/prompt_slots/`
Three-layer prompt overlay:
```
config/prompt_slots/*.yaml (Layer 1: defaults, ships with package)
↓
~/.everos/prompt_slots/*.yaml (Layer 2: app-level override)
↓
runtime override (Layer 3: per-call override)
```
everalgo receives PromptSlot as parameter — no hardcoded prompts in algorithm code.
### `core/observability/`
Three-piece observability:
- `metrics/` — Prometheus counter / gauge / histogram + global registry
- `logging/` — structlog with context processor (trace_id propagation)
- `tracing/` — OpenTelemetry tracer + span helpers
## Markdown layout
```
~/.everos/ # memory root (default; EVEROS_MEMORY__ROOT)
└── <app_id>/<project_id>/ # scope ("default" → default_app/default_project)
├── users/<user_id>/
│ ├── user.md # profile (single-file rewrite)
│ ├── episodes/episode-<YYYY-MM-DD>.md # daily-log append
│ ├── .atomic_facts/atomic_fact-<YYYY-MM-DD>.md # hidden, framework-derived
│ └── .foresights/foresight-<YYYY-MM-DD>.md # hidden, framework-derived
├── agents/<agent_id>/
│ ├── .cases/agent_case-<YYYY-MM-DD>.md # hidden, framework-derived
│ └── skills/skill_<name>/SKILL.md # named-dir
└── knowledge/ # global shared knowledge
```
System-managed entries (`.index/`, `.tmp/`) and `ome.toml` live directly
under the memory root.
Full tree + frontmatter chassis: [storage_layout.md](storage_layout.md) and
[how-memory-works.md](how-memory-works.md). Frontmatter has 4-tier field
protection (L1 read-only / L2 system / L3 business / L4 user).
## everalgo boundary
[`everalgo`](https://github.com/EverMind-AI/EverAlgo) is a separate Python library (published as the `everalgo-*` PyPI packages) holding **only memory extraction algorithms**:
- `everalgo.parser` — multi-modal parsing
- `everalgo.user_memory` — ConvMemCell / Episode / Foresight / AtomicFact / Profile extractors
- `everalgo.agent_memory` — AgentMemCell / Case / Skill extractors
- `everalgo.knowledge` — file-to-knowledge
everalgo is:
- **Stateless** — pure functions, no class hierarchy
- **No I/O** — does not touch md files / LanceDB / SQLite
- **No prompts inline** — receives `PromptSlot` parameter, project supplies defaults
This boundary lets everalgo be reused across product forms (this open-source build, EverOS Cloud, OpenClaw plugins, etc.).
## Further reading
- [docs/overview.md](overview.md) — vision and scope
- [docs/engineering.md](engineering.md) — engineering tooling and CI / CD
- [.claude/rules/architecture.md](../.claude/rules/architecture.md) — short-form rules for Claude Code

271
docs/cascade_runbook.md Normal file
View File

@ -0,0 +1,271 @@
# Cascade Runbook
The cascade daemon keeps LanceDB in sync with the markdown files under
the memory root. Service / entry points only ever write markdown; the
daemon is the **sole** writer of the LanceDB index. This runbook covers
the recurring operational questions.
## What runs where
When `everos server start` boots, the FastAPI lifespan wires four
providers in order:
1. **Metrics** — Prometheus collector.
2. **SQLite** — system DB + schema (`SQLModel.metadata.create_all`).
3. **LanceDB** — async connection + schema verification + FTS indexes.
4. **Cascade** — watcher + scanner + worker, all in-process tasks.
The cascade subsystem itself is three independent loops:
| Loop | Source signal | Effect |
|---|---|---|
| Watcher | `watchdog` filesystem events (sync thread) | `md_change_state.upsert` per registered kind |
| Scanner | Periodic walk (`scan_interval_seconds`, default 30 s) | Same — catches changes the watcher missed |
| Worker | `claim_pending_batch` polling (default 1 s when idle) | Handler dispatch → LanceDB upsert / delete |
Every loop talks to the same `md_change_state` sqlite table. The
worker's claim mode (`pending → processing → done/failed`) keeps
concurrent workers honest.
## Health: `everos cascade status`
```
queue:
pending: 3
done: 1247
failed (retryable=TRUE): 1 (eligible for `cascade fix --apply`)
failed (retryable=FALSE): 1 (fix md and re-save to recover)
lsn:
max: 1252
last_processed: 1250
lag: 2
```
- `lag > 0` means the worker is behind. Steady state should hover near
zero; sustained lag points at a slow handler or a stuck retry.
- `failed (retryable=FALSE)` is always user-actionable. Cascade will
never auto-clear these — they represent malformed md the user must
edit.
## Recovering from failures: `everos cascade fix`
`cascade fix` (no flag) lists every failed row. With `--apply`:
1. `UPDATE md_change_state SET status='pending', retry_count=0
WHERE status='failed' AND retryable=TRUE` (the partial index
`idx_md_change_retryable` makes this O(retryable)).
2. Drain the worker once so the retry runs synchronously.
Retryable failures cover transient embedding / HTTP errors (5xx, 429,
network resets) after the inline `MAX_RETRY=3` was exhausted. The
fix command resets the counter so a working backend gets a clean
start.
`retryable=FALSE` rows require the user to edit the md (typically a
YAML frontmatter issue) and re-save; the watcher picks the change up
naturally.
## One-shot replay: `everos cascade sync [PATH]`
Use this when the watcher missed an event (WSL mount, network share,
external editor with no inotify) or when you want a deterministic
flush before, say, a smoke test:
```bash
everos cascade sync # drain everything pending
everos cascade sync users/u1/episodes/X.md # re-enqueue + drain
```
The CLI builds the same `CascadeOrchestrator` as the daemon but only
calls `sync_once` / `drain_once` — no watcher / scanner background
task. So it's safe to run in parallel with a live `everos server`.
## Recovery paths
### LanceDB schema drift on startup
`LanceDBLifespanProvider.startup` calls `verify_business_schemas`. If
an on-disk table has columns the current Pydantic schema does not
declare (or vice versa), the boot fails with:
```
LanceDB table 'episode' schema drift: missing=[...], extra=[...].
The index is rebuildable from md — recover with
`rm -rf ~/.everos/.index/lancedb` and restart.
```
This is the documented recovery: delete the index, restart the
server, the scanner will pick up every md file on its first sweep and
the worker repopulates LanceDB. Markdown is the source of truth, so
no data is lost.
### inotify watch-limit exhaustion (Linux)
Default kernel limit is 8 192 watches per user. On a sizeable memory
root the watcher may silently miss events. Symptoms:
- Scanner catches the file changes but the watcher never logs an
event for the same path.
- `cat /proc/sys/fs/inotify/max_user_watches` is at the limit.
Fix by bumping the kernel parameter:
```bash
echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf
sudo sysctl -p
```
### WSL2 / network mounts
Filesystem events do not propagate from the Windows host into WSL2
(or across most SMB / NFS shares). The watcher will start without
error and silently see nothing.
Workarounds:
- Rely on the scanner — at default 30 s interval, throughput is
bounded but eventually-consistent.
- Drop the scan interval to ~5 s if the memory root is small.
- Run `everos cascade sync` explicitly after batch edits.
### Daemon process crash mid-batch
`claim_pending_batch` flips rows to `processing` *atomically*. If the
process dies before `mark_done` / `mark_failed`, those rows stay in
`processing` until the next boot. **The orchestrator auto-recovers**
on startup: `CascadeOrchestrator.start` calls
`md_change_state_repo.recover_orphan_processing()` before launching
the watcher / scanner / worker, which resets every `processing` row
back to `pending`. Single-process cascade means no race — at boot
time no other worker could legitimately own a `processing` row.
No operator action required; the structured log line
`cascade_recovered_orphan_processing` reports the count when it
fires.
### FD exhaustion (`os error 24` / EMFILE)
Symptoms (any of these on a long-running daemon):
- LanceDB query / index build fails with `lance error: ... Too many
open files (os error 24)`.
- `lsof -p <pid> | wc -l` grows monotonically over hours / days.
- Health log lines like `cascade_lancedb_optimize_failed` /
`cascade_lancedb_rebuild_failed` carrying `OSError: [Errno 24]`.
Cause (verified against `lance crate 4.0`): the LanceDB *index* cache
(`GlobalIndexCache`) holds one reader object per opened FTS / vector
/ scalar index, and each reader pins the file descriptors of its
`_indices/<uuid>/...` files. With a long-running daemon and steady-
state cascade ingest, every `optimize()` call adds new readers; with
LanceDB's own default (`index_cache_size_bytes=None`, unbounded), they
**are never evicted** and the FDs leak monotonically.
`drop_index` does **not** help — it is a manifest-only operation and
leaves the on-disk UUID directories untouched. Even an explicit
`optimize(cleanup_older_than=0)` `unlink()`-ing the files does not
release FDs: POSIX keeps the inode alive as long as a process holds
an open FD on it (the entries show as `(deleted)` in `lsof`). Only an
LRU eviction inside the cache (or a connection close) actually closes
the FDs.
Fix (already wired in `LanceDBSettings.index_cache_size_bytes` —
default 16 MB, ~290 FD ceiling): see
[Tuning knobs § LanceDB index cache](#lancedb-index-cache-index_cache_size_bytes)
for the sizing table and the env-var override path.
If you have already hit EMFILE in a running process, the cleanest
recovery is a daemon restart — the open connection closes, every FD
is released, and the next start comes up with the capped Session in
place.
## Tuning knobs
### Cascade scheduler knobs
All defaults live in `everos.memory.cascade.orchestrator.CascadeConfig`
and `everos.memory.cascade.worker.CascadeWorker`:
| Knob | Default | Effect |
|---|---|---|
| `scan_interval_seconds` | 30 | Scanner sweep cadence |
| `worker_batch_size` | 50 | Rows claimed per worker cycle |
| `worker_max_retry` | 3 | Inline retries before `mark_failed(retryable=TRUE)` |
| `worker_poll_interval_seconds` | 1 | Idle wait between empty drain attempts |
| `worker_retry_backoff_seconds` | 2 | Linear backoff seed; doubles per attempt |
Tuning surface is intentionally not in `Settings` yet — once we have
wall-clock numbers from real workloads, the values that need
operator override will surface there.
### LanceDB index cache (`index_cache_size_bytes`)
Lives in `LanceDBSettings`; overridable via the
`EVEROS_LANCEDB__INDEX_CACHE_SIZE_BYTES` environment variable. This
is the only knob that bounds the steady-state file-descriptor count
of a long-running EverOS daemon — see
[Recovery paths § FD exhaustion](#fd-exhaustion-os-error-24-emfile)
for why nothing else (prune, rebuild, `drop_index`) helps.
Measured cap → FD ceiling (30 add+optimize cycles + 100-query stress
on the real `Episode` schema):
| Cap | FD ceiling | Query latency (p50) | Safe under `ulimit -n` |
|---|---|---|---|
| `2 MB` | ~45 | ~5 ms | macOS default 256 (5× headroom) |
| `4 MB` | ~52 | ~3 ms | macOS default 256 |
| `8 MB` | ~140 | ~2.4 ms | macOS default 256 (1.8× headroom) |
| **`16 MB`** (default) | **~290** | **~2.3 ms** | **Linux default 1024 (3.5× headroom); macOS needs `ulimit -n 1024`** |
| `32 MB` | ~630 | ~1.4 ms | Linux default 1024 (1.6× headroom) |
| `unbounded` | grows forever | ~1.3 ms | NEVER use in a daemon |
EverOS's measured steady-state working set after a `rebuild_indexes`
cycle is roughly **50-100 readers / 3-6 MB resident** (5 tables × ~7
BM25 columns × ~10 `part_N` reader entries each), so the 16 MB default
provides ~3× headroom for burst traffic and stale-but-not-yet-evicted
readers.
When to override:
- **Tight `ulimit -n` environments** (containers; macOS dev boxes
that haven't bumped the default 256) → drop to `4 MB` or `8 MB`.
Query latency increases by ~1-3 ms but correctness is unaffected.
- **Larger working sets** (many more tables or much wider FTS
indexes than the default schema set) → bump to `32-64 MB`. Verify
your platform's `ulimit -n` covers the corresponding FD ceiling
with at least 2× headroom.
- **Diagnostic-only**: set to a tiny value (e.g. `1 MB`) to
*force* LRU thrashing and reproduce cache-miss latency in tests.
Do **not** set `metadata_cache_size_bytes` — it is intentionally left
at LanceDB's default (unbounded) because the metadata cache holds
parsed manifests / fragment stats and has zero effect on FD count;
capping it just thrashes parsing work without solving anything.
## Concurrency
The worker is async, not multi-process. Inside one drain cycle,
`asyncio.gather(*[_process_one(row) for row in batch])` runs every
claimed row concurrently — cascade is IO-bound (embedding HTTP calls
dominate wall time) so single-process coroutine concurrency saturates
the bottleneck. The `worker_batch_size` knob (default 50) caps
in-flight rows.
Multi-process workers are a scaling axis we'd reach for only if a
single process becomes CPU-bound, which the current design does not
anticipate. `claim_pending_batch` is already race-safe (the
``WHERE status='pending'`` filter ensures each row lands in exactly
one batch even if multiple workers raced), so adding processes later
is a deployment-side change with no schema work.
## What cascade does NOT do (yet)
- **Schema migration**: LanceDB column changes require `rm -rf`.
- **Parent-id back-link**: Episode rows currently carry
`parent_id=None`; the writer doesn't preserve the source memcell id
in the entry inline. Tracked separately.
- **Reference-file change detection (agent_skill)**: edits to
`references/*.md` siblings won't trigger a re-index — only changes
to `SKILL.md` itself fire the watcher. Workaround: run
`everos cascade sync agents/<a>/skills/skill_<n>/SKILL.md` after
editing references.

116
docs/cli.md Normal file
View File

@ -0,0 +1,116 @@
# CLI
The `everos` command-line entry point covers **setup and operations** —
generate a starter `.env` (`init`), run the HTTP API server (`server
start`), and operate the md → LanceDB index queue (`cascade`). Hot-path
business (`/add` `/flush` `/search` `/get`) is the **HTTP API**, not the
CLI.
CLI commands run **in-process** — they call into the `service/` /
infrastructure layers directly rather than the HTTP loopback.
## Installation
The script is exposed via `pyproject.toml`:
```toml
[project.scripts]
everos = "everos.entrypoints.cli.main:app"
```
After `uv sync` (or `pip install -e .`) the `everos` command resolves
to [`src/everos/entrypoints/cli/main.py`](../src/everos/entrypoints/cli/main.py),
a [Typer](https://typer.tiangolo.com/) app.
## Subcommand layout
```
everos
├── init Generate a starter .env from the packaged template
├── server
│ └── start Start the HTTP API server (uvicorn)
└── cascade Inspect / operate the md → LanceDB sync queue
├── status Queue / LSN summary
├── sync Drain the queue now (force md → LanceDB)
└── fix List failed rows / re-enqueue retryable ones
```
Each subcommand lives in its own module under
[`entrypoints/cli/commands/`](../src/everos/entrypoints/cli/commands/) and is
registered in `cli/main.py`. The CLI is intentionally small — hot-path
business (`/add` `/flush` `/search` `/get`) is the **HTTP API**, not the
CLI; the CLI covers setup (`init`), running the server, and index ops
(`cascade`). There is no `reindex` command — rebuild by deleting
`<root>/.index/lancedb` and restarting, or run `everos cascade sync`.
## `everos server start`
Wraps `uvicorn` to launch the FastAPI app from
[`entrypoints/api/app.py`](../src/everos/entrypoints/api/app.py)
in *factory* mode.
```bash
everos server start \
--host 127.0.0.1 \
--port 8000 \
--log-level info \
--env-file .env
```
| Flag | Env var | Default |
|---|---|---|
| `--host` | `EVEROS_API__HOST` | `127.0.0.1` (loopback only; binding `0.0.0.0` logs a warning — EverOS ships no auth) |
| `--port` | `EVEROS_API__PORT` | `8000` |
| `--log-level` | `EVEROS_LOG_LEVEL` | `INFO` |
| `--env-file` | — | searched: `./.env` → `$XDG_CONFIG_HOME/everos/.env` → `~/.everos/.env` |
| `--reload` | — | off (use in development) |
Lifespan startup wires the storage backends (SQLite engine + LanceDB
connection) on app boot; see
[`entrypoints/api/lifespans/`](../src/everos/entrypoints/api/lifespans/).
## Configuration via env vars
Both CLI and HTTP server read configuration from `pydantic-settings`:
| Env var | Settings field |
|---|---|
| `EVEROS_MEMORY__ROOT` | `Settings.memory.root` (memory-root path) |
| `EVEROS_MEMORY__TIMEZONE` | `Settings.memory.timezone` (e.g. `Asia/Shanghai`) |
| `EVEROS_SQLITE__BUSY_TIMEOUT_MS` | `Settings.sqlite.busy_timeout_ms` |
| `EVEROS_LANCEDB__READ_CONSISTENCY_SECONDS` | `Settings.lancedb.read_consistency_seconds` |
Pattern: `EVEROS_<SECTION>__<KEY>` (double underscore = nesting). See
[`config/settings.py`](../src/everos/config/settings.py).
## Logging
`configure_logging` runs at CLI startup and configures `structlog` with
the resolved log level. All in-process logs (CLI command bodies +
service / infra layers) flow through the same handler.
```bash
everos server start --log-level debug # see all sql / lance traffic
```
## API ↔ CLI division of labour
| Responsibility | API | CLI |
|---|---|---|
| Hot-path business (`/add` `/flush` `/search` `/get`) | ✅ | — (HTTP only) |
| Setup (generate `.env`) | — | `everos init` |
| Run the server | — | `everos server start` |
| Index ops (drain / inspect / fix the cascade queue) | — | `everos cascade {status,sync,fix}` |
| Health probe | `GET /health` | (use HTTP) |
| Metrics scrape | `GET /metrics` | (use HTTP) |
The CLI is the **shell-friendly** surface for ops + scripting; the
HTTP API is the **process-friendly** surface for clients (web UIs,
agents, automation).
## See also
- [architecture.md](architecture.md) — DDD layering between
entrypoints / service / memory / infra
- [`entrypoints/cli/main.py`](../src/everos/entrypoints/cli/main.py)
- [`entrypoints/cli/commands/server.py`](../src/everos/entrypoints/cli/commands/server.py)

263
docs/datetime.md Normal file
View File

@ -0,0 +1,263 @@
# Datetime & Timezones
> Audience: contributors. Read this once before touching any code that
> records a moment in time.
## Table of contents
- [The two-zone discipline](#the-two-zone-discipline)
- [Why two zones](#why-two-zones)
- [Helper reference](#helper-reference)
- [Field-type rules](#field-type-rules)
- [End-to-end data flow](#end-to-end-data-flow)
- [Common pitfalls](#common-pitfalls)
- [Testing guidance](#testing-guidance)
## The two-zone discipline
EverOS treats datetimes on **two separate rails**:
| Rail | Where it lives | Helper |
|---|---|---|
| **UTC** (storage) | SQLite, LanceDB, OME events — anything persisted to disk | `get_utc_now`, `ensure_utc`, `UtcDatetime` |
| **Display tz** | Markdown frontmatter, HTTP API responses, daily-log filename buckets, fallback zone for naive caller input | `get_now_with_timezone`, `today_with_timezone`, `to_display_tz` |
The display timezone is set by the `EVEROS_MEMORY__TIMEZONE`
environment variable (or `[memory] timezone` in TOML). Default `UTC`.
**Inviolable rule**: the display tz must **never** reach storage. Once
the user switches `EVEROS_MEMORY__TIMEZONE`, existing on-disk rows
must not misalign.
## Why two zones
### What goes wrong with a single "configured" zone
The naive design — "use one configured timezone everywhere" — has two
failure modes, both subtle:
1. **Configuration drift.** Day 1 the user configures
`EVEROS_MEMORY__TIMEZONE=Asia/Shanghai`. Everything stores
Shanghai-local datetimes. On Day 30 they switch to
`UTC`. SQLite (which strips tz on write and returns naive on read)
silently reinterprets the old Shanghai values as UTC — every old
row jumps eight hours into the future.
2. **Cross-region replication.** If two deployments share storage
but configure different display zones, both interpret the same
naive bytes against their own local zone and diverge by the
offset delta. There is no "true" reading.
UTC-only storage forecloses both: bytes on disk are zone-independent.
### Why not UTC everywhere then?
Users want to read timestamps in their wall-clock zone. Markdown
frontmatter that says `2026-05-29T06:00:00Z` for a meeting that
happened locally at 14:00 is jarring. The display rail solves this
without polluting storage: render UTC bytes through `to_display_tz`
at the boundary.
## Helper reference
All helpers live in [`everos.component.utils.datetime`](../src/everos/component/utils/datetime.py).
### Storage rail
| Helper | Behaviour |
|---|---|
| `get_utc_now() -> datetime` | Current UTC instant, `tzinfo=UTC`. Independent of any setting. Use as `default_factory` on any storage field. |
| `ensure_utc(d) -> datetime` | Naive → attach display tz → convert to UTC. Aware → `astimezone(UTC)`. Use at the storage boundary if you receive a datetime you didn't construct. |
| `UtcDatetime` | `Annotated[datetime, AfterValidator(ensure_utc)]`. Apply to any SQLite field. Pydantic auto-runs validation on both INSERT defaults and read-back rows. |
### Display rail
| Helper | Behaviour |
|---|---|
| `get_now_with_timezone() -> datetime` | Current instant in the configured display tz. `.isoformat()` produces e.g. `2026-05-29T14:00:00+08:00`. |
| `today_with_timezone() -> date` | Today's date in the display tz. Use for daily-log filename buckets. |
| `to_display_tz(d) -> datetime` | Convert any datetime to the display tz. Naive input is treated as already display-tz local. |
### Parsing & rendering
| Helper | Behaviour |
|---|---|
| `from_iso_format(value)` | Parse an ISO string / datetime / epoch. Naive input attaches **display tz** (the "if you didn't say a zone, assume your zone" rule). |
| `from_timestamp(ts)` | Parse epoch seconds / milliseconds (auto-detects). Returns display-tz aware. |
| `to_iso_format(d)` | `.isoformat()` after light validation. |
| `to_timestamp_ms(d)` | Milliseconds epoch (`int`). |
## Field-type rules
### SQLite tables
```python
from everos.component.utils.datetime import UtcDatetime, get_utc_now
from everos.core.persistence.sqlite import BaseTable, Field
class MyRow(BaseTable, table=True):
happened_at: UtcDatetime = Field(default_factory=get_utc_now)
```
Why `UtcDatetime` and not plain `datetime`? SQLAlchemy silently strips
tz on SQLite writes. `UtcDatetime`'s `AfterValidator` runs on
**construction** to make sure whatever the caller hands in gets
normalised to UTC before persistence.
SQLModel's ORM hydrate path (rows from `select(...)`) **bypasses**
the Pydantic validator — SQLAlchemy assigns column values straight
to instance attributes. To close that gap,
[core/persistence/sqlite/base.py](../src/everos/core/persistence/sqlite/base.py)
registers a SQLAlchemy `load` event listener that re-attaches
`tzinfo=UTC` to every `UtcDatetime` column after hydrate. Net effect:
**callers never see a naive datetime from a SQLite repo**, whatever
the code path.
`BaseTable.created_at` / `updated_at` already use `UtcDatetime` and
`get_utc_now` — any subclass inherits both the construction-time
validator **and** the load-time hook for free.
### LanceDB tables — zero configuration
```python
import datetime as _dt
class MyLanceRow(BaseLanceTable):
ts: _dt.datetime # automatically tz=UTC in the Arrow schema
```
LanceDB's Pydantic → PyArrow converter does not understand
`typing.Annotated` metadata; using `UtcDatetime` as the annotation
would raise `TypeError: Converting Pydantic type to Arrow Type`.
Instead, `BaseLanceTable.to_arrow_schema()` walks the inferred schema
and rewrites **every** naive `timestamp[us]` column to
`timestamp[us, tz=UTC]`. PyArrow then:
* **on write** — `astimezone(UTC)` any aware input automatically.
* **on read** — returns aware UTC datetimes (not naive).
No caller-side coercion needed, no per-table declaration. The
response shapers only run `to_display_tz(...)` to convert UTC to the
configured display zone.
If a future schema genuinely needs a naive datetime column (project
convention says storage is always UTC, so this would be unusual),
override `to_arrow_schema` on that subclass and skip the patch for
that one column.
### OME events / in-memory state
OME events are persisted-adjacent (the `run_record` / `counter` stores
serialise them). Use `get_utc_now()` for any `default_factory` on the
event payload.
## Two centralised defenses
| Backend | Defense | Where |
|---|---|---|
| **SQLite** | SQLAlchemy `load` event listener on `BaseTable` re-attaches `tzinfo=UTC` after every ORM hydrate | [core/persistence/sqlite/base.py](../src/everos/core/persistence/sqlite/base.py) |
| **LanceDB** | `BaseLanceTable.to_arrow_schema()` rewrites `UTC_DATETIME_FIELDS` columns to `timestamp[us, tz=UTC]`; PyArrow handles UTC end-to-end | [core/persistence/lancedb/base.py](../src/everos/core/persistence/lancedb/base.py) |
| **CI gate** | `scripts/check_datetime_discipline.py` fails the build on any code that bypasses `component/utils/datetime` | wired into `make lint` |
These defenses replace what used to be an "every consumer must call
`ensure_utc()`" shotgun discipline. With both in place, callers never
observe a naive datetime from either backend.
## End-to-end data flow
```
User input (any zone)
│
▼
from_iso_format ← naive → attach display tz
│
▼
ensure_utc ← storage boundary: → UTC
│
▼
┌────────────────┬────────────────┐
│ SQLite │ LanceDB │
│ (UtcDatetime │ (Arrow │
│ re-attaches │ stripped to │
│ UTC on read) │ UTC bytes) │
└────────────────┴────────────────┘
│
▼
from_iso_format ← read path normalises naive → display tz
│
▼
to_display_tz ← response boundary: → display tz
│
▼
Pydantic .isoformat() → "2026-05-29T14:00:00+08:00"
│
▼
HTTP API response / markdown frontmatter
```
The storage boundary and response boundary are the two points where
the zone discipline is enforced. Everything in between just passes
datetimes through.
## Common pitfalls
> [!WARNING]
> **`datetime.now()` without `tz=`.** Forbidden. Always use
> `get_utc_now()` (storage) or `get_now_with_timezone()` (display).
> Linted by `.claude/rules/datetime-handling.md` and CI.
> [!WARNING]
> **Calling `astimezone()` on a value just read from SQLite.** If the
> field isn't typed `UtcDatetime`, SQLite returns naive — and
> `astimezone()` on a naive datetime silently interprets it as
> **local process time**, not UTC. Always use `UtcDatetime` on SQLite
> fields.
> [!WARNING]
> **Storing `get_now_with_timezone()` directly.** That returns
> display-tz time. If the display tz later changes, your stored values
> are stranded. Use `get_utc_now()` for any persisted field.
> [!INFO]
> **Migrating existing rows.** Q2 was rolled out on a clean codebase
> with no production data. If you operate an instance where SQLite
> values were written with display-tz-aware values (pre-Q2), you must
> either drop the database or write a one-time migration that
> reinterprets each row's naive value against the old display tz
> before re-writing as UTC. The project does not ship such a
> migration.
## Testing guidance
For unit tests that depend on display-tz behaviour, both caches must
clear:
```python
import pytest
from everos.component.utils import datetime as dt_module
from everos.config import load_settings
@pytest.fixture(autouse=True)
def _isolate_tz(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.delenv("EVEROS_MEMORY__TIMEZONE", raising=False)
load_settings.cache_clear()
dt_module._display_tz.cache_clear()
```
The autouse fixture in [tests/conftest.py](../tests/conftest.py) does
exactly this — it runs for every test by default. If you write a
locally-scoped test that needs a non-default zone, monkeypatch the env
var **and** clear both caches:
```python
def test_my_thing(monkeypatch):
monkeypatch.setenv("EVEROS_MEMORY__TIMEZONE", "Asia/Shanghai")
load_settings.cache_clear()
dt_module._display_tz.cache_clear()
...
```
The full invariant set is covered in
[tests/unit/test_component/test_utils/test_datetime.py](../tests/unit/test_component/test_utils/test_datetime.py)
under the "Q2 two-zone discipline invariants" section. If you change
the storage / display contract, those tests are the first line of
defense — update them in lockstep.

553
docs/engineering.md Normal file
View File

@ -0,0 +1,553 @@
# Engineering & Dev-Efficiency Infrastructure
> Companions: business architecture lives in [architecture.md](architecture.md);
> hard coding constraints live in [../.claude/rules/](../.claude/rules/).
> This document covers the surrounding tooling, configuration, and processes
> — what we adopted, what role each piece plays, and how they fit together.
> CI runs on GitHub Actions; all checks are invoked through the `Makefile`.
---
## 1. Scope
Engineering / dev-efficiency infrastructure does not solve business problems —
it solves **team + code + time** problems:
```
┌──────────────────────────────────────────────────────────┐
│ │
│ Business architecture (docs/architecture.md) │
│ — answers "how to build the system" │
│ │
│ Engineering rules (.claude/rules/) │
│ — answers "how to write the code" │
│ │
│ Engineering / dev-efficiency infrastructure (this doc) │
│ — answers "how the team collaborates, │
│ how code is auto-checked, │
│ how releases are automated, │
│ how tools land in the project" │
│ │
└──────────────────────────────────────────────────────────┘
```
Reasons this is documented separately:
- **Cross-project reusable** — `CLAUDE.md` / rules / `pyproject.toml` are
patterns, not content. The next project can adopt them as-is.
- **Decoupled from business** — business architecture changes do not affect
these; upgrading these does not affect business.
- **Onboarding-oriented** — new contributors read this first to understand
what the tooling looks like.
---
## 2. Infrastructure overview
```
┌─────────────────────────────────────────────────────────────────────┐
│ Team collaboration / Code quality / CI/CD │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─ Claude Code engineering layer ────────────────────────────┐ │
│ │ │ │
│ │ CLAUDE.md ← team-shared context (auto loaded into │ │
│ │ system prompt) │ │
│ │ .claude/ │ │
│ │ ├── CLAUDE.md subdir context (optional) │ │
│ │ ├── rules/ (10) path-scoped hard coding rules │ │
│ │ ├── skills/ (3) slash command workflows │ │
│ │ └── settings.json permissions allowlist │ │
│ │ │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │
│ ┌─ Code quality gates ───────────────────────────────────────┐ │
│ │ │ │
│ │ pre-commit runs locally before commit │ │
│ │ ├ ruff (lint+fmt) │ │
│ │ ├ trailing-whitespace / end-of-file-fixer │ │
│ │ ├ check-yaml / check-toml │ │
│ │ ├ check-added-large-files (≥1MB warn) │ │
│ │ ├ detect-private-key │ │
│ │ └ gitlint (commit-msg stage) │ │
│ │ │ │
│ │ ruff lint + format │ │
│ │ (replaces black / isort / flake8) │ │
│ │ import-linter DDD layer-direction enforcement │ │
│ │ pytest unit / integration │ │
│ │ │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │
│ ┌─ Dependencies & build ─────────────────────────────────────┐ │
│ │ │ │
│ │ uv sole package manager │ │
│ │ (no `pip install`) │ │
│ │ pyproject.toml src layout + extras + groups │ │
│ │ uv.lock checked in; CI uses --frozen │ │
│ │ hatchling wheel build backend │ │
│ │ Makefile unified entry; CI calls it │ │
│ │ src/everos/templates/env.template │ │
│ │ environment variable template │ │
│ │ │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │
│ ┌─ CI/CD (GitHub Actions) ───────────────────────────────────┐ │
│ │ │ │
│ │ CI: .github/workflows/ci.yml lint / test / integ │ │
│ │ Docs: .github/workflows/docs.yml Markdown link check │ │
│ │ Both invoke Makefile targets; the Makefile is the │ │
│ │ single source of truth for commands. │ │
│ │ │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │
│ ┌─ Collaboration workflow ───────────────────────────────────┐ │
│ │ │ │
│ │ Branch model: dev / master (GitFlow Lite) │ │
│ │ PR template: .github/PULL_REQUEST_TEMPLATE.md │ │
│ │ ISSUE_TEMPLATE: bug / feature / use-case / docs / config │ │
│ │ CONTRIBUTING.md: contributor onboarding │ │
│ │ │ │
│ └────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────┘
```
---
## 3. Claude Code engineering layer
### 3.1 Loading mechanism
Claude Code automatically loads the following into the system prompt at
session start (no manual import):
```
┌────────────────────────┬──────────────────────────────────────────┐
│ File │ Purpose │
├────────────────────────┼──────────────────────────────────────────┤
│ CLAUDE.md (repo root) │ Team-shared context: architecture │
│ │ overview, commands, convention index │
│ .claude/rules/*.md │ Hard coding constraints │
│ │ (path-scoped on-demand load) │
│ .claude/settings.json │ Permissions allowlist (not in prompt) │
│ ~/.claude/CLAUDE.md │ User-level (personal preferences) │
│ CLAUDE.local.md │ Project-local personal (gitignored) │
└────────────────────────┴──────────────────────────────────────────┘
```
### 3.2 Rules (10 files, path-scoped)
| File | Paths (auto-load condition) |
|---|---|
| architecture.md | always loaded (no paths) |
| code-style.md | always loaded (no paths) |
| language-policy.md | always loaded (no paths) |
| imports.md | `src/**/*.py`, `tests/**/*.py` |
| init-py-and-reexport.md | `src/**/__init__.py`, `src/**/*.py` |
| module-docstring.md | `src/{infra,memory,service,component,core}/**/*.py` |
| async-programming.md | `src/**/*.py`, `tests/**/*.py` |
| datetime-handling.md | `src/**/*.py`, `tests/**/*.py` |
| logging-observability.md | `src/**/*.py` |
| testing.md | `tests/**/*.py` |
**Why path-scoped**: avoid loading 1000+ lines of rules every session
(~5–8K tokens). At startup only architecture + code-style + language-policy
load (~1.5–2K tokens); the rest load on demand when Claude Code reads a
matching `.py` file.
### 3.3 Skills (3 slash commands)
| Command | Purpose | When to use |
|---|---|---|
| `/commit` | Generate a Conventional Commits message | After a focused change, ready to commit |
| `/new-branch` | Create branch under dev/master strategy | Starting a new feat / fix / hotfix |
| `/pr` | Open a GitHub PR with the repo template | Ready to merge |
Skills and rules use **independent loading mechanisms**: rules auto-load
into the system prompt, skills only trigger when the user types `/<name>`.
### 3.4 settings.json
```json
{
"permissions": {
"allow": ["Bash(uv sync*)", "Bash(make*)", "Bash(uv run pytest*)", ...]
}
}
```
**Purpose**: reduce permission prompts. Team-shared config goes into
`settings.json` (in git); personal preferences go into `settings.local.json`
(gitignored).
---
## 4. Code quality gates
```
┌──────────────────────────────────────────────────────┐
│ Each stage can independently fail the change │
└──────────────────────────────────────────────────────┘
[Local editor]
│
▼
Stage 1: editor real-time feedback
├ ruff (lint + format) on save
└ path-relevant .claude/rules guide Claude Code
│
▼
Stage 2: pre-commit (triggered by `git commit`)
├ ruff fix + format
├ trailing-whitespace, end-of-file-fixer
├ check-yaml, check-toml
├ check-added-large-files (≥1MB)
├ detect-private-key
└ gitlint (commit-msg stage; rejects malformed messages)
│
▼
Stage 3: local `make ci` (manual, before push)
├ make lint (ruff check + ruff format --check + import-linter)
├ make test (pytest tests/unit)
└ make integration (pytest tests/integration)
│
▼
Stage 4: CI (GitHub Actions, push + PR triggered)
└ re-runs the same `make lint / test / integration` targets
│
▼
Stage 5: PR review
├ ≥ 1 approval
└ all threads resolved + all CI green
```
**Key design**: when any stage fails, **never merge** — there is no
`--no-verify` / `--allow-failure` escape hatch.
---
## 5. Dependencies & build
### 5.1 pyproject.toml overview
```toml
[project]
name = "everos"
requires-python = ">=3.12"
dependencies = [...] # runtime deps (minimal set)
[project.optional-dependencies]
multimodal = [...] # extras (install on demand)
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["src/everos"] # src layout
[project.scripts]
everos = "everos.entrypoints.cli.main:app" # exposes CLI command
[tool.ruff] # code style
[tool.pytest.ini_options] # tests
[tool.coverage.run] # coverage config (gate lives in `make cov`)
[tool.importlinter] # dependency direction
[dependency-groups]
dev = ["ruff", "pytest", "pytest-asyncio", "pytest-cov",
"import-linter", "pre-commit", "ipdb"]
```
**Single-file principle**: configuration that used to live in `pylintrc`,
`pytest.ini`, `.isort.cfg` is **all consolidated into `pyproject.toml`**.
### 5.2 Makefile commands
```
make help list all targets
make install uv sync --frozen
make format ruff fix + format
make lint ruff + import-linter + datetime discipline + openapi drift
make test pytest tests/unit
make integration pytest tests/integration
make cov pytest unit + integration, coverage gate (fail under 80%)
make ci lint + test + integration ← CI invokes these targets
make clean clear caches
```
**Single source of truth**: CI only invokes `make <target>`, so local and CI
run identical commands and cannot drift.
### 5.3 env.template (slimmed down)
The template lives at `src/everos/templates/env.template` (bundled
inside the wheel as package data, copied to `./.env` via `everos init`).
It groups settings by provider, each block sharing the OpenAI-protocol
`MODEL` / `API_KEY` / `BASE_URL` triple:
```
EVEROS_LLM__* # text model (model / api_key / base_url)
EVEROS_MULTIMODAL__* # vision model for image/office inputs
EVEROS_EMBEDDING__* # embedding model (vector index)
EVEROS_RERANK__* # cross-encoder reranker
EVEROS_MEMORY__ROOT # memory-root (md files + .index/{sqlite,lancedb}/)
EVEROS_LOG_LEVEL # DEBUG | INFO | WARNING | ERROR
EVEROS_LOG_FORMAT # json | text
TZ # display timezone (storage is always UTC)
```
Every key has a sensible default except the `API_KEY` fields, which you fill in.
---
## 6. CI/CD (GitHub Actions)
### 6.1 Strategy
```
┌──────────────────────────────────────────────────────────┐
│ │
│ GitHub Actions (.github/workflows/) │
│ ci.yml push (main/dev/master) + PR │
│ ├ make install-deps (uv sync --frozen) │
│ ├ make lint (ruff + import-linter + │
│ │ datetime + openapi drift) │
│ ├ make test (pytest tests/unit) │
│ └ make integration (pytest tests/integration) │
│ docs.yml Markdown link check + issue-template YAML │
│ │
│ Consistency: │
│ ├ astral-sh/setup-uv (cache keyed by uv.lock) │
│ ├ Makefile is the single source of CI commands │
│ └ pre-commit runs locally first to reduce CI churn │
│ │
└──────────────────────────────────────────────────────────┘
```
### 6.2 CI checklist
| Check | Tool | Failure condition |
|---|---|---|
| Lint | `make lint` (ruff check + ruff format --check) | any error |
| Layer direction | `make lint` (lint-imports inside) | layer violation |
| Datetime discipline | `make lint` (check_datetime_discipline.py) | bypasses helper module |
| OpenAPI drift | `make lint` (dump_openapi.py --check) | schema ≠ committed openapi.json |
| Unit | `make test` (pytest tests/unit) | any failure |
| Integration | `make integration` (pytest tests/integration) | any failure |
Integration tests run with a `FakeLLMClient` — no live credentials are needed in CI.
Commit message format is enforced **locally** via `gitlint` in the `commit-msg`
pre-commit stage; it does not run in CI.
### 6.3 Branch protection
| Branch | Rule |
|---|---|
| **master** | branch protection: PR + 1 review + green CI; no direct push |
| **dev** | same as above |
| feat / fix / hotfix | free push; rebase parent before merge |
---
## 7. Collaboration workflow
### 7.1 Branch model (GitFlow Lite)
```
v0.1 v0.2 v1.0
▲ ▲ ▲
│ release PR │ release PR │ release PR
│ (dev→master+tag) │ (dev→master+tag) │ (dev→master+tag)
master ●──────────────────────●─────────────●──────────────────●──────────────────────────────────●────► stable / released
│ ▲ │ │
│ │ merge hotfix │ │
│ │ │ │
│ ●──●──┘ │ │
│ │ hotfix branch │ │
│ │ (cut from master) │ │
│ │ │ │
│ ▼ sync to dev │ │
│ │ │ │
dev ●──●──●──●──●──●──●──●──●─●──●──●─●──●──●──●──●──●──●──●──●─●──●──●──●──●──●──●──●──●──●──●──●─────► integration
▲ ↑ ↑ ↑
│ release point release point release point
feat/A (dev HEAD → (dev HEAD → (dev HEAD →
●──●──● master + v0.1) master + v0.2) master + v1.0)
feat/* : cut from dev → PR → merge into dev
hotfix/* : cut from master → merge into master + sync into dev (double merge)
release : dev → master + tag on master (no separate release branch)
Vertical │ in the diagram = "dev HEAD merged into master via release PR + v0.x tag"
```
Details in [../.claude/skills/new-branch/SKILL.md](../.claude/skills/new-branch/SKILL.md).
### 7.2 PR template
A single PR template at [`.github/PULL_REQUEST_TEMPLATE.md`](../.github/PULL_REQUEST_TEMPLATE.md)
with five sections: **Summary / Area / Verification / Checklist / Notes for
Reviewers**. The `/pr` skill fills it in (see
[../.claude/skills/pr/SKILL.md](../.claude/skills/pr/SKILL.md)).
### 7.3 Commit convention (Conventional Commits)
Format: `<type>[(scope)][!]: <description>` per
[Conventional Commits](https://www.conventionalcommits.org).
```
feat: new feature
fix: bug fix
refactor: restructuring (no behavior change)
test: add / update tests
docs: documentation
style: formatting
perf: performance optimization
chore: configuration / build / tooling
build: build system or dependencies
ci: CI configuration
revert: revert a previous commit
```
`gitlint` enforces the format **locally** via its `contrib-title-conventional-commits`
rule in the commit-msg pre-commit stage. See
[../.claude/skills/commit/SKILL.md](../.claude/skills/commit/SKILL.md).
---
## 8. Issue templates / user support
```
.github/ISSUE_TEMPLATE/
├── bug_report.yml structured bug report (form)
├── feature_request.yml feature proposal (form)
├── use_case.yml share a use case / integration
├── docs.yml documentation issue
└── config.yml disable blank issues + community links
CONTRIBUTING.md contributor onboarding: setup / code style /
branch / commit / PR / testing
```
---
## 9. Infrastructure summary table
```
┌─────────────────────┬──────────────────────────────────────┬─────────────┐
│ Facility │ Location / file │ Failure │
│ │ │ impact │
├─────────────────────┼──────────────────────────────────────┼─────────────┤
│ CLAUDE.md │ /CLAUDE.md │ cc loses │
│ │ │ context │
│ Team rules │ /.claude/rules/ (10) │ cc unaware │
│ │ │ of conv. │
│ Team skills │ /.claude/skills/ (3) │ no slash │
│ │ │ workflows │
│ Permissions │ /.claude/settings.json │ cc prompts │
│ │ │ on each op │
├─────────────────────┼──────────────────────────────────────┼─────────────┤
│ pyproject │ /pyproject.toml │ build fail │
│ Lock file │ /uv.lock │ dep drift │
│ Makefile │ /Makefile │ no unified │
│ │ │ entry │
│ pre-commit │ /.pre-commit-config.yaml │ no local │
│ │ │ gate │
│ env template │ /src/everos/templates/env.template │ newcomers │
│ │ │ lost on env│
├─────────────────────┼──────────────────────────────────────┼─────────────┤
│ CI │ /.github/workflows/ci.yml │ PR cannot │
│ │ │ merge │
│ Docs CI │ /.github/workflows/docs.yml │ broken │
│ │ │ doc links │
│ PR template │ /.github/PULL_REQUEST_TEMPLATE.md │ no PR temp │
│ Issue templates │ /.github/ISSUE_TEMPLATE/ (5) │ scattered │
│ CONTRIBUTING │ /CONTRIBUTING.md │ contrib. │
│ │ │ confused │
└─────────────────────┴──────────────────────────────────────┴─────────────┘
```
---
## 10. Future extensions
```
Near-term
□ /new-module skill: scaffold a subpackage that complies with rules
□ ruff rule sets: add D (docstring), ANN (annotations)
□ Static type checking (pyright or mypy) once hot paths stabilize
Mid-term
□ release-please / Conventional Commits → automated changelog
□ Automated PyPI wheel upload on tag
□ Multi-Python version matrix (3.12 / 3.13)
□ Performance benchmark CI with historical comparison
Long-term
□ Mutation testing (mutmut)
□ Coverage ratchet (raise the 80% gate as the suite matures)
```
---
## 11. On investing in engineering infrastructure
```
┌──────────────────────────────────────────────────────────┐
│ │
│ Plain business code ≠ an engineering project │
│ │
│ Engineering project = business code + │
│ coding rules + │
│ quality gates (pre-commit + CI) + │
│ automation (Makefile + skills) + │
│ collaboration (branch + PR) + │
│ knowledge base (CLAUDE.md + │
│ rules + docs) │
│ │
│ The earlier this infrastructure lands, the faster and │
│ farther the team can run. │
│ │
└──────────────────────────────────────────────────────────┘
```
Old project vs. new project after this rewrite:
| Dimension | Old project | New project |
|---|---|---|
| Lint tools | black + isort + pylint | ruff (single tool) |
| Config files | pyproject + pylintrc + pyrightconfig + pytest.ini | unified pyproject.toml |
| pre-commit | basic | adds gitlint commit-msg + import / yaml / private-key checks |
| Layer direction | not enforced | import-linter enforced in CI |
| Commit format | freeform | gitlint pre-commit hook (Conventional Commits) |
| Claude Code integration | partial rules | rules + skills + settings (full) |
| CI platform | ad hoc | GitHub Actions calling Makefile targets |
| Tests | basic | unit + integration + e2e + coverage report |
These are not perfectionism — they are baseline requirements for
**multi-person collaboration, long-term maintenance, and sustainable
evolution**.
---
## 12. References
- Hard coding rules: [../.claude/rules/](../.claude/rules/) (auto-loaded by Claude Code)
- Slash command workflows: [../.claude/skills/](../.claude/skills/)
- Contributor onboarding: [../CONTRIBUTING.md](../CONTRIBUTING.md)
- Architecture: [architecture.md](architecture.md)
- Claude Code memory mechanism: [code.claude.com/docs/en/memory.md](https://code.claude.com/docs/en/memory.md)
- Claude Code skills: [code.claude.com/docs/en/skills.md](https://code.claude.com/docs/en/skills.md)
- ruff: [docs.astral.sh/ruff](https://docs.astral.sh/ruff/)
- import-linter: [import-linter.readthedocs.io](https://import-linter.readthedocs.io/)
- gitlint: [jorisroovers.com/gitlint](https://jorisroovers.com/gitlint/)
- uv: [docs.astral.sh/uv](https://docs.astral.sh/uv/)
- pre-commit: [pre-commit.com](https://pre-commit.com/)
- Conventional Commits: [conventionalcommits.org](https://www.conventionalcommits.org/)
- GitHub Actions: [docs.github.com/en/actions](https://docs.github.com/en/actions)

294
docs/how-memory-works.md Normal file
View File

@ -0,0 +1,294 @@
# How Memory Works
How EverOS turns a stream of messages into durable, searchable memory —
the storage stack, the path layout on disk, the write→index→read
pipeline, and the consistency guarantees.
This is the narrative companion to the reference docs: see
[storage_layout.md](storage_layout.md) for the exact file encoding,
[architecture.md](architecture.md) for the layer boundaries, and
[api.md](api.md) for the HTTP contract.
## Table of contents
- [The storage stack](#the-storage-stack)
- [Storage paths](#storage-paths)
- [How a memory is born](#how-a-memory-is-born)
- [Memory types & storage strategies](#memory-types--storage-strategies)
- [The cascade daemon](#the-cascade-daemon)
- [The Offline Memory Engine (OME)](#the-offline-memory-engine-ome)
- [Consistency model](#consistency-model)
- [Zero external services](#zero-external-services)
- [Operating it](#operating-it)
## The storage stack
Three embedded pieces, each owning what it is best at. Markdown is the
**source of truth**; the other two are **derived and rebuildable**.
| Layer | Backed by | Holds | Rebuildable? |
|---|---|---|---|
| **Markdown + YAML frontmatter** | plain `.md` files | the memory content itself — the only portable, human-editable asset | — (it *is* the truth) |
| **SQLite** (`aiosqlite`) | `.index/sqlite/*.db` | system state, audit log, the cascade queue, the boundary buffer, OME engine state | ✅ from markdown |
| **LanceDB** (Arrow) | `.index/lancedb/*.lance` | vector + BM25 + scalar columns for retrieval | ✅ from markdown |
!!! note "The one rule that follows from this"
Delete the entire `.index/` directory and **no memory is lost** — it
rebuilds from the `.md` tree. There is no separate "export"; the
markdown *is* the export. (How to trigger a rebuild:
[Operating it](#operating-it).)
## Storage paths
The default memory root is **`~/.everos/`** (override with
`EVEROS_MEMORY__ROOT` or `[memory] root` in TOML). Configuration (the
`.env` file) is separate from data (the memory root): the server searches
`./.env` → `$XDG_CONFIG_HOME/everos/.env` → `~/.everos/.env`.
Memory is partitioned by **`<app_id>/<project_id>`** *before* the
user-visible directories, so different `(app, project)` spaces never share
a directory or cross in search. The reserved id `"default"` materialises as
`default_app` / `default_project` on disk (so a default space stays
visually distinct from a user-named one).
```
~/.everos/ ← memory root (EVEROS_MEMORY__ROOT)
├── default_app/ ← <app_id> ("default" → default_app)
│ └── default_project/ ← <project_id> ("default" → default_project)
│ ├── users/ ← user-visible (source of truth)
│ │ └── <user_id>/
│ │ ├── user.md single-file (profile)
│ │ ├── episodes/
│ │ │ └── episode-<YYYY-MM-DD>.md daily-log append
│ │ ├── .atomic_facts/ daily-log (hidden)
│ │ │ └── atomic_fact-<YYYY-MM-DD>.md
│ │ └── .foresights/ daily-log (hidden)
│ │ └── foresight-<YYYY-MM-DD>.md
│ ├── agents/
│ │ └── <agent_id>/
│ │ ├── .cases/ daily-log (hidden)
│ │ │ └── agent_case-<YYYY-MM-DD>.md
│ │ └── skills/ skill-named dir
│ │ └── skill_<name>/SKILL.md (+ references/ scripts/)
│ └── knowledge/ ← shared / global (reserved)
│
├── .index/ ← system-managed, rebuildable (gitignore)
│ ├── sqlite/
│ │ ├── system.db state / audit / cascade queue (md_change_state) / buffer / LSN
│ │ ├── ome.db Offline Memory Engine state
│ │ ├── ome.aps.db APScheduler jobstore (split to avoid lock contention)
│ │ └── ome.db.lock OME single-engine guard (portalocker)
│ └── lancedb/
│ └── <kind>.lance/ one Arrow table per kind
│
├── ome.toml ← user-editable OME strategy overrides (hot-reloaded)
└── .tmp/ atomic-write staging
```
!!! warning "Differences from older PRD-era docs"
The index dir is **`.index/`** (dot-prefixed), not `_index/`. The
cascade queue and LSN/audit state live in **SQLite** (`system.db`,
table `md_change_state`) — there is no `.cascade.log` / `.manifest.json`
file in the current implementation. The `<app>/<project>` nesting is
real and always present (`default_app/default_project` for the default
scope). There is **no `everos reindex` command** (see
[Operating it](#operating-it)).
The path manager is
[`MemoryRoot`](../src/everos/core/persistence/memory_root.py); every path
above is a property on it. `MemoryRoot.ensure()` creates the runtime dirs
(`.index/{sqlite,lancedb}/`, `.tmp/`) and copies the OME template to
`ome.toml`; user-visible dirs appear on first write.
## How a memory is born
A message does not become memory immediately — it accumulates, a boundary
is detected, an LLM extracts a cell, writers persist markdown, and the
index catches up asynchronously.
```
POST /add ──▶ unprocessed_buffer (SQLite) ← messages accumulate per (session, app, project)
│
├─ boundary detector trips ─┐
POST /flush ─────────┤ (or you force it) │ one LLM call
│ ▼
│ extract MemCell ──▶ memcell row (SQLite)
│ │
│ ┌──────────────┴───────────────┐
│ ▼ ▼
│ UserMemoryPipeline (sync) AgentMemoryPipeline (fire-and-forget)
│ writes episode .md NOW emits AgentPipelineStarted
▼ │ │
(response returns once md is on disk) │
▼ ▼
┌─────────────────── Offline Memory Engine (OME) ───────────────────┐
│ async strategies write derived .md: │
│ atomic_facts · foresight · user profile · agent cases · agent skills │
└───────────────────────────────┬──────────────────────────────────────┘
▼
cascade daemon watches the .md tree
▼
md_change_state queue (SQLite, durable)
▼
rebuild LanceDB rows ──▶ searchable
```
- **`/add`** appends messages to a per-`(session_id, app_id, project_id)`
buffer and returns `accumulated` (or `extracted` if the boundary tripped
on this call). See [api.md](api.md).
- **`/flush`** forces the boundary now (one extraction LLM call), used at
the end of a chat/agent run.
- Episode markdown is written **synchronously** — when `/flush` returns
`extracted`, the episode file is already on disk.
- Everything else (atomic facts, foresight, profile, agent cases/skills)
is produced **asynchronously** by the OME — see
[the OME section](#the-offline-memory-engine-ome).
- The **cascade daemon** turns every `.md` write into LanceDB rows so the
content becomes searchable.
## Memory types & storage strategies
Six business memory kinds today, each user- or agent-owned, each picking
one of three on-disk patterns:
| Kind | Owner | Dir / file | Strategy | Produced by |
|---|---|---|---|---|
| **episode** | user | `episodes/episode-<date>.md` | daily-log | extraction (sync) |
| **atomic_fact** | user | `.atomic_facts/atomic_fact-<date>.md` (hidden) | daily-log | OME |
| **foresight** | user | `.foresights/foresight-<date>.md` (hidden) | daily-log | OME |
| **profile** | user | `user.md` | single-file rewrite | OME |
| **agent_case** | agent | `.cases/agent_case-<date>.md` (hidden) | daily-log | OME |
| **agent_skill** | agent | `skills/skill_<name>/SKILL.md` | skill-named dir | OME (clustering) |
The three strategies:
| Strategy | Shape | Why |
|---|---|---|
| **Daily-log append** | `<prefix>-<YYYY-MM-DD>.md`, one entry appended per memory | collapses thousands of per-entry files into one file per day |
| **Single-file rewrite** | a fixed filename overwritten in place | for a single evolving document (a user/agent profile) |
| **Skill-named dir** | one directory per skill | a skill is a richer unit (body + optional `references/` `scripts/`) |
!!! note
The single-file writer also supports `agent.md` / `soul.md` /
`tools.md` / `behaviors.md`, but no shipped OME strategy produces those
yet — today only `user.md` is written. Detailed frontmatter and
entry-id encoding live in [storage_layout.md](storage_layout.md).
## The cascade daemon
The cascade subsystem keeps LanceDB in sync with the markdown tree. It runs
**in-process** with the server (a coroutine started by the app lifespan),
not as a separate OS daemon.
1. A native filesystem watcher (`watchdog`: FSEvents on macOS, inotify on
Linux) sees a `.md` create/modify.
2. The change is enqueued in the **`md_change_state`** table (SQLite) —
durable, so a crash mid-sync replays on restart.
3. A worker drains the queue at **entry-level** granularity: it diffs the
file, re-embeds only changed entries (keyed by `content_sha256`), and
upserts the LanceDB rows.
Because markdown is the source of truth, **editing a file directly is
fully supported** — open an episode in VSCode / Obsidian / Vim, change an
entry, save, and the daemon re-indexes just that entry. Operate the queue
with `everos cascade` ([Operating it](#operating-it)); deeper runbook in
[cascade_runbook.md](cascade_runbook.md).
## The Offline Memory Engine (OME)
Most memory kinds are **not** extracted on the request path — they are
derived later by the OME, an in-process async strategy engine. When
extraction carves a MemCell, it emits an event; OME strategies pick it up
and write their markdown when ready:
- `extract_atomic_facts` — single-sentence facts from an episode
- `extract_foresight` — anticipatory notes
- `extract_user_profile` — the aggregated `user.md`
- `extract_agent_case` — a reusable agent trajectory (only when the cell is
substantive enough; thin trajectories are skipped by design)
- `extract_agent_skill` — clusters related cases into a named skill
Strategies are configurable without a code change via **`ome.toml`** at the
memory root (hot-reloaded within ~2 s). Example — turn two off:
```toml
[strategies.extract_foresight]
enabled = false
[strategies.extract_user_profile]
enabled = false
```
OME keeps its own state in `.index/sqlite/ome.db` (run records, counters)
and its scheduler jobstore in `.index/sqlite/ome.aps.db` (split so the sync
APScheduler writer and the async OME writer never contend for one file
lock).
!!! tip "Implication for clients"
After `/flush` returns `extracted`, the **episode** is queryable soon
(once cascade indexes it), but **atomic facts / profile / agent cases**
appear only after their OME strategy runs — typically seconds later.
Poll / retry if you need them immediately.
## Consistency model
Two paths, two guarantees:
| Path | Guarantee | Detail |
|---|---|---|
| **Write** (`/add`, `/flush`) | **strong** | the episode `.md` is on disk before the call returns `extracted`; never blocks on LanceDB |
| **Read** (`/search`, `/get`) | **eventual** | reads LanceDB, which lags md by the cascade processing time — sub-second typically, up to ~10–15 s under load |
So a `/search` immediately after the `/flush` that produced a record may
miss it. The markdown is durable regardless; index lag never loses data. If
you need read-your-write, retry with backoff, or force the queue with
`everos cascade sync`.
Integrity is anchored by a few invariants (details in
[storage_layout.md](storage_layout.md)): the frontmatter `id` /
`entry_id` is the immutable join key; `content_sha256` decides whether an
entry needs re-embedding; an LSN watermark (in `system.db`) orders
rebuilds; the durable `md_change_state` queue is the replayable audit
trail.
## Zero external services
No database server, message broker, or vector service to run. Vector ANN,
full-text BM25, and scalar filtering all execute inside the **embedded
LanceDB** engine in one query; SQLite is a local file. The whole stack is a
single directory you can copy, back up, or check the user-visible parts of
into git.
!!! note
There is no automatic "grep over markdown" search fallback today — if
the LanceDB index is unavailable, rebuild it from markdown (it is
derived and disposable) rather than relying on a degraded search path.
## Operating it
The CLI ([cli.md](cli.md)) is intentionally small:
| Command | What it does |
|---|---|
| `everos init` | write a starter `.env` |
| `everos server start` | run the HTTP API (cascade + OME start with it) |
| `everos cascade status` | queue / LSN summary |
| `everos cascade sync` | drain the cascade queue now (force md → LanceDB) |
| `everos cascade fix` | list failed rows / re-enqueue retryable ones |
!!! warning "There is no `everos reindex` or `everos flush`"
- **Reindex** = the index is rebuildable: stop the server,
`rm -rf <memory-root>/.index/lancedb`, restart — the cascade
rebuilds from markdown. For an incremental catch-up, use
`everos cascade sync`.
- **Flush** is an HTTP endpoint (`POST /api/v1/memory/flush`), not a
CLI command — it forces *extraction* of the session buffer, which is
a different thing from forcing *index sync* (`cascade sync`).
## References
- [storage_layout.md](storage_layout.md) — exact file encoding, frontmatter
chassis, entry-id format, atomic-write semantics
- [architecture.md](architecture.md) — DDD layers and dependency rules
- [api.md](api.md) — the HTTP contract (`/add` `/flush` `/search` `/get`)
- [cascade_runbook.md](cascade_runbook.md) — operating the sync queue

63
docs/index.md Normal file
View File

@ -0,0 +1,63 @@
# EverOS Documentation
Documentation for [EverOS](../README.md) — md-first memory extraction
framework. Organised by [Diátaxis](https://diataxis.fr/) — what kind of
question you have determines which section to read.
## Reference
Technical reference: contracts, commands, schemas — read these when you
already know what you want to do and need to know exactly how.
| Doc | Purpose |
|---|---|
| [api.md](api.md) | HTTP API v1 reference — endpoints, request / response, error contracts |
| [cli.md](cli.md) | `everos` CLI subcommands + env var conventions |
| [storage_layout.md](storage_layout.md) | Memory-root tree + frontmatter chassis + EntryId encoding |
| [prompt_slots.md](prompt_slots.md) | YamlConfigLoader + three-layer prompt override |
## Explanation
Design decisions and architectural concepts — read these to understand
why the system is shaped the way it is.
| Doc | Purpose |
|---|---|
| [overview.md](overview.md) | Project vision, scope, design philosophy |
| [how-memory-works.md](how-memory-works.md) | Storage stack + on-disk paths + write→index→read pipeline + consistency |
| [architecture.md](architecture.md) | DDD layered architecture + dependency rules |
| [datetime.md](datetime.md) | Two-zone discipline — UTC at storage, display tz at boundaries |
## How-to
Task-driven operational guides — read these when you need to do a
specific thing (drain a queue, recover from a stuck row, etc.).
| Doc | Purpose |
|---|---|
| [cascade_runbook.md](cascade_runbook.md) | Cascade subsystem ops — drain queue, recover stuck rows |
## Engineering / Internal
For maintainers and contributors working on the framework itself,
not for using it.
| Doc | Purpose |
|---|---|
| [engineering.md](engineering.md) | Engineering & dev-efficiency infrastructure (CI / tooling / Claude Code) |
## See also
Top-level project files live next to the repo root:
- [README.md](../README.md) — quick start & feature overview
- [QUICKSTART.md](../QUICKSTART.md) — 5-minute walkthrough (install → service → search)
- [CONTRIBUTING.md](../CONTRIBUTING.md) — how to contribute (issue-only model)
- [CHANGELOG.md](../CHANGELOG.md) — release notes
- [SECURITY.md](../SECURITY.md) — security policy & private vulnerability reporting
- [CITATION.md](../CITATION.md) — academic citation info
- [ACKNOWLEDGMENTS.md](../ACKNOWLEDGMENTS.md) — third-party acknowledgments
Coding conventions and slash command workflows are auto-loaded by
Claude Code from [.claude/rules/](../.claude/rules/) and
[.claude/skills/](../.claude/skills/).

126
docs/locomo_benchmark.md Normal file
View File

@ -0,0 +1,126 @@
# Running the LoCoMo Benchmark
This guide walks through reproducing EverOS's LoCoMo retrieval scores
locally using the `hybrid` and `agentic` search methods.
## Contents
- [Prerequisites](#prerequisites)
- [1. Prepare the dataset](#1-prepare-the-dataset)
- [2. Start the server](#2-start-the-server)
- [3. Run `hybrid`](#3-run-hybrid)
- [4. Run `agentic`](#4-run-agentic)
- [5. Where the results land](#5-where-the-results-land)
- [Notes](#notes)
---
## Prerequisites
- Python **3.12**, [uv](https://docs.astral.sh/uv/)
- A `.env` at the repo root with the LLM / embedding credentials EverOS
needs:
- `EVEROS_LLM__MODEL`, `EVEROS_LLM__API_KEY`, `EVEROS_LLM__BASE_URL`
- `EVEROS_EMBEDDING__*`
- `EVEROS_RERANK__*`
- The benchmark driver also reads `LLM_API_KEY` / `ANSWER_MODEL` /
`JUDGE_MODEL` for the answer + judge passes.
Install the project:
```bash
uv sync
```
## 1. Prepare the dataset
Place the LoCoMo file at `data/locomo10.json` (the dataset is
distributed by the LoCoMo authors, not this repo). Override the path
later with `--data-path` if you keep it elsewhere.
## 2. Start the server
```bash
EVEROS_MEMORY__ROOT=~/.everos \
uv run python -m everos.entrypoints.cli.main server start --port 8000
```
`EVEROS_MEMORY__ROOT` isolates one benchmark's corpus from another —
change it (or `rm -rf` it) whenever you want a clean run.
Leave the server running in one terminal; run the benchmark from
another.
## 3. Run `hybrid`
Single conversation:
```bash
bash tests/run_locomo_batch.sh \
--conv-indices 0 \
--methods hybrid \
--base-url http://localhost:8000 \
--top-k 10
```
All 10 conversations, 2-way parallel:
```bash
bash tests/run_locomo_batch.sh \
--conv-indices 0-9 \
--methods hybrid \
--base-url http://localhost:8000 \
--top-k 10 \
--concurrency 2
```
The wrapper picks up `EVEROS_MEMORY__ROOT` from the environment so the
cascade poll path matches the server's data root. If you set them
differently, pass `--corpus-path` explicitly.
## 4. Run `agentic`
Same wrapper, swap `--methods`:
```bash
bash tests/run_locomo_batch.sh \
--conv-indices 0-9 \
--methods agentic \
--base-url http://localhost:8000 \
--top-k 10 \
--concurrency 2
```
You can also benchmark multiple methods in one go — they share the
same ingested corpus:
```bash
bash tests/run_locomo_batch.sh \
--conv-indices 0-9 \
--methods hybrid,agentic \
--base-url http://localhost:8000 \
--top-k 10 \
--concurrency 2
```
## 5. Where the results land
Default output root is `benchmark_results/run_<timestamp>/`. Override
with `--output-root`:
```
<output_root>/
├── conv0.json … conv9.json # per-conv summary + per-question details
├── conv0.log … conv9.log # per-conv stdout (only in --concurrency >1 mode)
└── conv0_checkpoints/ … # incremental search/answer/eval JSON
```
An aggregate accuracy table prints at the end of the wrapper run.
## Notes
- **Re-running on the same corpus**: add `--skip-add` to skip ingest and
reuse what's already in `~/.everos`. Useful when comparing methods
side by side.
- **Judge variance**: `--judge-runs 3` runs the judge three times per
question and majority-votes; slower but reduces LLM-judge noise.

1664
docs/openapi.json Normal file
View File

File diff suppressed because it is too large Load Diff

85
docs/overview.md Normal file
View File

@ -0,0 +1,85 @@
# EverOS — Project Overview
## Vision
Build an open-source Python memory framework where **AI agents' long-term memory is plain Markdown files on the user's disk**, not opaque rows in a hosted database.
## Scope
**In scope (v1)**:
- Local deployment for personal agents or small teams
- Conversation, workflow, agent-trace, file-knowledge → structured memory
- Hybrid retrieval (BM25 + vector + scalar filter)
- Cascade index sync (md edit → LanceDB sub-second)
- Dual-track memory (user-track / agent-track)
- Offline memory evolution (Foresight / AtomicFact / Profile / Skill)
- CLI + HTTP API
**Out of scope (v1, future v2)**:
- Multi-tenant / group / community deployment (10K+ users)
- End-to-cloud sync (planned for v2)
- Distributed deployment / sharding
## Design philosophy
### 1. Markdown as Source of Truth
```
delete all LanceDB / SQLite files → can rebuild from md
delete any md file → memory is gone
```
User trust comes from physical visibility — the user can `cat` / `vim` / `grep` their own memory at any time.
### 2. Three-piece storage with clear job boundaries
| Component | Role | Does NOT do |
|---|---|---|
| Markdown files | Truth source — entries, frontmatter | Search (grep is degraded fallback only) |
| SQLite | Queue, cascade audit log, sensitive data isolation | Vector / full-text |
| LanceDB | Vector ANN + BM25 + scalar filter, single-query hybrid | Be the source of truth (loss = rebuild from md) |
### 3. Algorithm-orchestration separation
[`everalgo`](https://github.com/EverMind-AI/EverAlgo) (a separate library, published as the `everalgo-*` PyPI packages) holds the extraction algorithms (MemCell extraction, Episode generation, Profile evolution). EverOS calls everalgo via the PromptSlot interface; everalgo knows nothing about storage.
This boundary lets the same algorithm power both this open-source lightweight version and other product forms.
### 4. DDD layered architecture
```
entrypoints → service → memory → infra
↓
component / core / config
```
Strict single-direction dependency, enforced by `import-linter` in CI.
## Why src layout (`src/everos/`)
- Standard PyPA project structure used when shipping to PyPI
- Avoid namespace collision with system packages named `memory`, `infra`, etc.
- Avoid accidental import of working-tree code in dev (PyPA recommendation)
## Comparable projects (where EverOS differs)
| Project | Position | Difference |
|---|---|---|
| [mem0](https://github.com/mem0ai/mem0) | API-first memory service | mem0 stores in vector DB; we store in md files |
| [Letta](https://github.com/letta-ai/letta) | Agent OS w/ Core/Recall/Archival | Letta uses Postgres; we use markdown filesystem |
| [MemOS](https://github.com/MemTensor/MemOS) | Multi-classification memory | MemOS targets enterprise; we target lightweight (single-user / small team) |
| [memsearch](https://github.com/zilliztech/memsearch) | md-first search engine | Closest to us; we add memory extraction (not just search) |
## Roadmap
- **v0.1 (MVP)** — Phase 1 core loop: markdown + lancedb + cascade + episode extraction
- **v0.2** — Full extraction pipeline (workspace / agent / knowledge), evolution framework
- **v0.3** — Production hardening, full CLI, HTTP API, Obsidian demo
- **v1.0** — Stable API, PyPI release, comprehensive docs
- **v2** (future) — Edge-to-cloud sync via EverMe (separate project)
## Status
**Alpha — v0.1.0 in active development**. Core API may change before v1.0.

111
docs/prompt_slots.md Normal file
View File

@ -0,0 +1,111 @@
# PromptSlot
PromptSlot is the layer between the algorithm code (`everalgo`) and
the prompts it sends to LLMs. Algorithm code receives a `PromptSlot`
parameter; the *project* (EverOS) supplies defaults and lets operators
override.
> **Status (2026-05-07)**: the YAML loader is implemented; the higher-
> level `PromptSlot` model + sandbox dry-run + three-layer overlay
> resolution arrive when the memory layer ships (see Stage 2).
## Three-layer overlay
```
config/prompt_slots/<name>.yaml (Layer 1: defaults shipped with the package)
↓
~/.everos/prompt_slots/<name>.yaml (Layer 2: app-level override; per-deployment)
↓
runtime override (Layer 3: per-call override; e.g. "force model X")
```
Effective prompt = layer 3 wins → layer 2 → layer 1. Layer 1 is
loaded eagerly at startup; layer 2 is loaded on first reference (lazy);
layer 3 is supplied at the call site.
## Loader
The category loader lives at
[`src/everos/component/config/loader.py`](../src/everos/component/config/loader.py)
as `YamlConfigLoader`:
```python
from pathlib import Path
from everos.component.config import YamlConfigLoader
loader = YamlConfigLoader(
root=Path("src/everos/config"),
categories={"prompt_slots": None}, # subdir == category name
)
# Reads <root>/prompt_slots/episode_extract.yaml → dict
slot = loader.find("prompt_slots", "episode_extract")
# Refresh after on-disk edits.
loader.refresh() # drop the entire cache
loader.refresh("prompt_slots") # drop one category
loader.refresh("prompt_slots", "episode_extract") # drop one entry
```
Top-level YAML is required to be a mapping; a list / scalar root
raises `TypeError` to fail-fast (loud, not silent).
## YAML format (proposed; subject to change)
```yaml
# config/prompt_slots/episode_extract.yaml
template: |
Extract a single episode from this conversation:
{{ memcell.text }}
variables:
memcell: input memcell
output_schema:
type: object
properties:
summary: { type: string }
participants: { type: array }
llm:
model: gpt-4o-mini
temperature: 0.3
max_tokens: 2000
validation:
test_cases:
- input: { memcell: { text: "Hi" } }
expected: { summary: "...", participants: [] }
```
When layer 2 supplies an override the loader will be re-pointed at
`~/.everos/prompt_slots/`; the runtime resolution logic (currently TBD)
sandbox-runs the merged slot before returning it.
## Why YAML (not TOML)
Two reasons:
1. **Multiline templates** — TOML's basic-string grammar fights
prompt content (no easy `{{ jinja }}` variables, awkward escaping).
YAML's literal block scalar (`|`) preserves prompts as-is.
2. **Comment + reference ergonomics** — operators frequently inherit
slots, tweak a few keys, and leave inline notes. YAML is more
forgiving for hand-editing.
The Pydantic Settings file (`config/default.toml`) stays TOML — it's
machine-managed and type-validated; YAML's flexibility costs more
than it pays for that case.
## Why a separate loader (not Pydantic Settings)
Settings = **one** structured tree, validated at load time, tied to a
single source of truth. PromptSlots = **many** separate templates
discovered by name, layered per-deployment. They're different shapes;
forcing one model on the other gets clunky.
## See also
- [`src/everos/component/config/loader.py`](../src/everos/component/config/loader.py)
- [`tests/unit/test_component/test_config/test_loader.py`](../tests/unit/test_component/test_config/test_loader.py)
- [`docs/architecture.md`](architecture.md) — layer placement

222
docs/storage_layout.md Normal file
View File

@ -0,0 +1,222 @@
# Storage Layout
How `everos` lays out a memory-root on disk: directory tree, file
naming, frontmatter chassis, and entry-id encoding.
The contents are the **source of truth**; SQLite and LanceDB are
derived indexes that can be rebuilt from markdown alone.
## 1. Memory-root tree
A memory-root is a single directory holding all persisted memory. The
default location is `~/.everos/`; override via `EVEROS_MEMORY__ROOT`
env var or `[memory] root` in the TOML config.
Memory is partitioned by **`<app_id>/<project_id>`** *before* the
user-visible scope dirs, so different `(app, project)` spaces never share
a directory. The reserved id `"default"` materialises as `default_app` /
`default_project` on disk. The scope is encoded **in the path**, not in
the frontmatter (see [§3](#3-frontmatter-chassis-yaml)).
```
<memory-root>/ default ~/.everos
│
├── <app_id>/ user-visible; "default" → default_app
│ └── <project_id>/ "default" → default_project
│ ├── users/
│ │ └── <user_id>/
│ │ ├── user.md single-file rewrite (profile)
│ │ ├── episodes/ daily-log append
│ │ │ └── episode-<YYYY-MM-DD>.md
│ │ ├── .atomic_facts/ daily-log append (hidden)
│ │ │ └── atomic_fact-<YYYY-MM-DD>.md
│ │ └── .foresights/ daily-log append (hidden)
│ │ └── foresight-<YYYY-MM-DD>.md
│ ├── agents/
│ │ └── <agent_id>/
│ │ ├── .cases/ daily-log append (hidden)
│ │ │ └── agent_case-<YYYY-MM-DD>.md
│ │ └── skills/ skill-named dir
│ │ └── skill_<name>/
│ │ ├── SKILL.md
│ │ ├── references/ (optional)
│ │ └── scripts/ (optional)
│ └── knowledge/ user-visible (shared / global, reserved)
│
├── .index/ system-managed, rebuildable (gitignore)
│ ├── sqlite/
│ │ ├── system.db state / cascade queue (md_change_state) / buffer / audit / LSN (+ -wal / -shm)
│ │ ├── ome.db Offline Memory Engine state
│ │ ├── ome.aps.db APScheduler jobstore (split to avoid lock contention)
│ │ └── ome.db.lock OME single-engine guard (portalocker)
│ └── lancedb/
│ └── <kind>.lance/ one directory per LanceDB table
│
├── ome.toml user-editable OME strategy overrides (hot-reloaded)
└── .tmp/ staging dir for batch / multi-step writes
```
> Cascade queue state, the LSN watermark, and the change audit all live in
> SQLite (`system.db`, table `md_change_state`) — crash-recovery replays
> from that durable queue, not a log file. (`MemoryRoot` also exposes a
> `.lock` anchor for the `memory_root_lock` primitive; there is no
> `.cascade.log` / `.manifest.json`.)
The path manager is [`MemoryRoot`](../src/everos/core/persistence/memory_root.py),
exposing every path as a property. `MemoryRoot.ensure()` creates the
runtime-required dirs (`.index/{sqlite,lancedb}/`, `.tmp/`) and copies the
OME template to `ome.toml`; the user-visible dirs are *not* pre-created —
they appear on first write.
> The single-file writer also supports `agent.md` / `soul.md` / `tools.md`
> / `behaviors.md`, but no shipped strategy produces those today — only
> `user.md` is written. `memcell` is a SQLite-only kind (the boundary
> ledger); it has no markdown file.
## 2. Three storage strategies
Each business memory kind picks one of three on-disk patterns:
| Strategy | Filename | Mutation | Examples |
|---|---|---|---|
| **Daily-log append** | `<FILE_PREFIX>-<YYYY-MM-DD>.md` under `<DIR_NAME>/` | append entries | episode / atomic_fact / foresight / agent_case |
| **Skill-named dir** | `skills/skill_<name>/SKILL.md` (+ `references/` `scripts/`) | overwrite the file | agent skills (procedural memory) |
| **Single-file rewrite** | `user.md` (writer also supports `agent.md` / `soul.md` / `tools.md` / `behaviors.md`, not yet produced) | overwrite the file | user profile |
Markdown IO primitives live in
[`core/persistence/markdown/`](../src/everos/core/persistence/markdown/);
business-aware writers live in
[`infra/persistence/markdown/writers/`](../src/everos/infra/persistence/markdown/writers/)
and pick the right strategy via a base class.
To add a new memory kind, define its per-kind frontmatter schema under
[`infra/persistence/markdown/mds/`](../src/everos/infra/persistence/markdown/mds/)
and add a matching writer/reader pair under
[`writers/`](../src/everos/infra/persistence/markdown/writers/) and
[`readers/`](../src/everos/infra/persistence/markdown/readers/).
## 3. Frontmatter chassis (YAML)
Every markdown file carries a YAML frontmatter block at the top:
```markdown
---
id: episode_log_alice_2026-06-01
type: episode_daily
file_type: episode_daily
schema_version: 1
user_id: alice
track: user
date: '2026-06-01'
entry_count: 11
last_appended_at: '2026-06-01T09:12:13+00:00'
---
<!-- entry:ep_20260601_00000001 -->
...content...
<!-- /entry:ep_20260601_00000001 -->
```
Scope (`app_id` / `project_id`) is **not** a frontmatter field — it is
carried by the `<app>/<project>` path segments and recovered by the
cascade path parser. The frontmatter only holds the file-level owner
(`user_id` / `agent_id`) and `track`.
The chassis lives in [`core/persistence/markdown/frontmatter.py`](../src/everos/core/persistence/markdown/frontmatter.py)
(Pydantic v2):
```
BaseFrontmatter id / type / schema_version + SCOPE_DIR ClassVar
├─ UserScopedFrontmatter + user_id / track="user" + SCOPE_DIR="users"
└─ AgentScopedFrontmatter + agent_id / track="agent" + SCOPE_DIR="agents"
```
Concrete business schemas subclass one of the scope mixins and add
per-kind fields plus three more ClassVars that drive path resolution
+ entry-id assembly:
```python
class EpisodeDailyFrontmatter(DailyLogPathMixin, UserScopedFrontmatter):
ENTRY_ID_PREFIX: ClassVar[str] = "ep"
DIR_NAME: ClassVar[str] = "episodes"
FILE_PREFIX: ClassVar[str] = "episode"
type: Literal["episode_daily"] = "episode_daily"
date: dt.date
entry_count: int = 0
last_appended_at: dt.datetime | None = None
```
## 4. Entry-id encoding
Inside daily-log files each entry is bracketed by HTML-comment markers
so the raw markdown stays clean for human readers:
```
<!-- entry:<entry_id> -->
...content...
<!-- /entry:<entry_id> -->
```
`<entry_id>` is `<prefix>_<YYYYMMDD>_<NNNNNNNN>` (8-digit sequence),
e.g. `ep_20260601_00000001`:
| Segment | Source |
|---|---|
| `prefix` | `Frontmatter.ENTRY_ID_PREFIX` (declared by the schema subclass) |
| `<YYYYMMDD>` | The daily-log file's date bucket |
| `NNNNNNNN` | Per-file sequence, 8-digit zero-padded, restarts at `00000001` each day per scope |
Implementation: [`core/persistence/markdown/entries.py`](../src/everos/core/persistence/markdown/entries.py)
(`EntryId.parse / format / next_for`).
> **File-level seq, not global**: the same `ep_20260601_00000001` may
> appear across two different `user_id`s (each user has its own daily file).
> Cross-table joins must therefore key on **`(scope_id, entry_id)`**
> rather than `entry_id` alone — see SQLite/LanceDB tables that follow.
## 5. SQLite + LanceDB derived indexes
```
.index/
├── sqlite/
│ └── system.db state / audit log / task queue / LSN watermark
│ + per-kind business state tables (composite key)
└── lancedb/
└── <kind>.lance/ one Arrow-based table per kind
stores text / vector / tags / metadata
```
- **SQLite** schema lives in
[`infra/persistence/sqlite/tables/`](../src/everos/infra/persistence/sqlite/tables/);
every business table that joins back to markdown declares a
`UniqueConstraint("user_id", "entry_id")` (or `agent_id` symmetric).
- **LanceDB** schemas live in
[`infra/persistence/lancedb/tables/`](../src/everos/infra/persistence/lancedb/tables/);
`Vector(N)` dimension matches the embedding model output.
Both layers are **fully derivable from markdown** — wipe `.index/`
and the in-process cascade subsystem re-builds everything by scanning the
user-visible tree (the durable `md_change_state` SQLite queue covers
crash-recovery replay).
## 6. Atomic write semantics
`MarkdownWriter` uses a same-directory temp file
(`.<name>.tmp.<uuid>`) + `os.replace` for atomicity. Keeping the temp
file in the same directory guarantees `os.replace` is atomic on POSIX
(the rename is only atomic within a single filesystem).
`MarkdownWriter.append_entry` reads → merges frontmatter →
appends an entry block → atomic write back. The caller passes a full
`EntryId` (built via `EntryId.next_for(prefix, date, current_count)`);
this primitive is **schema-agnostic** — field-level semantics
(`entry_count` / `last_appended_at`) are a business writer's job
(see `BaseDailyAppender._frontmatter_updates` in
[`infra/persistence/markdown/writers/base.py`](../src/everos/infra/persistence/markdown/writers/base.py)).
## 7. References
- Code:
- [`core/persistence/memory_root.py`](../src/everos/core/persistence/memory_root.py) — memory-root resolution
- [`core/persistence/markdown/`](../src/everos/core/persistence/markdown/) — schema-agnostic read/write chassis
- [`infra/persistence/markdown/mds/`](../src/everos/infra/persistence/markdown/mds/) — per-kind frontmatter schemas
- [`infra/persistence/{markdown,sqlite,lancedb}/`](../src/everos/infra/persistence/) — business-aware adapters

221
pyproject.toml Normal file
View File

@ -0,0 +1,221 @@
[project]
name = "everos"
version = "1.0.0"
description = "EverOS — local-first markdown memory framework for AI agents and user chats; lightweight, dev-friendly, small-team"
license = {text = "Apache-2.0"}
readme = "README.md"
requires-python = ">=3.12"
authors = [
{name = "EverMind AI"},
]
keywords = ["memory", "ai-agent", "markdown", "lancedb", "rag", "everos"]
classifiers = [
"Development Status :: 3 - Alpha",
"Intended Audience :: Developers",
"License :: OSI Approved :: Apache Software License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.12",
"Topic :: Scientific/Engineering :: Artificial Intelligence",
]
dependencies = [
# Core data validation & config
"pydantic>=2.7.1",
"pydantic-settings>=2.0.0",
"python-dotenv>=1.0.1",
# Storage stack (md-first three-piece set)
"lancedb>=0.13.0", # Vector + BM25 + scalar filter (Arrow-based)
"aiosqlite>=0.20.0", # Async SQLite driver (used by SA async engine)
"sqlmodel>=0.0.22", # ORM (Pydantic + SQLAlchemy 2.0 async)
"alembic>=1.13.0", # SQLite schema migrations
"greenlet>=3.0", # Required by SQLAlchemy async
# LLM & embedding (one provider per file pattern)
"openai>=1.0.0",
# Markdown / file system
"PyYAML>=6.0", # YAML frontmatter parsing
"watchdog>=4.0.0", # Cross-platform file system events (cascade daemon)
# Web framework (entrypoints/api)
"fastapi>=0.104.0",
"uvicorn[standard]>=0.24.0",
# Observability (core/observability)
"structlog>=24.0.0",
"prometheus-client>=0.20.0",
# CLI
"typer>=0.12.0",
# Tokenization (BM25 Chinese support)
"jieba==0.42.1",
# OME (Offline Memory Evolution) async scheduler & file I/O
"apscheduler>=3.10.4,<4.0", # async strategy scheduler chassis
"portalocker>=2.8.2", # cross-platform file lock for single-engine guard
"watchfiles>=0.21.0", # native fs watcher for config hot reload
"anyio>=4.0", # Async file I/O (anyio.Path, to_thread.run_sync) for the markdown layer
# Algorithm library (everalgo monorepo, published on PyPI).
"everalgo-core==0.2.0",
"everalgo-boundary==0.2.0",
"everalgo-user-memory==0.2.0",
"everalgo-agent-memory==0.2.0",
"everalgo-rank==0.3.0",
]
[project.optional-dependencies]
multimodal = ["everalgo-parser[svg]>=0.1.0"] # [svg] bundles cairosvg → SVG works by default
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["src/everos"]
# ``env.template`` lives at src/everos/templates/env.template and is
# auto-included via ``packages``. ``everos init`` reads it through
# ``importlib.resources``. No force-include needed.
# sdist uses an include-whitelist (not exclude) so we don't accidentally ship
# things like CLAUDE.md, .claude/, CI configs, or IDE settings. New top-level
# files default to NOT-shipped — you have to opt them in here.
#
# exclude is kept as belt-and-suspenders for build artefacts that CI may
# generate inside the project tree (e.g. UV_CACHE_DIR=.uv-cache).
[tool.hatch.build.targets.sdist]
include = [
"/src",
"/tests",
"/data",
"/docs",
"/pyproject.toml",
"/uv.lock",
"/README.md",
"/QUICKSTART.md",
"/CHANGELOG.md",
"/CONTRIBUTING.md",
"/CODE_OF_CONDUCT.md",
"/SECURITY.md",
"/ACKNOWLEDGMENTS.md",
"/CITATION.md",
"/LICENSE",
"/NOTICE",
"/config.example.toml",
]
exclude = [".uv-cache", ".venv"]
[project.scripts]
everos = "everos.entrypoints.cli.main:app"
# ───────────────────────────────────────────────────────────────────────────────
# Tooling
# ───────────────────────────────────────────────────────────────────────────────
[tool.ruff]
line-length = 88
target-version = "py312"
extend-exclude = ["src_old"]
[tool.ruff.lint]
select = ["E", "F", "I", "N", "UP", "B", "SIM", "ASYNC"]
[tool.ruff.lint.per-file-ignores]
# LoCoMo benchmark embeds LLM prompts (ANSWER_PROMPT / JUDGE_*_PROMPT) and
# verbose argparse help strings as multi-line string literals. Line breaks
# would change what the LLM actually sees or noise the --help output, so
# E501 is silenced for this file only.
"tests/test_locomo.py" = ["E501"]
[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py"]
python_functions = ["test_*"]
asyncio_mode = "auto"
addopts = "-v --tb=short -m 'not slow and not live_llm'"
markers = [
"slow: runs that take >=10s (regardless of dependency); CI default excludes these. Run with `pytest -m slow`.",
"live_llm: requires real LLM / embedder / reranker credentials from .env; burns tokens. CI default excludes. Run with `pytest -m live_llm`.",
]
[tool.coverage.run]
source = ["src/everos"]
branch = true
omit = ["**/__init__.py"]
[tool.importlinter]
root_packages = ["everos"]
[[tool.importlinter.contracts]]
name = "Layered architecture"
type = "layers"
layers = [
"everos.entrypoints",
"everos.service",
"everos.memory",
"everos.infra",
]
[[tool.importlinter.contracts]]
name = "Subpackage internals are private"
type = "forbidden"
source_modules = ["everos.service", "everos.memory", "everos.entrypoints"]
# `**` matches any descendant (writer.py, reader.py, sub.foo.bar, ...).
# Public API must be reached via the subpackage `__init__.py`.
forbidden_modules = [
"everos.infra.persistence.markdown.**",
"everos.infra.persistence.lancedb.**",
"everos.infra.persistence.sqlite.**",
]
# `forbidden` contracts run a *transitive closure* — any path from a
# source module to a forbidden module fails the check, even when the
# source only imports the subpackage top-level. The whitelist below
# carves out **intra-subpackage wiring**: each storage subpackage's
# own ``__init__.py`` imports its own ``tables`` / ``repos`` / manager
# children to assemble the public API surface that outer layers reach.
# These edges are normal Python package wiring, *not* a privacy
# breach — without these ignores, even a legitimate
# ``from everos.infra.persistence.sqlite import X`` would be punished
# because the transitive scan walks one hop further into the package.
#
# Direct imports from outer layers into a subpackage internal module
# (e.g. ``from everos.infra.persistence.sqlite.tables import X`` written
# in ``service`` or ``entrypoints``) remain blocked — that edge is not
# in this whitelist.
ignore_imports = [
"everos.infra.persistence.sqlite -> everos.infra.persistence.sqlite.tables",
"everos.infra.persistence.sqlite -> everos.infra.persistence.sqlite.repos",
"everos.infra.persistence.sqlite -> everos.infra.persistence.sqlite.sqlite_manager",
"everos.infra.persistence.lancedb -> everos.infra.persistence.lancedb.tables",
"everos.infra.persistence.lancedb -> everos.infra.persistence.lancedb.repos",
"everos.infra.persistence.lancedb -> everos.infra.persistence.lancedb.lancedb_manager",
"everos.infra.persistence.markdown -> everos.infra.persistence.markdown.mds",
"everos.infra.persistence.markdown -> everos.infra.persistence.markdown.writers",
"everos.infra.persistence.markdown -> everos.infra.persistence.markdown.readers",
]
[[tool.importlinter.contracts]]
name = "OME does not depend on memory/service/entrypoints or sibling infra subpackages"
type = "forbidden"
source_modules = ["everos.infra.ome"]
forbidden_modules = [
"everos.infra.persistence",
"everos.memory",
"everos.service",
"everos.entrypoints",
]
[dependency-groups]
dev = [
"ruff>=0.8.0",
"pytest>=8.4.0",
"pytest-asyncio>=1.1.0",
"pytest-cov>=6.0.0",
"pytest-rerunfailures>=15.0",
"import-linter>=2.0",
"pre-commit>=4.0.0",
"ipdb>=0.13.13",
"pyinstrument>=5.0.0",
]

93
scripts/check_cjk.py Normal file
View File

@ -0,0 +1,93 @@
#!/usr/bin/env python3
"""Scan tracked text files for CJK characters outside the language-policy allowlist.
Replaces the grep-based reference command that used to live in
``.claude/rules/language-policy.md``. That command silently produced false
negatives on this repo: the ``grep -zZv`` + ``xargs -0`` NUL pipeline
mis-parsed the path list and reported "clean" even when violations existed.
Exit code 0 = clean, 1 = violations found (paths + line numbers printed).
Usage:
python scripts/check_cjk.py # scan all tracked files
python scripts/check_cjk.py a.py b.md # scan specific files (pre-commit)
python scripts/check_cjk.py --quiet # per-file counts only
"""
from __future__ import annotations
import argparse
import os
import re
import subprocess
# CJK / fullwidth code points: CJK symbols & ideographs, Hangul syllables,
# and halfwidth/fullwidth forms. Kept as escapes so this file stays ASCII.
_CJK = re.compile("[\\u3000-\\u9fff\\uac00-\\ud7af\\uff00-\\uffef]")
def _is_allowlisted(path: str) -> bool:
"""Return True if CJK is permitted in this path (see language-policy.md)."""
name = os.path.basename(path)
# 1. Tests: fixtures, sample inputs, and CJK-behavior assertions.
if path.startswith("tests/"):
return True
# 2. Tokenizer NLP resources (stopword lists, segmentation examples).
if path.startswith("src/everos/component/tokenizer/"):
return True
# 3. Locale-suffixed sample data, e.g. data/solo_chat_zh.json.
if re.match(r"data/.*_(zh|ja|ko)\.", path):
return True
# 4. Translated doc mirrors, e.g. README.zh.md.
if re.search(r"\.(zh|ja|ko)\.md$", path):
return True
# 5. Filenames explicitly marked with a CJK/locale token.
return bool(re.search(r"(^|[._-])(cjk|zh|ja|ko)([._-]|$)", name))
def _tracked_files() -> list[str]:
out = subprocess.check_output(["git", "ls-files"], text=True)
return out.splitlines()
def main() -> int:
parser = argparse.ArgumentParser(description="CJK language-policy scanner.")
parser.add_argument("files", nargs="*", help="files to scan (default: all tracked)")
parser.add_argument("--quiet", action="store_true", help="per-file counts only")
args = parser.parse_args()
paths = args.files or _tracked_files()
violations: list[tuple[str, int, str]] = []
for path in paths:
if _is_allowlisted(path):
continue
try:
with open(path, encoding="utf-8") as fh:
lines = fh.readlines()
except (UnicodeDecodeError, FileNotFoundError, IsADirectoryError):
continue # binary / missing / directory: nothing to scan
for i, line in enumerate(lines, start=1):
if _CJK.search(line):
violations.append((path, i, line.strip()))
if not violations:
print("CJK language-policy: clean")
return 0
by_file: dict[str, int] = {}
for path, _lineno, _text in violations:
by_file[path] = by_file.get(path, 0) + 1
print(f"CJK language-policy: {len(violations)} hit(s) in {len(by_file)} file(s)\n")
if args.quiet:
for path, count in sorted(by_file.items(), key=lambda kv: -kv[1]):
print(f" {count:4d} {path}")
else:
for path, lineno, text in violations:
print(f" {path}:{lineno}: {text[:100]}")
print("\nAllowed CJK locations are defined in .claude/rules/language-policy.md")
return 1
if __name__ == "__main__":
raise SystemExit(main())

350
scripts/check_consistency.py Executable file
View File

@ -0,0 +1,350 @@
#!/usr/bin/env python
"""Check md ↔ LanceDB consistency for an everos corpus.
Three checks per kind:
1. id set equality — md entry ids == LanceDB row entry_ids
2. content_sha256 equality — every shared id matches on both sides
3. id monotonicity (md-only) — within each daily-log md, the numeric
counter at the end of entry.id ascends
from 1 with no gap and no dupe
Two modes:
--mode lifespan (default) Full strict check through the everos app
lifespan stack (sqlite + lance + cascade +
ome). Safe ONLY on an idle corpus (no live
server writing). Covers every kind in
KIND_REGISTRY.
--mode readonly Bypass the lifespan stack, open LanceDB with
a fresh read connection, read md directly.
Safe even on an active corpus, but only
covers the three daily-log kinds (episode /
atomic_fact / foresight).
Examples:
scripts/check_consistency.py ~/.everos-locomo-all-kv-fast
scripts/check_consistency.py ~/.everos-locomo-all-kv-fast --mode readonly
scripts/check_consistency.py ~/.everos-locomo-all-kv-fast --owners joanna,nate
"""
# This script must mutate sys.path before importing everos/tests, and
# uses synchronous pathlib because it's a one-shot CLI, not server code.
# ruff: noqa: E402, ASYNC240
from __future__ import annotations
import argparse
import asyncio
import dataclasses
import os
import re
import sys
from collections import Counter
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(ROOT))
sys.path.insert(0, str(ROOT / "src"))
from dotenv import load_dotenv
load_dotenv(ROOT / ".env")
# ── shared: id counter parsing ──────────────────────────────────────────
_ID_NUM_RE = re.compile(r"_(\d+)$")
def _entry_counter(entry_id: str) -> int | None:
m = _ID_NUM_RE.search(entry_id)
return int(m.group(1)) if m else None
@dataclasses.dataclass
class MonotonicityReport:
path: str
total: int
not_sorted: bool
starts_at_1: bool
gaps: list[int]
dupes: list[int]
bad_format: list[str]
@property
def ok(self) -> bool:
return self.total == 0 or (
not self.not_sorted
and self.starts_at_1
and not self.gaps
and not self.dupes
and not self.bad_format
)
async def _scan_monotonicity(corpus: Path) -> list[MonotonicityReport]:
"""Walk all daily-log md files; report id-counter monotonicity per file."""
from everos.core.persistence import MarkdownReader
daily_dirs = ("/episodes/", "/.atomic_facts/", "/.foresights/", "/.agent_cases/")
reports: list[MonotonicityReport] = []
for md in sorted(corpus.rglob("*.md")):
rel = md.relative_to(corpus).as_posix()
if not (rel.startswith("users/") or rel.startswith("agents/")):
continue
if not any(d in "/" + rel for d in daily_dirs):
continue
parsed = await MarkdownReader.read(md)
counters: list[int] = []
bad_format: list[str] = []
for entry in parsed.entries:
c = _entry_counter(entry.id)
if c is None:
bad_format.append(entry.id)
else:
counters.append(c)
not_sorted = counters != sorted(counters)
starts_at_1 = bool(counters) and min(counters) == 1
gaps: list[int] = []
dupes: list[int] = []
if counters:
seen = set(counters)
for i in range(1, max(counters) + 1):
if i not in seen:
gaps.append(i)
cc = Counter(counters)
dupes = sorted(v for v, n in cc.items() if n > 1)
reports.append(
MonotonicityReport(
path=rel,
total=len(parsed.entries),
not_sorted=not_sorted,
starts_at_1=starts_at_1 if parsed.entries else True,
gaps=gaps,
dupes=dupes,
bad_format=bad_format,
)
)
return reports
def _print_monotonicity(reports: list[MonotonicityReport]) -> int:
issues = sum(1 for r in reports if not r.ok)
if issues == 0:
print(
f" all {len(reports)} daily-log md files have strictly ascending"
" ids from 1"
)
return 0
print(f" ⚠ {issues}/{len(reports)} md files have id-counter issues:")
for r in reports:
if r.ok:
continue
problems = []
if r.not_sorted:
problems.append("not-sorted")
if not r.starts_at_1 and r.total:
problems.append("not-from-1")
if r.gaps:
preview = r.gaps[:5]
problems.append(f"gaps={preview}{'...' if len(r.gaps) > 5 else ''}")
if r.dupes:
problems.append(f"dupes={r.dupes}")
if r.bad_format:
problems.append(f"bad-format×{len(r.bad_format)}")
print(f" {r.path}: total={r.total} {' '.join(problems)}")
return issues
# ── mode: lifespan ──────────────────────────────────────────────────────
async def run_lifespan_mode(corpus: Path) -> int:
"""Full strict check via app lifespan; covers every kind in KIND_REGISTRY."""
os.environ["EVEROS_MEMORY__ROOT"] = str(corpus)
from everos.config import load_settings
load_settings.cache_clear()
from everos.entrypoints.api.app import create_app
from tests._consistency_assertions import assert_md_lance_strict_consistent
app = create_app()
rc = 0
async with app.router.lifespan_context(app):
# 1+2. id set + sha
print("─── md ↔ LanceDB strict consistency ───")
try:
stats = await assert_md_lance_strict_consistent(corpus)
print(" PASS")
except AssertionError as e:
print(f" DRIFT:\n{e}")
rc = 1
stats = None
if stats is not None:
print()
print(
f" {'kind':<15s} {'md_files':>10s}"
f" {'md_entries':>12s} {'lance_rows':>12s}"
)
print(" " + "─" * 53)
for kind, s in stats.items():
print(
f" {kind:<15s} {s.md_file_count:>10d}"
f" {s.md_entry_count:>12d} {s.lance_row_count:>12d}"
)
# 3. id monotonicity
print()
print("─── id monotonicity ───")
reports = await _scan_monotonicity(corpus)
if _print_monotonicity(reports) > 0:
rc = max(rc, 2)
return rc
# ── mode: readonly ──────────────────────────────────────────────────────
async def run_readonly_mode(corpus: Path, owners_filter: list[str] | None) -> int:
"""Direct LanceDB read + md read; no lifespan / cascade / ome started.
Covers the three daily-log kinds; agent_case + user_profile + agent_skill
are NOT checked in this mode (use --mode lifespan on an idle corpus
snapshot for full coverage).
"""
import lancedb
from everos.core.persistence import MarkdownReader
from everos.memory.cascade.handlers.atomic_fact import AtomicFactHandler
from everos.memory.cascade.handlers.episode import EpisodeHandler
from everos.memory.cascade.handlers.foresight import ForesightHandler
from tests._consistency_assertions import _daily_log_sha_for_entry
db = lancedb.connect(str(corpus / ".index" / "lancedb"))
kinds = [
("episode", "episodes", "episode-", EpisodeHandler),
("atomic_fact", ".atomic_facts", "atomic_fact-", AtomicFactHandler),
("foresight", ".foresights", "foresight-", ForesightHandler),
]
# Pick owners
if owners_filter:
owners = owners_filter
else:
owners = (
sorted(p.name for p in (corpus / "users").iterdir() if p.is_dir())
if (corpus / "users").exists()
else []
)
print("─── md ↔ LanceDB consistency (readonly) ───")
rc = 0
for table_name, dir_name, prefix, handler_cls in kinds:
try:
table = db.open_table(table_name)
except FileNotFoundError:
print(f" {table_name}: table not in lancedb (skip)")
continue
for owner in owners:
md_dir = corpus / "users" / owner / dir_name
if not md_dir.exists():
continue
md_files = sorted(md_dir.glob(f"{prefix}*.md"))
md_sha_total: dict[str, str] = {}
for md in md_files:
parsed = await MarkdownReader.read(md)
for entry in parsed.entries:
md_sha_total[entry.id] = _daily_log_sha_for_entry(
handler_cls, entry.as_structured()
)
arr = (
table.search().where(f"owner_id = '{owner}'").limit(100_000).to_arrow()
)
lance_sha = dict(
zip(
arr["entry_id"].to_pylist(),
arr["content_sha256"].to_pylist(),
strict=True,
)
)
only_md = sorted(set(md_sha_total) - set(lance_sha))
only_lance = sorted(set(lance_sha) - set(md_sha_total))
sha_mismatch = sorted(
k
for k in set(md_sha_total) & set(lance_sha)
if md_sha_total[k] != lance_sha[k]
)
ok = not (only_md or only_lance or sha_mismatch)
status = "OK" if ok else "DRIFT"
if not ok:
rc = 1
print(
f" {table_name:<12s} owner={owner:<12s}"
f" md={len(md_sha_total):5d} lance={len(lance_sha):5d}"
f" {status}"
)
if only_md:
print(f" only_in_md (first 5): {only_md[:5]}")
if only_lance:
print(f" only_in_lance (first 5): {only_lance[:5]}")
if sha_mismatch:
print(f" sha_mismatch (first 5): {sha_mismatch[:5]}")
# id monotonicity (md-only, owner-filtered if provided)
print()
print("─── id monotonicity ───")
reports = await _scan_monotonicity(corpus)
if owners_filter:
owner_paths = tuple(f"users/{o}/" for o in owners_filter)
reports = [r for r in reports if any(r.path.startswith(p) for p in owner_paths)]
if _print_monotonicity(reports) > 0:
rc = max(rc, 2)
return rc
# ── main ────────────────────────────────────────────────────────────────
def _parse_args() -> argparse.Namespace:
p = argparse.ArgumentParser(
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
)
p.add_argument("corpus", help="memory root (e.g. ~/.everos-locomo-all-kv-fast)")
p.add_argument(
"--mode",
choices=("lifespan", "readonly"),
default="lifespan",
help="lifespan = full strict check (idle corpus only); "
"readonly = direct lance read (safe on active corpus)",
)
p.add_argument(
"--owners",
help="comma-separated owner filter (readonly mode only)",
)
return p.parse_args()
async def main() -> int:
args = _parse_args()
corpus = Path(args.corpus).expanduser().resolve()
if not corpus.exists():
print(f"ERROR: corpus does not exist: {corpus}")
return 1
owners = (
[o.strip() for o in args.owners.split(",") if o.strip()]
if args.owners
else None
)
print(f"corpus: {corpus}")
print(f"mode: {args.mode}")
if owners:
print(f"owners: {owners}")
print()
if args.mode == "lifespan":
return await run_lifespan_mode(corpus)
return await run_readonly_mode(corpus, owners)
if __name__ == "__main__":
sys.exit(asyncio.run(main()))

138
scripts/check_datetime_discipline.py Normal file
View File

@ -0,0 +1,138 @@
"""CI gate: enforce the two-zone discipline at the source-code level.
Scans `src/` for code patterns that bypass
:mod:`everos.component.utils.datetime` and would silently introduce
naive or local-tz datetimes. Exits non-zero on any hit.
Forbidden patterns (with a small allowlist):
1. ``datetime.now()`` / ``datetime.utcnow()`` / ``datetime.today()`` —
naive constructors / deprecated. Use ``get_utc_now()`` (storage) or
``get_now_with_timezone()`` (display).
2. ``time.time()`` / ``time.time_ns()`` — bypasses the helper module.
Use ``to_timestamp_ms(get_utc_now())`` if you really need ms epoch.
3. Direct ``datetime(YYYY, ...)`` constructor without ``tzinfo=`` —
produces naive datetimes; use ``ensure_utc(datetime(...))`` instead.
4. ``.astimezone(`` / ``.replace(tzinfo=`` outside the helper module —
should go through ``to_display_tz`` / ``ensure_utc``.
Allowlist (legitimate uses):
* ``src/everos/component/utils/datetime.py`` — the helper module itself.
* ``src/everos/core/persistence/sqlite/base.py`` — the SQLAlchemy ``load``
event listener that re-attaches UTC on hydrate.
Run::
python scripts/check_datetime_discipline.py
Wired into ``make ci``; any violation fails the build.
"""
from __future__ import annotations
import re
import sys
from pathlib import Path
_ROOT = Path(__file__).resolve().parent.parent
_SRC = _ROOT / "src"
_ALLOWLIST: set[Path] = {
_ROOT / "src/everos/component/utils/datetime.py",
_ROOT / "src/everos/core/persistence/sqlite/base.py",
}
# (regex, message) pairs. Each regex must match on a single line.
_HELPER_HINT = "use get_utc_now() / get_now_with_timezone()"
_PATTERNS: list[tuple[re.Pattern[str], str]] = [
(
re.compile(r"\bdatetime\.now\s*\(\s*\)"),
f"datetime.now() returns naive — {_HELPER_HINT}",
),
(
re.compile(r"\bdatetime\.utcnow\s*\("),
"datetime.utcnow() is deprecated and naive — use get_utc_now()",
),
(
re.compile(r"\bdatetime\.today\s*\("),
"datetime.today() returns naive — use today_with_timezone()",
),
(
re.compile(r"\bdt\.datetime\.now\s*\(\s*\)"),
f"dt.datetime.now() returns naive — {_HELPER_HINT}",
),
(
re.compile(r"\bdt\.datetime\.utcnow\s*\("),
"dt.datetime.utcnow() is deprecated and naive — use get_utc_now()",
),
(
re.compile(r"\b_dt\.datetime\.now\s*\(\s*\)"),
f"_dt.datetime.now() returns naive — {_HELPER_HINT}",
),
(
re.compile(r"\btime\.time(?:_ns)?\s*\("),
"time.time() bypasses the helper — use to_timestamp_ms(get_utc_now())",
),
(
re.compile(r"\.astimezone\s*\("),
".astimezone(...) outside helper — use to_display_tz() / ensure_utc()",
),
(
re.compile(r"\.replace\s*\(\s*tzinfo\s*="),
".replace(tzinfo=...) outside helper — use ensure_utc() / to_display_tz()",
),
]
# Skip lines that match these (comments, docstrings, `# tz-noqa`).
_COMMENT_RE = re.compile(r"^\s*#")
_DOCSTRING_TRIPLE = '"""'
def _scan_file(path: Path) -> list[tuple[int, str, str]]:
"""Return list of (line_no, line, message) violations in *path*."""
if path in _ALLOWLIST:
return []
hits: list[tuple[int, str, str]] = []
try:
text = path.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError):
return []
# Strip out triple-quoted blocks (docstrings + multi-line literals).
text_no_docstrings = re.sub(r'""".*?"""', "", text, flags=re.DOTALL)
text_no_docstrings = re.sub(r"'''.*?'''", "", text_no_docstrings, flags=re.DOTALL)
for lineno, line in enumerate(text_no_docstrings.splitlines(), start=1):
if _COMMENT_RE.match(line):
continue
if "# tz-noqa" in line:
continue
# Strip inline trailing comment to avoid false positives in
# comment text like ``# replace(tzinfo=...) — explanation``.
code_part = line.split("#", 1)[0]
for pat, msg in _PATTERNS:
if pat.search(code_part):
hits.append((lineno, line.rstrip(), msg))
break
return hits
def main() -> int:
rc = 0
for py in sorted(_SRC.rglob("*.py")):
violations = _scan_file(py)
if not violations:
continue
rel = py.relative_to(_ROOT)
for lineno, line, msg in violations:
print(f"{rel}:{lineno}: {msg}")
print(f" {line}")
rc = 1
if rc == 0:
print("OK — datetime discipline clean.")
return rc
if __name__ == "__main__":
sys.exit(main())

128
scripts/dump_openapi.py Normal file
View File

@ -0,0 +1,128 @@
"""Dump the FastAPI OpenAPI schema to ``docs/openapi.json``.
Static export — does **not** start the server. Calls ``app.openapi()``
directly on the FastAPI instance returned by ``create_app()``, which
the runtime ``GET /openapi.json`` handler returns verbatim. No lifespan
is run, so this is fast and side-effect-free.
Modes:
* default — write ``docs/openapi.json``.
* ``--check`` — write to a temp file and ``diff`` against the on-disk
copy. Exits non-zero on drift, so it can be wired into ``make lint``
to fail PRs that touch the API surface without regenerating the
committed schema. Same shape as ``check_datetime_discipline.py``.
Run::
python scripts/dump_openapi.py # write docs/openapi.json
python scripts/dump_openapi.py --check # CI gate
"""
from __future__ import annotations
import argparse
import json
import sys
import tempfile
from pathlib import Path
_ROOT = Path(__file__).resolve().parent.parent
_TARGET = _ROOT / "docs" / "openapi.json"
def _build_schema() -> dict:
"""Return the FastAPI app's full OpenAPI schema.
Force ``ENV=DEV`` so the ``openapi_url`` route is enabled — without
it the dev-mode endpoint check (see ``app.py``) shadows the route.
The schema content itself is identical in dev vs prod; the flag only
controls whether the runtime ``GET /openapi.json`` is exposed. We
flip it here so the static export matches the dev-mode endpoint
output the e2e test compares against.
"""
import os
os.environ["ENV"] = "DEV"
# Local import so an import-time evaluation of ``ENV`` (read inside
# ``create_app``) sees the override above.
from everos.entrypoints.api.app import create_app
# Pass an empty lifespan list so we don't pull up SQLite / LanceDB /
# OME — the schema is computed from static route declarations alone.
app = create_app(lifespan_providers=[])
return app.openapi()
def _render(schema: dict) -> str:
"""Pretty-print the schema as JSON with stable key order + trailing newline."""
return json.dumps(schema, indent=2, ensure_ascii=False, sort_keys=False) + "\n"
def _write_target(content: str) -> None:
_TARGET.parent.mkdir(parents=True, exist_ok=True)
_TARGET.write_text(content, encoding="utf-8")
def _check_against_target(content: str) -> int:
if not _TARGET.is_file():
print(
f"error: {_TARGET.relative_to(_ROOT)} does not exist; "
f"run `make openapi` to generate it.",
file=sys.stderr,
)
return 1
existing = _TARGET.read_text(encoding="utf-8")
if existing == content:
print(f"OK — {_TARGET.relative_to(_ROOT)} matches app.openapi() output.")
return 0
# Drift: print a unified diff to stderr so CI / reviewer can see what changed.
import difflib
diff = "".join(
difflib.unified_diff(
existing.splitlines(keepends=True),
content.splitlines(keepends=True),
fromfile=f"{_TARGET.relative_to(_ROOT)} (committed)",
tofile="app.openapi() (current)",
)
)
# Limit to first ~200 lines so a giant schema rewrite stays scannable.
capped = "".join(diff.splitlines(keepends=True)[:200])
print(
f"error: {_TARGET.relative_to(_ROOT)} is out of date.\n"
"Run `make openapi` and commit the result.\n\n" + capped,
file=sys.stderr,
)
if len(diff.splitlines()) > 200:
print(
f"... (truncated; full diff is {len(diff.splitlines())} lines)",
file=sys.stderr,
)
return 1
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument(
"--check",
action="store_true",
help="Compare against docs/openapi.json without writing; exit 1 on drift.",
)
args = parser.parse_args(argv)
schema = _build_schema()
content = _render(schema)
if args.check:
return _check_against_target(content)
_write_target(content)
print(f"wrote {_TARGET.relative_to(_ROOT)} ({len(content)} bytes)")
return 0
if __name__ == "__main__":
# Silence the unused-import warning on tempfile (kept for future use).
_ = tempfile
sys.exit(main())

123
scripts/e2e_memorize/README.md Normal file
View File

@ -0,0 +1,123 @@
# End-to-end memorize test
In-process driver that pushes a realistic fixture through `service.memorize`,
batching by 6 messages per `/add` call and then `/flush` at the end.
## What's here
| File | Purpose |
|---|---|
| `fixtures/chat_session.json` | 22 messages · 3 topic shifts · multi-user (Alice → Bob) — chat-mode fixture |
| `fixtures/agent_session.json` | 21 items · 2 task threads · interleaved `tool_calls` / `tool` results — agent-mode fixture |
| `run.py` | In-process runner (no HTTP) |
## Prereqs
1. **LLM client configured** in `.env`:
- `EVEROS_LLM__API_KEY=...`
- `EVEROS_LLM__BASE_URL=...` (OpenAI-compatible)
- `EVEROS_LLM__MODEL=...` (defaults to `gpt-4o-mini`)
- Without these, the boundary stage logs `memorize_no_llm_client` and skips the run.
2. **Memory root**: defaults to `~/.everos`; override with `EVEROS_MEMORY__ROOT=...`.
3. **Mode** is read from `settings.memorize.mode` (toml/env) before the first `memorize()` call.
## Run
```bash
# Chat mode — boundary uses everalgo.boundary.detect_boundaries
EVEROS_MEMORIZE__MODE=chat uv run python scripts/e2e_memorize/run.py \
--fixture scripts/e2e_memorize/fixtures/chat_session.json \
--expected-mode chat
# Agent mode — boundary uses everalgo.agent_memory.AgentBoundaryDetector
# (filter→detect→remap; tool items preserved in cells)
EVEROS_MEMORIZE__MODE=agent uv run python scripts/e2e_memorize/run.py \
--fixture scripts/e2e_memorize/fixtures/agent_session.json \
--expected-mode agent
# Dry run (print batch plan, no LLM calls)
uv run python scripts/e2e_memorize/run.py \
--fixture scripts/e2e_memorize/fixtures/chat_session.json --dry-run
```
## What to verify after a run
### 1. Console output
Each batch prints `status=` (`accumulated` while buffering, `extracted` when
cells got cut). Final `flush` should be `extracted` if any cell remained
in the tail. The trailing file walker lists md / sqlite files modified
in the last 10 minutes.
### 2. Episode md (sync — 4A)
```
~/.everos/users/<owner_id>/episodes/episode-YYYY-MM-DD.md
```
- Chat fixture: 2 owners (`u_alice`, `u_bob`) — expect Episodes split into
~3-4 cells aligned with topic shifts (Python bug → weekend ramen → Q3
review → SRE handoff/ramen wrap).
- Agent fixture: 1 user (`u_alice`) — expect ~2 Episodes aligned with the
two task threads (latency rollback → DB index fix).
### 3. SQLite memcell rows
```bash
sqlite3 ~/.everos/.index/sqlite/system.db \
"select memcell_id, track, owner_id, owner_type, json_array_length(sender_ids_json) as senders
from memcell order by timestamp"
```
- Chat run: rows with `track=user_memory`, `owner_type=user`.
- Agent run: parallel rows for both tracks (`user_memory` **and**
`agent_memory`) since agent mode dispatches both pipelines.
### 4. Unprocessed buffer
```bash
sqlite3 ~/.everos/.index/sqlite/system.db \
"select session_id, count(*) from unprocessed_buffer
where track='memorize' group by session_id"
```
After `flush` the buffer should be empty for the test session.
### 5. OME async output (only if subscribers exist)
- `users/<owner>/atomic_facts/atomic_fact-YYYY-MM-DD.md` (always; `extract_atomic_facts` is registered)
- `users/<owner>/foresights/foresight-YYYY-MM-DD.md` (always; `extract_foresight` is registered)
- `agents/<agent>/agent_cases/agent_case-YYYY-MM-DD.md` (**only after `extract_agent_cases` strategy is written + registered** — currently absent, the emit is a no-op)
### 6. Reset between runs
The fixture's session_id is randomised per invocation, so previous runs
don't pollute the new one. To wipe everything:
```bash
rm -rf ~/.everos/users ~/.everos/agents ~/.everos/.index/sqlite/system.db
```
## Boundary expectations cheat sheet
### Chat fixture topic shifts (timestamps ms)
| Range | Topic |
|---|---|
| msgs 1-6 (`1747396800–1747397010`) | Python KeyError debugging |
| msgs 7-12 (`1747400400–1747400610`) | Weekend ramen plans |
| msgs 13-16 (`1747407600–1747407720`) | Q3 revenue review meeting prep |
| msgs 17-22 (`1747411200–1747411410`) | Bob joins, SRE handoff + ramen + Q3 deck deadline |
Boundary detector should cut on topic gaps; 3 cuts → 4 cells is the most likely outcome.
### Agent fixture task threads
| Range | Task |
|---|---|
| items 1-13 (`1747396800–1747397140`) | API latency spike → identify keepalive pool regression → rollback |
| items 14-21 (`1747400400–1747400720`) | DB connection pool exhaustion → find unindexed query → CREATE INDEX CONCURRENTLY |
Boundary detector should cut between item 13 and item 14 (timestamp jump
~55 minutes, topic flip). Tool items inside each cell stay attached to
their initiating chat turn.

33
scripts/e2e_memorize/fixtures/agent_session.json Normal file
View File

@ -0,0 +1,33 @@
{
"_doc": "Agent-mode end-to-end fixture. 23 items across 2 task threads: (1) production latency diagnosis with rollback, (2) DB connection-pool exhaustion investigation. Includes ToolCallRequest + ToolCallResult interleaved with user/assistant text turns, parallel tool calls, one assistant turn with empty content (only tool_calls), one extended chain-of-thought reasoning, and one tool failure + retry in task 2. Run with EVEROS_MEMORIZE__MODE=agent. Expected boundary detector cuts ~2 cells aligned with the two task threads; tool items fold into the cell containing the preceding chat turn.",
"session_id_hint": "e2e_agent",
"messages": [
{"sender_id": "u_alice", "sender_name": "Alice", "role": "user", "timestamp": 1747396800000, "content": "Production API p99 latency spiked from 280ms to 2.5s starting around 21:00 UTC. Customer reports rolling in. Can you investigate?"},
{"sender_id": "ops_agent", "sender_name": "OpsAgent", "role": "assistant", "timestamp": 1747396820000, "content": "I'll pull the recent deploys and check for correlation with the spike timestamp.", "tool_calls": [{"id": "call_001", "type": "function", "function": {"name": "list_recent_deploys", "arguments": "{\"hours_back\": 4, \"service\": \"api-gateway\"}"}}]},
{"sender_id": "tool_runner", "sender_name": null, "role": "tool", "timestamp": 1747396825000, "tool_call_id": "call_001", "content": "[{\"deploy_id\": \"d-9f3a\", \"service\": \"api-gateway\", \"deployed_at\": \"2026-05-16T20:47:12Z\", \"author\": \"bob\", \"commit_sha\": \"a1b2c3d\", \"diff_summary\": \"refactor: switch upstream client to keepalive pool\"}, {\"deploy_id\": \"d-9f29\", \"service\": \"api-gateway\", \"deployed_at\": \"2026-05-16T15:12:03Z\", \"author\": \"carol\", \"commit_sha\": \"e4f5g6h\", \"diff_summary\": \"chore: bump otel-sdk to 1.31\"}]"},
{"sender_id": "ops_agent", "sender_name": "OpsAgent", "role": "assistant", "timestamp": 1747396870000, "content": "", "tool_calls": [{"id": "call_002", "type": "function", "function": {"name": "get_deploy_diff", "arguments": "{\"deploy_id\": \"d-9f3a\", \"files\": [\"upstream/client.go\", \"config/pool.yaml\"]}"}}]},
{"sender_id": "tool_runner", "sender_name": null, "role": "tool", "timestamp": 1747396880000, "tool_call_id": "call_002", "content": "config/pool.yaml: max_idle_conns_per_host changed from 32 → 4. upstream/client.go: new transport with IdleConnTimeout=10s (was 90s). The default Go transport now reuses far fewer connections."},
{"sender_id": "ops_agent", "sender_name": "OpsAgent", "role": "assistant", "timestamp": 1747396930000, "content": "Two changes in d-9f3a stand out, and the interaction between them likely explains the cliff. First, max_idle_conns_per_host went from 32 down to 4 — that alone caps how many warm sockets the upstream client can pin per backend. Second, IdleConnTimeout dropped from 90s to 10s, so even the few warm sockets we do keep get torn down between requests during normal traffic gaps. Combined effect: under sustained load p99 of ~250rps × ~6 backends, the pool is permanently saturated and every spillover request pays a TCP+TLS handshake (~800ms-1.5s on the affected region) instead of reusing a connection. That maps directly to the 280ms → 2.5s latency jump. Before I file a rollback recommendation I want to confirm the prediction against live pool metrics — if reuse_ratio collapsed and new-connection rate is up >10x, that's enough signal.", "tool_calls": [{"id": "call_003", "type": "function", "function": {"name": "query_pool_metrics", "arguments": "{\"service\": \"api-gateway\", \"window_min\": 15}"}}]},
{"sender_id": "tool_runner", "sender_name": null, "role": "tool", "timestamp": 1747396938000, "tool_call_id": "call_003", "content": "{\"idle_conns_avg\": 1.2, \"new_conn_rate_per_s\": 184, \"new_conn_rate_baseline\": 6, \"reuse_ratio\": 0.04, \"reuse_ratio_baseline\": 0.91}"},
{"sender_id": "ops_agent", "sender_name": "OpsAgent", "role": "assistant", "timestamp": 1747396980000, "content": "Confirmed: reuse_ratio collapsed from 0.91 to 0.04, new-connection rate is 30x baseline. This is the root cause. Recommend rollback of d-9f3a — the keepalive refactor needs a config review before re-deploying. Want me to initiate rollback?"},
{"sender_id": "u_alice", "sender_name": "Alice", "role": "user", "timestamp": 1747397040000, "content": "Yes, roll it back. Page Bob async so he sees the rollback when he comes online tomorrow — no need to wake him up."},
{"sender_id": "ops_agent", "sender_name": "OpsAgent", "role": "assistant", "timestamp": 1747397070000, "content": "Initiating rollback to d-9f29 and queuing a non-paging Slack DM for Bob.", "tool_calls": [{"id": "call_004", "type": "function", "function": {"name": "rollback_deploy", "arguments": "{\"service\": \"api-gateway\", \"to_deploy_id\": \"d-9f29\", \"reason\": \"p99 latency regression from keepalive pool refactor (max_idle_conns 32->4)\"}"}}, {"id": "call_005", "type": "function", "function": {"name": "slack_dm", "arguments": "{\"user\": \"bob\", \"channel\": \"async\", \"message\": \"Rolled back api-gateway d-9f3a at 21:04 UTC — keepalive pool change caused p99 spike. Logs in #incident-2026-05-16.\"}"}}]},
{"sender_id": "tool_runner", "sender_name": null, "role": "tool", "timestamp": 1747397115000, "tool_call_id": "call_004", "content": "{\"status\": \"ok\", \"rolled_back_to\": \"d-9f29\", \"completed_at\": \"2026-05-16T21:05:01Z\", \"verification\": {\"p99_5min_after\": \"312ms\", \"reuse_ratio_after\": 0.88}}"},
{"sender_id": "tool_runner", "sender_name": null, "role": "tool", "timestamp": 1747397116000, "tool_call_id": "call_005", "content": "{\"status\": \"queued\", \"message_id\": \"slack_msg_xyz\"}"},
{"sender_id": "ops_agent", "sender_name": "OpsAgent", "role": "assistant", "timestamp": 1747397140000, "content": "Rollback complete. p99 dropped back to 312ms within 5 minutes, reuse ratio recovered to 0.88. Slack DM queued for Bob. Incident-2026-05-16 is the channel I logged it to."},
{"sender_id": "u_alice", "sender_name": "Alice", "role": "user", "timestamp": 1747400400000, "content": "Different issue — we're seeing connection pool exhaustion warnings on the auth-service Postgres pool the last hour. Can you check whether it's a leak or just a sustained traffic spike?"},
{"sender_id": "ops_agent", "sender_name": "OpsAgent", "role": "assistant", "timestamp": 1747400420000, "content": "Pulling pool stats and recent traffic shape.", "tool_calls": [{"id": "call_006", "type": "function", "function": {"name": "query_db_pool", "arguments": "{\"service\": \"auth-service\", \"window_min\": 60}"}}, {"id": "call_007", "type": "function", "function": {"name": "query_request_rate", "arguments": "{\"service\": \"auth-service\", \"window_min\": 60}"}}]},
{"sender_id": "tool_runner", "sender_name": null, "role": "tool", "timestamp": 1747400430000, "tool_call_id": "call_006", "content": "{\"max_conns\": 50, \"in_use_now\": 49, \"in_use_p99_1h\": 50, \"idle_p99_1h\": 1, \"wait_queue_depth_p99\": 18, \"long_running_queries_above_30s\": 12}"},
{"sender_id": "tool_runner", "sender_name": null, "role": "tool", "timestamp": 1747400431000, "tool_call_id": "call_007", "content": "{\"rps_now\": 142, \"rps_baseline\": 138, \"rps_p99_1h\": 151}"},
{"sender_id": "ops_agent", "sender_name": "OpsAgent", "role": "assistant", "timestamp": 1747400470000, "content": "Traffic is flat (rps 142 vs baseline 138), but 12 long-running queries above 30s and wait queue depth p99 of 18. This is a leak / unbounded query, not a traffic surge. Let me find the offending queries.", "tool_calls": [{"id": "call_008", "type": "function", "function": {"name": "list_long_running_queries", "arguments": "{\"service\": \"auth-service\", \"min_duration_s\": 30, \"window_min\": 60}"}}]},
{"sender_id": "tool_runner", "sender_name": null, "role": "tool", "timestamp": 1747400478000, "tool_call_id": "call_008", "content": "{\"error\": \"timeout\", \"detail\": \"query against pg_stat_activity exceeded 15s while pool was saturated; retry with a narrower window\"}"},
{"sender_id": "ops_agent", "sender_name": "OpsAgent", "role": "assistant", "timestamp": 1747400490000, "content": "Tool itself is starved by the same pool exhaustion — fair enough. Retrying against the last 10 minutes only so pg_stat_activity has a tighter slice to scan.", "tool_calls": [{"id": "call_008b", "type": "function", "function": {"name": "list_long_running_queries", "arguments": "{\"service\": \"auth-service\", \"min_duration_s\": 30, \"window_min\": 10}"}}]},
{"sender_id": "tool_runner", "sender_name": null, "role": "tool", "timestamp": 1747400499000, "tool_call_id": "call_008b", "content": "[{\"query_sha\": \"q-71f\", \"sql_truncated\": \"SELECT u.* FROM users u JOIN sessions s ON s.user_id = u.id WHERE s.expired_at > now() - interval '30 day'\", \"avg_duration_s\": 84, \"calls_last_hour\": 11, \"missing_index\": \"sessions.expired_at\"}]"},
{"sender_id": "ops_agent", "sender_name": "OpsAgent", "role": "assistant", "timestamp": 1747400510000, "content": "Single offender — query q-71f is doing a full sessions-table scan because sessions.expired_at is unindexed. Each call takes ~84s and holds a connection. 11 calls in the last hour saturate the pool. Recommend: (1) add the index now (concurrent), (2) add a query timeout cap on this endpoint long-term. Want me to add the index?"},
{"sender_id": "u_alice", "sender_name": "Alice", "role": "user", "timestamp": 1747400560000, "content": "Yes please add the index. Use CREATE INDEX CONCURRENTLY so we don't block writes. Then post in #incident-2026-05-16 with the runbook entry."},
{"sender_id": "ops_agent", "sender_name": "OpsAgent", "role": "assistant", "timestamp": 1747400590000, "content": "On it. Creating the concurrent index and posting the runbook update.", "tool_calls": [{"id": "call_009", "type": "function", "function": {"name": "execute_sql", "arguments": "{\"db\": \"auth-service\", \"sql\": \"CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_sessions_expired_at ON sessions(expired_at)\", \"timeout_s\": 600}"}}]},
{"sender_id": "tool_runner", "sender_name": null, "role": "tool", "timestamp": 1747400650000, "tool_call_id": "call_009", "content": "{\"status\": \"ok\", \"duration_s\": 47, \"rows_indexed\": 1841229}"},
{"sender_id": "ops_agent", "sender_name": "OpsAgent", "role": "assistant", "timestamp": 1747400720000, "content": "Index created in 47s over 1.84M rows. Query q-71f should drop from ~84s to sub-second. Pool will drain in the next minute as long-running queries finish. Posted runbook to #incident-2026-05-16."}
]
}

31
scripts/e2e_memorize/fixtures/chat_session.json Normal file
View File

@ -0,0 +1,31 @@
{
"_doc": "Chat-mode end-to-end fixture. 22 messages across 3 topics + a multi-user handoff. Run with EVEROS_MEMORIZE__MODE=chat. Expected boundary detector cuts ~3-4 cells (Python bug → weekend plans → cross-team meeting).",
"session_id_hint": "e2e_chat",
"messages": [
{"sender_id": "u_alice", "sender_name": "Alice", "role": "user", "timestamp": 1747396800000, "content": "I'm getting a weird KeyError on a dict comprehension when the source list has None entries. The traceback points at the dict literal, not the comprehension itself."},
{"sender_id": "assistant", "sender_name": "Bot", "role": "assistant", "timestamp": 1747396830000, "content": "Can you share a minimal repro? Something like the comprehension expression plus 2-3 sample input items. I want to see whether the None is on the key side or the value side."},
{"sender_id": "u_alice", "sender_name": "Alice", "role": "user", "timestamp": 1747396890000, "content": "Sure: `{r['id']: r['name'] for r in rows}` where rows comes from a left-join and sometimes the right side returns None. So r is None for those rows."},
{"sender_id": "assistant", "sender_name": "Bot", "role": "assistant", "timestamp": 1747396920000, "content": "Got it. The None is the row itself, not a field. `None['id']` raises TypeError actually, but if you preprocess the rows to dicts upstream, the missing key on a partial dict would be KeyError. Either way, filter before the comprehension: `{r['id']: r['name'] for r in rows if r is not None and 'id' in r}`."},
{"sender_id": "u_alice", "sender_name": "Alice", "role": "user", "timestamp": 1747396980000, "content": "That worked, thanks. Also occasionally the SQL itself wraps the row in a tuple instead of dict — pyodbc behaviour. Got bitten by it last quarter too."},
{"sender_id": "assistant", "sender_name": "Bot", "role": "assistant", "timestamp": 1747397010000, "content": "pyodbc returns Row objects by default; you'd need `cursor.execute(...).fetchall()` then iterate. Or set `cursor.row_factory = lambda c, r: dict(zip([col[0] for col in c.description], r))` to normalise upstream."},
{"sender_id": "u_alice", "sender_name": "Alice", "role": "user", "timestamp": 1747400400000, "content": "Switching topics — any weekend plans? I'm thinking of trying the new ramen place on Carmine Street, supposedly the tonkotsu is decent."},
{"sender_id": "assistant", "sender_name": "Bot", "role": "assistant", "timestamp": 1747400430000, "content": "Tonkotsu sounds good. If you go Saturday before 1pm you'll skip the queue. There's also a coffee place two doors down that does decent cold brew if you want a pre-meal stop."},
{"sender_id": "u_alice", "sender_name": "Alice", "role": "user", "timestamp": 1747400490000, "content": "Cold brew yes. Bob mentioned wanting to come too, do you think the ramen place takes reservations for three?"},
{"sender_id": "assistant", "sender_name": "Bot", "role": "assistant", "timestamp": 1747400520000, "content": "Their site says walk-in only on weekends. For three people the wait is usually 25-40 minutes around noon — manageable but worth knowing."},
{"sender_id": "u_alice", "sender_name": "Alice", "role": "user", "timestamp": 1747400580000, "content": "OK we'll show up at 11:30 then. Will send Bob the address."},
{"sender_id": "assistant", "sender_name": "Bot", "role": "assistant", "timestamp": 1747400610000, "content": "Sensible. Reminder it's cash-only for the first 6 months apparently — saw that on a Reddit thread last week."},
{"sender_id": "u_alice", "sender_name": "Alice", "role": "user", "timestamp": 1747407600000, "content": "One more thing — the Q3 revenue review is next Thursday at 2pm, who from engineering needs to attend? I want to make sure platform team is represented since we're presenting the infra cost reductions."},
{"sender_id": "assistant", "sender_name": "Bot", "role": "assistant", "timestamp": 1747407630000, "content": "Standard attendees are eng director + relevant TL. For platform cost reductions, your eng lead Carol should be there since she owns the runbook. Add Dan as a backup in case Carol's on PTO."},
{"sender_id": "u_alice", "sender_name": "Alice", "role": "user", "timestamp": 1747407690000, "content": "Good call. Carol mentioned PTO that week actually. Let me ping Dan to confirm he can do the slides handoff."},
{"sender_id": "assistant", "sender_name": "Bot", "role": "assistant", "timestamp": 1747407720000, "content": "If Dan has bandwidth, also ask him to pre-share the deck Monday so finance has time to review the cost numbers before the meeting. Saves a lot of meeting time clarifying figures."},
{"sender_id": "u_bob", "sender_name": "Bob", "role": "user", "timestamp": 1747411200000, "content": "Hey, Alice forwarded me about the ramen plan and the Thursday meeting. I can do both but the Thursday slot conflicts with the SRE oncall handoff at 1:45 — anyone we can swap with?"},
{"sender_id": "assistant", "sender_name": "Bot", "role": "assistant", "timestamp": 1747411230000, "content": "Erin usually does the Wednesday → Thursday handoff for SRE. She owes you one from last cycle. Ask her to take the 1:45 slot and you can ack the comms async at 1:30."},
{"sender_id": "u_bob", "sender_name": "Bob", "role": "user", "timestamp": 1747411290000, "content": "Erin's good. I'll ping her on Slack today. About the ramen — Alice said 11:30 Saturday, that works for me, walking distance from my place."},
{"sender_id": "assistant", "sender_name": "Bot", "role": "assistant", "timestamp": 1747411320000, "content": "Great. Just remind Alice it's cash-only for now — she'll forget otherwise based on past experience."},
{"sender_id": "u_bob", "sender_name": "Bob", "role": "user", "timestamp": 1747411380000, "content": "Ha, true. I'll grab some cash on the way. Also can you remind me about the Q3 revenue deck — when does it need to be done by, EOD Tuesday?"},
{"sender_id": "assistant", "sender_name": "Bot", "role": "assistant", "timestamp": 1747411410000, "content": "Yes — Dan agreed to pre-share Monday, so practical deadline is EOD Tuesday for finance to review Wednesday. After Wednesday review they may ask for last-minute number tweaks; build slack into Thursday morning for that."}
]
}

186
scripts/e2e_memorize/run.py Normal file
View File

@ -0,0 +1,186 @@
"""End-to-end memorize runner — in-process call into ``service.memorize``.
Calls ``service.memorize.memorize()`` directly (not via HTTP) so this works
without ``everos server start``. Drives a fixture through ``/add`` in
N-sized batches, then triggers ``/flush`` to drain the tail.
Reads ``settings.memorize.mode`` from current env / toml — set the mode via
``EVEROS_MEMORIZE__MODE=chat|agent`` *before* invoking this script (the
config is cached after the first ``load_settings()`` call).
Usage:
EVEROS_MEMORIZE__MODE=chat uv run python scripts/e2e_memorize/run.py \\
--fixture scripts/e2e_memorize/fixtures/chat_session.json
EVEROS_MEMORIZE__MODE=agent uv run python scripts/e2e_memorize/run.py \\
--fixture scripts/e2e_memorize/fixtures/agent_session.json --batch-size 5
After it finishes, check:
~/.everos/users/<owner>/episodes/<date>.md (written sync by 4A)
~/.everos/.index/sqlite/system.db memcell rows (written by boundary)
~/.everos/agents/<agent>/agent_cases/<date>.md (written async by OME
- only if a consumer of AgentMemCellWritten is registered)
"""
from __future__ import annotations
import argparse
import asyncio
import json
import sys
import time
import uuid
from pathlib import Path
from sqlmodel import SQLModel
from everos.component.llm import get_llm_client
from everos.config import load_settings
from everos.core.persistence import MemoryRoot
from everos.infra.persistence.sqlite import dispose_engine, get_engine
from everos.service.memorize import _get_engine as _get_ome_engine
from everos.service.memorize import memorize
def _chunks(items: list[dict], n: int) -> list[list[dict]]:
return [items[i : i + n] for i in range(0, len(items), n)]
def _print_header(mode: str, fixture_path: Path, session_id: str) -> None:
print("=" * 72)
print(f" everos e2e memorize · mode={mode}")
print(f" fixture : {fixture_path.name}")
print(f" session_id : {session_id}")
print(f" memory root : {MemoryRoot.default().root}")
llm_state = "<configured>" if get_llm_client() else "<None — pipeline will skip>"
print(f" llm_client : {llm_state}")
print("=" * 72)
def _list_written_files(session_id: str, mode: str) -> None:
"""Walk memory root and print files touched in this run."""
root = MemoryRoot.default().root
cutoff = time.time() - 600 # files modified in the last 10 min
print()
print("─── files modified within the last 10 minutes under memory root ───")
interesting = ("users", "agents", "knowledge", ".index")
for sub in interesting:
base = Path(root) / sub
if not base.is_dir():
continue
for p in sorted(base.rglob("*")):
if p.is_file() and p.stat().st_mtime >= cutoff:
rel = p.relative_to(root)
size = p.stat().st_size
print(f" {rel} ({size}b)")
print()
print(f"Tip: grep '{session_id}' in any episode md to find this run's entries.")
async def _setup() -> None:
"""Create sqlite schema + start OME engine — the bits the HTTP lifespan
normally handles. LanceDB is not needed for the memorize sync path
(only cascade reads it), so we skip it.
"""
engine = get_engine()
async with engine.begin() as conn:
await conn.run_sync(SQLModel.metadata.create_all)
ome = _get_ome_engine()
await ome.start()
async def _teardown() -> None:
ome = _get_ome_engine()
await ome.stop()
await dispose_engine()
async def _run(args: argparse.Namespace) -> None:
settings = load_settings()
mode = settings.memorize.mode
if args.expected_mode and args.expected_mode != mode:
print(
f"!! expected mode={args.expected_mode!r} but "
f"settings.memorize.mode={mode!r}. "
"Set EVEROS_MEMORIZE__MODE before launching."
)
sys.exit(2)
fixture_path = Path(args.fixture).resolve() # noqa: ASYNC240
fixture = json.loads(fixture_path.read_text()) # noqa: ASYNC230
messages: list[dict] = fixture["messages"]
session_id = f"{fixture.get('session_id_hint', 'e2e')}_{uuid.uuid4().hex[:8]}"
_print_header(mode, fixture_path, session_id)
if args.dry_run:
for i, batch in enumerate(_chunks(messages, args.batch_size), start=1):
print(
f"[dry] batch {i}: {len(batch)} msgs "
f"(first content: {batch[0]['content'][:60]!r})"
)
print("[dry] would flush at the end")
return
await _setup()
try:
batches = _chunks(messages, args.batch_size)
for i, batch in enumerate(batches, start=1):
result = await memorize(
{"session_id": session_id, "messages": batch}, is_final=False
)
print(
f"add batch {i}/{len(batches)} ({len(batch)} msgs) → "
f"status={result.status:<11s} message_count={result.message_count}"
)
print()
print("flushing residual tail...")
flush_result = await memorize(
{"session_id": session_id, "messages": []}, is_final=True
)
print(
f"flush → status={flush_result.status:<11s} "
f"message_count={flush_result.message_count}"
)
# OME strategies are fire-and-forget; each cell fires 2 strategies
# (atomic_facts + foresight), each ~5-10s on a real LLM. Sleep long
# enough for ~8-10 invocations to finish before engine.stop() drains
# the scheduler — otherwise APS cancels in-flight LLM calls.
await asyncio.sleep(30)
_list_written_files(session_id, mode)
finally:
await _teardown()
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--fixture",
required=True,
help="path to fixture JSON (e.g. fixtures/chat_session.json)",
)
parser.add_argument(
"--batch-size",
type=int,
default=6,
help="how many messages per /add call (default 6 — 20 msgs across ~4 batches)",
)
parser.add_argument(
"--expected-mode",
choices=["chat", "agent"],
help="sanity check: fail fast if settings.memorize.mode mismatches",
)
parser.add_argument(
"--dry-run",
action="store_true",
help="print the batch plan without calling memorize",
)
args = parser.parse_args()
asyncio.run(_run(args))
if __name__ == "__main__":
main()

35
src/everos/README.md Normal file
View File

@ -0,0 +1,35 @@
# `everos` package
Source layout for the `everos` Python package. This README is a quick
orientation; full architectural detail lives elsewhere.
## Layout
```
everos/
├── entrypoints/ Presentation: cli + api
├── service/ Application: use case orchestration
├── memory/ Domain: extract + search + cascade + prompt_slots + models
├── infra/ Infrastructure: persistence/{markdown, sqlite, lancedb}
├── component/ Cross-cutting providers: llm / embedding / config / utils
├── core/ Runtime base: observability / lifespan / context
└── config/ Data: Settings + default.toml + prompt_slots templates
```
Each subpackage has a top-level `__init__.py` describing its responsibility
and public API.
## Dependency rule
```
entrypoints → service → memory → infra
↓
component / core / config
```
Single-direction; enforced by `import-linter` in CI.
## Further reading
- Architecture: [../../docs/architecture.md](../../docs/architecture.md)
- Coding rules (auto-loaded by Claude Code): [../../.claude/rules/](../../.claude/rules/)

11
src/everos/__init__.py Normal file
View File

@ -0,0 +1,11 @@
"""everos — md-first memory extraction framework."""
from importlib.metadata import PackageNotFoundError
from importlib.metadata import version as _pkg_version
try:
__version__ = _pkg_version("everos")
except PackageNotFoundError:
# Editable install without dist-info, or running from a source tree that
# was never installed. Fall back to a sentinel rather than crash imports.
__version__ = "0.0.0+unknown"

4
src/everos/component/__init__.py Normal file
View File

@ -0,0 +1,4 @@
"""Cross-cutting components.
Technical capabilities used by every layer; depend on no business layer.
"""

14
src/everos/component/config/__init__.py Normal file
View File

@ -0,0 +1,14 @@
"""Config processing capability.
YAML loader for category-organised config trees (PromptSlot templates,
etc.). Distinct from :mod:`everos.config` (configuration *data* + Settings
schema, which uses TOML for the Pydantic Settings file) — this subpackage
holds *capability* (how to load), the other holds *data* (what to load).
External usage:
from everos.component.config import YamlConfigLoader
"""
from .loader import YamlConfigLoader as YamlConfigLoader
__all__ = ["YamlConfigLoader"]

146
src/everos/component/config/loader.py Normal file
View File

@ -0,0 +1,146 @@
"""YAML config loader for category-organised file trees.
Concept: a project keeps several *categories* of YAML config files under
their own subdirectories — for example PromptSlot templates under
``config/prompt_slots/<name>.yaml``. The loader:
1. registers a category → subdirectory mapping
2. resolves ``find(category, name)`` to ``<root>/<subdir>/<name>.yaml``
3. caches parsed contents until ``refresh`` is called
Uses ``yaml.safe_load`` (no arbitrary tags) — PyYAML is already a project
dependency for markdown frontmatter, so no extra cost.
"""
from __future__ import annotations
from collections.abc import Mapping
from pathlib import Path
from typing import Any
import yaml
class YamlConfigLoader:
"""Load YAML files organised by category subdirectories.
Usage:
loader = YamlConfigLoader(root=Path("src/everos/config"))
loader.register_category("prompt_slots")
# → reads <root>/prompt_slots/episode.yaml
meta = loader.find("prompt_slots", "episode")
names = loader.list("prompt_slots")
loader.refresh() # next find() re-reads from disk
Cache semantics:
* ``find`` parses the file on first access and caches the dict.
* ``refresh()`` empties the entire cache.
* ``refresh(category)`` empties one category's entries.
* ``refresh(category, name)`` empties a single entry.
"""
def __init__(
self,
root: Path,
categories: Mapping[str, str | None] | None = None,
) -> None:
"""
Args:
root: Base directory containing the category subdirectories.
categories: Optional pre-registered category map (``name → subdir``).
When ``subdir`` is ``None`` the category name is used as-is.
"""
self._root = Path(root)
self._subdirs: dict[str, str] = {}
self._cache: dict[tuple[str, str], dict[str, Any]] = {}
if categories:
for name, subdir in categories.items():
self.register_category(name, subdir)
# ── Category management ────────────────────────────────────────────────
def register_category(self, name: str, subdir: str | None = None) -> None:
"""Register a category. ``subdir`` defaults to ``name``."""
self._subdirs[name] = subdir if subdir is not None else name
def categories(self) -> list[str]:
"""Return registered category names (sorted)."""
return sorted(self._subdirs)
# ── Lookup ─────────────────────────────────────────────────────────────
def find(self, category: str, name: str) -> dict[str, Any]:
"""Load ``<root>/<subdir>/<name>.yaml`` for ``category``.
Raises:
KeyError: if ``category`` was not registered.
FileNotFoundError: if the yaml file does not exist.
TypeError: if the parsed YAML is not a mapping.
"""
cache_key = (category, name)
if cache_key in self._cache:
return self._cache[cache_key]
path = self._path_for(category, name)
if not path.is_file():
raise FileNotFoundError(f"yaml not found: {path}")
with path.open("r", encoding="utf-8") as fh:
data = yaml.safe_load(fh)
if data is None:
data = {}
if not isinstance(data, dict):
raise TypeError(
f"yaml top-level must be a mapping, got {type(data).__name__}: {path}"
)
self._cache[cache_key] = data
return data
def list(self, category: str) -> list[str]:
"""Return sorted yaml stems available in ``category`` (no extension).
Raises:
KeyError: if ``category`` was not registered.
"""
directory = self._dir_for(category)
if not directory.is_dir():
return []
return sorted(p.stem for p in directory.glob("*.yaml"))
# ── Cache control ──────────────────────────────────────────────────────
def refresh(
self,
category: str | None = None,
name: str | None = None,
) -> None:
"""Invalidate cached entries.
- ``refresh()`` → drop every cached entry
- ``refresh(category)`` → drop everything in ``category``
- ``refresh(category, name)`` → drop a single entry
"""
if category is None:
self._cache.clear()
return
if name is not None:
self._cache.pop((category, name), None)
return
self._cache = {
(cat, n): v for (cat, n), v in self._cache.items() if cat != category
}
# ── Internals ──────────────────────────────────────────────────────────
def _dir_for(self, category: str) -> Path:
try:
subdir = self._subdirs[category]
except KeyError as exc:
raise KeyError(
f"category not registered: {category!r}; known: {sorted(self._subdirs)}"
) from exc
return self._root / subdir
def _path_for(self, category: str, name: str) -> Path:
return self._dir_for(category) / f"{name}.yaml"

33
src/everos/component/embedding/__init__.py Normal file
View File

@ -0,0 +1,33 @@
"""Embedding provider adapters (one provider per file).
Public surface:
- :class:`EmbeddingProvider` — Protocol every provider satisfies.
- :class:`EmbeddingError` — provider-side failure.
- :class:`OpenAIEmbeddingProvider` — concrete provider for any
OpenAI-protocol embeddings endpoint (DeepInfra, vLLM, OpenAI, …).
- :func:`build_embedding_provider` — settings-driven factory.
External usage::
from everos.component.embedding import build_embedding_provider
provider = build_embedding_provider(settings.embedding)
vec = await provider.embed("hello")
"""
from .accessor import EmbeddingNotConfiguredError as EmbeddingNotConfiguredError
from .accessor import get_embedder as get_embedder
from .factory import build_embedding_provider as build_embedding_provider
from .openai_provider import OpenAIEmbeddingProvider as OpenAIEmbeddingProvider
from .protocol import EmbeddingError as EmbeddingError
from .protocol import EmbeddingProvider as EmbeddingProvider
__all__ = [
"EmbeddingError",
"EmbeddingNotConfiguredError",
"EmbeddingProvider",
"OpenAIEmbeddingProvider",
"build_embedding_provider",
"get_embedder",
]

48
src/everos/component/embedding/accessor.py Normal file
View File

@ -0,0 +1,48 @@
"""Process-wide embedding provider accessor.
Lazy singleton mirror of :func:`everos.component.llm.get_llm_client`:
first call reads settings and builds the OpenAI-protocol embedding
client; subsequent calls return the cached instance. Strategies and
other components that need a process-wide embedder import this rather
than threading the provider through their constructors.
Raises :class:`EmbeddingNotConfiguredError` when credentials are missing
so misconfiguration surfaces at the call site (or at app startup via a
lifespan provider) instead of silently degrading.
"""
from __future__ import annotations
from everos.config import load_settings
from everos.core.observability.logging import get_logger
from .factory import build_embedding_provider
from .protocol import EmbeddingProvider
logger = get_logger(__name__)
class EmbeddingNotConfiguredError(RuntimeError):
"""Raised when ``settings.embedding`` lacks ``model``/``api_key``/``base_url``."""
_embedder: EmbeddingProvider | None = None
def get_embedder() -> EmbeddingProvider:
"""Return the singleton :class:`EmbeddingProvider`.
Raises:
EmbeddingNotConfiguredError: When required settings fields are
unset. See :func:`build_embedding_provider` for the exact
keys.
"""
global _embedder
if _embedder is not None:
return _embedder
try:
_embedder = build_embedding_provider(load_settings().embedding)
except ValueError as exc:
raise EmbeddingNotConfiguredError(str(exc)) from exc
logger.info("embedder_built")
return _embedder

56
src/everos/component/embedding/factory.py Normal file
View File

@ -0,0 +1,56 @@
"""Factory for building an embedding provider from :class:`EmbeddingSettings`."""
from __future__ import annotations
from everos.config import EmbeddingSettings
from .openai_provider import OpenAIEmbeddingProvider
from .protocol import EmbeddingProvider
# Vector dim for the LanceDB index column — see ``17_lancedb_tables_design.md``.
_DEFAULT_DIM = 1024
def build_embedding_provider(
settings: EmbeddingSettings,
*,
dim: int = _DEFAULT_DIM,
) -> EmbeddingProvider:
"""Build an OpenAI-compatible embedding provider from settings.
Args:
settings: The :class:`EmbeddingSettings` slice from
:func:`everos.config.load_settings`.
dim: Target vector dimension; defaults to 1024 to match the
LanceDB ``vector`` column shape.
Returns:
An :class:`EmbeddingProvider` ready to call ``embed`` /
``embed_batch``.
Raises:
ValueError: If ``model``, ``api_key`` or ``base_url`` is unset.
"""
if not settings.model:
raise ValueError(
"Embedding model is not configured "
"(set EVEROS_EMBEDDING__MODEL or [embedding] model in user toml)"
)
if settings.api_key is None:
raise ValueError(
"Embedding api_key is not configured (set EVEROS_EMBEDDING__API_KEY)"
)
if not settings.base_url:
raise ValueError(
"Embedding base_url is not configured (set EVEROS_EMBEDDING__BASE_URL)"
)
return OpenAIEmbeddingProvider(
model=settings.model,
api_key=settings.api_key.get_secret_value(),
base_url=settings.base_url,
dim=dim,
timeout=settings.timeout_seconds,
max_retries=settings.max_retries,
batch_size=settings.batch_size,
max_concurrent=settings.max_concurrent,
)

98
src/everos/component/embedding/openai_provider.py Normal file
View File

@ -0,0 +1,98 @@
"""OpenAI-compatible embedding provider.
Wraps :class:`openai.AsyncOpenAI` so any OpenAI-protocol endpoint
(DeepInfra, OpenAI, Together, Fireworks, …) works without per-provider
forks. Self-hosted vLLM also exposes the same shape; the only quirk it
imposes is that the ``dimensions`` request parameter is ignored — we
truncate client-side to ``dim`` so callers always see the declared
shape regardless of backend.
Concurrency model:
- ``embed_batch`` splits the inputs into chunks of ``batch_size``.
- An :class:`asyncio.Semaphore` capped at ``max_concurrent`` bounds
in-flight requests; remaining chunks queue and start as slots free.
- Retries / timeouts come from the openai SDK (``max_retries``,
``timeout`` constructor args).
"""
from __future__ import annotations
import asyncio
from collections.abc import Sequence
import openai
from .protocol import EmbeddingError
class OpenAIEmbeddingProvider:
"""OpenAI-compatible embedding provider with batching + concurrency.
Args:
model: Embedding model id (e.g. ``"Qwen/Qwen3-Embedding-4B"``).
api_key: Bearer credential as a plain ``str``.
base_url: OpenAI-protocol endpoint
(e.g. ``"https://api.deepinfra.com/v1/openai"``).
dim: Target vector dimension. Vectors longer than this are
truncated client-side (matches the LanceDB column shape —
see ``17_lancedb_tables_design.md``).
timeout: Per-request timeout, seconds.
max_retries: Retry budget exposed via the openai SDK.
batch_size: How many inputs per ``/embeddings`` call.
max_concurrent: Cap on in-flight chunked requests.
"""
def __init__(
self,
*,
model: str,
api_key: str,
base_url: str,
dim: int = 1024,
timeout: float = 30.0,
max_retries: int = 3,
batch_size: int = 10,
max_concurrent: int = 5,
) -> None:
self.dim = dim
self._model = model
self._batch_size = batch_size
self._semaphore = asyncio.Semaphore(max_concurrent)
self._client = openai.AsyncOpenAI(
api_key=api_key,
base_url=base_url,
timeout=timeout,
max_retries=max_retries,
)
async def embed(self, text: str) -> list[float]:
"""Embed a single string."""
vectors = await self._embed_chunk([text])
return vectors[0]
async def embed_batch(self, texts: Sequence[str]) -> list[list[float]]:
"""Embed many strings, preserving input order."""
if not texts:
return []
chunks = [
list(texts[i : i + self._batch_size])
for i in range(0, len(texts), self._batch_size)
]
results = await asyncio.gather(*(self._embed_chunk(chunk) for chunk in chunks))
# gather preserves order across awaitables, and each chunk preserves
# its internal order — so flattening yields the input order back.
return [vec for chunk in results for vec in chunk]
async def _embed_chunk(self, chunk: list[str]) -> list[list[float]]:
"""One ``/embeddings`` call, semaphore-guarded."""
async with self._semaphore:
try:
response = await self._client.embeddings.create(
model=self._model,
input=chunk,
)
except openai.OpenAIError as exc:
raise EmbeddingError(str(exc)) from exc
# OpenAI returns ``data`` indexed by request order; truncate to ``dim``.
return [list(item.embedding[: self.dim]) for item in response.data]

48
src/everos/component/embedding/protocol.py Normal file
View File

@ -0,0 +1,48 @@
"""Embedding provider protocol.
The cascade worker / retrieval pipeline depend on a single small
contract: turn a string (or list of strings) into a fixed-dimension
vector. Whether the backend is OpenAI, vLLM, DeepInfra, Ollama, or a
local model is the provider's business — the contract is invariant.
"""
from __future__ import annotations
from collections.abc import Sequence
from typing import Protocol, runtime_checkable
class EmbeddingError(Exception):
"""Raised on any provider-side embedding failure.
Wraps the upstream SDK exception via ``__cause__`` (PEP 3134) so
diagnostic loggers preserve the original error chain.
"""
@runtime_checkable
class EmbeddingProvider(Protocol):
"""Async embedding provider contract.
``dim`` is the post-truncation vector dimension every embed call
returns. Providers that don't natively support dimension truncation
must truncate client-side so callers see the declared shape.
"""
dim: int
async def embed(self, text: str) -> list[float]:
"""Embed a single string. Returns a ``[dim]`` vector."""
...
async def embed_batch(self, texts: Sequence[str]) -> list[list[float]]:
"""Embed a batch of strings preserving input order.
Implementations chunk by ``batch_size`` and bound in-flight
requests by ``max_concurrent`` (both from settings). On failure,
raises :class:`EmbeddingError` — the worker treats it as a
retryable / unrecoverable case per HTTP-status mapping.
"""
...

45
src/everos/component/llm/__init__.py Normal file
View File

@ -0,0 +1,45 @@
"""LLM provider adapters (one provider per file, mem0-style).
Public surface:
- :class:`LLMClient` — Protocol every provider satisfies (re-exported
from :mod:`everalgo.llm`; same shape so everos providers can be
handed to everalgo operators).
- :class:`ChatMessage` / :class:`ChatResponse` / :class:`Usage` — chat
payload types (re-exported from :mod:`everalgo.llm`).
- :class:`LLMError` — provider-side failure (re-exported).
- :class:`LLMNotConfiguredError` — raised when credentials are missing.
- :class:`OpenAIProvider` — concrete provider wrapping
``openai.AsyncOpenAI`` against any OpenAI-compatible endpoint.
- :func:`build_llm_provider` — settings-driven factory.
- :func:`get_llm_client` — process-wide lazy singleton accessor.
External usage::
from everos.component.llm import build_llm_provider, LLMClient
provider = build_llm_provider(settings.llm)
"""
from .client import LLMNotConfiguredError as LLMNotConfiguredError
from .client import get_llm_client as get_llm_client
from .client import get_multimodal_llm_client as get_multimodal_llm_client
from .factory import build_llm_provider as build_llm_provider
from .openai_provider import OpenAIProvider as OpenAIProvider
from .protocol import ChatMessage as ChatMessage
from .protocol import ChatResponse as ChatResponse
from .protocol import LLMClient as LLMClient
from .protocol import LLMError as LLMError
from .protocol import Usage as Usage
__all__ = [
"ChatMessage",
"ChatResponse",
"LLMClient",
"LLMError",
"LLMNotConfiguredError",
"OpenAIProvider",
"Usage",
"build_llm_provider",
"get_llm_client",
"get_multimodal_llm_client",
]

89
src/everos/component/llm/client.py Normal file
View File

@ -0,0 +1,89 @@
"""Process-wide LLM client accessor.
Lazy singleton — first call reads settings and builds the algo LLM
client; subsequent calls return the cached instance. Raises
:class:`LLMNotConfiguredError` when no credentials are present so
misconfiguration surfaces at app startup (via the LLM lifespan
provider) instead of silently failing per-request downstream.
"""
from __future__ import annotations
from everalgo.llm import build_client
from everalgo.llm.config import LLMConfig
from everalgo.llm.protocols import LLMClient
from everos.config import load_settings
from everos.core.observability.logging import get_logger
logger = get_logger(__name__)
class LLMNotConfiguredError(RuntimeError):
"""Raised when ``settings.llm`` is missing ``api_key`` or ``base_url``."""
_llm_client: LLMClient | None = None
_multimodal_client: LLMClient | None = None
def get_llm_client() -> LLMClient:
"""Return the singleton algo LLM client.
Raises:
LLMNotConfiguredError: When ``settings.llm.api_key`` or
``settings.llm.base_url`` is unset.
"""
global _llm_client
if _llm_client is not None:
return _llm_client
llm_cfg = load_settings().llm
api_key = (
llm_cfg.api_key.get_secret_value() if llm_cfg.api_key is not None else None
)
if not api_key or not llm_cfg.base_url:
raise LLMNotConfiguredError(
"LLM is required; set EVEROS_LLM__API_KEY + EVEROS_LLM__BASE_URL"
)
_llm_client = build_client(
LLMConfig(
model=llm_cfg.model,
api_key=api_key,
base_url=llm_cfg.base_url,
)
)
logger.info("llm_client_built", model=llm_cfg.model)
return _llm_client
def get_multimodal_llm_client() -> LLMClient:
"""Return the singleton multimodal LLM client (for everalgo.parser).
Reads the flat ``[multimodal]`` config — kept separate from the main
``[llm]`` so parsing can target a vision/audio-capable endpoint.
Raises:
LLMNotConfiguredError: When ``settings.multimodal.api_key`` or
``settings.multimodal.base_url`` is unset.
"""
global _multimodal_client
if _multimodal_client is not None:
return _multimodal_client
cfg = load_settings().multimodal
api_key = cfg.api_key.get_secret_value() if cfg.api_key is not None else None
if not api_key or not cfg.base_url:
raise LLMNotConfiguredError(
"Multimodal LLM is required for parsing; set "
"EVEROS_MULTIMODAL__API_KEY + EVEROS_MULTIMODAL__BASE_URL"
)
_multimodal_client = build_client(
LLMConfig(
model=cfg.model,
api_key=api_key,
base_url=cfg.base_url,
)
)
logger.info("multimodal_llm_client_built", model=cfg.model)
return _multimodal_client

45
src/everos/component/llm/factory.py Normal file
View File

@ -0,0 +1,45 @@
"""Factory for building an LLM provider from :class:`LLMSettings`."""
from __future__ import annotations
from everos.config import LLMSettings
from .openai_provider import OpenAIProvider
from .protocol import LLMClient
def build_llm_provider(settings: LLMSettings) -> LLMClient:
"""Build an OpenAI-compatible LLM provider from settings.
Unwraps :class:`pydantic.SecretStr` here so downstream callers never
touch the raw key directly. Fails fast if either ``api_key`` or
``base_url`` is missing — caller is expected to set them via
``.env`` / user toml / programmatic init before calling.
Args:
settings: The :class:`LLMSettings` slice from
:func:`everos.config.load_settings`.
Returns:
A provider that structurally satisfies
:class:`everalgo.llm.LLMClient` and can be passed to everalgo
operators via ``llm=``.
Raises:
ValueError: If ``api_key`` or ``base_url`` is unset.
"""
if settings.api_key is None:
raise ValueError(
"LLM api_key is not configured "
"(set EVEROS_LLM__API_KEY or [llm] api_key in user toml)"
)
if not settings.base_url:
raise ValueError(
"LLM base_url is not configured "
"(set EVEROS_LLM__BASE_URL or [llm] base_url in user toml)"
)
return OpenAIProvider(
model=settings.model,
api_key=settings.api_key.get_secret_value(),
base_url=settings.base_url,
)

114
src/everos/component/llm/openai_provider.py Normal file
View File

@ -0,0 +1,114 @@
"""OpenAI-compatible LLM provider for everos.
Implements the :class:`everalgo.llm.LLMClient` structural contract by
wrapping :class:`openai.AsyncOpenAI` — the same backbone everalgo's own
``OpenAICompatClient`` uses, but defined here in everos so the
provider can be constructed from :class:`everos.config.LLMSettings`
and handed to everalgo extractors via the ``llm=`` per-call parameter.
Keeps the provider lean (matches the everalgo minimum-viable shape):
no multi-key rotation, no scenario-level routing, no token-usage
collector — those are deployment concerns layered on top.
"""
from __future__ import annotations
from collections.abc import Mapping
from typing import Any, Literal
import openai
from .protocol import ChatMessage, ChatResponse, LLMError, Usage
class OpenAIProvider:
"""Thin async wrapper over ``openai.AsyncOpenAI``.
Structurally satisfies :class:`everalgo.llm.LLMClient` (PEP 544);
instances can be passed directly to everalgo operators that accept
``llm: LLMClient | None``.
Args:
model: Default model id (override per-call with ``model=`` on
:meth:`chat`).
api_key: Bearer credential. Pass as plain ``str`` — settings
should unwrap :class:`pydantic.SecretStr` at the factory
boundary.
base_url: OpenAI-compatible endpoint (e.g.
``"https://openrouter.ai/api/v1"``).
timeout: Per-request timeout in seconds.
temperature: Default sampling temperature (overridable per call).
max_tokens: Default max-tokens cap (overridable per call).
"""
def __init__(
self,
*,
model: str,
api_key: str,
base_url: str | None = None,
timeout: float = 60.0,
temperature: float = 0.0,
max_tokens: int | None = None,
) -> None:
self._model = model
self._temperature = temperature
self._max_tokens = max_tokens
self._client = openai.AsyncOpenAI(
api_key=api_key,
base_url=base_url,
timeout=timeout,
)
async def chat(
self,
messages: list[ChatMessage],
*,
model: str | None = None,
temperature: float | None = None,
max_tokens: int | None = None,
response_format: Mapping[str, Any] | None = None,
**extra: Any,
) -> ChatResponse:
"""Send a chat completion request and return the parsed response."""
request: dict[str, Any] = {
"model": model or self._model,
"messages": [m.model_dump() for m in messages],
"temperature": (
temperature if temperature is not None else self._temperature
),
}
effective_max = max_tokens if max_tokens is not None else self._max_tokens
if effective_max is not None:
request["max_tokens"] = effective_max
if response_format is not None:
request["response_format"] = dict(response_format)
request.update(extra)
try:
completion = await self._client.chat.completions.create(**request)
except openai.OpenAIError as exc:
raise LLMError(str(exc)) from exc
choice = completion.choices[0]
usage: Usage | None = None
if completion.usage is not None:
usage = Usage(
prompt_tokens=completion.usage.prompt_tokens,
completion_tokens=completion.usage.completion_tokens,
)
return ChatResponse(
content=choice.message.content or "",
model=completion.model,
usage=usage,
finish_reason=_normalise_finish_reason(choice.finish_reason),
raw=None,
)
def _normalise_finish_reason(
value: str | None,
) -> Literal["stop", "length", "content_filter"] | None:
if value in ("stop", "length", "content_filter"):
return value # type: ignore[return-value]
return None

39
src/everos/component/llm/protocol.py Normal file
View File

@ -0,0 +1,39 @@
"""LLM client protocol re-export.
The structural contract every everos LLM provider satisfies is the same
:class:`everalgo.llm.LLMClient` Protocol — everos providers must be
pass-through-compatible with the everalgo extractors that accept an
``llm=`` parameter. Re-exporting the type here keeps the import path
stable (``everos.component.llm``) even if the everalgo namespace
shifts later.
The :class:`ChatMessage` / :class:`ChatResponse` / :class:`Usage`
shapes are likewise re-exported so callers can build / inspect chat
payloads without reaching into the everalgo package directly.
"""
from __future__ import annotations
from everalgo.llm import (
ChatMessage as ChatMessage,
)
from everalgo.llm import (
ChatResponse as ChatResponse,
)
from everalgo.llm import (
LLMClient as LLMClient,
)
from everalgo.llm import (
LLMError as LLMError,
)
from everalgo.llm import (
Usage as Usage,
)
__all__ = [
"ChatMessage",
"ChatResponse",
"LLMClient",
"LLMError",
"Usage",
]

34
src/everos/component/rerank/__init__.py Normal file
View File

@ -0,0 +1,34 @@
"""Rerank provider adapters (one provider per file).
Public surface:
- :class:`RerankProvider` — Protocol every provider satisfies.
- :class:`RerankResult` / :class:`RerankError` — value type + error.
- :class:`DeepInfraRerankProvider` — DeepInfra inference-API rerank.
- :class:`VllmRerankProvider` — OpenAI-compat ``/v1/rerank`` (vLLM,
self-hosted, other compatible servers).
- :func:`build_rerank_provider` — settings-driven factory that picks
the concrete provider via ``settings.rerank.provider``.
External usage::
from everos.component.rerank import build_rerank_provider
provider = build_rerank_provider(settings.rerank)
scored = await provider.rerank("how to file a claim", documents)
"""
from .deepinfra_provider import DeepInfraRerankProvider as DeepInfraRerankProvider
from .factory import build_rerank_provider as build_rerank_provider
from .protocol import RerankError as RerankError
from .protocol import RerankProvider as RerankProvider
from .protocol import RerankResult as RerankResult
from .vllm_provider import VllmRerankProvider as VllmRerankProvider
__all__ = [
"DeepInfraRerankProvider",
"RerankError",
"RerankProvider",
"RerankResult",
"VllmRerankProvider",
"build_rerank_provider",
]

196
src/everos/component/rerank/deepinfra_provider.py Normal file
View File

@ -0,0 +1,196 @@
"""DeepInfra inference-API rerank provider.
DeepInfra exposes reranker models (e.g. ``Qwen/Qwen3-Reranker-4B``) at::
POST {base_url}/{model}
Authorization: Bearer <api_key>
Content-Type: application/json
The request shape is the inference-API convention used across DeepInfra
reranker / classifier models:
{
"queries": ["<query>"],
"documents": ["<doc 1>", "<doc 2>", ...]
}
The response carries one ``scores`` array per query:
{
"scores": [[0.12, 0.87, 0.43, ...]],
"request_id": "...",
"inference_status": {...}
}
We submit one query at a time (matches the :class:`RerankProvider`
contract) and unwrap the inner score list. Documents longer than the
model's input window are silently truncated server-side.
"""
from __future__ import annotations
import asyncio
from collections.abc import Sequence
from typing import Any
import httpx
from .protocol import RerankError, RerankResult
# Qwen3-Reranker chat template. The DeepInfra inference API treats the reranker
# as a yes/no generator, so the prompt scaffolding must be supplied client-side
# (verbatim mirror of the EverCore benchmark's reranker client). Without it the
# model scores raw text off-template and returns uncalibrated relevance.
_QWEN3_PREFIX = (
"<|im_start|>system\n"
"Judge whether the Document meets the requirements based on the Query and "
'the Instruct provided. Note that the answer can only be "yes" or "no".'
"<|im_end|>\n<|im_start|>user\n"
)
_QWEN3_SUFFIX = "<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"
_DEFAULT_RERANK_INSTRUCTION = (
"Given a question and a passage, determine if the passage contains "
"information relevant to answering the question."
)
def _format_qwen3_inputs(
query: str, documents: list[str], instruction: str | None
) -> tuple[str, list[str]]:
"""Wrap query + documents in the Qwen3-Reranker chat template."""
instr = instruction or _DEFAULT_RERANK_INSTRUCTION
formatted_query = f"{_QWEN3_PREFIX}<Instruct>: {instr}\n<Query>: {query}\n"
formatted_docs = [f"<Document>: {doc}{_QWEN3_SUFFIX}" for doc in documents]
return formatted_query, formatted_docs
class DeepInfraRerankProvider:
"""Rerank provider for the DeepInfra inference API.
Args:
model: Reranker model id (e.g. ``"Qwen/Qwen3-Reranker-4B"``).
api_key: Bearer credential as plain ``str``.
base_url: Inference endpoint root
(e.g. ``"https://api.deepinfra.com/v1/inference"``). The
``/{model}`` suffix is appended at request time.
timeout: Per-request timeout, seconds.
max_retries: Soft retry count on transport errors / 5xx.
batch_size: Cap on documents per request (large doc lists are
split, scores merged in input order).
max_concurrent: Cap on in-flight requests across all batches.
"""
def __init__(
self,
*,
model: str,
api_key: str,
base_url: str,
timeout: float = 30.0,
max_retries: int = 3,
batch_size: int = 10,
max_concurrent: int = 5,
) -> None:
self._model = model
self._api_key = api_key
self._url = f"{base_url.rstrip('/')}/{model}"
self._timeout = timeout
self._max_retries = max_retries
self._batch_size = batch_size
self._semaphore = asyncio.Semaphore(max_concurrent)
async def rerank(
self,
query: str,
documents: Sequence[str],
*,
instruction: str | None = None,
) -> list[RerankResult]:
"""Score every document against ``query``; return sorted desc."""
if not documents:
return []
chunks: list[tuple[int, list[str]]] = [
(offset, list(documents[offset : offset + self._batch_size]))
for offset in range(0, len(documents), self._batch_size)
]
chunk_scores = await asyncio.gather(
*(self._score_chunk(query, docs, instruction) for _, docs in chunks)
)
scored: list[RerankResult] = []
for (offset, _), scores in zip(chunks, chunk_scores, strict=True):
scored.extend(
RerankResult(index=offset + i, score=score)
for i, score in enumerate(scores)
)
scored.sort(key=lambda r: r.score, reverse=True)
return scored
async def _score_chunk(
self, query: str, documents: list[str], instruction: str | None
) -> list[float]:
formatted_query, formatted_docs = _format_qwen3_inputs(
query, documents, instruction
)
payload: dict[str, Any] = {
"queries": [formatted_query],
"documents": formatted_docs,
}
headers = {
"Authorization": f"Bearer {self._api_key}",
"Content-Type": "application/json",
}
async with self._semaphore:
for attempt in range(self._max_retries + 1):
try:
async with httpx.AsyncClient(timeout=self._timeout) as client:
response = await client.post(
self._url, json=payload, headers=headers
)
except httpx.HTTPError as exc:
if attempt == self._max_retries:
raise RerankError(
f"DeepInfra rerank transport failure: {exc}"
) from exc
continue
if response.status_code == 200:
return _extract_scores(response.json(), len(documents))
# Retry on 5xx / 429 only; surface 4xx immediately.
if response.status_code >= 500 or response.status_code == 429:
if attempt == self._max_retries:
raise RerankError(
f"DeepInfra rerank HTTP {response.status_code}: "
f"{response.text[:200]}"
)
continue
raise RerankError(
f"DeepInfra rerank HTTP {response.status_code}: "
f"{response.text[:200]}"
)
raise RerankError(
f"DeepInfra rerank exhausted retries ({self._max_retries})"
)
def _extract_scores(body: dict[str, Any], expected_len: int) -> list[float]:
"""Unwrap ``scores`` from the DeepInfra response body.
Inference API returns ``scores`` as either:
- ``[[s1, s2, ...]]`` — one score row per query (current single-query
shape); take row 0.
- ``[s1, s2, ...]`` — flat list (fallback for providers that drop
the outer list when only one query is sent).
"""
raw = body.get("scores")
if not isinstance(raw, list):
raise RerankError(f"DeepInfra rerank response missing scores: {body!r}")
row = raw[0] if raw and isinstance(raw[0], list) else raw
if len(row) != expected_len:
raise RerankError(
f"DeepInfra rerank returned {len(row)} scores, expected {expected_len}"
)
return [float(s) for s in row]

74
src/everos/component/rerank/factory.py Normal file
View File

@ -0,0 +1,74 @@
"""Factory for building a rerank provider from :class:`RerankSettings`.
The ``provider`` field on :class:`RerankSettings` selects which concrete
implementation to build:
- ``"deepinfra"`` → :class:`DeepInfraRerankProvider`
- ``"vllm"`` → :class:`VllmRerankProvider`
Adding a new provider = one match arm here + one new file under
:mod:`everos.component.rerank`.
"""
from __future__ import annotations
from everos.config import RerankSettings
from .deepinfra_provider import DeepInfraRerankProvider
from .protocol import RerankProvider
from .vllm_provider import VllmRerankProvider
def build_rerank_provider(settings: RerankSettings) -> RerankProvider:
"""Build a rerank provider from settings.
Args:
settings: The :class:`RerankSettings` slice from
:func:`everos.config.load_settings`.
Returns:
A :class:`RerankProvider` ready to call ``rerank``.
Raises:
ValueError: If ``model`` or ``base_url`` is unset, or if
``provider`` does not match a known implementation.
``api_key`` is required for ``deepinfra``; optional (empty
string) for ``vllm`` self-hosted endpoints.
"""
if not settings.model:
raise ValueError(
"Rerank model is not configured "
"(set EVEROS_RERANK__MODEL or [rerank] model in user toml)"
)
if not settings.base_url:
raise ValueError(
"Rerank base_url is not configured (set EVEROS_RERANK__BASE_URL)"
)
api_key = settings.api_key.get_secret_value() if settings.api_key else ""
if settings.provider == "deepinfra":
if not api_key:
raise ValueError(
"DeepInfra rerank api_key is not configured "
"(set EVEROS_RERANK__API_KEY)"
)
return DeepInfraRerankProvider(
model=settings.model,
api_key=api_key,
base_url=settings.base_url,
timeout=settings.timeout_seconds,
max_retries=settings.max_retries,
batch_size=settings.batch_size,
max_concurrent=settings.max_concurrent,
)
if settings.provider == "vllm":
return VllmRerankProvider(
model=settings.model,
api_key=api_key,
base_url=settings.base_url,
timeout=settings.timeout_seconds,
max_retries=settings.max_retries,
batch_size=settings.batch_size,
max_concurrent=settings.max_concurrent,
)
raise ValueError(f"unknown rerank provider: {settings.provider!r}")

62
src/everos/component/rerank/protocol.py Normal file
View File

@ -0,0 +1,62 @@
"""Rerank provider protocol.
The contract every rerank provider satisfies: given a query and a list
of candidate documents, return a re-ordered list of ``(index, score)``
pairs (highest relevance first). The provider does **not** filter —
that's the caller's job (e.g. drop scores below a threshold, take
``top_k``). Returning every input pair keeps the contract stable
across providers whose backends may not natively support ``top_n``.
"""
from __future__ import annotations
from collections.abc import Sequence
from typing import NamedTuple, Protocol, runtime_checkable
class RerankError(Exception):
"""Raised on any provider-side rerank failure."""
class RerankResult(NamedTuple):
"""One scored document from a rerank call.
``index`` is the position of the document in the *input* list (so
callers can map back to the original document text). ``score`` is
provider-defined; higher = more relevant.
"""
index: int
score: float
@runtime_checkable
class RerankProvider(Protocol):
"""Async rerank provider contract."""
async def rerank(
self,
query: str,
documents: Sequence[str],
*,
instruction: str | None = None,
) -> list[RerankResult]:
"""Score and re-order ``documents`` against ``query``.
Args:
query: The search query.
documents: Passage texts to score against ``query``.
instruction: Task instruction for instruction-tuned rerankers
(e.g. Qwen3-Reranker). Providers that wrap the model's chat
template fold this into the prompt; providers backed by a
dedicated rerank endpoint that handles templating server-side
may ignore it. ``None`` defers to the provider's default.
Returns:
One :class:`RerankResult` per input document, sorted by
``score`` descending. The returned list length equals
``len(documents)``.
"""
...

173
src/everos/component/rerank/vllm_provider.py Normal file
View File

@ -0,0 +1,173 @@
"""vLLM rerank provider.
Self-deployed vLLM (and other OpenAI-compatible rerank servers) expose
the OpenAI-style rerank endpoint::
POST {base_url}/rerank
Authorization: Bearer <api_key> # optional for self-hosted ("EMPTY")
Content-Type: application/json
Request body:
{
"model": "<model>",
"query": "<query>",
"documents": ["<doc 1>", "<doc 2>", ...]
}
Response body:
{
"results": [
{"index": 0, "relevance_score": 0.87},
{"index": 1, "relevance_score": 0.43},
...
],
"id": "...",
...
}
We pass documents through as-is — caller is responsible for any
prompt-template formatting required by the underlying reranker. Output
ordering may already be score-descending; we sort defensively to honour
the :class:`RerankProvider` contract regardless of server behaviour.
"""
from __future__ import annotations
import asyncio
from collections.abc import Sequence
from typing import Any
import httpx
from .protocol import RerankError, RerankResult
class VllmRerankProvider:
"""Rerank provider for vLLM / OpenAI-compat ``/v1/rerank`` endpoints.
Args:
model: Reranker model id (e.g. ``"Qwen/Qwen3-Reranker-4B"``).
api_key: Bearer credential. Pass ``""`` (empty string) for
self-hosted endpoints that don't require auth — the
``Authorization`` header is omitted in that case.
base_url: API root that *contains* the ``/v1`` prefix
(e.g. ``"http://localhost:8000/v1"``). The ``/rerank``
suffix is appended at request time.
timeout: Per-request timeout, seconds.
max_retries: Soft retry count on transport errors / 5xx.
batch_size: Cap on documents per request.
max_concurrent: Cap on in-flight requests across all batches.
"""
def __init__(
self,
*,
model: str,
api_key: str,
base_url: str,
timeout: float = 30.0,
max_retries: int = 3,
batch_size: int = 10,
max_concurrent: int = 5,
) -> None:
self._model = model
self._api_key = api_key
self._url = f"{base_url.rstrip('/')}/rerank"
self._timeout = timeout
self._max_retries = max_retries
self._batch_size = batch_size
self._semaphore = asyncio.Semaphore(max_concurrent)
async def rerank(
self,
query: str,
documents: Sequence[str],
*,
instruction: str | None = None,
) -> list[RerankResult]:
"""Score every document against ``query``; return sorted desc.
``instruction`` is accepted for protocol parity but not transmitted:
the OpenAI-compatible ``/rerank`` endpoint applies the reranker's chat
template (including any task instruction) server-side, so unlike the
DeepInfra completion-style API there is no client-side template to fill.
"""
if not documents:
return []
chunks: list[tuple[int, list[str]]] = [
(offset, list(documents[offset : offset + self._batch_size]))
for offset in range(0, len(documents), self._batch_size)
]
chunk_results = await asyncio.gather(
*(self._score_chunk(query, docs) for _, docs in chunks)
)
scored: list[RerankResult] = []
for (offset, _), partial in zip(chunks, chunk_results, strict=True):
scored.extend(
RerankResult(index=offset + r.index, score=r.score) for r in partial
)
scored.sort(key=lambda r: r.score, reverse=True)
return scored
async def _score_chunk(
self, query: str, documents: list[str]
) -> list[RerankResult]:
payload: dict[str, Any] = {
"model": self._model,
"query": query,
"documents": documents,
}
headers: dict[str, str] = {"Content-Type": "application/json"}
if self._api_key:
headers["Authorization"] = f"Bearer {self._api_key}"
async with self._semaphore:
for attempt in range(self._max_retries + 1):
try:
async with httpx.AsyncClient(timeout=self._timeout) as client:
response = await client.post(
self._url, json=payload, headers=headers
)
except httpx.HTTPError as exc:
if attempt == self._max_retries:
raise RerankError(
f"vLLM rerank transport failure: {exc}"
) from exc
continue
if response.status_code == 200:
return _parse_rerank_results(response.json())
if response.status_code >= 500 or response.status_code == 429:
if attempt == self._max_retries:
raise RerankError(
f"vLLM rerank HTTP {response.status_code}: "
f"{response.text[:200]}"
)
continue
raise RerankError(
f"vLLM rerank HTTP {response.status_code}: {response.text[:200]}"
)
raise RerankError(f"vLLM rerank exhausted retries ({self._max_retries})")
def _parse_rerank_results(body: dict[str, Any]) -> list[RerankResult]:
items = body.get("results")
if not isinstance(items, list):
raise RerankError(f"vLLM rerank response missing results: {body!r}")
parsed: list[RerankResult] = []
for item in items:
try:
parsed.append(
RerankResult(
index=int(item["index"]),
score=float(item["relevance_score"]),
)
)
except (KeyError, TypeError, ValueError) as exc:
raise RerankError(f"malformed rerank result entry: {item!r}") from exc
return parsed

24
src/everos/component/tokenizer/__init__.py Normal file
View File

@ -0,0 +1,24 @@
"""Tokenizer provider — sync app-layer tokenisation for BM25 indexing.
Public surface:
- :class:`Tokenizer` — Protocol every provider satisfies.
- :class:`JiebaTokenizer` — default jieba-backed implementation.
- :func:`build_tokenizer` — factory returning the configured tokenizer.
External usage::
from everos.component.tokenizer import build_tokenizer
tk = build_tokenizer()
tokens = tk.tokenize("hello 世界") # ['hello', '世界']
"""
from .factory import build_tokenizer as build_tokenizer
from .jieba_provider import JiebaTokenizer as JiebaTokenizer
from .protocol import Tokenizer as Tokenizer
__all__ = [
"JiebaTokenizer",
"Tokenizer",
"build_tokenizer",
]

17
src/everos/component/tokenizer/factory.py Normal file
View File

@ -0,0 +1,17 @@
"""Factory for the cascade-time tokenizer.
Single implementation today (``JiebaTokenizer``). Lifting this into a
factory keeps callers (cascade handler) decoupled from the concrete
choice, so swapping to char-bigram / hf tokenizer later is a one-file
change — see ``17_lancedb_tables_design.md`` §2.4.1.
"""
from __future__ import annotations
from .jieba_provider import JiebaTokenizer
from .protocol import Tokenizer
def build_tokenizer() -> Tokenizer:
"""Build the default tokenizer (``JiebaTokenizer``)."""
return JiebaTokenizer()

141
src/everos/component/tokenizer/jieba_provider.py Normal file
View File

@ -0,0 +1,141 @@
"""Jieba-based tokenizer — covers CJK + English mixed content.
Uses ``jieba.cut_for_search`` (search-mode segmentation: yields both the
greedy max-match segment and its finer sub-segments for compound CJK
words). Same mode as the legacy enterprise keyword-search path uses on
the query side — keeping cascade write and search query symmetric is
the hard contract for BM25 recall to work.
After segmentation we drop:
* whitespace / empty tokens (so the join-on-space output stays clean),
* tokens shorter than ``min_token_length`` (default 2 — same threshold
enterprise's ``filter_stopwords(min_length=2)`` uses; single-char
fragments mostly hurt BM25 precision),
* tokens in a small bilingual stopword set (Chinese function words +
English articles / prepositions / aux verbs).
"""
from __future__ import annotations
from collections.abc import Sequence
from typing import Final
import jieba
# Small bilingual stopword set. Intentionally tight (not a full
# Chinese stopword list) so the behaviour is predictable; callers
# tuning recall can subclass / extend.
_DEFAULT_STOPWORDS: Final[frozenset[str]] = frozenset(
{
# English — articles / prepositions / aux verbs that dominate BM25
# idf-noise but add no recall value.
"the",
"a",
"an",
"and",
"or",
"but",
"if",
"of",
"to",
"in",
"on",
"at",
"by",
"for",
"with",
"as",
"is",
"are",
"was",
"were",
"be",
"been",
"being",
"do",
"does",
"did",
"has",
"have",
"had",
"this",
"that",
"these",
"those",
"it",
"its",
# Chinese — function words / particles. ``cut_for_search`` emits
# these as single-char tokens anyway, and the min_length=2 floor
# would drop most; listing them explicitly makes the intent clear
# and is a no-op when min_length filtering also kicks in.
"的",
"了",
"和",
"是",
"在",
"我",
"你",
"他",
"她",
"它",
"也",
"都",
"就",
"还",
"或",
"及",
"与",
"对",
"把",
"被",
"有",
"没",
"不",
"啊",
"吗",
"呢",
"吧",
"哦",
}
)
_DEFAULT_MIN_TOKEN_LENGTH: Final[int] = 2
class JiebaTokenizer:
"""Tokenizer that calls into ``jieba.cut_for_search`` and filters."""
def __init__(
self,
*,
min_token_length: int = _DEFAULT_MIN_TOKEN_LENGTH,
extra_stopwords: frozenset[str] | None = None,
) -> None:
# Touching ``jieba.initialize()`` here would force eager dict load
# at import time and balloon test-collection latency. ``jieba.cut*``
# lazy-loads on first call instead.
self._min_len = min_token_length
self._stopwords = (
_DEFAULT_STOPWORDS | extra_stopwords
if extra_stopwords
else _DEFAULT_STOPWORDS
)
def tokenize(self, text: str) -> list[str]:
if not text:
return []
out: list[str] = []
for raw in jieba.cut_for_search(text):
tok = raw.strip().lower()
if not tok or tok.isspace():
continue
if len(tok) < self._min_len:
continue
if tok in self._stopwords:
continue
out.append(tok)
return out
def tokenize_batch(self, texts: Sequence[str]) -> list[list[str]]:
return [self.tokenize(t) for t in texts]

37
src/everos/component/tokenizer/protocol.py Normal file
View File

@ -0,0 +1,37 @@
"""Tokenizer protocol.
App-layer tokenisation gates every BM25-indexed field in LanceDB
(``17_lancedb_tables_design.md`` §2.4.1): the source surface form lives
in ``<field>`` while the space-joined token stream lives in
``<field>_tokens``, and the FTS index reads only the latter using a
whitespace tokenizer. Keeping the tokenizer decision in the app layer
means it can swap (jieba → unigram → hf) without re-indexing or
touching LanceDB schemas.
The protocol is sync — every concrete tokenizer in scope today (jieba,
char-bigram, regex word-split) is CPU-bound with no IO, so an async
wrapper would just shuffle work onto the event loop. If a future GPU
or remote tokenizer needs IO it should add an async method explicitly.
"""
from __future__ import annotations
from collections.abc import Sequence
from typing import Protocol, runtime_checkable
@runtime_checkable
class Tokenizer(Protocol):
"""Sync tokeniser contract used by the cascade handler."""
def tokenize(self, text: str) -> list[str]:
"""Return the ordered token list for ``text``.
Implementations must drop empty / whitespace-only tokens so the
resulting space-joined string never carries adjacent spaces.
"""
...
def tokenize_batch(self, texts: Sequence[str]) -> list[list[str]]:
"""Tokenise many strings, preserving input order."""
...

22
src/everos/component/utils/__init__.py Normal file
View File

@ -0,0 +1,22 @@
"""Common utilities (datetime, tokenization, etc.).
Public API:
from everos.component.utils.datetime import (
UtcDatetime,
ensure_utc,
from_iso_format,
from_timestamp,
get_now_with_timezone,
get_utc_now,
to_date_str,
to_display_tz,
to_iso_format,
to_timestamp_ms,
today_with_timezone,
)
from everos.component.utils.tokenize import (
tokens_for_index,
tokens_for_query,
join_tokens,
)
"""

263
src/everos/component/utils/datetime.py Normal file
View File

@ -0,0 +1,263 @@
"""Timezone-aware datetime helpers.
EverOS follows a **two-zone discipline**:
* **Storage** (SQLite + LanceDB) is always UTC. Use :func:`get_utc_now`
for any ``default_factory`` / write-path timestamp; if you accept a
``datetime`` from a caller, normalise with :func:`ensure_utc` before
it crosses the persistence boundary.
* **Display** (markdown frontmatter, HTTP API response, date buckets for
daily-log filenames) uses the configured "display timezone" from
:attr:`everos.config.MemorySettings.timezone` (``EVEROS_MEMORY__TIMEZONE``).
Use :func:`get_now_with_timezone` / :func:`today_with_timezone` /
:func:`to_display_tz` here.
The display timezone also serves as the **fallback timezone for naive
input**: if a caller hands us a string / datetime without offset (e.g.
a hand-written ISO timestamp), :func:`from_iso_format` attaches the
display timezone before further processing — that matches a human's
intuition ("if I didn't say a zone, you should assume my zone").
Never call :func:`datetime.datetime.now` /
:func:`datetime.datetime.utcnow` directly — see
:doc:`.claude/rules/datetime-handling`.
Cache invalidation in tests::
load_settings.cache_clear()
_display_tz.cache_clear()
"""
from __future__ import annotations
import datetime as _dt
from functools import cache
from typing import Annotated
from zoneinfo import ZoneInfo
from pydantic import AfterValidator
_MS_THRESHOLD = 1e12 # ts >= this is treated as milliseconds
@cache
def _display_tz() -> _dt.tzinfo:
"""Resolve the configured **display timezone** (cached).
Reads :attr:`everos.config.MemorySettings.timezone`; that field
validates the name with :class:`zoneinfo.ZoneInfo` at load time, so
by the time we reach here the value is guaranteed valid. This
timezone governs:
1. ISO output rendered in markdown / API responses.
2. The fallback zone attached to naive-input datetimes.
It does **not** govern storage — see :func:`get_utc_now`.
"""
# Lazy import to avoid pulling in pydantic-settings at module load.
from everos.config import load_settings
return ZoneInfo(load_settings().memory.timezone)
def get_utc_now() -> _dt.datetime:
"""Return the current time as a UTC-aware datetime.
Use for any **storage** write-path (SQLite ``default_factory``,
LanceDB row construction, OME event ``ts``, any internal "when
did this happen" record). Independent of the display timezone — a
new deployment that switches ``EVEROS_MEMORY__TIMEZONE`` will not
misalign existing rows.
Display-side code should use :func:`get_now_with_timezone` instead,
or render via :func:`to_display_tz`.
"""
return _dt.datetime.now(tz=_dt.UTC)
def get_now_with_timezone() -> _dt.datetime:
"""Return the current time in the **display timezone** (configured).
Use for **display** write-paths only — markdown frontmatter values,
daily-log date buckets, places where a human will see the literal
string. The returned datetime carries the display timezone offset
so ``.isoformat()`` produces something like
``2026-05-29T14:00:00+08:00``.
For storage / internal "when did this happen" timestamps use
:func:`get_utc_now` instead — display timezone must not bleed into
persisted rows.
"""
return _dt.datetime.now(tz=_display_tz())
def today_with_timezone() -> _dt.date:
"""Return today's date in the **display timezone**.
Use this anywhere a *date bucket* is needed (e.g. daily-log file
boundaries) — it normalises ``get_now_with_timezone().date()`` so
the timezone fallback rules are applied consistently.
"""
return get_now_with_timezone().date()
def ensure_utc(d: _dt.datetime | None) -> _dt.datetime | None:
"""Normalise any datetime to UTC at the **storage boundary**.
Semantics:
* ``None`` → ``None`` (nullable-column convenience: lets callers
pipe ``ensure_utc(row.last_attempt_at)`` without an outer guard).
* Aware input → ``astimezone(UTC)``.
* **Naive input → assume UTC** (attach ``tzinfo=UTC``); no
display-tz fallback.
Why naive→UTC rather than naive→display→UTC? Every caller of this
function sits at the storage boundary, and the dominant naive
source is SQLite reads: SQLAlchemy strips tz on write so what
comes back is a naive value whose bytes are UTC. Treating those
naive reads as display-tz would drift by the configured offset on
every round trip — exactly the bug Q2 prevents.
Caller-supplied datetimes that may genuinely be naive in display
tz (e.g. ISO strings from HTTP request bodies that omitted the
offset) should be funnelled through :func:`from_iso_format` first,
which encodes the "if you didn't say a zone, assume your zone"
rule. The aware result then passes through ``ensure_utc`` as a
pure ``astimezone(UTC)``.
Use the :data:`UtcDatetime` ``Annotated`` type to apply this
automatically on Pydantic model fields.
"""
if d is None:
return None
if d.tzinfo is None:
return d.replace(tzinfo=_dt.UTC)
return d.astimezone(_dt.UTC)
def to_display_tz(d: _dt.datetime | None) -> _dt.datetime | None:
"""Convert a datetime to the **display timezone** (configured).
Used at the **response render boundary**: any datetime leaving the
system through an API response or markdown body passes through
here so the user sees their wall-clock time with the matching
``+HH:MM`` offset.
* ``None`` → ``None`` (nullable-column convenience).
* Naive input is treated as already display-tz local (the fallback
rule) — attach the zone and return as-is.
* Aware input is ``astimezone(...)``-d to the display tz.
"""
if d is None:
return None
if d.tzinfo is None:
return d.replace(tzinfo=_display_tz())
return d.astimezone(_display_tz())
UtcDatetime = Annotated[_dt.datetime, AfterValidator(ensure_utc)]
"""Pydantic-friendly ``datetime`` type that normalises to UTC.
Apply to any SQLModel / Pydantic ``datetime`` field that maps to a
storage column. Both INSERT default values and post-read values pass
through :func:`ensure_utc`, so SQLite's tz-stripping behaviour is
neutralised: rows go in as UTC and come out as UTC-aware.
Usage::
from everos.component.utils.datetime import UtcDatetime, get_utc_now
class MyRow(BaseTable, table=True):
happened_at: UtcDatetime = Field(default_factory=get_utc_now)
"""
def from_timestamp(ts: int | float) -> _dt.datetime:
"""Parse a Unix timestamp into a timezone-aware datetime.
Auto-detects seconds vs milliseconds: values ``>= 1e12`` are treated as
milliseconds. Returned datetime is in the default timezone.
"""
seconds = ts / 1000.0 if ts >= _MS_THRESHOLD else float(ts)
return _dt.datetime.fromtimestamp(seconds, tz=_display_tz())
def from_iso_format(value: _dt.datetime | int | float | str) -> _dt.datetime:
"""Parse a value into a timezone-aware datetime (strict).
Accepted inputs:
* ``datetime`` — naive values get the default timezone attached.
* ``int`` / ``float`` — Unix timestamp (auto-detect seconds vs ms).
* ``str`` — ISO-8601, including ``"Z"`` suffix for UTC.
Raises:
TypeError: On unsupported input type.
ValueError: On malformed string / negative timestamp.
"""
if isinstance(value, _dt.datetime):
if value.tzinfo is None:
return value.replace(tzinfo=_display_tz())
return value
if isinstance(value, bool): # bool is an int subclass — reject explicitly
raise TypeError("from_iso_format does not accept bool")
if isinstance(value, int | float):
return from_timestamp(value)
if isinstance(value, str):
s = value.strip()
# Python's fromisoformat accepts "+HH:MM" but not the "Z" suffix; map it.
if s.endswith("Z"):
s = s[:-1] + "+00:00"
parsed = _dt.datetime.fromisoformat(s)
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=_display_tz())
return parsed
raise TypeError(
f"from_iso_format: unsupported type {type(value).__name__}; "
"expected datetime / int / float / str"
)
def to_iso_format(
value: _dt.datetime | int | float | str | None,
) -> str | None:
"""Render a value as an ISO-8601 string (timezone-aware).
Accepted inputs:
* ``None`` — returns ``None`` (nullable column convenience).
* ``datetime`` — rendered as-is (must already be tz-aware).
* ``int`` / ``float`` — interpreted via :func:`from_timestamp`.
* ``str`` — re-validated through :func:`from_iso_format`.
"""
if value is None:
return None
if isinstance(value, _dt.datetime):
return value.isoformat()
if isinstance(value, bool): # bool is an int subclass
raise TypeError("to_iso_format does not accept bool")
if isinstance(value, int | float):
return from_timestamp(value).isoformat()
if isinstance(value, str):
if not value:
return None
return from_iso_format(value).isoformat()
raise TypeError(
f"to_iso_format: unsupported type {type(value).__name__}; "
"expected datetime / int / float / str / None"
)
def to_date_str(d: _dt.datetime | None) -> str | None:
"""Render the date portion of a datetime as ``YYYY-MM-DD``.
Accepts ``None`` for nullable database columns. When the input is
already a :class:`datetime.date`, call ``d.isoformat()`` directly.
"""
if d is None:
return None
return d.date().isoformat()
def to_timestamp_ms(d: _dt.datetime) -> int:
"""Convert a datetime to a Unix timestamp in milliseconds."""
return int(d.timestamp() * 1000)

37
src/everos/config/__init__.py Normal file
View File

@ -0,0 +1,37 @@
"""Configuration data and Settings schema.
Public API:
from everos.config import (
Settings, MemorySettings, SqliteSettings, LanceDBSettings,
LLMSettings, EmbeddingSettings, RerankSettings,
BoundaryDetectionSettings,
load_settings,
)
Distinct from ``everos.component.config`` (which is a *capability* —
loader / merger / env reader).
"""
from .settings import BoundaryDetectionSettings as BoundaryDetectionSettings
from .settings import EmbeddingSettings as EmbeddingSettings
from .settings import LanceDBSettings as LanceDBSettings
from .settings import LLMSettings as LLMSettings
from .settings import MemorySettings as MemorySettings
from .settings import MultimodalSettings as MultimodalSettings
from .settings import RerankSettings as RerankSettings
from .settings import Settings as Settings
from .settings import SqliteSettings as SqliteSettings
from .settings import load_settings as load_settings
__all__ = [
"BoundaryDetectionSettings",
"EmbeddingSettings",
"LLMSettings",
"LanceDBSettings",
"MemorySettings",
"MultimodalSettings",
"RerankSettings",
"Settings",
"SqliteSettings",
"load_settings",
]

137
src/everos/config/default.toml Normal file
View File

@ -0,0 +1,137 @@
# everos default configuration.
#
# Lookup order (later overrides earlier):
# 1. This file (shipped defaults; lowest priority)
# 2. ~/.everos/config.toml — user-level overrides (optional;
# path is overridable via EVEROS_CONFIG_FILE)
# 3. .env file in the working directory
# 4. Environment variables — EVEROS_<SECTION>__<KEY>
# e.g. EVEROS_SQLITE__BUSY_TIMEOUT_MS=10000
# 5. Programmatic init args (highest priority)
#
# `null` (omitted in TOML) means "use the Pydantic default declared in code".
[memory]
# memory-root is the single directory holding all persisted memory.
# `~` is expanded; the path is resolved when MemoryRoot is constructed.
root = "~/.everos"
# Effective timezone for date buckets and timestamps. Drives
# component.utils.datetime; this is the SOLE source — OS `TZ` is not
# read. Override via `EVEROS_MEMORY__TIMEZONE` env var if needed.
timezone = "UTC"
[api]
# HTTP server bind for ``everos server start``. Default ``127.0.0.1``
# keeps the API on loopback only — EverOS ships no built-in auth (see
# SECURITY.md threat model). Only set ``host = "0.0.0.0"`` after you
# have placed your own gateway / auth layer in front of the server.
# Override via EVEROS_API__HOST and EVEROS_API__PORT.
host = "127.0.0.1"
port = 8000
[sqlite]
# PRAGMA journal_mode — WAL is the recommended high-concurrency mode.
journal_mode = "WAL"
# PRAGMA synchronous — NORMAL is safe under WAL and ~2x faster than FULL.
synchronous = "NORMAL"
# PRAGMA foreign_keys — must be explicitly enabled per connection.
foreign_keys = true
# PRAGMA temp_store — MEMORY keeps query intermediates in RAM (no IO impact
# on durability — only affects sort/group/temp-table calculation buffers).
temp_store = "MEMORY"
# PRAGMA busy_timeout — milliseconds to wait on a locked DB before erroring.
busy_timeout_ms = 5000
# PRAGMA journal_size_limit — cap WAL/journal at ~64 MB.
journal_size_limit_bytes = 67108864
# PRAGMA cache_size — KB of page cache (per connection).
cache_size_kb = 2048
[lancedb]
# Read consistency interval in seconds.
# omitted / null -> no consistency check (fastest reads)
# 0 -> strict (every read checks updates)
# >0 -> eventual (interval seconds between checks)
# Uncomment to override:
# read_consistency_seconds = 5.0
[llm]
# Provider-agnostic OpenAI-protocol client config. Override via env:
# EVEROS_LLM__MODEL, EVEROS_LLM__API_KEY, EVEROS_LLM__BASE_URL
# Or via a ``.env`` file next to the project root (auto-loaded).
model = "gpt-4o-mini"
# api_key = ""
# base_url = ""
[multimodal]
# Independent LLM for multimodal parsing (everalgo-parser); must accept
# image / pdf / audio image_url parts. Override via env:
# EVEROS_MULTIMODAL__MODEL, EVEROS_MULTIMODAL__API_KEY, EVEROS_MULTIMODAL__BASE_URL
model = "google/gemini-3-flash-preview"
max_concurrency = 4
# api_key = ""
# base_url = ""
# file:// content-item support (read locally by EverOS, not everalgo).
# file_uri_allow_dirs: empty = allow any readable file (local-first default);
# list base dirs to confine reads when the API is exposed.
# file_uri_allow_dirs = ["/srv/uploads"]
# file_uri_max_bytes = 52428800 # 50 MiB cap per file:// asset
[embedding]
# OpenAI-compatible embedding endpoint. Override via env:
# EVEROS_EMBEDDING__MODEL, EVEROS_EMBEDDING__API_KEY, EVEROS_EMBEDDING__BASE_URL
# model / api_key / base_url have no shipped defaults — must be set
# (env or user toml) before the embedding capability is used.
# model = "Qwen/Qwen3-Embedding-4B"
# api_key = ""
# base_url = "https://api.example.com/v1"
timeout_seconds = 30.0
max_retries = 3
batch_size = 10
max_concurrent = 5
[rerank]
# Rerank provider. Override via env:
# EVEROS_RERANK__PROVIDER, EVEROS_RERANK__MODEL, EVEROS_RERANK__API_KEY,
# EVEROS_RERANK__BASE_URL
# `provider` picks the request-shape:
# - "deepinfra" -> POST {base_url}/{model} (DeepInfra inference API)
# - "vllm" -> POST {base_url}/rerank (OpenAI-compat rerank endpoint)
provider = "deepinfra"
# model = "Qwen/Qwen3-Reranker-4B"
# api_key = ""
# base_url = "https://api.deepinfra.com/v1/inference"
timeout_seconds = 30.0
max_retries = 3
batch_size = 10
max_concurrent = 5
[boundary_detection]
# Passed through to ``everalgo.BoundaryDetector.adetect``.
hard_token_limit = 65536
hard_msg_limit = 500
[search]
# Vector retrieval strategy when SearchMethod.VECTOR is selected.
# "maxsim_atomic" (default): ANN over atomic_fact.vector (pool=top_k*20),
# max-pool the per-fact cosine by parent memcell, then reverse-resolve
# to episode rows. MaxSim over atomic facts; +0.6pp over the legacy
# episode-vector path on LoCoMo, at the cost of one extra LanceDB scan.
# "episode": single-vector ANN over episode.vector (legacy path).
# Override via EVEROS_SEARCH__VECTOR_STRATEGY.
vector_strategy = "maxsim_atomic"
[memorize]
# Conversation mode. Selects the boundary detector and which pipelines run:
# "chat" -> BoundaryDetector + user_memory only
# "agent" -> AgentBoundaryDetector + user_memory + agent_memory
# A single service process serves one mode at a time; switching mode
# requires a restart. Override via EVEROS_MEMORIZE__MODE.
mode = "agent"
# Maximum wall-clock for one memorize() invocation while holding the
# per-session lock. On timeout the outer asyncio.timeout cancels the call
# and the lock auto-releases so subsequent concurrent /add on the same
# session aren't deadlocked. Covers boundary LLM + memcell writes +
# synchronous portion of pipeline dispatch.
# Override via EVEROS_MEMORIZE__SESSION_LOCK_TIMEOUT_SECONDS.
session_lock_timeout_seconds = 360.0

64
src/everos/config/default_ome.toml Normal file
View File

@ -0,0 +1,64 @@
# everos OME (Offline Memory Engine) — per-strategy overrides.
#
# This file is materialised at ``<memory-root>/ome.toml`` by
# ``MemoryRoot.ensure()`` on first server start. Edit it to toggle
# individual strategies or tweak their gate / retry / cron without
# restarting the server; the engine watches this file and hot-reloads
# changes within ~2 seconds. Re-running ``ensure()`` will NOT overwrite
# your edits — the file is only materialised when absent.
#
# Overrides are partial: only the keys you set replace the in-code
# defaults; omitted keys keep each strategy's coded value. Unknown
# keys (typos) raise StartupValidationError, so you cannot silently
# misconfigure a strategy.
#
# Per-strategy schema (StrategyOverride):
# enabled = bool # disable a strategy entirely
# max_retries = int >= 0 # re-fire on failure
# cron = str # replace the @cron(...) trigger
# idle_seconds = int > 0 # replace @idle(...) idle window
# scan_interval_seconds = int > 0 # paired with idle_seconds; must be <= idle/2
# [strategies.<name>.gate] # only valid on @counter-gated strategies
# threshold = int > 0 # counter trigger threshold
# cooldown_seconds = int >= 0 # min seconds between fires
# event_field = str # dispatch field for counter increment
# ── User-memory pipeline ────────────────────────────────────────────────
# Atomic fact extraction (runs per memcell). Disable to skip fact mining.
# [strategies.extract_atomic_facts]
# enabled = true
# Foresight extraction (runs per memcell). Heavy LLM call — common to
# disable in evaluation / benchmark runs.
# [strategies.extract_foresight]
# enabled = false
# Profile clustering trigger (counter-gated; fires once N user memcells
# accumulate). Lower the threshold to cluster more aggressively.
# [strategies.trigger_profile_clustering]
# enabled = true
# [strategies.trigger_profile_clustering.gate]
# threshold = 5
# User-profile extraction (runs after clustering trigger fires). Common
# to disable in evaluation runs where ground-truth profiles aren't measured.
# [strategies.extract_user_profile]
# enabled = false
# ── Agent-memory pipeline ───────────────────────────────────────────────
# Agent case extraction (runs per agent memcell). One per tool call cycle.
# [strategies.extract_agent_case]
# enabled = true
# Skill clustering trigger (counter-gated; fires once N agent cases
# accumulate per agent).
# [strategies.trigger_skill_clustering]
# enabled = true
# [strategies.trigger_skill_clustering.gate]
# threshold = 5
# Agent skill extraction (runs after skill clustering trigger fires).
# [strategies.extract_agent_skill]
# enabled = true

0
src/everos/config/prompt_slots/.gitkeep Normal file
View File

0
src/everos/config/prompt_slots/__init__.py Normal file
View File

20
src/everos/config/prompt_slots/boundary_detection.yaml Normal file
View File

@ -0,0 +1,20 @@
# Custom prompt slot for BoundaryDetector.adetect.
#
# Default behaviour
# Leave this slot disabled (``enabled: false``). The pipeline will pass
# ``prompt=None`` through to algo, which falls back to the everalgo
# bundled default prompt — see:
# ~/everalgo/packages/everalgo-boundary/src/everalgo/boundary/prompts/en/chat.py
# (constant ``CHAT_BOUNDARY_DETECT_PROMPT_EN``)
#
# To customise
# 1. Read the algo default at the path above; note the required
# placeholders ``{messages}`` and ``{token_count}``.
# 2. Replace the ``template`` body below with your prompt.
# 3. Flip ``enabled`` to ``true``.
#
# When ``enabled: false`` or ``template`` is empty, the pipeline sends
# ``prompt=None`` and the algo default is used (zero override cost).
enabled: false
template: ""

23
src/everos/config/prompt_slots/episode_extract.yaml Normal file
View File

@ -0,0 +1,23 @@
# Custom prompt slot for EpisodeExtractor.aextract.
#
# Default behaviour
# Leave this slot disabled (``enabled: false``). The pipeline will pass
# ``prompt=None`` through to algo, which falls back to the everalgo
# bundled default prompt — see:
# everalgo/user_memory/prompts/en/episode.py
# (the pipeline calls ``aextract`` with ``sender_id=None``, so the
# whole-memcell ``EPISODE_GENERATION_PROMPT`` is used, not the
# per-user ``USER_EPISODE_GENERATION_PROMPT``)
#
# To customise
# 1. Read the algo default at the path above; note the required
# placeholders ``{conversation_start_time}``, ``{conversation}`` and
# ``{custom_instructions}``.
# 2. Replace the ``template`` body below with your prompt.
# 3. Flip ``enabled`` to ``true``.
#
# When ``enabled: false`` or ``template`` is empty, the pipeline sends
# ``prompt=None`` and the algo default is used (zero override cost).
enabled: false
template: ""

403
src/everos/config/settings.py Normal file
View File

@ -0,0 +1,403 @@
"""Application settings.
Loaded by :func:`load_settings`. Source priority (later wins):
1. ``config/default.toml`` (shipped values; lowest priority)
2. ``~/.everos/config.toml`` (user-level overrides; optional)
3. ``.env`` file in the working directory (secrets / machine-specific)
4. ``EVEROS_<SECTION>__<KEY>`` environment variables
5. Init args passed programmatically (highest priority)
The user-level toml path defaults to ``~/.everos/config.toml``. Override
with the ``EVEROS_CONFIG_FILE`` environment variable. The file is
optional — if it does not exist, the source is silently skipped.
The settings tree mirrors the TOML structure: ``settings.sqlite.busy_timeout_ms``
maps to ``[sqlite].busy_timeout_ms`` and to ``EVEROS_SQLITE__BUSY_TIMEOUT_MS``.
``load_settings`` is ``functools.cache``-d so callers in hot paths (e.g.
:mod:`everos.component.utils.datetime`) don't re-parse the TOML on every
call. Tests that mutate environment variables must call
``load_settings.cache_clear()`` after the mutation to invalidate.
"""
from __future__ import annotations
import os
from functools import cache
from pathlib import Path
from typing import Literal
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
from pydantic import BaseModel, Field, SecretStr, field_validator
from pydantic_settings import (
BaseSettings,
PydanticBaseSettingsSource,
SettingsConfigDict,
TomlConfigSettingsSource,
)
_DEFAULT_TOML_PATH = Path(__file__).parent / "default.toml"
_USER_TOML_ENV_VAR = "EVEROS_CONFIG_FILE"
_DEFAULT_USER_TOML_PATH = Path("~/.everos/config.toml").expanduser()
def _resolve_user_toml_path() -> Path:
"""Resolve the user-level ``config.toml`` path.
Defaults to ``~/.everos/config.toml``; override with the
``EVEROS_CONFIG_FILE`` environment variable.
"""
override = os.environ.get(_USER_TOML_ENV_VAR)
return Path(override).expanduser() if override else _DEFAULT_USER_TOML_PATH
class MemorySettings(BaseModel):
"""memory-root configuration."""
root: Path = Path("~/.everos")
timezone: str = "UTC"
"""Effective timezone for date buckets and timestamps.
Default ``"UTC"``. Override via ``[memory] timezone = "..."`` in
TOML or ``EVEROS_MEMORY__TIMEZONE`` env var. Validated against
:class:`zoneinfo.ZoneInfo` at load time, so an invalid name fails
fast (no silent fallback). This is the **sole** source of truth for
the project's effective timezone — the OS ``TZ`` env var is *not*
consulted, keeping the configuration deterministic.
"""
@field_validator("timezone")
@classmethod
def _validate_timezone(cls, v: str) -> str:
try:
ZoneInfo(v)
except (ZoneInfoNotFoundError, ValueError) as exc:
raise ValueError(f"invalid timezone: {v!r}") from exc
return v
class ApiSettings(BaseModel):
"""HTTP API server bind configuration.
Default ``host = "127.0.0.1"`` keeps the server on loopback only,
matching the threat model in ``SECURITY.md``: EverOS ships **no
built-in authentication**, so binding to a routable interface
(``0.0.0.0`` etc.) without your own gateway / auth layer in front
is unsupported.
Env binding:
EVEROS_API__HOST
EVEROS_API__PORT
"""
host: str = "127.0.0.1"
port: int = Field(default=8000, ge=1, le=65535)
class SqliteSettings(BaseModel):
"""SQLite tunables applied as PRAGMAs on every new connection."""
journal_mode: Literal["WAL", "DELETE", "MEMORY", "OFF", "TRUNCATE", "PERSIST"] = (
"WAL"
)
synchronous: Literal["FULL", "NORMAL", "OFF", "EXTRA"] = "NORMAL"
foreign_keys: bool = True
temp_store: Literal["DEFAULT", "FILE", "MEMORY"] = "MEMORY"
busy_timeout_ms: int = Field(default=5000, ge=0)
journal_size_limit_bytes: int = Field(default=64 * 1024 * 1024, ge=0)
cache_size_kb: int = Field(default=2048, ge=0)
class LLMSettings(BaseModel):
"""LLM client configuration.
Read by the service layer when lazily constructing the LLM client
handed to algo extractors. Provider-agnostic field names — the
project follows the OpenAI API protocol so any OpenAI-compatible
endpoint plugs in via ``base_url``.
Env binding (via parent ``Settings``):
EVEROS_LLM__MODEL
EVEROS_LLM__API_KEY
EVEROS_LLM__BASE_URL
"""
model: str = "gpt-4o-mini"
api_key: SecretStr | None = None
base_url: str | None = None
class MultimodalSettings(BaseModel):
"""Multimodal parsing LLM config (everalgo-parser).
Flat section mirroring ``[llm]``. The model must accept multimodal
``image_url`` parts (image / pdf / audio); it is kept independent from
the main ``[llm]`` so parsing can target a vision/audio-capable
endpoint without affecting boundary / extraction.
Env binding (via parent ``Settings``):
EVEROS_MULTIMODAL__MODEL
EVEROS_MULTIMODAL__API_KEY
EVEROS_MULTIMODAL__BASE_URL
EVEROS_MULTIMODAL__MAX_CONCURRENCY
EVEROS_MULTIMODAL__FILE_URI_ALLOW_DIRS
EVEROS_MULTIMODAL__FILE_URI_MAX_BYTES
"""
model: str = "google/gemini-3-flash-preview"
api_key: SecretStr | None = None
base_url: str | None = None
max_concurrency: int = 4
# ``file://`` content-item support (read locally by EverOS, not everalgo).
file_uri_allow_dirs: list[str] = []
"""Allowlisted base dirs for ``file://`` uris. Empty = allow any readable
file (local-first default); set to confine reads when the API is exposed."""
file_uri_max_bytes: int = 50 * 1024 * 1024
"""Max size (bytes) of a ``file://`` asset; larger files are rejected."""
class EmbeddingSettings(BaseModel):
"""Embedding client configuration.
OpenAI-compatible embedding endpoint. ``model`` / ``api_key`` /
``base_url`` are required at runtime when the embedding capability
is enabled; the runtime knobs (``timeout`` etc.) have sensible
defaults.
Env binding:
EVEROS_EMBEDDING__MODEL
EVEROS_EMBEDDING__API_KEY
EVEROS_EMBEDDING__BASE_URL
EVEROS_EMBEDDING__TIMEOUT_SECONDS
EVEROS_EMBEDDING__MAX_RETRIES
EVEROS_EMBEDDING__BATCH_SIZE
EVEROS_EMBEDDING__MAX_CONCURRENT
"""
model: str | None = None
api_key: SecretStr | None = None
base_url: str | None = None
timeout_seconds: float = Field(default=30.0, gt=0)
max_retries: int = Field(default=3, ge=0)
batch_size: int = Field(default=10, ge=1)
max_concurrent: int = Field(default=5, ge=1)
class RerankSettings(BaseModel):
"""Rerank client configuration.
Unlike LLM / embedding (single OpenAI-compatible shape), rerank API
schemas differ between providers — DeepInfra uses ``POST {base_url}/
{model}`` with a custom body, vLLM uses ``POST {base_url}/rerank``
with ``{model, query, documents}``. ``provider`` picks which client
implementation the factory builds.
Env binding:
EVEROS_RERANK__PROVIDER
EVEROS_RERANK__MODEL
EVEROS_RERANK__API_KEY
EVEROS_RERANK__BASE_URL
EVEROS_RERANK__TIMEOUT_SECONDS
EVEROS_RERANK__MAX_RETRIES
EVEROS_RERANK__BATCH_SIZE
EVEROS_RERANK__MAX_CONCURRENT
"""
provider: Literal["deepinfra", "vllm"] = "deepinfra"
model: str | None = None
api_key: SecretStr | None = None
base_url: str | None = None
timeout_seconds: float = Field(default=30.0, gt=0)
max_retries: int = Field(default=3, ge=0)
batch_size: int = Field(default=10, ge=1)
max_concurrent: int = Field(default=5, ge=1)
class BoundaryDetectionSettings(BaseModel):
"""Hard limits passed through to ``everalgo`` BoundaryDetector."""
hard_token_limit: int = Field(default=65536, ge=1)
hard_msg_limit: int = Field(default=500, ge=1)
class MemorizeSettings(BaseModel):
"""Memorize use-case configuration.
``mode`` selects which boundary detector runs and which pipelines are
dispatched. A service process serves one mode at a time; toggling
requires a restart.
- ``"chat"`` -> ``everalgo.user_memory.BoundaryDetector`` and only the
user-memory pipeline runs.
- ``"agent"`` -> ``everalgo.agent_memory.AgentBoundaryDetector`` and
both user-memory + agent-memory pipelines run.
``session_lock_timeout_seconds`` caps how long one ``memorize()``
invocation can hold the per-session lock. Covers boundary LLM call +
memcell DB writes + (synchronous portion of) pipeline dispatch. Stops
a stuck LLM from deadlocking subsequent concurrent calls on the same
session_id: on timeout the outer ``asyncio.timeout`` cancels the task
and the lock auto-releases.
Env binding:
EVEROS_MEMORIZE__MODE
EVEROS_MEMORIZE__SESSION_LOCK_TIMEOUT_SECONDS
"""
mode: Literal["chat", "agent"] = "agent"
session_lock_timeout_seconds: float = Field(default=360.0, gt=0)
class SearchSettings(BaseModel):
"""Search-pipeline policy knobs.
``vector_strategy`` selects the read path taken by
``SearchMethod.VECTOR``:
- ``"maxsim_atomic"`` (default) — ANN over ``atomic_fact.vector``
(recall pool ``top_k * 20``, capped at 2000), max-pool the per-fact
cosine by parent memcell, then reverse-resolve the top memcells back
to episode rows. MaxSim over atomic facts; trades one extra LanceDB
scan for finer-grained semantic match on long episodes.
- ``"episode"`` — single-vector ANN over ``episode.vector`` (one vector
per episode = the embedded Content section). The legacy path; kept
so deployments can opt out via env.
Env binding:
EVEROS_SEARCH__VECTOR_STRATEGY={episode,maxsim_atomic}
"""
vector_strategy: Literal["episode", "maxsim_atomic"] = "maxsim_atomic"
class LanceDBSettings(BaseModel):
"""LanceDB tunables.
``read_consistency_seconds``:
``None`` (omitted) → no consistency check (highest performance).
``0`` → strict consistency (every read).
``>0`` → eventual (interval between checks).
``index_cache_size_bytes``:
Upper bound on LanceDB's global *index* cache (``GlobalIndexCache``
in lance crate). Each cached entry is one opened FTS / vector /
scalar index reader and **holds the file descriptors of its on-disk
``_indices/<uuid>/...`` files**.
LanceDB's own default is ``None`` (unbounded), which on a long-
running daemon means every new index UUID created by an
``optimize()`` call adds a fresh reader to the cache, and its
FDs are never released — they leak monotonically until
``EMFILE`` (os error 24). Verified locally: 30 optimize cycles
take FD usage from 0 to ~960 against macOS's default ``ulimit -n``
of 256 / Linux's 1024.
Setting a byte cap turns the cache into a real LRU: when it
exceeds the cap, the oldest readers are dropped, Rust ``Drop``
runs ``close(fd)``, and the FD pressure resolves itself.
Cap → steady-state FD upper bound (measured under 30 add+optimize
cycles with the real ``Episode`` schema and 100-query stress):
=========== ================= ===================
cap FD upper bound query latency (100q)
=========== ================= ===================
``2 MB`` ~45 ~5 ms
``4 MB`` ~52 ~3 ms
``8 MB`` ~140 ~2.4 ms
``16 MB`` ~290 ~2.3 ms ← default
``32 MB`` ~630 ~1.4 ms
``unbound`` >960 (leaks) ~1.3 ms
=========== ================= ===================
EverOS's measured steady-state working set after a 12 h
``rebuild_indexes`` cycle is ~50-100 readers / 3-6 MB resident
(5 tables × ~7 BM25 columns × ~10 part_N entries each), so
``16 MB`` gives ~3× headroom for burst traffic and stale-but-not-
yet-evicted readers, while the FD ceiling (~290) stays well below
common ulimits (macOS default 256 needs ``ulimit -n 1024`` first;
Linux default 1024 is fine out of the box).
Override via ``EVEROS_LANCEDB__INDEX_CACHE_SIZE_BYTES`` if your
working set is much larger (heavier table count or much wider
indexes) or if you hit a tighter ``ulimit -n`` (containers / dev
boxes).
Note: the *metadata* cache (``metadata_cache_size_bytes``) is
**not** exposed — experiment showed it caches in-memory parsed
manifests / fragment stats with zero impact on FD count; leaving
it unbounded (lancedb default) is fine.
"""
read_consistency_seconds: float | None = None
index_cache_size_bytes: int = 16 * 1024 * 1024
class Settings(BaseSettings):
"""Top-level application settings."""
memory: MemorySettings = MemorySettings()
api: ApiSettings = ApiSettings()
sqlite: SqliteSettings = SqliteSettings()
lancedb: LanceDBSettings = LanceDBSettings()
llm: LLMSettings = LLMSettings()
embedding: EmbeddingSettings = EmbeddingSettings()
rerank: RerankSettings = RerankSettings()
boundary_detection: BoundaryDetectionSettings = BoundaryDetectionSettings()
memorize: MemorizeSettings = MemorizeSettings()
search: SearchSettings = SearchSettings()
multimodal: MultimodalSettings = MultimodalSettings()
model_config = SettingsConfigDict(
env_prefix="EVEROS_",
env_nested_delimiter="__",
env_file=".env",
env_file_encoding="utf-8",
toml_file=_DEFAULT_TOML_PATH,
extra="ignore",
)
@classmethod
def settings_customise_sources(
cls,
settings_cls: type[BaseSettings],
init_settings: PydanticBaseSettingsSource,
env_settings: PydanticBaseSettingsSource,
dotenv_settings: PydanticBaseSettingsSource,
file_secret_settings: PydanticBaseSettingsSource,
) -> tuple[PydanticBaseSettingsSource, ...]:
"""Layer TOML sources between env / dotenv and the secret store.
Order (earlier wins in pydantic-settings):
init_args > env > .env > user_toml > default_toml > secrets
The user-level toml (default ``~/.everos/config.toml``) is only
registered when the file exists, so the source list stays tight.
"""
sources: list[PydanticBaseSettingsSource] = [
init_settings,
env_settings,
dotenv_settings,
]
user_toml_path = _resolve_user_toml_path()
if user_toml_path.is_file():
sources.append(
TomlConfigSettingsSource(settings_cls, toml_file=user_toml_path)
)
sources.append(TomlConfigSettingsSource(settings_cls))
sources.append(file_secret_settings)
return tuple(sources)
@cache
def load_settings() -> Settings:
"""Load settings from default.toml + environment variables (cached).
Cached at the module level — every caller sees the same instance until
something explicitly clears the cache (``load_settings.cache_clear()``).
Tests that monkeypatch environment variables must call
``cache_clear`` after each mutation to pick the new env up.
"""
return Settings()

0
src/everos/core/__init__.py Normal file
View File

Some files were not shown because too many files have changed in this diff Show More