Add Hermes memory evaluation framework with LoCoMo dataset support

Browse Source

- Implement HermesClient for interacting with the Hermes CLI.
- Create judge module for grading QA outputs from Hermes memory.
- Develop LoCoMo dataset parsing and formatting utilities.
- Introduce run_eval script to facilitate memory evaluation using LoCoMo-style datasets.

...

This commit is contained in:

tomtan

2026-05-27 17:06:26 +08:00

parent ba59133d80

commit c173fa45a7

11 changed files with 68338 additions and 0 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

66751

eval/hermes_memory_eval/datasets/locomo10.json Normal file

File diff suppressed because one or more lines are too long