ivanwu/jarvis-models
1
0
Fork 0
mirror of https://github.com/BoardWare-Genius/jarvis-models.git synced 2025-12-13 16:53:24 +00:00
Code Issues Packages Projects Releases Wiki Activity

modety vlms with lmdeploy

Browse Source
This commit is contained in:
Ivan087
2024-08-20 09:41:10 +08:00
parent 30dedc935f
commit 4c3756811d
3 changed files with 357 additions and 290 deletions
Download Patch File Download Diff File Expand all files Collapse all files

187
README.md
View File

@ -1,92 +1,95 @@
# jarvis-models
## Conda Environment and Python Library Requirement
```bash
conda create -n jarvis-models python==3.10.11
pip install -r sample/requirement_out_of_pytorch.txt
pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 --index-url https://download.pytorch.org/whl/cu118
```
## More Dependencies
| System | package | web | install command |
| --- |-----------------------| --- | --- |
| python | filetype | https://pypi.org/project/filetype/ | pip install filetype |
| python | fastAPI | https://fastapi.tiangolo.com/ | pip install fastapi |
| python | python-multipart | https://pypi.org/project/python-multipart/ | pip install python-multipart |
| python | uvicorn | https://www.uvicorn.org/ | pip install "uvicorn[standard]" |
| python | SpeechRecognition | https://pypi.org/project/SpeechRecognition/ | pip install SpeechRecognition |
| python | gtts | https://pypi.org/project/gTTS/ | pip install gTTS |
| python | PyYAML | https://pypi.org/project/PyYAML/ | pip install PyYAML |
| python | injector | https://github.com/python-injector/injector | pip install injector |
| python | langchain | https://github.com/langchain-ai/langchain | pip install langchain |
| python | chromadb | https://docs.trychroma.com/getting-started | pip install chromadb |
| python | lagent | https://github.com/InternLM/lagent/blob/main/README.md | pip install lagent |
| python | sentence_transformers | https://github.com/InternLM/lagent/blob/main/README.md | pip install sentence_transformers |
## Start
Start the jarvis-models service via
```bash
uvicorn main:app --reload
```
or
```bash
python main.py
```
## Configuration
Create ".env.yaml" at the root of jarvis-models, and copy the following yaml configuration
```yaml
env:
version: 0.0.1
host: 0.0.0.0
port: 8000
log:
level: debug
time_format: "%Y-%m-%d %H:%M:%S"
filename: "D:/Workspace/Logging/jarvis/jarvis-models.log"
melotts:
mode: local # or docker
url: http://10.6.44.16:18080/convert/tts
speed: 0.9
device: 'cuda'
language: 'ZH'
speaker: 'ZH'
tesou:
url: http://120.196.116.194:48891/chat/
TokenIDConverter:
token_path: src/asr/resources/models/token_list.pkl
unk_symbol: <unk>
CharTokenizer:
symbol_value:
space_symbol: <space>
remove_non_linguistic_symbols: false
WavFrontend:
cmvn_file: src/asr/resources/models/am.mvn
frontend_conf:
fs: 16000
window: hamming
n_mels: 80
frame_length: 25
frame_shift: 10
lfr_m: 7
lfr_n: 6
filter_length_max: -.inf
dither: 0.0
Model:
model_path: src/asr/resources/models/model.onnx
use_cuda: false
CUDAExecutionProvider:
device_id: 0
arena_extend_strategy: kNextPowerOfTwo
cudnn_conv_algo_search: EXHAUSTIVE
do_copy_in_default_stream: true
batch_size: 3
blackbox:
lazyloading: true
```
# jarvis-models
## Conda Environment and Python Library Requirement
```bash
conda create -n jarvis-models python==3.10.11
pip install -r sample/requirement_out_of_pytorch.txt
pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 --index-url https://download.pytorch.org/whl/cu118
```
## More Dependencies
| System | package | web | install command |
| --- |-----------------------| --- | --- |
| python | filetype | https://pypi.org/project/filetype/ | pip install filetype |
| python | fastAPI | https://fastapi.tiangolo.com/ | pip install fastapi |
| python | python-multipart | https://pypi.org/project/python-multipart/ | pip install python-multipart |
| python | uvicorn | https://www.uvicorn.org/ | pip install "uvicorn[standard]" |
| python | SpeechRecognition | https://pypi.org/project/SpeechRecognition/ | pip install SpeechRecognition |
| python | gtts | https://pypi.org/project/gTTS/ | pip install gTTS |
| python | PyYAML | https://pypi.org/project/PyYAML/ | pip install PyYAML |
| python | injector | https://github.com/python-injector/injector | pip install injector |
| python | langchain | https://github.com/langchain-ai/langchain | pip install langchain |
| python | chromadb | https://docs.trychroma.com/getting-started | pip install chromadb |
| python | lagent | https://github.com/InternLM/lagent/blob/main/README.md | pip install lagent |
| python | sentence_transformers | https://github.com/InternLM/lagent/blob/main/README.md | pip install sentence_transformers |
## Start
Start the jarvis-models service via
```bash
uvicorn main:app --reload
```
or
```bash
python main.py
```
## Configuration
Create ".env.yaml" at the root of jarvis-models, and copy the following yaml configuration
```yaml
env:
version: 0.0.1
host: 0.0.0.0
port: 8000
log:
level: debug
time_format: "%Y-%m-%d %H:%M:%S"
filename: "D:/Workspace/Logging/jarvis/jarvis-models.log"
melotts:
mode: local # or docker
url: http://10.6.44.16:18080/convert/tts
speed: 0.9
device: 'cuda'
language: 'ZH'
speaker: 'ZH'
tesou:
url: http://120.196.116.194:48891/chat/
TokenIDConverter:
token_path: src/asr/resources/models/token_list.pkl
unk_symbol: <unk>
CharTokenizer:
symbol_value:
space_symbol: <space>
remove_non_linguistic_symbols: false
WavFrontend:
cmvn_file: src/asr/resources/models/am.mvn
frontend_conf:
fs: 16000
window: hamming
n_mels: 80
frame_length: 25
frame_shift: 10
lfr_m: 7
lfr_n: 6
filter_length_max: -.inf
dither: 0.0
Model:
model_path: src/asr/resources/models/model.onnx
use_cuda: false
CUDAExecutionProvider:
device_id: 0
arena_extend_strategy: kNextPowerOfTwo
cudnn_conv_algo_search: EXHAUSTIVE
do_copy_in_default_stream: true
batch_size: 3
blackbox:
lazyloading: true
vlms:
url: http://10.6.80.87:23333
```