You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Please if somebody can explain.
Why is UI on load writing to virtual memory and not using RAM or VRAM from start?
I76700
32GB RAM
12GB VRAM
When model loaded or i give input, then the ssd storage is recovered and i guess moved to VRAM?
But RAM is not used much, only like +-8GB
anon8231489123/vicuna-13b-GPTQ-4bit-128g
**
call python server.py --model_type LLaMa --gpu-memory 10737418240 --cpu-memory 21474836480 --auto-devices --chat --wbits 4 --groupsize 128 --pre_layer 40
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Please if somebody can explain.
Why is UI on load writing to virtual memory and not using RAM or VRAM from start?
I76700
32GB RAM
12GB VRAM
When model loaded or i give input, then the ssd storage is recovered and i guess moved to VRAM?
But RAM is not used much, only like +-8GB
anon8231489123/vicuna-13b-GPTQ-4bit-128g
**
call python server.py --model_type LLaMa --gpu-memory 10737418240 --cpu-memory 21474836480 --auto-devices --chat --wbits 4 --groupsize 128 --pre_layer 40
Beta Was this translation helpful? Give feedback.
All reactions