I have the following build:
- 7800x3d
- 4090
- 64GB DDR5 EXPO
- x670e MS MAG Tomahawk WiFi
- RM1000x PSU
- Fractal something or other (the airflow one) XL case
I just remembered I have a 4080 collecting dust in the closet. I’m planning on installing it tomorrow right underneath the 4090. I think there will be ~1/4" clearance between the two.
Wanted to ask if there’s anything I need to do in nVidia control panel settings or OobaBooga that will allow me to use both graphics cards (and give me an additional 16GB VRAM)?
Also, do you think the speeds will increase noticeably? I’m getting 1 token/second on Llama-3-70B-Instruct-Q5_K_M. Do you think the speeds will go up another ~5 tokens per second?
I know I should be using a smaller quant—but I value intelligence over speed. Even though the speed currently sucks.