Which EPYC CPU for inferencing? Self-hosted build

Intermediate
1

Unsure if this is the correct forum to post this in…

I’m making plans to build a self-hosted system with five 3090s. The only goal is inferencing (at this point in time).

I am considering the Gigabyte MZ32-AR0 motherboard due to the 5 x PCIe Gen4 x16 slots.

I’m looking at the following (used) processors:

AMD EPYC 7282
AMD EPYC 7302P
AMD EPYC 7352
AMD EPYC™ 7F52

Seems as if the AMD EPYC 7352 is the most powerful. However, I plan on running the LLM entirely on the GPU. So as long as it CPU is 7200 and above…does single core or multi core really matter? Do number of cores matter 18 vs 24?

Also, does RAM speed (2400 vs 3200) and RAM quantity matter when the GPU is doing all the heavy lifting?

1 Like
3

I think you’ve already decided on this because the post is old, but assuming that you have enough VRAM, you can get by with 16GB of RAM. As long as you have 4 cores, you can get by.

Anyway, VRAM, preferably GeForce. You don’t need to think about anything else. Recently, there are more and more cases where you can get by comfortably using CUDA-compatible programs such as ZULDA even with AMD GPUs, but basically, I recommend CUDA GPUs. You shouldn’t have to bear any unnecessary hardships.