I am trying to calculate GPU memory bandwidth for an inference flow of llama-3-8B model running on GPU. To check GPU memory usage, I am using nvidia-smi command. How to calculate GPU memory bandwidth?
Related Topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| GPU memory calculator | 2 | 920 | July 5, 2024 | |
| VRAM keeps increasing during sequential llama2-13b inferencing | 1 | 264 | July 15, 2024 | |
| Llama 70b model not using GPU | 0 | 1048 | September 13, 2023 | |
| Estimate training compute for 150B LLM | 0 | 490 | June 30, 2023 | |
| Llama3 OutOfMemory on an A100 when doing CausalLM | 0 | 140 | June 5, 2024 |