Technical Note: VRAM Thermal Saturation during Flux.1 / SDXL Inference on Laptops

Models
1

Hi everyone! :waving_hand:

I’ve been spending some time profiling how the new Flux.1 and SD 3.5 Large models impact laptop thermals during sustained local inference runs.

What I found is a pretty significant “telemetry gap” on several RTX 30 and 40-series mobile chips. Even when the GPU core stays at a stable ~75°C, the Memory Junction (VRAM) often rockets to the 105°C – 108°C threshold within just a few minutes of generation.

This usually triggers a silent firmware-level throttle that most standard monitoring tools don’t even flag. The memory clocks drop by up to 40%, and your it/s takes a massive hit without any obvious warning from the GPU core temperature.

I found that global undervolting wasn’t providing the stability I needed for long batches – it often led to CUDA errors or general instability. Instead I’ve been experimenting with a “Pulse Throttling” approach. By using the Windows API to introduce millisecond-level process suspensions (specifically NtSuspendProcess), I can give the shared heat pipes enough time to shed thermal energy before the firmware slams on the brakes.

I actually ended up building a free utility called VRAM Shield to automate this logic as managing the duty cycles manually was a nightmare.

Is anyone else seeing these kinds of deltas between Core and Junction temps during long Flux runs? I’d love to compare some HWiNFO logs or hear how others are managing this thermal soak on mobile hardware.

1 Like
2

Some of this phenomenon seems to be backed up by nVidia itself:

The phenomenon described is technically plausible.

What is probably happening

This is best understood as a steady-state inference problem, not just a “GPU core temperature” problem. On supported NVIDIA devices, nvidia-smi exposes GPU Current Temp, Memory Current Temp, GPU Max Operating Temp, and Memory Max Operating Temp as separate fields. NVML also defines SW Thermal Slowdown as a condition where clocks are reduced so that GPU temperature stays below GPU Max Operating Temp and memory temperature stays below Memory Max Operating Temp. In plain English: memory-side heat can become the limiter even when the usual GPU temperature number looks acceptable. (NVIDIA Docs)

That is why the symptom can feel “silent.” Many consumer overlays focus on core temperature and utilization, but the more decisive signals are often the clock-event reasons, power limit state, and, where supported, memory temperature. NVIDIA’s own telemetry model separates thermal slowdown from power-cap slowdown, so a big drop in iterations per second can happen even when the simple headline metrics still look normal. (NVIDIA Docs)

Why Flux and SDXL make this show up

Flux is unusually heavy. Hugging Face’s current Diffusers docs say Flux is a very large model and can require roughly 50 GB of RAM/VRAM to load all components before optimization. Their memory guide also says modern diffusion models like Flux have billions of parameters and often need offloading, quantization, or other memory-saving methods to fit on common GPUs. That makes these models very good at exposing any weakness in a laptop’s long-run thermal or power behavior. (Hugging Face)

There is also a broader systems reason. NVIDIA’s TensorRT guidance says thermal throttling shows up as a workload that starts normally, temperature rises under sustained inference, and then clocks drop once thresholds are reached. The same guide also notes that poor cooling can reduce the stabilized clock even before obvious hard throttling, because hotter silicon leaks more power at a given clock. So “fast at first, much slower after a few minutes” fits a real and documented pattern. (NVIDIA Docs)

Why the “telemetry gap” part is believable

That complaint has real history. NVIDIA’s own developer forum has a long-running request to expose memory junction temperature through nvidia-smi or NVML, driven by users who were seeing throttling while ordinary GPU temperature looked fine. Current NVIDIA docs now show Memory Current Temp and Memory Max Operating Temp, but they also say those fields are available only on supported devices. So the clean conclusion is not “the telemetry is missing everywhere.” It is “telemetry is still uneven across devices and tools.” (NVIDIA Developer Forums)

What I would be careful about

I would not assume this is always a VRAM thermal story. Public issue trackers show very similar “first run is fine, later runs collapse” symptoms from other causes:

  • A Diffusers issue on an RTX 4060 Laptop GPU reported Flux going from about 8 minutes to about 30 minutes on later runs while VRAM usage stayed the same. (GitHub)
  • A ComfyUI issue on an RTX 4090 Mobile GPU reported extreme slowdown after updating ComfyUI, PyTorch, xFormers, and Triton-Windows; the user later said the culprit was triton-windows, and later that a clean reinstall fixed the environment. (GitHub)
  • NVIDIA’s TensorRT docs also say that on Windows, WDDM mode tends to produce worse and less stable inference performance than TCC, and a display-attached GPU cannot be switched to TCC. On many laptops that means you are stuck with the less stable mode by design. (NVIDIA Docs)

So the clean interpretation is this: the symptom is real, but the root cause can be thermal, power-limited, software-regression, or Windows driver-state related. The surface behavior alone does not prove which one you have. (NVIDIA Docs)

About the “pulse throttling” idea

As an experiment, it is understandable. As a primary fix, I would treat it as a last resort.

The reason is simple: NVIDIA already documents supported ways to shape behavior more cleanly:

  • power limit with -pl
  • locked GPU clocks with -lgc
  • locked memory clocks with -lmc

NVIDIA also documents the exact signals you should watch while doing this: power draw, enforced power limit, thermal slowdown reasons, and power-cap reasons. That is a better control loop than periodically suspending the whole process from user space. (NVIDIA Docs)

In other words, if the real problem is “the laptop cannot hold the peak board state for long,” then the clean fix is usually to lower the sustained load a little so the machine stays out of the cliff, not to let it hit the cliff and then pause it after the fact. NVIDIA’s performance docs explicitly separate power throttling from thermal throttling, and explain that fully loaded inference with no gaps can reveal a lower real steady-state clock than bursty testing suggests. (NVIDIA Docs)

What this means in plain English

The strongest version of the claim is:

  • Yes, a laptop can lose a lot of Flux/SDXL throughput because the memory side or the whole board steady state becomes the bottleneck while the usual GPU temperature still looks “fine.” NVIDIA’s current telemetry model supports that mechanism. (NVIDIA Docs)
  • No, that is not the only explanation for long-run slowdown. Public Flux and image-gen issues show similar collapses from software stack changes and Windows behavior. (GitHub)
  • The most accurate framing is: heavy diffusion inference is pushing a Windows laptop into a lower sustained-performance state, and the job is to identify whether the limiter is memory thermals, power limits, WDDM behavior, or a broken software path. (NVIDIA Docs)

The simplest practical takeaway

If someone describes this pattern, my first reaction is:

  1. Believe the symptom. It is plausible. (NVIDIA Docs)
  2. Do not trust core temperature alone. Check memory temp if exposed, clocks, power draw, and clock-event reasons. (NVIDIA Docs)
  3. Reduce sustained pressure first. For Flux, use supported memory reductions like offloading, group offloading, quantization, VAE tiling, and VAE slicing. (Hugging Face)
  4. Use supported board controls before process suspension. Power limits and clock locks are the official levers. (NVIDIA Docs)
  5. Keep software regressions in play. Flux slowdowns on laptops are not always thermal. (GitHub)

My bottom line: the core idea is credible, but the safest version is “long Flux/SDXL runs can expose memory-side or board-level steady-state limits on laptops, and basic GPU temp readouts often do not tell the whole story.” That is the part I would keep. The part I would treat more cautiously is using process suspension as the main remedy instead of first proving whether the limiter is thermal, power, or software. (NVIDIA Docs)

1 Like
3

Thanks for the solid breakdown, @John6666. You’re spot on about the telemetry gap – it’s exactly what pushed me to look for a workaround in the first place.

You’re right that official levers like -pl or -lgc via nvidia-smi are the “cleaner” way to handle GPU behavior in a perfect world. But while I was developing VRAM Shield, I ran into a few practical walls that made process-level modulation (Pulse Throttling) a necessity for the laptop segment.

The biggest issue is the locked VBIOS on so many consumer laptops. If you try running nvidia-smi -lgc on a locked Lenovo Legion or certain Acer models, half the time the commands are either flat-out ignored or the allowed range is so narrow it doesn’t actually stop the VRAM from hitting that 105°C wall during a heavy Flux run.

Then there’s the “global vs. surgical” problem. Official driver limits are a blanket fix. If I cap the GPU power globally to save the VRAM during a background render I’m nerfing the entire system. Pulse Throttling lets me target the specific compute-heavy PID while keeping the rest of the OS and other GPU-accelerated apps responsive.

And honestly, most people running local SD or LLMs aren’t CLI wizards. I wanted to build a “thermal safety net” that just works in the background and reacts to real-time sensor data, regardless of how restrictive the factory firmware is.

I completely agree that process suspension is an “extreme” measure from a pure systems architecture perspective. But in the “wild west” of laptop cooling designs and locked-down firmware it’s often the only reliable way to prevent that 40% silent firmware throttle you mentioned.

1 Like
4

It’s probably a limiter designed to prevent hardware damage, so it’s essentially designed to be hard to bypass, but well, there are times when we want to bypass it…

The next step is to make this falsifiable.

Right now the idea is technically coherent. NVIDIA exposes separate GPU temp, memory temp, power limits, and thermal/power throttle reasons, so the basic mechanism is real. But to move from “plausible workaround” to “credible tool,” you need to prove three things:

  1. What is actually causing the slowdown on each machine.
  2. Whether per-process modulation improves the long-run steady state, not just the first few minutes.
  3. Whether the workaround adds new failure modes that are worse than the original problem. (NVIDIA Docs)

What I think is happening

My current model is:

  • Hardware creates the pressure. Flux/SDXL are heavy enough to push a laptop into a bad steady state. Hugging Face documents Flux as expensive on consumer hardware and recommends offloading and other memory reductions for exactly that reason. (NVIDIA Docs)
  • Firmware enforces the cliff. NVIDIA says locked clocks only hold until power or thermal throttling occurs, and the enforced power ceiling is the minimum of several limits, not just the one requested by software. (NVIDIA Docs)
  • Windows adds variance. NVIDIA’s TensorRT guidance says inference performance under Windows display-driver conditions is less stable than compute-focused TCC setups, and most display-attached consumer laptops are stuck on the display path. (NVIDIA Docs)
  • Software determines how hard you hit the wall and can also imitate it when the stack regresses. Public Flux slowdown issues on laptops show both real steady-state collapse and stack-caused collapse. (NVIDIA Docs)

So I would investigate it as a layered steady-state problem, not as a single “VRAM is hot” story. (NVIDIA Docs)

The highest-value investigation plan

1. Build a baseline matrix, not a single anecdote

Test the same workload in these modes:

  • stock behavior
  • stock after reboot
  • -pl if supported
  • -lgc or -lmc if supported
  • your per-process modulation

Run each case for a no-gap window like 15 to 30 minutes, not one or two prompts. NVIDIA explicitly warns that gaps between inferences can make power throttling look less severe and can inflate apparent performance. That means short or bursty tests are the wrong benchmark for your use case. (NVIDIA Docs)

What to record each second or each few seconds:

  • throughput
  • per-step latency
  • GPU clock
  • memory clock
  • power draw
  • enforced power limit
  • GPU temp
  • memory temp if exposed
  • throttle reason state

That dataset will tell you whether you are actually preventing a cliff or just moving it. NVIDIA documents all of those observables. (NVIDIA Docs)

2. Separate thermal from power-cap early

This is the most important split.

NVIDIA’s throttle reasons make the distinction explicit:

  • SwThermalSlowdown means clocks are being reduced because GPU temp or memory temp has crossed the max operating threshold.
  • SwPowerCap means clocks are being reduced to stay under the current power limit. (NVIDIA Docs)

So the first question for each laptop should be:

When performance collapses, which reason activates first?

If thermal fires first, your tool is really a thermal guardrail.
If power cap fires first, you are actually compensating for board power policy, not memory temperature.
If neither fires, start suspecting Windows or the software stack. (NVIDIA Docs)

3. Use NVIDIA’s own recommended telemetry path during runs

NVIDIA’s TensorRT best-practices guide recommends using nvidia-smi -q before the run and nvidia-smi dmon -s pcu during the run to capture power, clocks, temperature, and utilization. That is the simplest official way to get a baseline without building your whole telemetry stack first. (NVIDIA Docs)

I would standardize on:

  • one snapshot from nvidia-smi -q
  • one continuous dmon log during the workload
  • your own per-PID intervention log with timestamps

Then align them on a single clock. That lets you answer: “When my controller intervened, did the GPU stop spending time in thermal slowdown, or did it just shift where the slowdown happened?” (NVIDIA Docs)

4. Add Windows ETW tracing to prove the “surgical” claim

Your strongest claim is not just “I lowered temperatures.”
It is “I protected one heavy PID without wrecking the rest of the system.”

To prove that, add Windows ETW tooling:

  • GPUView reads ETW logs and is designed to analyze GPU and CPU activity on Windows. (Microsoft Learn)
  • WPR/WPA are Microsoft’s standard tools for recording and analyzing ETW traces. (Microsoft Learn)
  • PresentMon is useful for high-level Windows graphics responsiveness and frame timing. It captures CPU/GPU/display frame metrics across DirectX, OpenGL, and Vulkan. (GitHub)

What I would look for:

  • does the foreground desktop remain responsive during modulation
  • do other GPU apps keep normal frame pacing
  • do you create bursts of scheduler starvation or long stalls around each intervention

That is the cleanest way to prove the “global vs surgical” argument instead of just asserting it. (Microsoft Learn)

5. Add a CUDA timeline so you know where you are interrupting

Use Nsight Systems next.

NVIDIA documents that Nsight Systems can trace:

  • CUDA API calls
  • CUDA kernel execution
  • CUDA memory usage over time
  • thread scheduling
  • child processes
  • GPU metrics sampling on supported systems (NVIDIA Docs)

This is important because your control loop may look effective at the board level while still interrupting the process at terrible moments, for example:

  • during heavy host-device memory activity
  • while holding CPU-side locks
  • during an allocator spike
  • during a runtime call that causes downstream instability

Nsight Systems will not solve everything, but it will show whether your pauses line up with kernel bursts, memory bursts, and context boundaries. NVIDIA also notes that CUDA memory tracking and tracing can add overhead, and that crashes can lose trace data if the device is not finalized cleanly. So use it for short diagnostic captures, not as your default telemetry path. (NVIDIA Docs)

6. Investigate safety as aggressively as performance

This is where the real engineering burden is.

Microsoft is very clear: SuspendThread is primarily for debuggers, not for synchronization, and suspending a thread that owns a mutex or critical section can deadlock other threads. (Microsoft Learn)

So I would treat safety as a first-class test matrix:

  • long batch runs, not short demos
  • multi-process scenarios
  • app close / cancel / interrupt behavior
  • pause during model load vs pause during steady generation
  • pause during save/export
  • crash recovery
  • repeated suspend/resume cycles over hours

What I would want to know:

  • do hangs ever occur
  • do CUDA errors rise
  • does shutdown become flaky
  • do orphaned suspended threads or stuck child processes appear
  • does the app remain stable after hundreds or thousands of cycles

If you do not have a strong answer here, the tool may still be useful, but it stays in “clever workaround” territory instead of “reliable safety net.” (Microsoft Learn)

The most useful experiments

Experiment A: prove that the limiter is thermal, not just power

Success condition:

  • SwThermalSlowdown decreases materially
  • sustained clocks stabilize at a higher long-run average
  • 20-minute average throughput improves
  • crash rate does not increase (NVIDIA Docs)

Failure condition:

  • only SwPowerCap changes
  • memory temp is unavailable and no thermal reason fires
  • throughput gain disappears over a long run
  • errors increase (NVIDIA Docs)

Experiment B: prove that the tool is better than a mild official limit

Even if official controls are too global, compare against the best available low-friction baseline anyway:

  • mild -pl
  • mild lower locked clock where supported
  • your pulse controller

If your controller only beats a stock run, that is not enough.
It should beat the best feasible official fallback on locked laptops often enough to justify the extra complexity. NVIDIA’s docs support using those controls where available, while also making clear that thermal and power throttling can still override requested clocks. (NVIDIA Docs)

Experiment C: prove the “surgical” benefit

Run a foreground graphics workload or normal desktop activity while the heavy PID is running.

If PresentMon and ETW show that the foreground workload remains smooth while the target PID is modulated, that is a real differentiator. If everything still stutters, then you have not actually solved the global-vs-surgical problem. (GitHub)

What I would investigate next, specifically

I would split the next phase into four tracks.

Track 1. Generality

Find out which laptops actually need this class of workaround.

Because NVIDIA says memory temperature reporting is only available on supported devices, you need to know whether your tool is most useful on machines with weak observability, weak clock control, or both. That is how you avoid overgeneralizing from a few laptop families. (NVIDIA Docs)

Track 2. Architecture

I would explore whether you can move from external forced suspension toward cooperative self-throttling where possible.

That means integrating at safer boundaries:

  • between generations
  • between denoise phases
  • at explicit checkpoints in supported apps

That is not because the current approach cannot work. It is because Microsoft’s debugger warning never goes away. A cooperative pause path, where available, is structurally safer than forcing arbitrary thread suspension. The official warning is the reason to explore this, even if you keep the external fallback. (Microsoft Learn)

Track 3. Explainability

Expose why the controller intervened.

For each intervention, log something like:

  • near memory max operating temp
  • thermal slowdown active
  • power cap active
  • clocks dropping without reason visibility

That makes the tool much more defensible, because NVIDIA’s telemetry model already supports those categories. (NVIDIA Docs)

Track 4. Boundaries

Be explicit about where the tool should not run.

Given Microsoft’s guidance, I would strongly consider:

  • opt-in allowlist only
  • no default system-wide injection behavior
  • conservative maximum duty cycle
  • watchdog and automatic disable on hangs or repeated CUDA errors

That keeps the tool framed as a targeted mitigation rather than a generic background optimizer. (Microsoft Learn)

The cleanest success criteria

I would call the investigation successful if you can show all of this:

  1. On affected laptops, the collapse correlates with NVIDIA thermal or power throttle evidence, not just a subjective feeling. (NVIDIA Docs)
  2. Per-process modulation reduces time spent in the bad state and improves long-run, not short-run, throughput. (NVIDIA Docs)
  3. Foreground responsiveness remains acceptable, proven with ETW or PresentMon rather than impression. (Microsoft Learn)
  4. Stability remains acceptable across long runs, despite Microsoft’s documented suspend risks. (Microsoft Learn)

My blunt recommendation

Do not spend the next phase arguing about whether the workaround is elegant.
It is not. Microsoft’s API guidance settles that. (Microsoft Learn)

Spend the next phase proving four claims:

  • what the limiter is
  • what your controller changes
  • what it costs
  • where it is actually better than the official alternatives (NVIDIA Docs)

That is the path from “interesting workaround” to “credible laptop-specific control layer.”

5

Appreciate the rigorous breakdown, @John6666. You’ve essentially outlined our internal engineering and QA pipeline.

Regarding the safety concerns around SuspendThread (CUDA deadlocks, thread state issues) and the need for “surgical” proof: this is exactly why VRAM Shield evolved from a basic script into a PID-controller-based system (Smart Throttling). We actively use ETW tracing and a LibreHardwareMonitor sidecar to micro-adjust the load safely, avoiding the exact crash conditions and application hangs you mentioned.

Our telemetry confirms that on locked mobile VBIOS, this micro-burst approach is often the only practical way to prevent SwThermalSlowdown on the memory junction without globally crippling the GPU power limit (since nvidia-smi is often restricted anyway).

1 Like