Continous increase in Memory usage

batch_size=8,

I see, so the pipeline is suspicious. Then, this is about the only thing that is suspicious. There is also the fact that the batch increases the memory required, but it is also simply prone to bugs.