proto-tools also handles device management, so you can fully utilize your local hardware. It places each tool on an available GPU, and when memory runs low it evicts the least recently used model to make room.
1 0 5
Ben Viggiano
@bviggiano_13
Replying to @bviggiano_13
When you have a batch of work and several GPUs sitting idle, proto-tools can spread the batch across all of them automatically. The pieces run in parallel, and the results come back in the original order, so the multi-GPU execution is invisible to your code.
1 5