PMLE Question 102

Community Discussion

Written by other learners. Comments can disagree with the marked answer — check the official documentation when in doubt.

• Chose D5

  When using distributed training with tf.distribute.MirroredStrategy, each GPU processes a slice of the batch. If you keep the batch size constant, each GPU receives a smaller effective batch, which might not fully utilize the computational power of each device. Increasing the batch size allows each GPU to process more data in parallel, which can lead to improved training speed and better resource utilization without modifying your training loop or switching strategies

• Chose A1

  I will go with A. By using tf.distribute.Strategy.experimental_distribute_dataset we can ensure that the dataset is effectively split across the GPUs, which will help fully utilize the GPUs and achieve faster training times. Increasing the batch size can improve training performance on GPUs by allowing them to process more data in parallel. However, if the dataset is not properly distributed across GPUs, simply increasing the batch size won't lead to improved training times. In fact, using a larger batch size can lead to memory bottlenecks if not handled correctly. The key here is to first ensure proper data distribution before tweaking batch size.

• Chose A1

  A is the right answer