PDE Question 87

Community Discussion

Written by other learners. Comments can disagree with the marked answer — check the official documentation when in doubt.

• Chose A1

  Why this is the better choice: Larger Parquet files reduce the number of splits and metadata overhead. This leads to fewer tasks, less shuffle, and better parallelism. It does not increase infrastructure cost—just improves how data is structured. It aligns with Google’s best practices for Spark on Dataproc.

• Chose C1

  C is correct. It cannot be D as increasing boot disk size does not impact shuffle performance much. We need local SSDs specifically attached for shuffle storage (temporary fast storage), not just a bigger persistent boot disk. C is the correct answer because SSD + HDFS shuffle layer = fastest for Spark shuffle-heavy jobs, while still using preemptibles and keeping costs down. The job as mentioned is shuffle-intensive. In Spark, shuffling (moving data between nodes) is heavily disk I/O intensive. Faster local storage (i.e., SSDs) can dramatically speed up shuffle operations compared to using standard HDDs. GCS is great for object storage, not shuffle storage.

• Chose C1

  C is the correct answer