DATABRICKS-SPARK Question 108

Community Discussion

Written by other learners. Comments can disagree with the marked answer — check the official documentation when in doubt.

• Chose A0

  ,B - Correct answer s : A, B — `coalesce()` can only decrease a DataFrame's partition count and can never raise it beyond however many partitions already exist.; `coalesce()` avoids a full shuffle by combining existing partitions on the same executor, moving less data than `repartition()`. A . Coalesce is designed only to merge existing partitions together, so it cannot raise the partition count above what the DataFrame already has; requesting more partitions than currently exist has no effect. B . By combining nearby partitions on the same executor instead of redistributing every row across the network, coalesce moves far less data than a full shuffle, which is exactly why it is the cheaper way to reduce partition count. C . Because coalesce merges whatever partitions already exist rather than hashing rows into new ones, the resulting partitions can be uneven in size, unlike the more even distribution a full shuffle in `repartition` typically produces. D . A full shuffle moving every row across the cluster is the behavior of `repartition`, not `coalesce`; coalesce specifically avoids that cost when it is only reducing the partition count. E . Coalesce only changes how a DataFrame's existing partitions are grouped together; it has no effect on the DataFrame's schema and does not drop or recompute any columns.