DATABRICKS-DEP Question 94

Community Discussion

Written by other learners. Comments can disagree with the marked answer — check the official documentation when in doubt.

• Chose A0

  ,B,C - Correct answer s : A, B, C — Open the Executors tab to check whether the executor running the slow task is under GC pauses or memory pressure that would compound the skew.; Open the associated SQL/DataFrame tab to inspect the physical plan and confirm which operator produced the oversized shuffle partition.; Enable adaptive query execution's skew join optimization, or increase `spark.sql.shuffle.partitions`, so the oversized partition splits across more tasks. A . Checking the executor running the outlier task for GC pauses or memory pressure is a standard Spark UI workflow: a struggling executor can turn ordinary skew into an extreme long tail, and this rules that factor in or out before changing the job. B . The physical plan in the SQL/DataFrame tab identifies exactly which shuffle-producing operator (such as a join or aggregation) created the oversized partition, which is necessary before deciding how to rewrite or reconfigure the query. C . Adaptive query execution's skew join handling, or a higher shuffle partition count, directly targets an oversized shuffle partition by splitting it into smaller pieces so no single task carries a disproportionate share of the data. D . Driver memory pressure affects planning and coordination, not the size of an individual executor task's shuffle partition, so a larger driver node would not address a single skewed reducer task. E . Structured Streaming micro-batches run as ordinary Spark jobs and expose the same Stages tab with per-task shuffle metrics as batch jobs, so this claim about batch-only visibility is incorrect. F . Speculative execution reruns slow tasks on other executors as a mitigation, but disabling it removes that safety net without doing anything to rebalance the skewed partition itself, so the underlying skew remains.