A data engineer has a small Python dictionary `country_codes` (well under 1 MB) that maps ISO codes to country names, and needs to look it up inside a PySpark UDF applied to a 500 million row DataFrame. To avoid Spark serializing and shipping a fresh copy of `country_codes` with every task, which change should the engineer make to the following code? ```python def lookup_country(code): return country_codes.get(code, "Unknown") lookup_udf = udf(lookup_country, StringType()) df = df.withColumn("country_name", lookup_udf(col("iso_code"))) ```
Select one answer.
Purchase Full Access to view answer options & explanation
The first 118 questions in this bank are completely free. Unlock Full Access on the exam page to view all answer choices, verified answers, explanations, and community discussions for all 392 questions.
