A data engineer is using Auto Loader to read incoming JSON data as it arrives. They have configured Auto Loader to quarantine invalid JSON records but notice that over time, some records are being quarantined even though they are well-formed JSON. The code snippet is: df = (spark.readStream .format("cloudFiles") .option("cloudFiles.format", "json") .option("badRecordsPath", "/tmp/somewhere/badRecordsPath") .schema("a int, b int") .load("/Volumes/catalog/schema/raw_data/")) What is the cause of the missing data?
Select one answer.
Purchase Full Access to view answer options & explanation
The first 119 questions in this bank are completely free. Unlock Full Access on the exam page to view all answer choices, verified answers, explanations, and community discussions for all 395 questions.
