Databricks Certified Data Engineer Professional Practice Mode

Community Discussion

Written by other learners. Comments can disagree with the marked answer — check the official documentation when in doubt.

• Chose C2

  Since is using SDC Type 1 so the this will not keep the historical change. Option C is more makesense.

• Chose C1

  Correct: C - We process only tens of thousands of new records instead of scanning millions of historical records. Using the max 'last_login' per user ensures merging only the most recent state for each user from that batch. The MERGE satement efficiently handles both updates and inserts (SCD Type 1). Incorrect: A - The 'account_history' is a table, not a file directory. Auto Loader subscribes to file directories in object storage, not Delta Tables. B - Overwritting the entire table by scanning all of 'account_history' records every hour is extremely inefficient when processing millions of records to update only tens of thousands. D - Using Delta version history adds unnecessary complexity and overhead compared to direct timestamp filtering. E - Username may not be unique across users. The 'user_id' is defind as the primary key in the schema.

• Chose C1

  Many people who are fixated on the "overwrite" in option B have overlooked the fact, that the overwrite is happening by querying the history table, grouping on user_id and then filtering (probably a having clause) the records on the max(last_updated). What do you think will happen to the other records which have not been updated in the last hour?. they will get overwritten and this will lead to missing records. Option C is the best option here.