ADP Question 18

Community Discussion

Written by other learners. Comments can disagree with the marked answer — check the official documentation when in doubt.

• Chose B1

  B, Since you can clean the data and analyse it using SQL. Option A, even though you can use PythonOperator to clean the data, the tool of composer doesn't have to do anything with this question itself.

• Chose B4

  The best option is B. Use BigQuery to batch load the data into BigQuery and use SQL for cleaning and analysis. Loading directly into BigQuery and using SQL provides the optimal balance of efficiency and simplicity for cleaning large datasets before analysis by leveraging BigQuery's scalable processing for both loading and transformation. Option A (Cloud Composer + PythonOperator) adds unnecessary complexity of workflow orchestration and external processing before loading, reducing efficiency. Option C (Storage Transfer Service + Cloud Run) overcomplicates the process with extra data movement and event-driven functions, making it less direct for data cleaning. Option D (Cloud Run functions) is less efficient for large-scale data cleaning compared to BigQuery SQL's parallel processing and adds complexity before data is in BigQuery for analysis. Therefore, loading into BigQuery and using SQL is the most efficient and straightforward approach for cleaning data before analysis in this scenario.

• Chose A2

  PythonOperator allows leveraging Python libraries (e.g., Pandas, PySpark) to perform robust data cleaning tasks: Handle missing values (e.g., imputation, filtering). Fix incorrect data types (e.g., string-to-date conversions). Remove duplicates (e.g., using deduplication logic).