A financial services company receives a regular data feed from its credit card servicing partner. Approximately 5,000 records are sent every 15 minutes in plaintext, delivered over HTTPS directly into an Amazon S3 bucket with server-side encryption. This feed contains sensitive credit card primary account number (PAN) data. The company needs to automatically mask the PAN before sending the data to another S3 bucket for additional internal processing. The company also needs to remove and merge specific fields, and then transform the record into JSON format. Additionally, extra feeds are likely to be added in the future, so any design needs to be easily expandable. Which solutions will meet these requirements?
Select one answer.
Marked answer: C
Community Discussion
Written by other learners. Comments can disagree with the marked answer — check the official documentation when in doubt.
Chose C1
When you need to process and transform sensitive batch data from S3 (mask, remove fields, reformat) and make it scalable for future feeds, AWS Glue is ideal. It’s serverless, integrates natively with S3, and supports complex ETL in a single managed job. Avoid multi-step Lambda/SQS/Fargate setups unless streaming or ultra-low latency is required, and avoid EMR unless big data scale or specialized frameworks are needed.
Chose C1
Option C
Chose C3
Option C is the most suitable solution for the described scenario: 1) AWS Glue Crawler and Custom Classifier: Use AWS Glue to create a crawler and custom classifier to understand and catalogue the data feed formats. This step ensures that AWS Glue can work with the incoming data effectively. 2) AWS Glue ETL Job: Create an AWS Lambda function that triggers an AWS Glue ETL job when a new data file is delivered. This ETL job can perform the required transformation, including masking, field removal, and converting records to JSON format. AWS Glue is a suitable service for data preparation and transformation. 3) Output to S3 Bucket. This approach is scalable, easily expandable to handle additional feeds in the future, and leverages AWS Glue's capabilities for data transformation and processing. It also maintains a clear separation of tasks, making it a robust and efficient solution for the given requirements.
