CCAR-P Question 38

Question

You are assessing a Claude-based system whose dominant risk is silent quality drift on safety relevant outputs after a model-version upgrade. Which assessment activity most directly addresses this risk?
A. Disable adversarial evaluation entirely during model-version upgrade cycles to reduce evaluation cost, accepting that safety drift will go undetected until it appears in production.
B. Skip evaluation on each model-version upgrade and rely on user-submitted complaints to surface safety drift after the upgraded model has already served production traffic.
C. Rotate adversarial inputs randomly so no two upgrades are scored on the same set.
D. Maintain an adversarial evaluation set with version-attributed scoring so each upgrade is measured against the same set before promotion.

Accepted Answer

Marked answer: D. Maintain an adversarial evaluation set with version-attributed scoring so each upgrade is measured against the same set before promotion.

Preparing Practice Exam…

CCAR-P Question 38

Community Discussion