You have deployed a deep learning model to an Agent Platform endpoint using a machine type with NVIDIA GPUs. You initially configured the endpoint to autoscale based on a target CPU utilization of 60%. During a load test, you observe that prediction latency increases significantly as traffic rises, but the number of replicas remains constant. Cloud Monitoring indicates that CPU utilization stays below 40%, while the GPU utilization consistently exceeds 90%. You need to ensure that the endpoint scales efficiently to handle the increased load. What should you do?
Select one answer.
Purchase Full Access to view answer options & explanation
The first 110 questions in this bank are completely free. Unlock Full Access on the exam page to view all answer choices, verified answers, explanations, and community discussions for all 364 questions.
