ExamPark
Course menuMLA-C02

Exams / MLA-C02 / Study guide / Real-time, serverless, async and batch inference

Real-time, serverless, async and batch inference: MLA-C02 study notes and practice questions

What to know about Real-time, serverless, async and batch inference for the MLA-C02 exam, from the Deployment and Orchestration of ML and AI Workflows domain (24% of the exam guide), then 3 free questions to test it. Part of the MLA-C02 study guide.

Last verified against current AWS documentation ·How we write our questions

What to know for the exam

This is one of the most tested choices on the exam. The question describes traffic and payloads; you pick the option.

The four options

Option Fits Key facts
Real-time endpoint Steady traffic needing low latency, such as checkout fraud scoring Instances run all the time; auto scaling; GPU support; short synchronous requests
Serverless inference Intermittent or unpredictable traffic with small payloads, where some cold-start delay is acceptable Pay per use, scales to zero; memory size set per endpoint; no GPUs; provisioned concurrency reduces cold starts
Asynchronous inference Large payloads (up to 1 GB) or long processing (up to an hour), such as video or long documents Requests are queued; results go to S3 with SNS notifications; can scale to zero when the queue is empty
Batch transform Scoring a whole dataset offline, such as a nightly scoring of all customers No endpoint; reads from S3, writes results to S3, then releases compute

Reading the clues

  • “Milliseconds”, “user is waiting”: real-time.
  • “A few requests an hour”, “pay only when used”: serverless.
  • “Files of several hundred MB”, “processing takes 10 minutes”: asynchronous.
  • “Every night, millions of records”, “no persistent endpoint”: batch transform.
  • Inference components let several models share one endpoint’s instances with their own scaling, and can scale to zero.
  • Inference pipelines chain preprocessing, prediction and postprocessing containers behind one endpoint.
  • Data capture saves requests and responses to S3 for monitoring.
  • Inference Recommender load-tests instance types and suggests the best fit.
  • Bedrock batch inference is the batch option for foundation models.

Common traps

  • Real-time endpoint for occasional requests, paying for idle instances.
  • Serverless for large payloads or GPUs.
  • An endpoint for a nightly job that batch transform does without one.

Try the 3 practice questions

Practice Real-time, serverless, async and batch inferenceTake the free practice exam

Question 1

Medium ·Link to this question (question 1)

A pet insurer lets customers upload short videos of an injury when filing a claim. Each video is up to a few hundred megabytes and takes three to five minutes for the vision model to score, and customers do not need the result in the same request.

The model should scale to zero when no claims are pending, so that the insurer does not pay for idle instances, and the customer app should receive a job status to poll.

Which SageMaker AI inference option meets these requirements?

Choose one answer.

  1. A. A real-time endpoint with a fixed number of GPU instances, sized for the largest video, with the app waiting on the open HTTP connection for each result.
  2. B. An asynchronous inference endpoint, which queues large requests, scales to zero when idle, and writes results to S3 for the app to poll.
  3. C. A serverless inference endpoint, which lets the app send each video directly and wait for the scored result in the response.
  4. D. A batch transform job started once an hour over all videos uploaded in that hour, with the app reading results after each job finishes.
Show the answer and explanation

Answer: B

  • A. Wrong: A real-time endpoint keeps instances running when no claims are pending, and it holds the connection open for minutes, which is not reliable. The stem asks for scale to zero and a status to poll.
  • B. Right: Correct. Asynchronous inference accepts large payloads, processes requests from a queue, and can scale to zero when there is no work. Results are written to S3, and the app checks for completion. Learn more: Asynchronous inference in SageMaker AI (opens in a new tab)
  • C. Wrong: Serverless endpoints suit smaller payloads and intermittent traffic, but this workload needs large payloads and multi-minute processing with polling. The option does not provide the queued, pollable pattern.
  • D. Wrong: Batch transform is designed for offline scoring of a dataset, and an hourly job adds delay to each claim. It does not give each customer a status they can poll.

Explanation

Asynchronous inference handles large payloads and long processing by queuing requests, writing results to S3, and it can scale to zero when idle. Real-time endpoints stay running and hold connections open, serverless endpoints target smaller payloads, and batch transform runs offline rather than per claim.

Study

Question 2

Easier ·Link to this question (question 2)

A logistics broker has an internal model that scores carrier invoices for duplicate billing. The tool is used by a few analysts during business hours, and it receives only a handful of requests per day with small payloads.

The broker wants the lowest cost for this light, unpredictable traffic and does not want to manage instance capacity, while still accepting that a request after an idle period may take a little longer.

Which endpoint type should the team choose?

Choose one answer.

  1. A. A real-time endpoint on a GPU instance that runs at all times, so that every request is answered at the same speed.
  2. B. A batch transform job that is started by hand each time an analyst wants a score for a new invoice.
  3. C. A serverless inference endpoint, which charges for the compute used during requests and needs no instance capacity to be managed.
  4. D. An Amazon EC2 instance running a web server that loads the model in memory and is managed by an auto scaling group with a minimum of two instances.
Show the answer and explanation

Answer: C

  • A. Wrong: A running GPU instance is paid for all day even though the traffic is a handful of requests. This is the highest cost for the light workload.
  • B. Wrong: Batch transform works for a set of records processed together, and it is not a request-time service for a single invoice. Starting a job by hand for each invoice adds delay and effort.
  • C. Right: Correct. Serverless inference fits intermittent traffic with small payloads, and the broker pays for the compute used rather than for idle instances. Requests after an idle period can see a slightly longer start. Learn more: Deploy models with SageMaker Serverless Inference (opens in a new tab)
  • D. Wrong: Two always-on instances cost more than the light traffic justifies, and the broker would manage the servers itself. The stem asks for lower cost with no capacity management.

Explanation

Serverless inference suits intermittent traffic with small payloads, because the broker pays for compute used and does not manage instances. A running real-time endpoint or two instances pay for idle time, and batch transform is not built for single on-demand requests.

Study

Question 3

Easier ·Link to this question (question 3)

A logistics broker scores about 40 million shipments every night to predict which ones will miss their delivery window. Results are used the next morning in a planning report, and nobody needs an answer during the day.

The team wants the scores in Amazon S3, with no endpoint running between the nightly runs.

Which approach meets these requirements?

Choose one answer.

  1. A. A real-time endpoint that runs all day, with the scoring service calling it once for each shipment in the nightly batch.
  2. B. A serverless inference endpoint that the nightly job calls for each shipment, one request at a time.
  3. C. A multi-model endpoint that loads the shipment model from S3 when a request arrives, with the nightly job sending one request per shipment.
  4. D. A batch transform job that reads the shipment files from S3, scores them with the model, and writes the predictions back to S3.
Show the answer and explanation

Answer: D

  • A. Wrong: A running endpoint is paid for all day, and calling it once per shipment is slower than a bulk job designed for offline scoring. The stem asks for no endpoint between runs.
  • B. Wrong: Calling a serverless endpoint once for each of 40 million shipments is slow and expensive, and it is a request-time pattern. Offline scoring of a dataset is what batch transform is built for.
  • C. Wrong: A multi-model endpoint is a real-time endpoint for many models, and it runs continuously. It does not remove the need for an always-on serving layer.
  • D. Right: Correct. Batch transform scores a large dataset offline, starts and stops its own compute, and writes results to S3. No endpoint needs to stay running between the nightly runs. Learn more: Batch transform for inference with SageMaker AI (opens in a new tab)

Explanation

Batch transform scores a large dataset offline, then releases its compute and writes results to S3. Real-time and serverless endpoints serve requests one at a time, and a multi-model endpoint still runs continuously.

Study

Want more on Real-time, serverless, async and batch inference? The 5 full-length practice sets have 8 more Real-time, serverless, async and batch inference questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets

Study Real-time, serverless, async and batch inference

More Deployment and Orchestration of ML and AI Workflows topics