Guardrails and responsible AI: MLA-C02 study notes and practice questions
What to know about Guardrails and responsible AI for the MLA-C02 exam, from the Operating, Monitoring, and Securing ML and AI Solutions domain (24% of the exam guide), then 3 free questions to test it. Part of the MLA-C02 study guide.
A regional bank deploys a customer chatbot powered by a Bedrock model. The compliance team has said that the chatbot must not give investment advice, and that it must not reveal a customer's account numbers, Social Security numbers or home addresses in its responses.
The protections should apply to the model's inputs and outputs in one place, so that they do not depend on each application team writing its own filters.
Which solution meets these requirements?
Choose one answer.
AA. Add a line to the system prompt that tells the model never to give investment advice or reveal account numbers or other personal data.
BB. Have each application team write a regular expression filter in its own code to remove account numbers from the responses.
CC. Restrict the chatbot to a smaller model, since smaller models do not discuss investments or personal data.
DD. Create a Bedrock Guardrail with a denied investment-advice topic and a sensitive information filter for account numbers and addresses, attached to the model calls.
Show the answer and explanation
Answer: D
A. Wrong: A prompt instruction depends on the model following it and is not enforced on inputs or outputs. The stem asks for controls that do not depend on each application.
B. Wrong: Filters written in each application depend on every team, and they do not cover topic restrictions such as investment advice. The stem asks for one central control.
C. Wrong: Model size does not determine which topics a model discusses or whether it reveals personal data. The option does not enforce the rules.
Bedrock Guardrails can deny a topic, such as investment advice, and filter or mask sensitive information such as account numbers and addresses in both inputs and outputs. Attaching the guardrail to the model calls applies the policy centrally, while prompt rules, per-team filters and smaller models do not enforce it.
A public library runs a retrieval-augmented assistant that answers patron questions from its policy documents. Staff have found answers that are fluent but not supported by the retrieved passages, such as a fine amount that appears nowhere in the policy.
The team wants responses that are not supported by the retrieved text to be blocked before they reach patrons, and it wants answers that do not address the question to be blocked too. Neither check should require changing the policy documents.
Which Bedrock Guardrails feature should the team configure?
Choose one answer.
AA. A contextual grounding check, with a grounding threshold and a relevance threshold, applied to the response and the retrieved source passages.
BB. A word filter that blocks the words and phrases the policy documents never use, so that unsupported answers are rejected because of their vocabulary.
CC. A sensitive information filter that masks fine amounts and dollar values in each response before it is shown.
DD. Lower the model's temperature to zero and trust that the model will repeat only what the passages say.
B. Wrong: Word filters match specific words and do not compare the answer with the retrieved passages. An answer can avoid any blocked word and still be unsupported.
C. Wrong: The sensitive information filter masks personal data types, and it does not check whether an answer is supported by the retrieved text. The stem asks about grounding.
D. Wrong: Temperature reduces randomness but does not guarantee grounding, and it does not block unsupported answers. The stem asks for a check that blocks them.
Explanation
The contextual grounding check scores a response for grounding in the source passages and for relevance to the query, and it blocks responses below the configured thresholds. Word filters, personal data filters and temperature settings do not check the answer against the retrieved text.
A logistics broker has built an AI agent that reads carrier emails and then looks up shipment records and changes delivery dates in its internal system. Some incoming emails contain instructions aimed at the agent, such as telling it to ignore its rules and change a shipment's carrier.
The security team wants the agent to resist these injected instructions, and it wants the agent to be unable to change delivery dates for any shipment outside the requesting carrier's own account, even if the model is persuaded to try.
Which TWO controls meet these requirements? (Choose two.)
Choose two answers.
AA. Rely on the system prompt alone to tell the agent to ignore any instructions found inside emails, since the model follows its system prompt.
BB. Screen each email with an Amazon Bedrock guardrail that has the prompt attack filter enabled, and block emails the guardrail flags before the agent acts on them.
CC. Give the tool that changes delivery dates an IAM role and a code check that allow updates only to shipments in the requesting carrier's account.
DD. Raise the model's temperature so that the agent is less predictable, which makes injected instructions less likely to work.
EE. Give the agent full administrator access to the internal system, so that it can always complete the carrier requests without errors.
Show the answer and explanation
Answers: B, C
A. Wrong: A system prompt is a helpful instruction but is not an enforced control, and injected text can still persuade the model. The security team needs controls outside the model.
C. Right: Correct. Least-privilege permissions and checks inside the action function limit what the agent can change, even if the model is persuaded. The limit holds whatever the model decides.
D. Wrong: Higher temperature makes outputs more random and does not make the agent resist injected instructions. The option also reduces predictability in a way that works against the team.
E. Wrong: Full administrator access gives the agent far more power than it needs and would let an injected instruction change any shipment. The option is the opposite of the security goal.
Explanation
A guardrail's prompt attack filter screens incoming text for injected instructions, and least-privilege permissions plus checks in the tool limit what the agent can change, whatever the model decides. A system prompt alone is not enforced, and higher temperature or administrator access does not address the risk.
Want more on Guardrails and responsible AI? The 5 full-length practice sets have 10 more Guardrails and responsible AI questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →