CloudWatch agent and log collection: SOA-C03 study notes and practice questions
What to know about CloudWatch agent and log collection for the SOA-C03 exam, from the Monitoring, Logging, Analysis, Remediation, and Performance Optimization domain (22% of the exam guide), then 2 free questions to test it. Part of the SOA-C03 study guide.
Questions on this topic usually begin with “the memory metric is missing” or “logs aren’t arriving”.
What EC2 reports by default
From the hypervisor: CPU utilization, network in and out, disk reads and writes for instance store, and status checks. Not memory, disk space used, swap or process data. Those come from inside the OS, so they need the CloudWatch agent.
The CloudWatch agent
Collects system metrics (memory, disk, swap, network, processes) and log files, on Linux and Windows, on EC2 or on premises.
Its configuration is a JSON file. Store it in Systems Manager Parameter Store so every instance uses the same settings.
Install and configure it at scale with Systems Manager (the AWS-ConfigureAWSPackage document, then AmazonCloudWatch-ManageAgent), or with State Manager to keep it configured.
Can collect custom application metrics through StatsD and collectd.
What it needs
Permissions: an instance profile role with the CloudWatchAgentServerPolicy managed policy (and Systems Manager permissions if you manage it that way).
Network: a route to the CloudWatch and CloudWatch Logs endpoints. In a private subnet without a NAT gateway, add interface VPC endpoints for monitoring and logs (and the SSM endpoints if you use Systems Manager).
On premises: an IAM user or role credentials configured on the server, or Systems Manager hybrid activation.
Troubleshooting
Symptom
Likely cause
Connection timeouts in the agent log
No route to the endpoint: missing NAT, endpoint or security group rule
AccessDenied errors
Instance role missing the CloudWatch agent policy
Metrics arrive but logs don’t
Log file path wrong in the config, or the log group permissions
Compute Optimizer ignores memory
The agent isn’t publishing memory metrics
Common traps
Turning on detailed monitoring to get memory. Detailed monitoring only makes the default metrics more frequent.
Fixing IAM for a timeout. Timeouts are network problems.
A solar installer runs its quoting application on 40 Amazon Linux EC2 instances.
The operations team needs to alarm when memory utilization or root-volume disk usage passes 85 percent. They notice that these metrics are not in the AWS/EC2 namespace, even with detailed monitoring turned on. The instances are already managed by AWS Systems Manager.
Which TWO actions will provide the metrics? (Choose two.)
Choose two answers.
AA. Attach the CloudWatchAgentServerPolicy managed policy to the IAM role in the instances' instance profile.
BB. Turn on Systems Manager Inventory on the instances so that it collects memory and disk usage and publishes them to CloudWatch.
CC. Turn on Compute Optimizer enhanced infrastructure metrics so that memory and disk usage appear in CloudWatch for each instance.
DD. Install the CloudWatch agent on all instances with Systems Manager, and configure it to collect mem_used_percent and disk_used_percent.
EE. Create a CloudWatch Logs metric filter on /var/log/messages that extracts memory and disk usage values into custom metrics.
Memory and disk usage are guest operating system metrics, so EC2 cannot report them from outside the instance. The CloudWatch agent collects them and publishes them to the CWAgent namespace, using an instance role that includes CloudWatchAgentServerPolicy. Systems Manager makes installing and configuring the agent on 40 instances a single operation.
A pet insurer moved its claims workers to private subnets that have no NAT gateway and no internet gateway. The CloudWatch agent on the instances should send application logs to CloudWatch Logs. The instance role includes CloudWatchAgentServerPolicy, and Systems Manager works through existing interface endpoints.
No logs arrive, and the agent's own log shows repeated connection timeouts to logs.us-east-2.amazonaws.com.
What should a CloudOps engineer do to fix this with the LEAST change?
Choose one answer.
AA. Create an interface VPC endpoint for CloudWatch Logs with private DNS, and allow HTTPS to it from the instances.
BB. Attach the CloudWatchLogsFullAccess managed policy to the instance role so that the agent can create log groups and streams.
CC. Create a gateway VPC endpoint for CloudWatch Logs and add it to the route tables of the private subnets.
DD. Add a NAT gateway in a public subnet, add an internet gateway to the VPC, and route the private subnets' 0.0.0.0/0 traffic to it.
Show the answer and explanation
Answer: A
A. Right: Correct. The timeouts show the agent cannot reach the public Logs endpoint. An interface endpoint with private DNS makes the usual service name resolve to private IP addresses inside the VPC, so the agent works without configuration changes. Learn more: Using CloudWatch Logs with interface VPC endpoints (opens in a new tab)
C. Wrong: Gateway endpoints exist only for Amazon S3 and DynamoDB. CloudWatch Logs is reached privately through an interface endpoint, so this endpoint type cannot be created for it. Learn more: Gateway endpoints (opens in a new tab)
D. Wrong: A NAT gateway would let the agent reach the public endpoint, but it adds internet egress the design deliberately removed. It is also more change and cost than one interface endpoint. Learn more: NAT gateways (opens in a new tab)
Explanation
Connection timeouts point at network reachability, not permissions. In subnets with no internet path, every AWS service the instance calls needs a VPC endpoint, just as Systems Manager already has. CloudWatch Logs supports interface endpoints, and with private DNS the agent keeps using the standard endpoint name.
The agent's calls to CloudWatch Logs time out until an interface endpoint gives the private subnets a path to the service.
Want more on CloudWatch agent and log collection? The 5 full-length practice sets have 6 more CloudWatch agent and log collection questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →