A game studio's matchmaking service has unit tests, integration tests against a staging database and a load test. All of them run as stages of an AWS CodePipeline pipeline that builds the service with CodeBuild.
Developers read the test output in the build logs. A failing test has sometimes still left the build green, because one script step ignores the exit code of the test command. The release lead wants a failed test to stop the pipeline, and wants each run to show per-test results and code coverage in the CodeBuild console.
The team does not want to write a log parser or run a separate reporting service.
Which change meets these requirements with the LEAST operational overhead?
Choose one answer.
AA. Add a reports section to the buildspec that publishes the JUnit XML test output and a coverage report, and make the test command exit non-zero when a test fails.
BB. Pipe the test output to a Lambda function that searches for the word FAILED, writes a summary to Amazon S3, and sends an SNS message to the release lead when it finds a match.
CC. Run the tests in a separate CodeBuild project that sits outside the pipeline, so that failures do not block deployments, and review its logs by hand before each release.
DD. Set every build command to continue after errors, and export the test counts as pipeline variables that a manual approval action displays to the release lead.
Show the answer and explanation
Answer: A
A. Right: Correct. CodeBuild reads JUnit XML and coverage files from the reports section and shows them in the console. A test command that exits non-zero fails the build, and the pipeline stops before the next stage. Learn more: Test reports in AWS CodeBuild (opens in a new tab)
C. Wrong: Tests outside the pipeline cannot stop a release, which is the outcome the release lead asked to prevent. Reviewing logs by hand is slow and easy to skip.
When a buildspec declares reports, CodeBuild turns JUnit XML and coverage output into per-test results in the console without custom parsing. A test command that exits non-zero fails the build, and the pipeline stops before later stages run. Log parsers, separate projects and ignored errors each add work or let failures through.
Want more on Automated tests and test reports in pipelines? The 5 full-length practice sets have 8 more Automated tests and test reports in pipelines questions, 375 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →