A dental chain's data lake stores one folder for each clinic under a shared S3 prefix. Every clinic's folder holds the same Parquet table, but a crawler splits the folders into dozens of separate catalog tables because it treats each folder as a different schema.
The analysts want one table with the clinic as a partition, and the crawler should keep finding new clinic folders each night.
Which crawler setting meets these requirements?
Choose one answer.
AA. Create one crawler for each clinic folder, and point each crawler at its own table name so that the catalog stays organized.
BB. Turn off the crawler's schedule, and delete the extra tables by hand after each run, keeping the table with the most rows.
CC. Set the crawler to create a single schema for each S3 path, so folders with compatible schemas map to one partitioned table.
DD. Rename each clinic folder so that the crawler sees the same name, which makes the crawler merge the folders into one table.
Show the answer and explanation
Answer: C
A. Wrong: One crawler per folder multiplies the crawlers to maintain and still creates a table per clinic. It also breaks the goal of a single table that picks up new folders automatically.
B. Wrong: Manual cleanup after every run is error-prone, and turning off the schedule stops the crawler from finding new clinic folders. The option fails both requirements.
C. Right: Correct. This crawler setting groups compatible S3 paths into one table, and the folder layout becomes partitions. New clinic folders with the same schema are added to the same table on the next run. Learn more: Customizing crawler behavior (opens in a new tab)
D. Wrong: Renaming folders does not change the schemas the crawler compares, and the folder names still differ. The option does not address the grouping behavior.
Explanation
A crawler groups compatible files into one table only when it is told to create a single schema for each S3 path. With that setting, clinic folders that share a schema map to one table, and the folder becomes a partition. Per-folder crawlers, manual cleanup and renaming folders all add work or miss the grouping setting.
Want more on Glue Data Catalog and crawlers? The 5 full-length practice sets have 8 more Glue Data Catalog and crawlers questions, 325 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →