Refresh Table (AWS)¶
Amazon S3 directories often back the Hive partitions and only running the
recover partitions command creates Hive
partitions. The Refresh Table template can create Hive partitions.
It is important to understand the concept of
Stability Interval in the context of the Refresh Table template.
After directories in Amazon S3 buckets are created, upload data into them. Do not create a Hive partition
corresponding to such S3 directories until the process of populating the directory is complete. The
defines the number of minutes that must elapse since the last modification to a directory before the Hive partition
corresponding to it is created. See Composing a Refresh Table Query (AWS) for more information.
As an example, consider a process where data is loaded into an Amazon S3 directory at midnight every day. The process of
uploading the logs takes about an hour. So, you can configure the Refresh Table template with a
of 2 hours (120 minutes) to ensure that the process of uploading the data is complete before the Hive partitions are