Partitions and Data Tiers

Creating a partition allows you to improve search performance by searching over a smaller number of messages. Use the Partitions page to set up and manage partitions. To access the Partitions page, go to Manage Data > Logs > Partitions.

In this section, we will introduce the following concepts:

📄️ About Partitions

A partition stores your data in an index separate from the rest of your account's data so you can optimize searches, manage variable retention, and specify certain data to forward to S3.

📄️ Add a Partition

Partitions provide three primary functions:

📄️ Edit a Partition

This page has instructions for editing a partition.

📄️ Run a Search Against a Partition

Running a search against the data in a Partition is almost exactly the same as running any other query. The difference you'll notice is the speed at which results are returned, especially if you're searching over a large amount of data.

📄️ View Details About a Partition

To view details about a Partition:

📄️ Edit Data Forwarding Destinations for a Partition

You can specify data forwarding settings for a partition so that the messages that were routed to an index can be forwarded to an existing or new Amazon S3 destination.

📄️ Manage Indexes with Variable Retention

Your company log data comes in many types, and different types of data may require different retention periods. For example, some sensitive information may require a short life span, while other vital data, such as customer logins, may require a longer lifespan for daily use. Still other data may need to be available for historical analysis and to gather metrics to make business decisions, though it is not accessed every day.

📄️ Decommission a Partition

Once they are created, partitions cannot be deleted, and partition names cannot be reused. This is due to the fact that a partition may include log messages that are not stored anywhere else, and if the partition were to be deleted, the log messages would be lost.