AWS Cost Explorer Source

The AWS Cost Explorer Source collects cost and usage reports from AWS Cost Explorer. You have the option to collect from one or more specific AWS cost types and set how often reports are collected.

note

This feature is in Beta. To participate, contact your Sumo account executive.

Create an AWS Cost Explorer Source

When you create an AWS Cost Explorer collector Source, you add it to an existing Sumo Logic hosted collector. Before creating the Source, identify the hosted collector you want to use or simply create a new hosted collector. For further instructions, see Create a Hosted Collector.

To configure an AWS Cost Explorer Source:

1. On the Manage Data > Collection > Collection page, click Add Source next to a Hosted Collector.

2. Select AWS Cost Explorer.

   

3. Enter a Name for the Source in the Sumo Logic console. The Description is optional.

   

4. For Source Category (Optional), enter any string to tag the output collected from the Source. Category metadata is stored in a searchable field called \_sourceCategory.

5. Fields. Click the +Add Field link to define the fields you want to associate, each field needs a name (key) and value.

   •  A green circle with a check mark is shown when the field exists in the Fields table schema.

   •  An orange triangle with an exclamation point is shown when the field doesn't exist in the Fields table schema. In this case, an option to automatically add the nonexistent fields to the Fields table schema is provided. If a field is sent to Sumo that does not exist in the Fields schema it is ignored, known as dropped.

     It's recommended to add an account field and assign it a friendly name to identify the corresponding AWS account.

     

6. For the AWS Access Key and AWS Secret Key, provide the IAM User access key and secret key you want to use to authenticate collection requests. See how to grant access to an AWS product for details.

7. For the Cost Type, select one or more of the following types to collect. For details on the types, see Amazon's Understanding your AWS Cost Datasets: A Cheat Sheet and Exploring your data using Cost Explorer.

   • AmortizedCost
   • BlendedCost
   • NetAmortizedCost
   • NetUnblendedCost
   • UnblendedCost

8. For Granularity, select one or both of the available time intervals to pull data.

   • Daily Costs (Polled every 12h)
   • Monthly Costs (Polled every day)

9. Processing Rules. Configure any desired filters, such as allowlist, denylist, hash, or mask, as described in Create a Processing Rule.

10. Click Submit when complete.

States

The AWS Cost Explorer Source reports errors, its health, and initialization status. Other than indicating that the source is healthy, you are also informed, in real-time, if the source is running into trouble communicating with AWS API, or if there's an error that requires user action indicated by Sumo Logic Health Events.

An AWS Cost Explorer Source goes through the following states when created:

1. Pending: Once the Source is submitted, details are stored and the source is placed in a Pending state.
2. Started: A collection task is created on the hosted collector.
3. Initialized: Task configuration is complete in Sumo Logic.
4. Authenticated: The Source has successfully authenticated with AWS
5. Collecting: The Source is actively collecting data from AWS accounts.

If the Source has any issues during any one of these states it is placed in an Error state.

Hover your mouse over the status icon to view a tooltip with details on the detected issue.

When you delete the source it is placed in a Stopping state, when it has successfully stopped it is deleted from your Hosted Collector.

On the Collection page, the Health and Status for Sources is displayed. Use Health Events to investigate issues with collection.

Error types

When Sumo Logic detects an issue it is tracked by Health Events. The following table shows the three possible error types, the reason the error would occur, if the Source attempts to retry, and the name of the event log in the Health Event Index.

Type	Reason	Retries	Retry Behavior	Health Event Name
ThirdPartyConfig	Normally due to an invalid configuration. You'll need to review your Source configuration and make an update.	No retries are attempted until the Source is updated.	Not applicable	ThirdPartyConfigError
ThirdPartyGeneric	Normally due to an error communicating with the third party service APIs.	Yes	The Source will retry for up to 90 minutes, after which retries will be attempted every 60 minutes.	ThirdPartyGenericError
FirstPartyGeneric	Normally due to an error communicating with the internal Sumo Logic APIs.	Yes	The Source will retry for up to 90 minutes, after which retries will be attempted every 60 minutes.	FirstPartyGenericError

JSON Configuration

Sources can be configured using UTF-8 encoded JSON files with the Collector Management API. See how to use JSON to configure Sources for details.

Parameter	Type	Required	Description	Access
config	JSON Object	Yes	Contains the configuration parameters for the Source.
schemaRef	JSON Object	Yes	Use {"type":"AWS Cost Explorer"}  for an AWS Cost Explorer Source.	Not modifiable
sourceType	String	Yes	Use Universal for an AWS Cost Explorer  Source.	Not modifiable

The following table shows the config parameters for an AWS Cost Explorer Source.

Parameter	Type	Required?	Default	Description	Access
name	String	Yes		Type a desired name of the Source. The name must be unique per Collector. This value is assigned to the metadata field _source.	modifiable
description	String	No	null	Type a description of the Source.	modifiable
category	String	No	null	Type a category of the source. This value is assigned to the metadata field _sourceCategory. See best practices for details.	modifiable
fields	JSON Object	No		JSON map of key-value fields (metadata) to apply to the Collector or Source. Use the field account to tag logs with a friendly AWS account name.	modifiable
accessID	String	Yes		Provide the AWS IAM User access key ID you want to use to authenticate collection requests.	modifiable
accessKey	String	Yes		Provide the AWS Secret Key you want to use to authenticate collection requests.	modifiable
granularity	String	array	Yes		Provide a comma separated list, such as ["daily","monthly"]
costMetrics	String	array	Yes		Provide a comma separated list, such as ["AmortizedCost","BlendedCost","NetAmortizedCost", "NetUnblendedCost","UnblendedCost"]

AWS Cost Explorer Source JSON Example:

{
    "api.version": "v1",
    "source": {
        "schemaRef": {
            "type": "AWS Cost Explorer"
        },
        "config": {
            "accessID": "********",
            "name": "billing",
            "description": "billing",
            "fields": {
                "_siemForward": false,
                "account": "prod"
            },
            "accessKey": "********",
            "granularity": ["daily", "monthly"],
            "costMetrics": ["AmortizedCost", "BlendedCost", "NetAmortizedCost", "NetUnblendedCost", "UnblendedCost"],
            "category": "aws/billing"
        },
        "state": {
            "state": "Collecting"
        },
        "sourceType": "Universal"
    }
}