JSON Parameters for Hosted Sources

This topic describes JSON Source parameters for Hosted Collectors. See the following topics for additional information:

Use JSON to Configure Sources. The topic includes a list of common parameters for all log Source types. For Sources, the common parameter name must be unique per Collector.
Source API Methods and Examples for information on creating Sources.
JSON Source Parameters for Installed Collectors.
View or Download Collector or Source JSON Configuration from Sumo Logic.

note

JSON files need to be UTF-8 encoded following RFC 8259.

Common Parameters

See Use JSON to Configure Sources for a list of common parameters.

Source types for Hosted Collectors

Each Source can have its own unique fields in addition to the generic fields listed in Use JSON to Configure Sources. The sourceType field determines the type of Source (and the associated parameters). The next table lists the valid field types. The sections that follow list the unique parameters for each and associated JSON examples.

The G Suite Apps Audit Source cannot be created with JSON. This Source requires manual 0Auth confirmation which is only supported in the Sumo web interface.

Log Sources for Installed Collectors

Field Type	Type Value
Local File Source	LocalFile
Remote File Source	RemoteFileV2
Local Windows Event Log Source	LocalWindowsEventLog
Remote Windows Event Log Source	RemoteWindowsEventLog
Local Windows Performance Source	LocalWindowsPerfMon
Remote Windows Performance Source	RemoteWindowsPerfMon
Windows Active Directory Inventory Source	ActiveDirectory
Syslog Source	Syslog
Script Source	Script
Docker Log Source	DockerLog
Docker Stats Source	DockerStats

Metric Sources for Installed Collectors

Field Type	Type Value
Host Metrics Source	SystemStats
Streaming Metrics Source	StreamingMetrics

Log Sources for Hosted Collectors

Field Type	Type Value
Akamai SIEM API Source	Universal
Amazon S3 Source	Polling
AWS S3 Archive Source	Polling
AWS CloudFront Source	Polling
AWS CloudTrail Source	Polling
AWS Elastic Load Balancing Source	Polling
AWS Kinesis Firehose for Logs Source	HTTP
AWS S3 Audit Source	Polling
AWS Metadata (Tag) Source	Polling
Azure Event Hubs Source	Universal
Carbon Black Cloud Source	Universal
Carbon Black Inventory Source	Universal
Cloud Syslog Source	Cloudsyslog
Cisco AMP Source	Universal
Crowdstrike FDR Source	Universal
CrowdStrike Source	Universal
CSE AWS EC2 Inventory Source	Universal
Cybereason Source	Universal
Duo Source	Universal
Google Cloud Platform Source	HTTP
HTTP Source	HTTP
Microsoft Graph Security API Source	Universal
Mimecast Source	Universal
Netskope Source	Universal
Okta Source	Universal
Palo Alto Cortex XDR	Universal
Proofpoint On Demand Source	Universal
Proofpoint TAP Source	Universal
Salesforce Source	Universal
Sophos Central Source	Universal
Tenable Source	Universal

Metrics sources for hosted collectors

Field Type	Type Value
AWS CloudWatch Source	Polling

Log Source parameters for Hosted Collectors

HTTP Source

In addition to the common parameters, the following parameters are for an HTTP Source.

Parameter	Type	Required?	Default	Description	Access
`fields`	JSON Object	No		Enable Extended HTTP Metadata Collection by adding `_convertHeadersToFields=true`.	modifiable
`sourceType`	String	Yes		HTTP	not modifiable
`messagePerRequest`	Boolean	Yes		When set to `true`, only a single message will be sent for each HTTP request. To disable this feature, set to `false`. You need to specify the common parameter `multilineProcessingEnabled` as false when setting `messagePerRequest` to `true`.	modifiable
`url`	String	No	URL assigned by Sumo for Source	The Source's unique HTTP endpoint web address.	not modifiable

HTTP Source JSON example
{
    "api.version": "v1",
    "source": {
        "sourceType": "HTTP",
        "name": "Example1",
        "fields":{
          "_convertHeadersToFields":"true"
        },
        "messagePerRequest": true,
        "multilineProcessingEnabled": false
    }
}

Cloud Syslog Source

In addition to the common parameters, the following parameters are for a Cloud Syslog Source.

Parameter	Type	Required?	Default	Description	Access
sourceType	String	Yes		Cloudsyslog	not modifiable

Cloud Syslog Source JSON example:

{
   "api.version":"v1",
   "source":{
      "sourceType":"Cloudsyslog",
      "name":"Example1"
   }
}

Google Cloud Platform Source

In addition to the common parameters, the following parameters are for a Google Cloud Platform Source.

Parameter	Type	Required?	Default	Description	Access
`sourceType`	String	Yes	HTTP	not modifiable
`thirdPartyRef`	Nested	JSON	Yes		Specify the the serviceType as GoogleCloudLogs.

Google Cloud Platform Source JSON example:

{
  "source": {
    "name": "Example-GCP",
    "thirdPartyRef":{
      "resources":[
         {
            "serviceType":"GoogleCloudLogs"
         }
      ]
    },
    "sourceType": "HTTP"
  }
}

AWS Log Sources

In addition to the common parameters, the following parameters are for all AWS log Sources except Kinesis.

Parameter	Type	Required?	Default	Description	Access
`sourceType`	String	Yes		Polling	not modifiable
`contentType`	String	No		Define based on the AWS Source you are creating. Archive: AwsS3ArchiveBucket S3: AwsS3Bucket S3 Audit: AwsS3AuditBucket CloudFront: AwsCloudFrontBucket CloudTrail: AwsCloudTrailBucket ELB: AwsElbBucket Metadata: AwsMetadata Kinesis: KinesisLog	not modifiable
`scanInterval`	Long	Yes	300000	Time interval of S3 bucket scans for new data, in milliseconds. Minimum value: 1000 For Automatic assign to: -1	modifiable
`paused`	Boolean	Yes	false	When set to true, the scanner for S3 bucket items is paused. To disable, set to false.	modifiable
`url`	String	No	URL	assigned by Sumo for Source	Used to set up Event Based Notifications with AWS. This value is created and assigned by Sumo when the Source is created.
`thirdPartyRef`	Nested	JSON	Yes		Includes all required information for third-party integration, including the S3 bucket name, path expression for the S3 objects, and access credentials. See examples below table.
`snsTopicOrSubscriptionArn` This parameter goes in the thirdPartyRef nested JSON.	String	No		SNS topic/subscription ARN. If SNS has been successfully configured and has received a subscription confirmation request isSuccess will be true. See example below table.	not modifiable
`bucketName` This parameter goes in the thirdPartyRef nested JSON.	String	Yes		Name of your AWS S3 bucket.	Modifiable
`pathExpression` This parameter goes in the thirdPartyRef nested JSON.	String	Yes		Wildcard pattern that matches the S3 objects you'd like to collect.	Modifiable

IAM User authentication example:

"authentication": {
  "type": "S3BucketAuthentication",
  "awsId": "AKIAIOSFODNN7EXAMPLE",
  "awsKey": "*******"
}

IAM Role authentication example:

"authentication": {
  "type": "AWSRoleBasedAuthentication",
  "roleARN": "arn:aws:iam::123456789012:role/myrole"
}

snsTopicOrSubscriptionArn example

"thirdPartyRef": {
        "resources": [{
            "serviceType": "#",
            "path": {
                "type": "#",
                "bucketName": "#",
                "pathExpression": "*",
                "snsTopicOrSubscriptionArn": {
                    "isSuccess": true,
                    "arn": "arn:aws:sns:#:#:SumoSNSTopic-#:#"
                }
            },
            "authentication": {...
            }
        }]
    }

AWS S3 Archive Source

The parameters for this Source can be referenced in the AWS Log Sources section. This is an Amazon S3 Archive Source JSON example:

{
  "api.version":"v1",
  "source":{
    "name":"my_new_archive",
    "automaticDateParsing":true,
    "multilineProcessingEnabled":true,
    "useAutolineMatching":true,
    "contentType":"AwsS3ArchiveBucket",
    "forceTimeZone":false,
    "filters":[],
    "cutoffTimestamp":0,
    "encoding":"UTF-8",
    "fields":{

    },
    "thirdPartyRef":{
      "resources":[{
        "serviceType":"AwsS3ArchiveBucket",
        "path":{
          "type":"S3BucketPathExpression",
          "bucketName":"awsbucketname",
          "pathExpression":"*"
        },
        "authentication":{
          "type":"AWSRoleBasedAuthentication",
          "roleARN":"arn:aws:iam::123456789012:role/myrole"
        }
      }]
    },
    "scanInterval":300000,
    "paused":false,
    "sourceType":"Polling"
  }
}

Amazon S3 Source

The parameters for this Source can be referenced in the AWS Log Sources section. This is an Amazon S3 Source JSON example:

{
  "api.version":"v1",
  "source":{
    "sourceType": "Polling",
    "name": "Example1",
    "contentType": "AwsS3Bucket",
    "thirdPartyRef":{
      "resources":[
        {
          "serviceType": "AwsS3Bucket",
          "path": {
            "type": "S3BucketPathExpression",
            "bucketName": "Bucket1",
            "pathExpression": "*"
          },
          "authentication": {
            "type": "AWSRoleBasedAuthentication",
            "roleARN": "arn:aws:iam::123456789012:role/myrole"
          }
        }    
      ]
    },
    "scanInterval": 300000,
    "paused": false
  }
}

AWS Elastic Load Balancing Source

The parameters for this Source can be referenced in the AWS Log Sources section. This is an AWS Elastic Load Balancing Source JSON example:

{
    "api.version": "v1",
    "source": {
        "sourceType": "Polling",
        "name": "Example1",
        "contentType": "AwsElbBucket",
        "thirdPartyRef": {
            "resources": [{
                "serviceType": "AwsElbBucket",
                "path": {
                    "type": "S3BucketPathExpression",
                    "bucketName": "Bucket1",
                    "pathExpression": "*"
                },
                "authentication": {
                    "type": "AWSRoleBasedAuthentication",
                    "roleARN": "arn:aws:iam::123456789012:role/myrole"
                }
            }]
        },
        "scanInterval": 300000,
        "paused": false
    }
}

AWS CloudFront Source

The parameters for this Source can be referenced in the AWS Log Sources section. This is an AWS CloudFront Source JSON example:

{
    "api.version": "v1",
    "source": {
        "sourceType": "Polling",
        "name": "Example1",
        "contentType": "AwsCloudFrontBucket",
        "thirdPartyRef": {
            "resources": [{
                "serviceType": "AwsCloudFrontBucket",
                "path": {
                    "type": "S3BucketPathExpression",
                    "bucketName": "Bucket1",
                    "pathExpression": "*"
                },
                "authentication": {
                    "type": "AWSRoleBasedAuthentication",
                    "roleARN": "arn:aws:iam::123456789012:role/myrole"
                }
            }]
        },
        "scanInterval": 300000,
        "paused": false
    }
}

AWS CloudTrail Source

The parameters for this Source can be referenced in the AWS Log Sources section. This is an AWS CloudTrail Source JSON example:

{
    "api.version": "v1",
    "source": {
        "sourceType": "Polling",
        "name": "Example1",
        "contentType": "AwsCloudTrailBucket",
        "thirdPartyRef": {
            "resources": [{
                "serviceType": "AwsCloudTrailBucket",
                "path": {
                    "type": "S3BucketPathExpression",
                    "bucketName": "Bucket1",
                    "pathExpression": "*"
                },
                "authentication": {
                    "type": "AWSRoleBasedAuthentication",
                    "roleARN": "arn:aws:iam::123456789012:role/myrole"
                }
            }]
        },
        "scanInterval": 300000,
        "paused": false
    }
}

Amazon S3 Audit Source

The parameters for this Source can be referenced in the AWS Log Sources section. This is an AWS S3 Audit Source JSON example:

{
    "api.version": "v1",
    "source": {
        "sourceType": "Polling",
        "name": "Example1",
        "contentType": "AwsS3AuditBucket",
        "thirdPartyRef": {
            "resources": [{
                "serviceType": "AwsS3AuditBucket",
                "path": {
                    "type": "S3BucketPathExpression",
                    "bucketName": "Bucket1",
                    "pathExpression": "*"
                },
                "authentication": {
                    "type": "AWSRoleBasedAuthentication",
                    "roleARN": "arn:aws:iam::123456789012:role/myrole"
                }
            }]
        },
        "scanInterval": 300000,
        "paused": false
    }
}

AWS Metadata (Tag) Source

The parameters for this Source can be referenced in the AWS Log Sources section. This is an AWS Metadata Source JSON example:

{
    "api.version": "v1",
    "source": {
        "sourceType": "Polling",
        "name": "exampleName",
        "description": "Poll metadata for data",
        "automaticDateParsing": false,
        "multilineProcessingEnabled": false,
        "useAutolineMatching": false,
        "contentType": "AwsMetadata",
        "forceTimeZone": false,
        "filters": [],
        "cutoffTimestamp": 0,
        "encoding": "UTF-8",
        "fields": {

        },
        "thirdPartyRef": {
            "resources": [{
                "serviceType": "AwsMetadata",
                "path": {
                    "type": "AwsMetadataPath",
                    "limitToRegions": ["ap-southeast-2", "us-east-1", "us-west-2"],
                    "limitToNamespaces": ["AWS/EC2"],
                    "tagFilters": ["tag1", "tag2", "tag3", "tag4", "name"]
                },
                "authentication": {
                    "type": "AWSRoleBasedAuthentication",
                    "roleARN": "arn:aws:iam::123456789012:role/myrole"
                }
            }]
        },
        "scanInterval": 60000,
        "paused": false
    }
}

AWS Kinesis Firehose for Logs Source

This is an AWS Kinesis Firehose for Logs Source JSON example. It uses the thirdPartyRef parameter to enable S3 Replay. Leave thirdPartyRef empty if you don't want to enable S3 replay.

{
  "source":{ "name":"test-source", "automaticDateParsing":true, "multilineProcessingEnabled":true, "useAutolineMatching":true, "contentType":"AwsS3Bucket", "forceTimeZone":false, "filters":[], "cutoffTimestamp":1634972400000, "encoding":"UTF-8", "fields":{}, "thirdPartyRef":{   "resources":[{     "serviceType":"AwsS3Bucket",     "path":{       "type":"S3BucketPathExpression",       "bucketName":"test-kinesis-bucket-name",       "pathExpression":"http-endpoint-failed/*",       "useVersionedApi":true     },     "authentication":{       "type":"S3BucketAuthentication",       "awsId":"XXX",       "awsKey":"XXX"     }   }] }, "sourceType":"HTTP"
  }
}

To disable S3 Replay use the NoPathExpression placeholder for path and authentication, for example:

{
    "source":
        {
            "name": "test-kinesis-source",
            "automaticDateParsing": true,
            "multilineProcessingEnabled": true,
            "useAutolineMatching": true,
            "contentType": "KinesisLog",
            "forceTimeZone": false,
            "filters": [],
            "cutoffTimestamp": 0,
            "encoding": "UTF-8",
            "fields": {},
            "thirdPartyRef": {
                "resources": [
                    {
                        "serviceType": "KinesisLog",
                        "path": {
                            "type": "NoPathExpression"
                        },
                        "authentication": {
                            "type": "NoAuthentication"
                        }
                    }
                ]
            },
            "messagePerRequest": false,
            "sourceType": "HTTP"
        }
}

Metrics Source parameters for Hosted Collectors

AWS CloudWatch Source

The following parameters are for an AWS CloudWatch Source.

Parameter	Type	Required?	Default	Description	Access
`name`	String	Yes		Type a desired name of the Source. The name must be unique per Collector. This value is assigned to the metadata field `_source`.	modifiable
`description`	String	No	null	Type a description of the Source.	modifiable
`category`	String	No	null	Type a category of the source. This value is assigned to the metadata field `_sourceCategory`. See best practices for details.	modifiable
`sourceType`	String	Yes		Polling	not modifiable
`contentType`	String	No		AwsCloudWatch	not modifiable
`scanInterval`	Long	Yes	300000	Time interval of S3 bucket scans for new data, in milliseconds. Minimum value: 1000	modifiable
`paused`	Boolean	Yes	false	When set to `true`, the scanner for metrics is paused. To disable, set to `false`.	modifiable
`thirdPartyRef`	Nested	JSON	Yes		Includes all required information for third-party integration, including the relevant Amazon regions, namespaces, and access credentials.

AWS CloudWatch Source JSON example:

{
   "api.version":"v1",
   "source":{
      "sourceType": "Polling",
      "name": "Example1",
      "contentType": "AwsCloudWatch",
      "thirdPartyRef":{
         "resources":[
            {
               "serviceType": "AwsCloudWatch",
               "path": {
                  "type": "CloudWatchPath",
                  "limitToRegions": ["region-1", "region-2"],
                  "limitToNamespaces": ["AWS/ELB", "AWS/Route53", "AWS/OpsWork"]
               },
               "authentication": {
                  "type": "S3BucketAuthentication",
                  "awsId": "AKIAIOSFODNN7EXAMPLE",
                  "awsKey": "*******"
               }
            }
         ]
      },
      "scanInterval": 300000,
      "paused": false
   }
}

AWS Kinesis Firehose for Metrics Source

This is an AWS Kinesis Firehose for Metrics Source JSON example:

{
  "api.version": "v1",
  "source": {
    "sourceType": "HTTP",
    "name": "CloudWatch Metrics via Kinesis",
    "contentType": "KinesisMetric",
    "description": "Managed by Terraform in content-management / myDeployment",
    "fields": {},
    "thirdPartyRef": {
      "resources": [
        {
          "serviceType": "KinesisMetric",
          "authentication": {
            "roleARN": "arn:aws:iam::123456789012:role/myrole",
            "type": "AWSRoleBasedAuthentication"
          },
          "path": {
            "type": "KinesisMetricPath",
            "tagFilters": [
              {
                "namespace": "AWS/EBS",
                "tags": [
                  "Deployment=myDeployment",
                  "Cluster=my_cluster_a;my_cluster_b",
                  "VolumeSet=commitlog;data"
                ],
                "type": "TagFilters"
              }
          }
        }
      ]
    }
  }
}

Source types for Hosted Collectors​

Log Sources for Installed Collectors​

Metric Sources for Installed Collectors​

Log Sources for Hosted Collectors​

Metrics sources for hosted collectors​

Log Source parameters for Hosted Collectors​

HTTP Source​

Cloud Syslog Source​

Google Cloud Platform Source​

AWS Log Sources​

AWS S3 Archive Source​

Amazon S3 Source​

AWS Elastic Load Balancing Source​

AWS CloudFront Source​

AWS CloudTrail Source​

Amazon S3 Audit Source​

AWS Metadata (Tag) Source​

AWS Kinesis Firehose for Logs Source​

Metrics Source parameters for Hosted Collectors​

AWS CloudWatch Source​

AWS Kinesis Firehose for Metrics Source​