Skip to main content

Sumo Logic App for MongoDB

Thumbnail icon

MongoDB is a source-available cross-platform document-oriented database program. The Sumo Logic app for MongoDB supports logs and metrics from the open source version of MongoDB. The App is tested on the 4.4.4 version of MongoDB.

Log Types​

The MongoDB logs are generated in files as configured in the configuration file /var/log/mongodb/mongodb.log. For more details on MongoDB logs, see this link.

The Sumo Logic App for MongoDB supports metrics generated by the MongoDB plugin for Telegraf. The app assumes prometheus format Metrics.

Sample Log Message​

{
    "t":{
        "$date":"2021-05-21T10:22:57.373+00:00"
    },
    "s":"I",
    "c":"NETWORK",
    "id":51800,
    "ctx":"conn500659",
    "msg":"client metadata",
    "attr":{
        "remote":"127.0.0.1:49472",
        "client":"conn500659",
        "doc":{
            "application":{
                "name":"MongoDB Shell"
            },
            "driver":{
                "name":"MongoDB Internal Client",
                "version":"4.4.4"
            },
            "os":{
                "type":"Linux",
                "name":"PRETTY_NAME=\"Debian GNU/Linux 10 (buster)\"",
                "architecture":"x86_64",
                "version":"Kernel 4.4.0-62-generic"
            }
        }
    }
}

Sample Query​

Dashboard: MongoDB - Errors and Warnings, Panel: Errors by Component
environment=* db_cluster=* db_system=mongodb  | json "log" as _rawlog nodrop
| if (isEmpty(_rawlog), _raw, _rawlog) as _raw
| json field=_raw "t.$date" as timestamp
| json field=_raw "s" as severity
| json field=_raw "c" as component
| json field=_raw "ctx" as context
| json field=_raw "msg" as msg
| where severity in ("E")
| count by component

Collecting Logs and Metrics for MongoDB​

This section provides instructions for configuring log and metric collection for the Sumo Logic App for MongoDB.

Step 1: Configure Fields in Sumo Logic​

Create the following Fields in Sumo Logic prior to configuring collection. This ensures that your logs and metrics are tagged with relevant metadata, which is required by the app dashboards. For information on setting up fields, see Sumo Logic Fields.

If you're using MongoDB in a Kubernetes environment, create the fields:

  • pod_labels_component
  • pod_labels_environment
  • pod_labels_db_system
  • pod_labels_db_cluster

Step 2: Configure Collection for MongoDB​

In Kubernetes environments, we use the Telegraf Operator, which is packaged with our Kubernetes collection. You can learn more about it here.The diagram below illustrates how data is collected from MongoDB in a Kubernetes environment. In the architecture shown below, there are four services that make up the metric collection pipeline: Telegraf, Prometheus, Fluentd and FluentBit.
mongodb_on_k8s

The first service in the pipeline is Telegraf. Telegraf collects metrics from MongoDB. Note that we’re running Telegraf in each pod we want to collect metrics from as a sidecar deployment for example, Telegraf runs in the same pod as the containers it monitors. Telegraf uses the MongoDB input plugin to obtain metrics. (For simplicity, the diagram doesn’t show the input plugins.) The injection of the Telegraf sidecar container is done by the Telegraf Operator. We also have Fluentbit that collects logs written to standard out and forwards them to FluentD, which in turn sends all the logs and metrics data to a Sumo Logic HTTP Source.

Prerequisites

It’s assumed that you are using the latest helm chart version. If not, upgrade using the instructions here.

Configure Metrics Collection​

This section explains the steps to collect MongoDB metrics from a Kubernetes environment.

  1. Set up Kubernetes Collection with the Telegraf Operator.
  2. On your MongoDB Pods, add the following annotations:
annotations:
  telegraf.influxdata.com/class: sumologic-prometheus
  prometheus.io/scrape: "true"
  prometheus.io/port: "9273"
  telegraf.influxdata.com/inputs: |+
[[inputs.mongodb]]
  servers = ["mongodb://<username-CHANGEME>:<password-CHANGEME>@127.0.0.1:27017"]
  gather_perdb_stats = true
  gather_col_stats = true
  [inputs.mongodb.tags]
    environment="ENV_TO_BE_CHANGED"
    component="database"
    db_system="mongodb"
    db_cluster="ENV_TO_BE_CHANGED"
    db_cluster_address = "ENV_TO_BE_CHANGED"
    db_cluster_port = "ENV_TO_BE_CHANGED"
  1. Please enter values for the following parameters (marked ENV_TO_BE_CHANGED above):
    • telegraf.influxdata.com/inputs - This contains the required configuration for the Telegraf MongoDB Input plugin. Please refer to this doc for more information on configuring the MongoDB input plugin for Telegraf. Note: As telegraf will be run as a sidecar the host should always be localhost.
       * In the input plugins section (`[inputs.MongoDB]`):
          * `servers` - The URL to the MongoDB server. This can be a comma-separated list to connect to multiple MongoDB servers. Please see [this doc](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/mongodb) for more information on additional parameters for configuring the MongoDB input plugin for Telegraf.
       * In the tags section (`[inputs.MongoDB.tags])`:
          * `environment` - This is the deployment environment where the MongoDB cluster identified by the value of **servers** resides. For example: dev, prod or qa. While this value is optional we highly recommend setting it.
          * `db_cluster` - Enter a name to identify this MongoDB cluster. This cluster name will be shown in the Sumo Logic dashboards.
                  * `db_cluster_address` - Enter the cluster hostname or ip address that is used by the application to connect to the database. It could also be the load balancer or proxy endpoint.
                  * `db_cluster_port` - Enter the database port. If not provided, a default port will be used.
      note

      db_cluster_address and db_cluster_port should reflect the exact configuration of DB client configuration in your application, especially if you instrument it with OT tracing. The values of these fields should match exactly the connection string used by the database client (reported as values for net.peer.name and net.peer.port metadata fields).

For example, if your application uses “mongodb-prod.sumologic.com:3306” as the connection string, the field values should be set as follows: db_cluster_address=mongodb-prod.sumologic.com db_cluster_port=3306

If your application connects directly to a given mongodb node, rather than the whole cluster, use the application connection string to override the value of the “host” field in the Telegraf configuration: host=mongodb-prod.sumologic.com

Pivoting to Tracing data from Entity Inspector is possible only for “MongoDB address” Entities. :::

  • Here’s an explanation for additional values set by this configuration that we request you do not modify as they will cause the Sumo Logic apps to not function correctly.
    • telegraf.influxdata.com/class: sumologic-prometheus - This instructs the Telegraf operator what output to use. This should not be changed.
    • prometheus.io/scrape: "true" - This ensures our Prometheus will scrape the metrics.
    • prometheus.io/port: "9273" - This tells prometheus what ports to scrape on. This should not be changed.
    • telegraf.influxdata.com/inputs
      • In the tags section ([inputs.mongodb.tags]):
        • component: “database” - This value is used by Sumo Logic apps to identify application components.
        • db_system: “mongodb” - This value identifies the database system.
  • See this doc for more parameters that can be configured in the Telegraf agent globally.
  1. Sumo Logic Kubernetes collection will automatically start collecting metrics from the pods having the labels and annotations defined in the previous step.
  2. Verify metrics in Sumo Logic.

Configure Logs Collection​

This section explains the steps to collect MongoDB logs from a Kubernetes environment.

  1. Add labels on your MongoDB pods to capture logs from standard output on Kubernetes. Make sure that the logs from MongoDB are sent to stdout. For more details, see this doc.
    1. Apply following labels to the MongoDB pods:
      labels:
          environment: "prod"
          component: "database"
          db_system: "mongodb"
          db_cluster: "mongodb_prod_cluster01"
    2. Enter in values for the following parameters:
     * `environment`. This is the deployment environment where the MongoDB cluster identified by the value of **servers** resides. For example: dev, prod or qa. While this value is optional we highly recommend setting it.
     * `db_cluster`. Enter a name to identify this MongoDB cluster. This cluster name will be shown in the Sumo Logic dashboards.
         * `db_cluster_address` - Enter the cluster hostname or ip address that is used by the application to connect to the database. It could also be the load balancer or proxy endpoint.
         * `db_cluster_port` - Enter the database port. If not provided, a default port will be used.
    note

    db_cluster_address and db_cluster_port should reflect exact configuration of DB client configuration in your application, especially if you instrument it with OT tracing. The values of these fields should match exactly the connection string used by the database client (reported as values for net.peer.name and net.peer.port metadata fields).

For example if your application uses “mongodb-prod.sumologic.com:3306” as the connection string, the field values should be set as follows: db_cluster_address=mongodb-prod.sumologic.com db_cluster_port=3306

If your application connects directly to a given mongodb node, rather than the whole cluster, use the application connection string to override the value of the “host” field in the Telegraf configuration: host=mongodb-prod.sumologic.com.

Pivoting to Tracing data from Entity Inspector is possible only for “MongoDB address” Entities. :::

 * **Do not modify the following values** as they will cause the Sumo Logic apps to not function correctly.
   * `component: “database”`. This value is used by Sumo Logic apps to identify application components.
   * `db_system: “mongodb”`. This value identifies the database system.
 * See [this doc](/docs/send-data/collect-from-other-data-sources/collect-metrics-telegraf/install-telegraf#Configuring-Telegraf) for more parameters that can be configured in the Telegraf agent globally.
  1. Collecting MongoDB Logs from a Log File (Optional). Follow the steps below to capture MongoDB logs from a log file on Kubernetes.
    1. Determine the location of the MongoDB log file on Kubernetes. This can be determined from the MongoDB.conf for your MongoDB cluster along with the mounts on the MongoDB pods.
    2. Install the Sumo Logic tailing sidecar operator.
    3. Add the following annotation in addition to the existing annotations.
      annotations:
        tailing-sidecar: sidecarconfig;<mount>:<path_of_MongoDB_log_file>/<MongoDB_log_file_name>
      Example:
       annotations:
        tailing-sidecar: sidecarconfig;data:/mongo-prim-data/MongoDB.log
    4. Make sure that the MongoDB pods are running and annotations are applied by using the command:
      kubectl describe pod <MongoDB_pod_name>
    5. Sumo Logic Kubernetes collection will automatically start collecting logs from the pods having the annotations defined above.
  2. Add an FER to normalize the fields in Kubernetes environments. This step is not needed if one is using application components solution terraform script. Labels created in Kubernetes environments automatically are prefixed with pod_labels. To normalize these for our app to work, we need to create a Field Extraction Rule if not already created for Database Application Components. To do so:
    1. Go to Manage Data > Logs > Field Extraction Rules.
    2. Click the + Add button on the top right of the table.
    3. The Add Field Extraction Rule form will appear:
    4. Enter the following options:
    • Rule Name. Enter the name as App Observability - Database.
    • Applied At. Choose Ingest Time
    • Scope. Select Specific Data
    • Scope: Enter the following keyword search expression:
      pod_labels_environment=* pod_labels_component=database \
      pod_labels_db_system=* pod_labels_db_cluster=*
    • Parse Expression.Enter the following parse expression:
      | if (!isEmpty(pod_labels_environment), pod_labels_environment, "") as environment
      | pod_labels_component as component
      | pod_labels_db_system as db_system
      | pod_labels_db_cluster as db_cluster
  3. Click Save to create the rule.

Installing MongoDB Monitors​

The next few sections have instructions for installing Sumo Logic Monitors for MongoDB, the app, and descriptions of each of the app dashboards. These instructions assume you have already set up collection as described in the Collect Logs and Metrics for MongoDB App page.

Sumo Logic has provided pre-packaged alerts available through Sumo Logic monitors to help you proactively determine if a MongoDB cluster is available and performing as expected. These monitors are based on metric and log data and include pre-set thresholds that reflect industry best practices and recommendations. For more information about individual alerts, see MongoDB Alerts.

To install these monitors, you must have the Manage Monitors role capability.

You can install monitors by importing a JSON file or using a Terraform script.

There are limits to how many alerts can be enabled. For more information, see Monitors for details.

Method A: Importing a JSON file​

  1. Download the JSON file that describes the monitors.
  2. Replace $$mongodb_data_source with a custom source filter. To configure alerts for a specific database cluster, use a filter like db_system=mongodb or db_cluster=dev-mongodb. To configure the alerts for all of your clusters, set $$mongodb_data_source to blank ("").
  3. Go to Manage Data > Alerts > Monitors.
  4. Click Add.
  5. Click Import.
  6. On the Import Content popup, enter MongoDB in the Name field, paste in the JSON into the the popup, and click Import.
  7. The monitors are created in a "MongoDB" folder. The monitors are disabled by default. See the Monitors topic for information about enabling monitors and configuring notifications or connections.

Method B: Using a Terraform script​

  1. Generate a Sumo Logic access key and ID for a user that has the Manage Monitors role capability. For instructions see Access Keys.

  2. Download Terraform 0.13 or later, and install it.

  3. Download the Sumo Logic Terraform package for MongoDB monitors. The alerts package is available in the Sumo Logic github repository. You can either download it using the git clone command or as a zip file.

  4. Alert Configuration. After extracting the package , navigate to the terraform-sumologic-sumo-logic-monitor/monitor_packages/MongoDB/ directory. Edit the MongoDB.auto.tfvars file and add the Sumo Logic Access Key and Access ID from Step 1 and your Sumo Logic deployment. If you're not sure of your deployment, see Sumo Logic Endpoints and Firewall Security.

    access_id   = "<SUMOLOGIC ACCESS ID>"
    access_key  = "<SUMOLOGIC ACCESS KEY>"
    environment = "<SUMOLOGIC DEPLOYMENT>"

    The Terraform script installs the alerts without any scope filters, if you would like to restrict the alerts to specific clusters or environments, update the mongodb_data_source variable. For example:

    • To configure alerts for A specific cluster set mongodb_data_source to something like db_cluster = mongodb.prod.01
    • To configure alerts for All clusters in an environment set mongodb_data_source to something like environment = prod
    • To configure alerts for Multiple clusters using a wildcard set mongodb_data_source to something like db_cluster = mongodb-prod*
    • To configure alerts for A specific cluster within a specific environment, set mongodb_data_source to something like db_cluster = mongodb-1 and environment = prod. This assumes you have configured and applied Fields as described in Step 1: Configure Sumo Logic Fields.

    All monitors are disabled by default on installation. To enable all of the monitors, set the monitors_disabled parameter to false. By default, the monitors will be located in a "MongoDB" folder on the Monitors page. To change the name of the folder, update the monitor folder name in the folder variable in the MongoDB.auto.tfvars file.

  5. If you want the alerts to send email or connection notifications, edit the MongoDB_notifications.auto.tfvars file to populate the connection_notifications and email_notifications sections. Examples are provided below. In the variable definition below, replace <CONNECTION_ID> with the connection ID of the Webhook connection. You can obtain the Webhook connection ID by calling the Monitors API.

Pagerduty connection example
connection_notifications = [
    {
      connection_type       = "PagerDuty",
      connection_id         = "<CONNECTION_ID>",
      payload_override      = "{\"service_key\": \"your_pagerduty_api_integration_key\",\"event_type\": \"trigger\",\"description\": \"Alert: Triggered {{TriggerType}} for Monitor {{Name}}\",\"client\": \"Sumo Logic\",\"client_url\": \"{{QueryUrl}}\"}",
      run_for_trigger_types = ["Critical", "ResolvedCritical"]
    },
    {
      connection_type       = "Webhook",
      connection_id         = "<CONNECTION_ID>",
      payload_override      = "",
      run_for_trigger_types = ["Critical", "ResolvedCritical"]
    }
  ]

For information about overriding the payload for different connection types, see Set Up Webhook Connections.

Email notifications example
email_notifications = [
    {
      connection_type       = "Email",
      recipients            = ["abc@example.com"],
      subject               = "Monitor Alert: {{TriggerType}} on {{Name}}",
      time_zone             = "PST",
      message_body          = "Triggered {{TriggerType}} Alert on {{Name}}: {{QueryURL}}",
      run_for_trigger_types = ["Critical", "ResolvedCritical"]
    }
  ]
  1. Install Monitors
    1. Navigate to the terraform-sumologic-sumo-logic-monitor/monitor_packages/MongoDB/ directory and run terraform init. This will initialize Terraform and download the required components.
    2. Run terraform plan to view the monitors that Terraform will create or modify.
    3. Run terraform apply.

Installing the MongoDB App​

Now that you have set up collection for MongoDB, install the Sumo Logic App for MongoDB to use the preconfigured searches and dashboards to analyze your data.

Locate and install the app you need from the App Catalog. If you want to see a preview of the dashboards included with the app before installing, click Preview Dashboards.

  1. From the App Catalog, search for and select the app.
  2. Select the service version you're using and click Add to Library. Version selection applies only to a few apps currently. For more information, see the Install the Apps from the Library.
  3. To install the app, complete the following fields.
    • App Name. You can retain the existing name or enter the app's name of your choice.

    • Advanced. Select the Location in the Library (the default is the Personal folder in the library), or click New Folder to add a new folder.
  4. Click Add to Library.

Once an app is installed, it will appear in your Personal folder or another folder that you specified. From here, you can share it with your organization.

Panels will start to fill automatically. It's important to note that each panel slowly fills with data matching the time range query received since the panel was created. Results won't immediately be available, but with a bit of time, you'll see full graphs and maps.

Viewing MongoDB Dashboards​

If no events have occurred within the time range of the Panel, the Panel will be empty.

Overview​

The MongoDB - Overview dashboard provides an at-a-glance view of MongoDB health, performance and problems causing errors.

Use this dashboard to:

  • Identify Slow Queries impacting the performance.
  • Gain insights into Replication and Sharding operations.
  • Verify Page Faults generated to determine the root cause of the problems.
MongoDB dashboards

Resource​

The MongoDB - Resource dashboard shows resource utilization by the MongoDB component.

Use this dashboard to:

  • Determine Memory and Disk Usage.
  • Identify potential resource constraints and issues.
MongoDB dashboards

Errors and Warnings​

The MongoDB - Errors and Warnings dashboard shows errors and warnings by the MongoDB component.

Use this dashboard to:

  • Determine components producing multiple errors or warnings.
MongoDB dashboards

Logins and Connections​

The MongoDB - Logins and Connections dashboard shows geo location of client connection requests, failed connection logins by geo location, and count of failed login attempts.

Use this dashboard to:

  • Determine potential hacking attempts.
  • Determine location of attacks.
MongoDB dashboards

Query Logs​

The MongoDB - Queries dashboard shows read and write query trends.

Use this dashboard to:

  • Monitor abnormal spikes in Query volume
  • Identify the read versus write ratio of your application queries. ing or adjusting indexes to improve query performance.
MongoDB dashboards

Replication Logs​

The MongoDB - Replication dashboard shows replica deletes/updates/inserts trend and replica state.

Use this dashboard to:

  • Monitor replication state and replication events like inserts/updates/commands per second.
  • Track Replication Oplog window to identify replication delay
MongoDB dashboards

Sharding​

The MongoDB - Sharding dashboard dashboard shows sharding related errors, events, failures and number of chunks moving between shards.

Use this dashboard to:

  • Identify Sharding errors and warnings.
  • Gain insights into Chunk operations.
MongoDB dashboards

MongoDB Alerts​

Sumo Logic provides out-of-the-box alerts available via Sumo Logic monitors. These alerts are built based on logs and metrics datasets and have preset thresholds based on industry best practices and recommendations.

NameDescriptionTrigger TypeAlert ConditionsRecover Conditions
MongoDB - Too Many Cursors TimeoutsThis alert fires when we detect that there are too many cursors (100) timing out on a MongoDB server within a 5 minute time interval.Warning>= 100< 100
MongoDB - Too Many Cursors OpenThis alert fires when we detect that there are too many cursors (>10K) opened by MongoDB.Warning>= 10000< 10000
MongoDB - Missing PrimaryThis alert fires when we detect that a MongoDB cluster has no node marked as primary.Critical<= 0> 0
MongoDB - Instance DownThis alert fires when we detect that the MongoDB instance is down.Missing Data----
MongoDB - Replication LagThis alert fires when we detect that the replica lag for a given MongoDB cluster is greater than 60 seconds. Please review the replication configuration.Warning> 60<= 60
MongoDB - Replication Heartbeat ErrorThis alert fires when we detect that the MongoDB Replication Heartbeat request has errors, which indicates replication is not working as expected.Warning> 0<= 0
MongoDB - Too Many ConnectionsThis alert fires when we detect a given MongoDB server has too many connections (over 80% of capacity).Warning>= 80< 80
MongoDB - Secondary Node Replication FailureThis alert fires when we detect that a MongoDB secondary node is out of sync for replication.Warning> 0<= 0
MongoDB - Slow QueriesThis alert fires when we detect that a MongoDB cluster is executing slow queries.Warning> 0<= 0
MongoDB - Sharding WarningThis alert fires when we detect warnings in MongoDB sharding operations.Warning> 0<= 0
MongoDB - Sharding Chunk Split FailureThis alert fires when we detect that a MongoDB chunk not been split during sharding.Warning> 0<= 0
MongoDB - Sharding ErrorThis alert fires when we detect errors in MongoDB sharding operations.Critical> 0<= 0
MongoDB - Replication ErrorThis alert fires when we detect errors in MongoDB replication operations.Warning> 0<= 0
MongoDB - Sharding Balancer FailureThis alert fires when we detect that data balancing failed on a MongoDB Cluster with 1 mongos instance and 3 mongod instances.Warning> 0<= 0
Sumo Logic YouTubeSumo Logic Twitter
Legal
Privacy Statement
Terms of Use

Copyright © 2022 by Sumo Logic, Inc.