Monitoring your AWS deployment

GraphDB exposes several endpoints that return metrics in the Prometheus text format. These endpoints could be scraped using the CloudWatch agent so that you can monitor the metrics in CloudWatch and create alarms based on them.

In order for the CloudWatch agent to have permission to push metrics to CloudWatch, your EC2 instances will need to have the CloudWatchAgentServerPolicy permission attached to their instance profile. See the official AWS documentation as well as the GraphDB documentation on Creating an IAM role and attaching policies for more information.

Once the required permissions are attached, follow these steps to push metrics to CloudWatch:

  1. Install CloudWatch agent on the EC2 instances

  2. Create a Prometheus config on each instance, e.g. /etc/prometheus/prometheus.yml and paste the following script:

    global:
    scrape_interval: 1m
    scrape_timeout: 10s
    scrape_configs:
    - job_name: graphdb_infrastructure_monitor
        metrics_path: /rest/monitor/infrastructure
        scrape_interval: 5s
        static_configs:
        - targets: [ 'localhost:7200' ]
    - job_name: graphdb_structures_monitor
        metrics_path: /rest/monitor/structures
        scrape_interval: 5s
        static_configs:
        - targets: [ 'localhost:7200' ]
    

    Note

    This step is necessary so that the CloudWatch agent knows which endpoints to scrape.

    Hint

    Check the Monitoring and troubleshooting documentation for more information on the various metrics and endpoints.

  3. Create a file named config.json and paste the following example content:

    {
        "agent": {
            "debug": true
        },
        "logs":{
        "metrics_collected":{
            "prometheus":{
                "log_group_name": "graphdb",
                "prometheus_config_path": "/etc/prometheus/prometheus.yml",
                "emf_processor": {
                    "metric_declaration_dedup": true,
                    "metric_namespace": "CWAgent-Prometheus",
                    "metric_unit":{
                    "graphdb_class_count": "Count"
                    },
                    "metric_declaration":[
                        {
                            "source_labels": [
                                "job"
                            ],
                            "label_matcher": "graphdb_hw_monitor",
                            "dimensions": [["host"]],
                            "metric_selectors":[
                                "^graphdb_class_count$"
                            ]
                    }
                    ]
                }
            }
        }
        }
    }
    

    Note

    The agent also requires a configuration file for itself. This is where we reference the prometheus configuration and specify the metrics that we are interested in. Check the AWS documentation for more details.

  4. Use the following command to start the CloudWatch agent:

    amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:config.json
    
  5. Go to the CloudWatch console and check if the metrics are being sent over