Administering Command Center Agents
The Command Center metrics collection agent, ccagent, runs on segment hosts and receives real-time metrics emitted by the metrics collection database extension. Each segment host has one ccagent process. The metrics collection extension connects to ccagent using Unix Domain Sockets (UDS) to transfer metrics from Greenplum Database. Starting Greenplum Command Center with the gpcc start command starts the Command Center agent on each segment host. Stopping Command Center with gpcc stop ends these processes. The Command Center backend monitors these agents and restarts them when necessary.
Installing the gpperfmon database configures Greenplum Database to run agents on the master host and each segment host. The agents are started automatically when the database system starts up. The Greenplum Database postmaster process monitors the agents and restarts them when necessary.
gpmmongpmmon agent runs on the Greenplum Database master host. It collects initial query information from the master. Every 15 seconds, by default, it prompts the gpsmon agents to send their accumulated data. The gpmmon process saves the metrics data and logs to CSV text files that feed the external tables in the gpperfmon database. gpsmongpsmon agent runs on each Greenplum Database segment host. It listens on a UDP port for metrics emitted by Greenplum Database, gathers additional metrics from the operating system, and forwards data to the gpmmon agent when requested. This topic describes basic agent administration tasks, including adding hosts and viewing agent log files.
Configuring gpmon Role Logging Options
The metrics collection agent runs queries on Greenplum Database on behalf of Command Center to perform activities such as retrieving information to display in the Command Center UI, saving state in the gpperfmon and postgres databases, inserting alert event records, and harvesting query history for the gpmetrics history tables. The agent runs these queries using the gpmon database role.
If you do not want all of this query activitiy to be logged in the pg_log log file, you can set logging options for the gpmon role using the ALTER ROLE command. For example, these commands set the gpmon role to log only DDL statements (CREATE, ALTER, DROP) and to log only fatal and panic errors.
ALTER ROLE gpmon SET log_statement TO DDL;
ALTER ROLE gpmon SET log_min_messages to FATAL;
See the log_statement and log_min_messages server configuration parameters in the Greenplum Database documentation for logging options.
Adding and Removing Hosts
When you add or replace Greenplum Database hosts, you must reinstall the Greenplum Command Center software to ensure the software is installed on the new hosts. It is not necessary to uninstall Command Center before reinstalling. Stop Command Center and restart it to start agents on the new hosts.
The gpperfmon gpsmon agents on new hosts are detected automatically by the master agent, gpmmon. Whenever the gp_enable_gpperfmon server configuration parameter is enabled on the master, the gpmmon agent automatically detects, starts, and begins harvesting data from new segment agents.
Viewing and Maintaining Agent Log Files
Log messages for the Command Center metrics collector agents are saved in the logs directory of the Command Center installation directory.
Log messages for the gpperfmon master agent are written to the following file by default:
$MASTER_DATA_DIRECTORY/gpperfmon/logs/gpmmon.log
To change the gpperfmon log file location, edit the log_location parameter in gpperfmon.conf.
On the segment hosts, gpperfmon agent log messages are written to a gpsmon.log file in the segment instance’s data directory. For a host with multiple segments, the agent log file is located in the data directory of the first segment, as listed in the gp_configuration table by dbid. If the segment agent is unable to log into this directory, it will log messages to the home directory of the user running Command Center (typically gpadmin).
Configuring gpperfmon Log File Rollover
At higher logging levels, the size of the gpperfmon log files may grow dramatically. To prevent the log files from growing to excessive size, you can add an optional log rollover parameter to gpperfmon.conf. The value of this parameter is measured in bytes. For example:
max_log_size = 10485760
With this setting, the log files will grow to 10MB before the system rolls over the log file. The timestamp is added to the log file name when it is rolled over. Administrators must periodically clean out old log files that are no longer needed.