Pivotal Greenplum® Command Center 4.5.0 Release Notes
This document contains release information about Pivotal Greenplum Command Center 4.5.0. Greenplum Command Center 4.5.0 is a minor release that adds some features and resolves some issues.
See Enhancements and Changes in Greenplum Command Center 4.5.0 for information about new features and changes in this Command Center release.
Greenplum Command Center 4.5.0 is compatible with the following platforms.
- Pivotal Greenplum Database 5.14.0 and higher
- Red Hat Enterprise Linux 6.x1 and 7.x
- CentOS 6.x1 and 7.x
- SUSE Enterprise Linux 11 SP42
1If you use resource groups and workload management on Red Hat or CentOS 6.x, upgrade your kernel to 2.6.32-696 or higher to benefit from improvements in the Linux cgroups service.
2Greenplum Command Center workload management features are not supported on SUSE Enterprise Linux 11.
Greenplum Command Center 4.5.0 contains the following new features:
On the new Admin> Alerts page, Command Center users with superuser permission can configure alert rules. When an alert rule is matched, a record is logged in the
gpmetrics.gpcc_alert_log table and, optionally, an email is sent to a list of addresses you specify. You can also write a custom shell script to send alerts to other messaging systems.
Alert rules can detect the following events:
- Segment failure
- Out of memory errors
- Average memory usage percentage of segment hosts exceeds a specified percentage for a specified number of minutes
- Memory usage on the master host exceeds a specified percentage for a specified number of minutes
- Total disk space used on all hosts exceeds a specified percentage
- Number of Greenplum Database connections exceeds a specified percentage
- Average CPU usage of segment hosts exceeds a specified percentage for a specified number of minutes
- Spill files for an active query exceed a specified number of gigabytes
- Runtime for a query exceeds a specified number of minutes
- A query is blocked for more than a specified number of minutes
Query Text Download for Long Queries
On the Query Details page, if the query text is longer than 100K characters, Command Center only shows the first 100K characters and, when you click COPY, copies only the first 100K characters to the clipboard. When the query text is longer than 100K characters, Command Center adds a button, Retrieve full query text, which you can click to download a text file containing the entire text of the query. The file is available to download for 24 hours, or until the query is saved to history, when history collection is enabled.
161436520 - Panels displaying help text can now be scrolled vertically when the text does not fit in the window.
158278655 - The displayed metrics for a query could be inaccurate if Command Center received the query “done” status before the final metrics arrived. Now the final metrics are updated even when they arrive after the “done” status.
161074984 - Repeated assertion errors logged in the
agent.logfile caused log files to grow quickly and consume too much disk space. This is fixed. Each type of assertion error will be logged no more than once every 10 minutes.
162114428 - When a user visits Command Center with a browser, the Command Center web server (gpccws process) establishes a websocket connection and runs two goroutines to service the websocket. When the websocket closes, one of the goroutines does not exit. After many connections the gpccws process can occupy gigabytes of memory. This memory leak is fixed.
- The Greenplum Database metrics collector extension is now enabled only when Command Center agents are running. Previously, if an agent process terminated, the metrics collector continued to collect and send data.
Greenplum Command Center 4.4.2 contains the following resolved issues:
160195564 - When Command Center is restarted after a user has submitted a query, and the user submits another query in the same session, the database name and user name are missing from the second query in the Command Center interface. This is fixed.
161077294 - The Authentication view now prevents the user from saving an undefined entry to the
161274227 - The Command Center installer,
gpccinstall, no longer prints errors to the
pg_loglog file about missing
161273865 - The Query Monitor no longer shows runtimes flashing to 0 seconds when a query is cancelled through the user interface.
The Admin> Authorization view allows entering host names in the Address field.
On the visual query plan, a data motion showed a finished status when the send data operation completed. This is changed so that the node completes only after the corresponding data receive operation has also completed.
Enabled cross-site request forgery prevention during login.
Resolved Issues in Greenplum Command Center 4.4.1
160679694 - If Command Center is restarted while on the Query Monitor view and the browser is then refreshed, an extra web socket connection is created. This is fixed.
MPP-29539 - Command Center agent (ccagent) logging is disabled for a known issue with PL/pgSQL queries.
Greenplum Command Center 4.4.0 contains the following features and enhancements.
Command Center has a new user interface to assist administrators in enabling resource groups in Greenplum Database, importing existing resource queues to resource groups, and enabling workload management with Command Center. The option to import resource queues to resource groups is presented if no resource groups have been created (other than
admin_group) and Greenplum Database has resource queues to convert (other than
pg_default). Once the administrator has imported resource queues, or chosen to skip importing resource queues, the option to import queues is no longer presented.
The resource group list on the Workload Management view has a new column to show the minimum (fixed) amount of memory that will be allocated to a query for each resource group. This value is recalculated when you enter new values while editing resource groups.
Administrators can now define resource group assignment rules and idle session kill rules with an interactive interface. It is no longer necessary to edit the JSON document for workload management rules. The JSON text field is removed.
The Workload Management view changed to a light theme.
Command Center users with the Self Only permission level can:
- see all queries on the query monitor, including queries owned by other users
- cancel their own queries
- access the query details view for their own queries
- hover on a query to see query text for their own queries
- hover on locking/blocking queries and access details of locking/blocking queries that do not belong to other users
Users with Self Only permission level cannot see query text or access the details views of others’ queries.
The Greenplum Database roles
gpcc_operator_basicare created during Command Center installation if they do not already exist.
- Fixed a bug where the database name and role name were missing from query details when queries are executed in a session after restarting Command Center.
Greenplum Command Center 4.3.0 contains the following enhancements.
Resource Group Management
The Admin>Workload Mgmt view has a new user interface you can use to add and remove resource groups and to change the Concurrency, CPU %, and Memory % attributes of resource groups.
Resource Group Role Assignments
The Admin>Workload Mgmt view has a new user interface you can use to view and change Greenplum Database roles’ default resource groups.
Details Added to Visual Query Plan Steps
The metrics collector extension in Greenplum Database release 5.9 is updated to submit additional information about each step in the query plan to the Command Center backend. Command Center displays this information in the visual query plan when you expand a step in the query plan. The new information displayed depends on the operation the step performs and includes details such as hash key, merge key, join condition, or filter condition. Previously, you could only see this information by generating the textual query plan.
Greenplum Command Center 4.2.0 contains the following enhancements.
Visual Query Plan
The Command Center Query Details view now includes a visual query plan.
Idle Session Kill Rules
Idle session kill rules can include optional
- The value of the
exemptedRolesparameter is a list of Greenplum Database roles that are exempted from the rule. The list can include Posix regular expressions to match Greenplum Database role names.
- The value of the
messageparameter is a string to include in the message that is displayed when a session is killed.
Command Center Can Run on the Master Host or Standby Master Host
The Greenplum Command Center web server and backend may now be executed on the master host or on the standby master host. Running GPCC on the standby master host is recommended to avoid adding load to the master server, but it is no longer a requirement. After the GPCC software is installed, log in to the host where you want to run GPCC, source the
gpcc_path.sh file in the GPCC installation directory, and run the
gpcc start command.
Greenplum Command Center 4.1.0 contains the following enhancements.
- Command Center administrators can set permission levels for Command Center users. Permissions are enforced as described in the documentation.
On the Query Detail view, clicking Copy in the query text or query plan panel copies the text in the panel to the clipboard.
A help icon and in-app help have been added on the Query Monitor and Query Detail views.
The following workload management features, improvements, and bug fixes have been added in the workload management extension included with Greenplum Database 5.8.0.
On the Admin>Workload Mgmt view, you can add idle session kill rules for each resource group. When you add these rules, the Greenplum Database workload management extension kills a session after it has been idle for the number of seconds you specify. See Workload Management for syntax and examples.
Optimizations have been implemented to reduce the impact on Greenplum Database when the workload management extension is disabled.
The workload management extension takes advantage of resource group name-to-id caching added in Greenplum Database.
Fixed a bug in the workload management extension that caused errors to print to the
psqlprompt, even when the extension was disabled.
Greenplum Command Center 4.1.0 contains the following bug fixes.
- In the History Detail view, when either of the Disk R or Disk W metrics is 0, both are reported to be 0. This is fixed.
- When using the Kerberos gpmon-only authentication mode, generating an explain plan failed. This is fixed.
- The Command Center agent failed with the message “Error: can’t find gpcc.query_metrics_port, metrics_collector is not correctly installed.” This occurs when running Command Center on a Greenplum Database system that was upgraded from an earlier Greenplum Database 5.x release. The
gp_wlmextensions are installed with the upgrade, but the upgrade process does not perform the required configuration changes in the
postgresql.confconfiguration file. The Command Center installation instructions now include steps to manually configure and restart an upgraded Greenplum Database system.
- When Command Center starts, an error message is written in the Greenplum Database log file: “function gpcc_schema.read_pghba(unknown) does not exist.” This is fixed.
- A change to time zone handling in Greenplum Database 5.7 can cause Greenplum Command Center 4.0.0 to display an incorrect time if the master host operating system time and Greenplum Database use different time zones. Now Command Center times are displayed using the time zone of the Greenplum Database master host operating system. The current time, last sync time, and timestamps in alert logs, cluster metrics, and query history are all displayed using the master host’s system time zone.
- On the Admin> Authorization view, when the authorization method is
gssand the options field contains text, changing the method to
trustdoes not clear the options field and it is not possible to save changes to the
- The Command Center web server,
gpccws, spawns ssh processes but does not reap them in a timely manner, leading to many zombie processes. This is fixed.
- On the Admin> Authentication view, if the number of users listed in the user column is longer than can be displayed, Command Center truncates the list and adds ellipsis (
...) to the end. Only administrators can view the complete value, by editing the field. Now any user with access to the view can see the full list of users by hovering over the field.
Command Center Installation Changes
Greenplum Command Center 4.x software, unlike previous releases, is installed on every host in the Greenplum Database cluster. The Command Center web server and backend run on the standby master, if your Greenplum Database cluster has a standby master. If there is no standby master, Command Center runs on the master host instead.
To modify the Command Center installation—for example to enable or disable SSL or install Command Center on new or replaced hosts—just re-execute the installer and restart Command Center. It is not necessary to uninstall Command Center before reinstalling.
There is one Command Center installation per Greenplum Database cluster. It is no longer necessary to create Command Center instances after installing the software.
gpcmdr command-line utility is replaced with the new
gpcc utility. Use the
gpcc utility to start and stop Command Center and metrics collection agents, check Command Center status, and enable or disable Kerberos authentication.
In previous releases, the gpmon role required only local connections to databases on the Greenplum master host. In Greenplum Command Center 4.x, the gpmon user must be able to connect to databases from the host running the Command Center web server and backend, which requires adding a host entry to the
pg_hba.conf authentication configuration file.
Real-time Query Metrics
Greenplum Command Center 4.0 introduces real-time query metrics for Pivotal Greenplum Database 5.7 and above. This new feature combines the following new features in Greenplum Database and Greenplum Command Center:
- Greenplum Database saves query execution metrics in shared memory while queries execute.
- A new Greenplum Database metrics collection extension, included with Pivotal Greenplum Database, emits the saved metrics as UDP datagrams.
- A new Greenplum Command Center metrics collection agent running on each Greenplum Database host receives the datagrams and posts metrics to the Greenplum Command Center backend. The Command Center backend starts and manages the metrics collection agents.
The Command Center Query monitor view updates in real time so you can see queries that are waiting to execute and the current status and resource usage for queries that are running.
Metrics collection now includes lock and spill file information. On the Query Monitor, you can see which queries are blocked and which queries hold the locks blocking them. The Query Monitor shows the total size of spill files created on all segments for each query.
Installing the gpperfmon database remains a prerequisite for Command Center. The gpperfmon database is the source for query history displayed in the Command Center user interface. The new real-time metrics are not persisted and are not directly related to the metrics collected and persisted in the gpperfmon database.
Workload management is now an integrated Command Center feature rather than a separate product. Workload management is available in Command Center only after resource groups have been enabled in Greenplum Database by changing the
gp_resource_manager server configuration parameter from
'group' and enabling Linux control groups (cgroups).
In Command Center 4.0, workload management allows you to assign transactions to Greenplum Database resource groups at execution time by evaluating the current database role and query tags against workload assignment filters you define in Command Center. Query tags are user-defined
name=value parameters that you define in the
gpcc.query_tags database session parameter. You can define multiple query tags separated by semicolons. Set query tags in a Greenplum Database session either as a parameter in the database connection URL or by executing
SET gpcc.query_tags TO '<tag1>=<val1>;<tag2>=<val2>;...' in the database session.
When a transaction is about to execute, the current database role and query tags are compared to the workload assignment filters that you have created in Command Center. If a match is found, the transaction is assigned to a resource group according to the workload management filter. Otherwise, the transaction is assigned to the database user’s resource group, which is the default behavior when Command Center workload management is not enabled.
Workload management uses the
gp_wlm database extension included with Pivotal Greenplum Database.
Some features available in previous Greenplum Command Center releases have been removed or are not yet implemented in Command Center 4.x.
- The ability for a Command Center admin to post a message to the Query Monitor view is not yet implemented.
- The multi-cluster view has been removed.
The following are known issues in the current Greenplum Command Center release.
Unable to View Real Time Queries After Upgrading From a Previous Command Center Release
If you install a new version of Greenplum Command Center using the same port number as the previous version, and you use the Chrome web browser, you may be unable to view real-time queries until after you clear the browser’s cache. See the note in Connecting to the Command Center Console for steps to clear the browser cache.