How to Remove Ganglia and Again Install

Chapter 4. The Ganglia Web Interface

So far, this book has dealt with the drove of data. Now we volition hash out visualizing information technology. Visualization of these data is the primary responsibleness of a web-based application known as gweb. This chapter is an introduction to gweb and its features. Whether the job is understanding how a problem began in your cluster or convincing management that more than hardware is required, a picture is worth a chiliad data points.

Navigating the Ganglia Web Interface

gweb is organizaed into a number of summit-level tabs: Main, Search, Views, Aggregated Graphs, Compare Hosts, Events, Automated Rotation, Live Dashboard, and Mobile. These tabs allow you to easily jump right to the information you demand.

The gweb Main Tab

Effigy 4-ane. gweb navigation overview

gweb's navigation scheme is organized effectually Ganglia'south core concepts: grids, clusters, and nodes. As y'all click deeper into the bureaucracy, breadcrumb-style navigation links let you to render to college-level views. Figure four-1 shows how you lot tin hands navigate to exactly the view of the information you want.

Grid View

The grid view (Figure 4-ii) provides the highest-level view available. Grid graphs summarize information across all hosts known to a single gmetad process. Filigree view is the jumping-off betoken for navigating into more than details displays dealing with individual clusters and the hosts that compose those clusters:

1. Clicking on any grid-level summary graphs brings upwards the all time periods brandish. Clicking once again enlarges the graph yous're interested in.

2. Clicking on any cluster-level graph displays the cluster view.

Cluster View

A cluster is a drove of gmonds. They may be grouped past concrete location, common workload, or any other criteria. The top of the cluster view (Effigy iv-three) displays summary graphs for the entire cluster. A quick view of each private host is farther down the page.

Figure 4-ii. Filigree view

Figure iv-3. Cluster view

1. Clicking on a cluster summary shows you that summary of a range of time periods.

2. Clicking on an individual host takes you lot to the host display.

The background color of the host graphs is determined by their one-infinitesimal load average. The metric displayed for each host tin can be changed using the Metric select box near the top of the page.

The utilization heatmap provides an alternate display of the one-infinitesimal load averages. This is a very quick way to get a feeling for how evenly balanced the workload is in the cluster at the present time. The heatmap can exist disabled by setting

$conf["heatmaps_enabled"]=0

in conf.php .

When working with a cluster with thousands of nodes, or when using gweb over a wearisome network connexion, loading a graph for each node in the cluster tin can have a meaning amount of time. $conf["max_graphs"] can be defined in conf.php to accost this problem: to set an upper limit on the number of host graphs that will exist displayed in cluster view.

Physical view

Cluster view as well provides an alternative display known as physical view (Figure 4-4), which is too very useful for large clusters. Physical view is a compressed text-simply brandish of all the nodes in a cluster. By omitting images, this view can return much more rapidly than the main cluster view.

Effigy four-iv. Physical view

Clicking on a hostname in physical view takes you to the node view for that host. Node view is another text-only view, and is covered in more detail in Host View.

Adjusting the time range

Grid, cluster, and host views permit you to specify the time bridge (Figure iv-five) y'all'd similar to see. Monitoring an ongoing upshot usually involves watching the final few minutes of information, but questions like "what is normal?" and "when did this start?" are often best answered over longer time scales.

Effigy 4-5. Choosing a time range

You are free to define your own fourth dimension spans every bit well via your conf.php file. The defaults (defined in conf_default.php ) await like this:

                #   # Fourth dimension ranges   # Each value is the # of seconds in that range.   #   $conf['time_ranges'] = assortment(      'hour' => 3600,      '2hr'  => 7200,      '4hr'  => 14400,      'mean solar day'  => 86400,      'week' => 604800,      'month'=> 2419200,      'year' => 31449600   );

All of the born time ranges are relative to the electric current time, which makes it difficult to see (for instance) five minutes of data from two days agone, which can be a very useful view to accept when doing postmortem research on load spikes and other issues. The time range interface allows manual entry of brainstorm and end times and also supports zooming via mouse gestures.

In both cluster and host views, it is possible to click and drag on a graph to zoom in on a particular time frame (Effigy 4-6). The interaction causes the entire page to reload, using the desired time menses. Note that the resolution of the information displayed is express past what is stored in the RRD database files. Subsequently zooming, the time frame in utilize is reflected in the custom time frame display at the top of the folio. You can clear this by clicking clear and so go . Zoom support is enabled past default just may exist disabled by setting $conf["zoom_support"] = 0 in conf.php .

Figure 4-6. Zooming in on an interesting time frame

Host View

Metrics from a single gmond procedure are displayed and summarized in the host view (Effigy iv-seven). Summary graphs are displayed at the height, and private metrics are grouped together lower down.

Host Overview contains textual information about the host, including any string metrics being reported by the host, such as final boot time or operating system and kernel version.

Figure 4-7. Host view

Viewing private metrics

The "inspect" choice for individual metrics, which is also available in the "all time periods" display, allows you to view the graph data interactively:

1. Raw graph information tin can be exported equally CSV or JSON.

2. Events can be turned off and on selectively on all graphs or specific graphs.

3. Trend assay tin make predictions about future metric values based on past data.

4. Graph tin can exist time-shifted to show overlay of previous period'south data.

Node view

Node view (Effigy 4-8) is an alternative text-simply display of some very basic information near a host, similar to the physical view provided at the cluster level.

Effigy four-8. Node view

Graphing All Fourth dimension Periods

Clicking on a summary graph at the top of the grid, cluster, or host views leads to an "all fourth dimension periods" view of that graph. This brandish shows the same graph over a diverseness of time periods: typically the last hour, day, week, month, and year. This view is very useful when determining when a item trend may have started or what normal is for a given metric.

Many of the options described for viewing individual metrics are as well available for all time periods, include CSV and JSON export, interactive inspection, and consequence display.

The gweb Search Tab

Search allows you to find hosts and metrics speedily. It has multiple purposes:

• Detect a detail metric, which is especially useful if a metric is rare, such every bit outgoing_sms_queue.

• Apace observe a host regardless of a cluster.

Figure 4-9 shows how gweb search autocomplete allows you to find metrics beyond your unabridged deployment. To use this feature, click on the Search tab and start typing in the search field. Once you stop typing, a listing of results will appear. Results volition incorporate:

• A listing of matching hosts.

• A listing of matching metrics. If the search term matches metrics on multiple hosts, all hosts will be shown.

Figure 4-9. Searching for load_one metrics

Click on any of the links and a new window will open that will take you directly to the outcome. Yous tin keep clicking on the results; for each consequence, a new window will open.

The gweb Views Tab

Views are an arbitrary collection of metrics, host report graphs, or aggregate graphs. They are intended to be a way for a user to specify things of which they desire to take a single overview. For example, a user might want to come across a view that contains aggregate load on all servers, aggregate throughput, load on the MySQL server, and and then on. There are two ways to create/modify views: one is via the web GUI, and the other past programatically defining views using JSON.i

Creating views using the GUI

To create views click the Views tab, then click Create View. Type your name, then click Create.

Calculation metrics to views using the GUI

Click the plus sign in a higher place or below each metric or composite graph; a window volition pop up in which y'all tin select the view you desire the metric to exist added. Optionally, y'all can specify warning and critical values. Those values will announced as vertical lines on the graph. Echo the process for consecutive metrics. Figure iv-10 shows the UI for adding a metric to a view.

Figure 4-ten. Metric actions dialog

Defining views using JSON

Views are stored equally JSON files in the conf_dir directory. The default for the conf_dir is /var/lib/ganglia/conf . You can alter that by specifying an alternating directory in conf.php :

$conf['conf_dir'] = "/var/world wide web/html/conf";

You tin can create or edit existing files. The filename for the view must start with view_ and end with .json (as in, view_1.json or view_jira_servers.json ). It must be unique. Here is an example definition of a view that will result with a view with iii dissimilar graphs:

{  "view_name":"jira",  "items":[   { "hostname":"web01.domain.com","graph":"cpu_report"},   { "hostname":"web02.domain.com","graph":"load_report"},   { "aggregate_graph":"truthful",       "host_regex":[         {"regex":"web[2-7]"},         {"regex":"web50"}       ],       "metric_regex":[         {"regex":"load_one"}       ],       "graph_type":"stack",       "title":"Location Web Servers load"   }   ],   "view_type":"standard"  }

Tabular array 4-one lists the top-level attributes for the JSON view definition. Each particular can have the attributes listed in Tabular array four-2.

Table 4-ane. View items

Central	Value
view_name	Name of the view, which must be unique.
view_type	Standard or Regex. Regex view allows y'all to specify regex to match hosts.
items	An array of hashes describing which metrics should be part of the view.

Tabular array 4-2. Items configuration

Key	Value
hostname	Hostname of the host that we desire metric/graph displayed.
metric	Proper name of the metric, such as load_one.
graph	Graph name, such as cpu_report or load_report. You tin apply metric or graph keys but not both.
aggregate_graph	If this value exists and is set to true, the item defines an aggregate graph. This particular needs a hash of regular expressions and a clarification.
warning	(Optional) Adds a vertical yellow line to provide visual cue for a alarm country.
critical	(Optional) Adds a vertical red line to provide visual cue for a disquisitional state.

Once you compose your graphs, it is often useful to validate JSON—for example, that you don't have extra commas. To validate your JSON configuration, utilize Python's json.tool:

$ python -yard json.tool my_report.json

This command will written report whatever issues.

The gweb Aggregated Graphs Tab

Aggregate graphs (Figure 4-11) let you to create composite graphs combining different metrics. At a minimum, you lot must supply a host regular expression and metric regular expression. This is an extremely powerful feature, as it allows you to quickly and easily combine all sorts of metrics. Figure 4-12 includes two aggregate graphs showing all metrics matching host regex of loc and metric regex of load .

Figure 4-11. Aggregate line graph

Figure four-12. Aggregate stacked graph

Decompose Graphs

Related to aggregate graphs are decompose graphs, which decompose aggregate graphs past taking each metric and putting it on a split up graph. This feature is useful when you have many different metrics on an aggregate graph and colors are blending together. You will find the Decompose button above the graph.

The gweb Compare Hosts Tab

The compare hosts feature allows you to compare hosts across all their matching metrics. Information technology will basically create amass graphs for each metric. This characteristic is helpful when y'all want to observe why a particular host (or hosts) is behaving differently than other hosts.

The gweb Events Tab

Events are user-specified "vertical markers" that are overlaid on acme of graphs. They are useful in providing visual cues when certain events happen. For example, yous might want to overlay software deploys or backup jobs then that yous tin quickly acquaintance alter in behavior on certain graphs to an external upshot, equally in Figure 4-thirteen. In this example, we wanted to meet how increased rrdcached write delay would touch on our CPU expect IO percentage, and then we added an consequence when nosotros made the alter.

Figure 4-xiii. Upshot line overlay

Alternatively, you can overlay a timeline to indicate the duration of a particular event. For example, Figure four-xiv shows the full timeline for a backup chore.

Figure 4-14. Event timeline overlay

Past default, Ganglia stores event in a JSON hash that is stored in the events.json file. This is an instance JSON hash:

[  { "event_id":"1234",      "start_time":1308496361,      "end_time":1308496961,      "summary":"DB Backup",      "description":"Prod daily db backup",      "grid":"*",      "cluster":"*",      "host_regex":"centos1"  },  { "event_id":"2345",      "start_time":1308497211,      "summary":"FS cleanup",      "grid":"*",      "cluster":"*",      "host_regex":"centos1"  } ]

It is also possible to use a different backend for events, which can be useful if y'all need to scale upwardly to hundreds or thousands of events without incurring the processing penalty associated with JSON parsing. This characteristic is configured with two configuration options in your conf_default.php file. You lot should have PHP support for MySQL installed on your gweb server before attempting to configure this support. The database schema can be imported from conf/sql/ganglia.mysql :

# What is the provider used to provide events # Examples: "json", "mdb2" $conf['overlay_events_provider'] = "mdb2"; # If using MDB2, connection string: $conf['overlay_events_dsn'] = "mysql://dbuser:dbpassword@localhost/ganglia";

Alternatively, you can add events through the web UI or the API.

Events API

An easy way to manipulate events is through the Ganglia Events API, which is available from your gweb interface at /ganglia/api/events.php. To use it, invoke the URL along with cardinal/value pairs that define events. Key/value pairs can be supplied equally either GET or POST arguments. The full list of key/value pairs is provided in Tabular array 4-three.

Table 4-three. Events options

Primal	Value
action	add to add a new event, edit to edit, remove or delete to remove an event.
start_time	Offset fourth dimension of an result. Allowed options are now (uses current system time), UNIX timestamp, or any other well-formed date, every bit supported by PHP's strtotime function.
end_time	Optional. Same format equally start_time.
summary	Summary of an event. It will be shown in the graph legend.
host_regex	Host regular expression, such as web-|app-.

Examples

To add an upshot from your cron task, execute a command such equally:

curl "http://mygangliahost.com/ganglia/api/events.php?\   action=add&start_time=now&\   summary=Prod DB Fill-in&host_regex=db02"

or:

roll -X POST --data " activity=add&start_time=now\   &summary=Prod DB Backup&host_regex=db02" \   http://mygangliahost.com/ganglia/api/events.php

API volition return a JSON-encoded condition message with either status = ok or status = error.

If you are adding an event, you will likewise go the event_id of the event that was merely added in case you want to edit it after, such as to add an end_time.

The gweb Automated Rotation Tab

Automatic rotation is a feature intended for people in data centers who need to continuously rotate metrics to help spot early signs of trouble. It is intended to piece of work in conjunction with views. To activate it, click Automated Rotation and and so select the view you want rotated. Metrics will be rotated until the browser window is closed. Yous can modify the view while the view is rotated; changes will be reflected inside 1 total rotation. Graphs rotate every 30 seconds by default. You can adjust the rotation delay in the GUI.

Another powerful aspect of automatic rotation is that if you have multiple monitors, you can invoke different views to be rotated on dissimilar monitors.

The gweb Mobile Tab

gweb mobile represents the Ganglia web interface optimized for mobile devices. This mobile view is plant by visiting /ganglia/mobile.php on your gweb host. Information technology is intended for any mobile browsers supported by the jQueryMobile toolkit. This display covers virtually WebKit implementations, including Android, iPhone iOS, HP webOS, Blackberry Os 6+, and Windows Phone seven. The mobile view contains but a subset of features, including views optimized for a small-scale screen, host view, and search.

Custom Composite Graphs

Ganglia comes with a number of congenital-in composite graphs, such as a load report that shows current load, number of processes running, and number of CPUs; a CPU written report that shows system CPU, user CPU, and wait IO CPU all on the same graph; and many others. You tin define your ain composite graphs in two ways: PHP or JSON.

Defining graphs via PHP is more complex but gives you consummate control over every aspect of the graph. See the example PHP report for more than details.

For typical use cases, JSON is definitely the easiest way to configure graphs. For instance, consider the following JSON snippet, which will create a composite graph that shows all load indexes equally lines on one graph:

{    "report_name" : "load_all_report",    "title" : "Load All Report",    "vertical_label" : "load",    "series" : [       { "metric": "load_one", "color": "3333bb", "label": "Load one",         "line_width": "2", "blazon": "line" },       { "metric": "load_five", "color": "ffea00", "label": "Load v",         "line_width": "two", "type": "line" },       { "metric": "load_fifteen", "color": "dd0000", "label": "Load 15",         "type": "line" }    ] }

To use this snippet, save it every bit a file and put it in the graph.d subdirectory of your gweb installation. The filename must contain _report.json in information technology to be considered by the web UI. So y'all can save this file in your gweb install as load_all_report.json .

There are two main sections to the JSON written report. The first is a set of configurations for the overall report, and the 2nd is a list of options for the specific data serial that you wish to graph. The configuration options passed to the report are shown in Table 4-iv.

Table 4-four. Graph configuration

Cardinal	Value
report_name	Proper noun of the report that web UI uses.
title	Title of the report to show on a graph.
vertical_label	Y-axis clarification (optional).
series	An assortment of metrics to use to compose a graph. More nearly how those are divers in Table 4-5.

Options for series array are listed in Table iv-5. Note that each series has its own case of the dissimilar options.

Tabular array 4-5. Series options

Central	Value
metric	Name of a metric, such as load_one and cpu_system. If the metric doesn't exist it will be skipped.
color	A 6 hex-decimal colour code, such as 000000 for blackness.
characterization	Metric characterization, such as Load 1.
blazon	Particular type. It tin can be either line or stack.
line_width	If blazon is ready to line, this value will exist used equally a line width. If this value is not specified, it defaults to 2. If type is stack, it's ignored even if set.

Once y'all compose your graphs, it is oft useful to validate JSON. One case would be to verify that there are no extra commas, etc. To validate your JSON configuration, utilize Python's json.tool:

$ python -g json.tool my_report.json

This command will report any problems.

Other Features

There are a number of features in gweb that are turned off past default or can be adjusted:

Metric groups initially collapsed

By default, when you lot click on a host view, all of the metric groups are expanded. You tin change this view and then that simply metric graph titles are shown and yous have to click on the metric grouping to expand the view. To make this collapsed view the default beliefs, add the post-obit setting to conf.php :

$conf['metric_groups_initially_collapsed'] = truthful;

Filter hosts in cluster view

If you'd like to display simply certain hosts in the cluster view, it is possible to filter them out using the text box that is located next to the "Bear witness Node" dropdown. The filter accepts regular expressions, so it is possible to testify any host that has "web" in its name by entering web in the filter box; to show only webservers web10−web17, blazon web1[0-7]; or, to show web03 and web04 and all MySQL servers, blazon (web0[34]|mysql). Notation that the aggregate graphs will notwithstanding include information from all hosts, including those not displayed due to filters.

Default refresh menstruum

The host and cluster view volition refresh every 5 minutes (300 seconds). To adjust information technology, set the following value in conf.php :

$conf['default_refresh'] = 300;

Strip domain name from hostname in graphs

By default, the gweb interface will display fully qualified domain names (FQDN) in graphs. If all your machines are on the same domain, you can strip the domain name by setting the strip_domainname option in conf.php :

$conf['strip_domainname'] = true;

Set default fourth dimension menstruation

You tin adjust the default time period shown by adjusting the following variable:

$conf['default_time_range'] = 'hour';

Authentication and Authorization

Ganglia contains a uncomplicated authorization organisation to selectively allow or deny users admission to certain parts of the gweb application. We rely on the web server to provide authentication, so any Apache authentication arrangement (htpasswd, LDAP, etc.) is supported. Apache configuration is used for examples in this section, only the system works with whatever web server that can provide the required environs variables.

Configuration

The authorisation organisation has three modes of operation:

$conf['auth_system'] = 'readonly';

Anyone is immune to view any resource. No 1 can edit anything. This is the default setting.

$conf['auth_system'] = 'disabled';

Anyone is allowed to view or edit any resource.

$conf['auth_system'] = 'enabled';

Anyone may view public clusters without login. Authenticated users may gain elevated privileges.

If you wish to enable or disable authorization, add the modify to your conf.php file.

When a user successfully authenticates, a hash is generated from the username and a secret central and is stored in a cookie and fabricated available to the rest of gweb. If the cloak-and-dagger fundamental value becomes known, it is possible for an attacker to presume the identity of whatever user.

You can change this underground value at any time. Users who have already logged in will need to log in again.

Enabling Hallmark

Enabling authentication requires two steps:

1. Configure your spider web server to require authentication when accessing gweb/login.php , and to provide the $_SERVER['REMOTE_USER'] variable to gweb/login.php . (This variable is not needed on any other gweb page.)

2. Configure your spider web server to provide $_SERVER['ganglia_secret']. This is a secret value used for hashing authenticated user names.

If login.php does not require authentication, the user will come across an mistake message and no dominance volition exist allowed.

Sample Apache configuration

More than data about configuring authentication in Apache tin be establish hither. Annotation that Apache need only provide authentication; dominance is provided by gweb configuration. A sample Apache configuration is provided here:

SetEnv ganglia_secret yourSuperSecretValueGoesHere <Files "login.php">   AuthType Basic   AuthName "Ganglia Access"   AuthUserFile /var/lib/ganglia/htpasswd   Require valid-user </Files>

Other web servers

Sample configurations for other web servers such as Nginx and Lighttpd are bachelor on the gweb wiki.

Admission Controls

The default access control setup has the post-obit properties:

• Guests may view all public clusters.

• Admins may view all public and private clusters and edit configuration (views) for them.

• Guests may not view individual clusters.

Additional rules may be configured as required. This configuration should get in conf.php . The GangliaAcl configuration property is based on the Zend_Acl belongings. More documentation is available hither.

Annotation that in that location is no built-in stardom betwixt a user and a grouping in Zend_Acl. Both are implemented as roles. The system supports the configuration of hierarchical sets of ACL rules. Nosotros implement user/group semantics by making all user roles children of the GangliaAcl::Guest role, and all clusters children of GangliaAcl::ALL:

Name	Meaning
GangliaAcl::ALL_CLUSTERS	Every cluster should descend from this role. Guests have view access on GangliaAcl::ALL_CLUSTERS.
GangliaAcl::Guest	Every user should descend from this office. (Users may also take other roles, but this one grants global view privileges to public clusters.)
GangliaAcl::ADMIN	Admins may admission all private clusters and edit configuration for any cluster.
GangliaAcl::VIEW	This permission is granted to guests on all clusters, and then selectively denied for individual clusters.
GangliaAcl::EDIT	This permission is used to determine whether a user may update views and perform whatsoever other configuration tasks.

Actions

Currently, we only support two actions, view and edit . These are applied on a per-cluster basis. Then i user may have view access to all clusters, simply edit access to only one.

Configuration Examples

These should get in your conf.php file. The usernames you use must be the ones provided past whatever authentication system you are using in Apache. If yous want to explicitly allow/deny access to sure clusters, you lot need to spell that out here.

All later examples assume you have this code to start with:

$acl = GangliaAcl::getInstance();

Making a user an admin

$acl->addRole( 'username', GangliaAcl::ADMIN );

Defining a private cluster

$acl->addPrivateCluster( 'clustername' );

Granting certain users access to a individual cluster

$acl->addPrivateCluster( 'clustername' ); $acl->addRole( 'username', GangliaAcl::Guest ); $acl->allow( 'username', 'clustername', GangliaAcl::VIEW );

Granting users admission to edit some clusters

$acl->addRole( 'username', GangliaAcl::GUEST ); $acl->add together( new Zend_Acl_Resource( 'clustername' ), GangliaAcl::ALL_CLUSTERS ); $acl->permit( 'username', 'clustername', GangliaAcl::EDIT );

wolfswentorme.blogspot.com

Source: https://www.oreilly.com/library/view/monitoring-with-ganglia/9781449330637/ch04.html

Share this post