dashing

Dynamically create dashboards in Dashing.io

We use Skytap for many of our VMs and Dashing is a great tool to display data. We wanted to combine these by providing a dynamic way to create a new dashboard for each Skytap environment. The VMs all report their information up to Dashing automatically (I'll cover this in a separate post), allowing us to make some assumptions on what widgets we have.

This case may not work for you out of the box (unless you use Skytap), but the idea of dynamically available dashboards could work as well for AWS or other services without too much change to this sort of thing.

We'll need two files to create this. One is a job that runs periodically looking for what environments we have. We then create symlinks for that environment number to our main template. This allows us to easily update one file and it'll apply to all environments' dashboards. The job also cleans itself up, so if there's an environment that's been destroyed, the symlink is removed.

The second is the template itself. This is the meat of things, and is what is accessed whenever someone pulls up any environment dashboard. Because each dashboard URL includes the name of the environment of interest, we can use that to determine what VMs are that environment and display things accordingly.

First, the job file:

environments.erb

Skynet.py is a script we use to access the Skytap API. This just returns a list of environments.

This job essentially just gets the list of environments and loops through it to create symlinks to the template file. It also creates its own widget giving us the total number of Skytap environments we're using.

This piece is basically the meat of it. Parsed is a variable that has the list of environments in it and we just loop through it and add symlinks.

parsed.each do |value|
  total += 1
  erb = env_path + value.to_s + '.erb'
  next if File.file?(erb)
  puts 'Creating template link: ' + erb.to_s
  File.symlink(template, erb)
end

 

NEXT, THE DASHBOARD TEMPLATE:

environment.erb

The symlinks created all point to this template.

It does a few things I think are interesting. 

env = File.basename(__FILE__, File.extname(__FILE__)) 
skynet = `/opt/skynet/skynet.py -a vms #{env.to_s}`
parsed = JSON.parse(skynet)

This gets the environment name from the filename of the symlink. We use this environment name to go to our skynet script to get a list of VMs in that environment.

widgets = Sinatra::Application.settings.history
widget_list = []
widgets.each do |key, value|
  if key.to_s.index(vm.to_s) != nil
  .
  .
  .

This section gets the full list of widgets and loops through them all, using any that have the VM id in the name of the widget. We push all of our widget data from these machines named vmid_<name>, we can do this to get a list of widgets without even knowing in advance what widgets are available for a given VM.

widget_list.sort! { |a,b| a.order <=> b.order }
col = 0
widget_list.each do |w|
    col = col + 1
    %>
      <li data-row="<%=row%>" data-col="<%=col%>" data-sizex="1" data-sizey="1">
      <div data-id="<%=w.id%>" data-view="<%=w.view%>" ></div>
      </li>
    <%

This takes the list of widgets made from the prior loop and sorts them by order (this allows a VM to order the widgets if there's a logical way to order them, again without us knowing about it here). We then loop through this list and builds the actual HTML widget information. 

And with that, Dashing doesn't know about our environments, or what widgets are out there, but we can still display great dashboards for each environment. To have a VM give more information about itself, we can just have it update a new widget with the right name and the rest will be automatically done for us, with no changes to the Dashing server.

To make things even easier, we use Puppet to deploy the scripts to the server that are needed to post the widgets, so a new server will be updating the dashboard nearly immediately after creation.

Click for enlarged detail. This sort of dashboard any of our engineers or QA folk can get about any of our server sets with just a click.


The actual files:

Both scripts are below for review, or you can get the from gist with the links, also below.

 

 

Dashing widget to show Internet bandwidth usage

Our network guy likes to regularly check on the company's overall network usage. It's a bit of a tic of his, but those sorts of checks are a good way to see if something has gone sideways and is gobbling up our network even before users complain about a problem.

This widget now lets us all get that same information just at a glance. No more logging onto the ASA to see how it's doing, and now we get the info simply at a glance.

One of the two graphs: our internet usage over the last five minutes.

One of the two graphs: our internet usage over the last five minutes.

In all, four widgets are created with this dashing.io job:

  • inbound_bandwidth: simply a number of the current bandwidth usage (good for a maxmeter)
  • outbound_bandwidth: the same as inbound_bandwidth, but for outbound data.
  • bandwidth_short: bandwidth (in and out) over the last five minutes
  • bandwidth_long: bandwidth (in and out) over the last eight hours

The two graphable data sets are best served with the rickshaw graph (that's what I'm using, here), but any other graph able to read similar data could produce something for you.

It's still fairly early in the day here...

It's still fairly early in the day here...

In the end, you'll need to change some of the settings in there (IP address and community names need to be set to legitimate things), and the timing can be changed to what you want, but this can be used fairly directly if you want a simple view of your overall bandwidth.

Update: Our dashboard itself (the erb file) entries to display this data look like this:

<li data-row="2" data-col="4" data-sizex="3" data-sizey="1">
<div data-id="bandwidth_long_smooth" data-view="Rickshawgraph" data-suffix="M" data-unstack="true" data-renderer="line" data-legend="true" data-min="0" data-max="350" data-title="Bandwidth (8 hr)"></div>
</li>

<li data-row="3" data-col="4" data-sizex="3" data-sizey="1">
<div data-id="bandwidth_short" data-legend="true" data-view="Rickshawgraph" data-renderer="line" data-suffix="M" data-unstack="true" data-min="0" data-max="350" data-title="Bandwidth (5 min)"></div>
</li>

Exposing Dashing widget data

When troubleshooting Dashing widgets, I love being able to see what data Dashing really has about various widgets. This really helped troubleshoot the ability to check for stale widget data, among other things.

What I did was create a very simple dashboard called widgetdata. It has this code in it:

Which then creates a dashboard with some global widgets, followed by the raw data of all of the widgets. This list can get long, but I've found it invaluable to be able to search through this data to see what's going on. At the top are two widgets built from prior posts: Marking a widget as stale and Dashing widget to show widget count.

A small clip of some of the raw widget data

A small clip of some of the raw widget data

Dashing widgets for Active Directory

A list of about to expire passwords.

A list of about to expire passwords.

Here's a set of Dashing widgets that give us some visibility to users with expiring passwords. This should run as a scheduled job on a domain controller. It queries through PowerShell the users and their password expirations.

The three widgets created are:

expiring_users

A list of expiring users, defaulting to all users within the next 14 days

expired_users

All expired users

locked_users

All users who have locked out their accounts

For expiring_users and expired_users, the widget doesn't need to update very often, but if you want to use locked_users, you may want to have the scheduled job run more frequently so you can respond more quickly when a user locks themselves out.

Additionally, a fourth widget is made that is essentially a set of all three of those in one:

Looks like Brian reset his password in time, but not Patrick.

Looks like Brian reset his password in time, but not Patrick.

active_directory_users

This one then will turn to a yellow/warning status if there's an expired user, and red/critical if there is a locked out user.

 

Below is the PowerShell script. Then just add the list widget to your Dashing dashboard as desired.

Mark Dashing widget data as stale

I wanted an easy way to know if a Dashing widget hadn't been updated in a while to know if that data was stale. There's a little "last updated" line in the widget, but it's only really useful if I walk up to our dashboard. I want something that's obvious from a distance, but invisible when things are going well.

The below gist files can help set this up. The stale-widgets.rb file can go in your jobs/ folder and will run and go through each widget. If it's older than the threshold (set, here, at two hours), then it changes the status of that widget to "stale". The application.scss changes (just add those lines into your application.css file, or change to suit) then create a new status called "status-stale" which determine what a stale widget looks like.

Beware, though, that a status isn't changed until a new status shows up. All of our widgets push "status:normal" when all is good, which clears other status messages. If your widget updates don't push something similar, the widget may stay marked stale if it gets a future update.


Displaying Nagios in a Dashing dashboard

We have a nice Dashing dashboard, but also have Nagios checking our general network environment, and wanted a way to combine them both - Dashing can display the general statistics, and Nagios can tell us if something is wrong on the network somewhere.

This is fairly easily accomplished using Dashing's iframe widget.

This makes a widget take up three columns (our NOC dashboards are three tiles wide...) and shows our Naglite3 Nagios board. Everything's in one, nice, easy-to-glance-at screen.

Nagios status integrated into the Dashing dashboard.

Nagios status integrated into the Dashing dashboard.


Bus info in Dashing dashboards

The "Next Bus" widget

The "Next Bus" widget

We have a Dashing dashboard here at work, and there was some extra room on the dashboard, so I added a new widget to let me know when I should be leaving for the bus. This only works if you're in the Seattle area (it uses the One Bus Away data, which I think is Seattle-specific), but can take an estimate of walk-time into account so the timer displays "when to leave the office", not "when the bus arrives", roughly.

Some changes would need to be made to make it work for other cities, but hopefully this'd get you a bit of a head start.

This is the oba.rb file that collects the widget data, living in our jobs/ folder.

The bus widget along with the rest of this section of the dashboard.

The bus widget along with the rest of this section of the dashboard.

Dashing widget to show widget count

We build a lot of Dashing widgets dynamically, and discard just as many (see clearing Dashing widgets). As a part of that, I like seeing how many widgets we currently have, since each dashboard only displays a small faction of our total widgets.

We have the following file in our jobs/ directory:

 

This goes through the widget list and updates a widget called widget_count with the total number, which can be displayed in a nice little number widget.

Delete a Dashing dashboard widget

Every once in a while it's nice to be able to remove a Dashing widget. Maybe the data's stale, or otherwise may have bad data. This is a short snip that can be added into another Dashing job. Obviously this sort of job isn't what you'd want, but it should give you the info enough to put this into your own solution.

Widgets using this data will still exist, but just not have that data to pull from. Forcing a refresh of the dashboard (if it's up constantly) can then have it re-poll for the non-existant data, causing the widgets to revert to their "no data" state.

Clear Dashing Widgets

There are a handful of ways of clearing old widget data from Dashing, but none are well documented. Here are a few options, all of which boil down to the line:

Sinatra::Application.settings.history.clear()

This can go in a few places, depending on when and how you want the history cleared. We are dynamically making and discarding tons of widgets, so regularly clearing out the stale data is useful for us.

The main options are:

Clear widgets when Dashing starts:

Find the config.ru file, and add the line before the "run Sinatra::Application" line.

Clear widgets on a schedule:

Make a new job file in the jobs/ folder and create a file that looks something like:

The above will clear the history every midnight. The '0 0 * * *' is the schedule, using the format for cron jobs.

The Dashing Dashboard

Experts in DevOps are always interested in new ways to communicate technological innovations to interested industries. For many companies, the dashboard, which should present essential information in an easy to read graphical user interface (GUI), instead becomes a headache and a hassle with utility and design competing for supremacy rather than working together in harmony. The Dashing dashboard works to solve these problems by providing data in a simple, elegant solution.

What is Dashing and How Does it Work?

Dashing is a Sinatra based framework, which is a bit like Ruby without the Rails. Dashing allows developers to utilize premade widgets or design their own with HTML, SCSS, and data bindings from batman.js depending on their individual needs. To get your data to your widgets, you create a widget id then pass in the data with jobs (by using rufus scheduler) or API (directly over HTTP). A drag and drop interface makes it easy to rearrange your widgets quickly and efficiently.

Why Should I Use Dashing as a Dashboard?

The simplicity of the interface allows developers to focus on personalization elements (such as matching the company's design and tailoring the specific information the widgets will display) rather than muddling through overly-complicated coding requirements. Dashing dashboards update in real-time to make sure your business is displaying the latest data. The Dashing frontend also displays in any browser so it stays adaptable to a wide variety of situations.

Dashing has been used for a variety of diverse situations due to its simplicity and adaptability. The Dashing widget challenge produced a gallery of exciting examples and possibilities for the dashboard's potential.