Blog Archives

How to parse Nest info (and graph it using MRTG)

How to parse Nest info (and graph it using MRTG)

This is the continuation of my previous post where I talked about setting up my Nest Thermostat.

Data Flow

Since Nest does not offer an open API, we have to cheat a little (ok, a LOT!). By using Scott Baker’s Python code, you can query the Nest web service to get your current status on the Nest (using your nest login, password and serial number).

Data is sent from the local Nest to the web. Python is used to grab the data from the web and put into my own MRTG compatible format. From the Nest data, I am graphing Temperature and Humidity, Heating/Cooling State and the Battery Levels for each of 2 thermostats.

This is all done using a perl program that creates an array from the Python program output, parses the array line by line, then creates a file that MRTG can read to do the actual graphing.

Perl Code

This perl script runs every 5 minutes on my Raspberry Pi to create the files needed for MRTG.

#!/usr/bin/perl -w
use strict;

###
### Perl Nest (using Python to scrape the web) to mrtg log file script
### Written by Kyle Platts
### Nest Gen 1 Thermostat

my $nest1Humidity;  # Nest Humidity
my $nest1Temp;      # Nest Temperature
my $nest1Voltage;   # Nest Battery Voltage
my $nest1HeatState; # Nest call for heat
my $nest1AcState;   # Nest call for A/C
my $nest2Humidity;  # Nest Humidity
my $nest2Temp;      # Nest Temperature
my $nest2Voltage;   # Nest Battery Voltage
my $nest2HeatState; # Nest call for heat
my $uptime = `uptime`;
chomp $uptime;
my @uptimeSplit = split (/ /, $uptime);
chop $uptimeSplit[5];
$uptime = ($uptimeSplit[3] . " " . $uptimeSplit[4] . " " . $uptimeSplit[5]);
## Nest 1 = Upstairs
my @nest1 = `/usr/local/bin/nest.py --user user\@you.com --password password --serial 123456789 show`;
## Nest 2 = Downstairs
my @nest2 = `/usr/local/bin/nest.py --user user\@you.com --password password --serial 123456780 show`;

foreach my $line (@nest1) {
 chomp $line;
 if ($line =~ /current_humidity/) { # line contains the humidity, split it out
 my @nest1HumSplit = split (/\:/, $line);
 $nest1Humidity = ($nest1HumSplit[1] * 100);
 }
 if ($line =~ /current_temperature/) { # line contains the temperature in C
 my @nest1TempSplit = split (/\:/, $line);
 $nest1Temp = (((($nest1TempSplit[1] * 9) / 5) + 32) * 100);
 }
 if ($line =~ /battery_level/) { # line contains the Battery Voltage
 my @nest1VoltageSplit = split (/\:/, $line);
 $nest1Voltage = ($nest1VoltageSplit[1] * 1000);
 }
 if ($line =~ /hvac_heater_state/) { # line contains the Heat State
 my @nest1HeatSplit = split (/\:/, $line);
 $nest1HeatState = $nest1HeatSplit[1];
 if ($nest1HeatState =~ /True/) {
 $nest1HeatState = 1;
 }
 else {
 $nest1HeatState = 0;
 }
 }
 if ($line =~ /hvac_ac_state/) { # line contains the A/C State
 my @nest1AcSplit = split (/\:/, $line);
 $nest1AcState = $nest1AcSplit[1];
 if ($nest1AcState =~ /True/) {
 $nest1AcState = 1;
 }
 else {
 $nest1AcState = 0;
 }
 }
}
foreach my $line (@nest2) {
 chomp $line;
 if ($line =~ /current_humidity/) { # line contains the humidity, split it out
 my @nest2HumSplit = split (/\:/, $line);
 $nest2Humidity = ($nest2HumSplit[1] * 100);
 }
 if ($line =~ /current_temperature/) { # line contains the temperature in C
 my @nest2TempSplit = split (/\:/, $line);
 $nest2Temp = (((($nest2TempSplit[1] * 9) / 5) + 32) * 100);
 }
 if ($line =~ /battery_level/) { # line contains the Battery Voltage
 my @nest2VoltageSplit = split (/\:/, $line);
 $nest2Voltage = ($nest2VoltageSplit[1] * 1000);
 }
 if ($line =~ /hvac_heater_state/) { # line contains the Heat State
 my @nest2HeatSplit = split (/\:/, $line);
 $nest2HeatState = $nest2HeatSplit[1];
 if ($nest2HeatState =~ /True/) {
 $nest2HeatState = 1;
 }
 else {
 $nest2HeatState = 0;
 }
 }
}

open (STATE, '>', '/home/pi/scripts/nest.log') || die $!;
 if ($nest1AcState or $nest1HeatState == 1) {
 print (STATE "1\n");
 }
 else {
 print (STATE "0\n");
 }
 if ($nest2HeatState == 1) {
 print (STATE "1\n");
 }
 else {
 print (STATE "0\n");
 }
 print (STATE "$uptime\n");
 print (STATE "Nest States\n");

open (VOLTAGE, '>', '/home/pi/scripts/nest_bat.log') || die $!;
 print (VOLTAGE "$nest1Voltage\n");
 print (VOLTAGE "$nest2Voltage\n");
 print (VOLTAGE "$uptime\n");
 print (VOLTAGE "Nest Battery Voltage\n");

open (NEST1, '>', '/home/pi/scripts/nest_1.log') || die $!;
 print (NEST1 "$nest1Temp\n");
 print (NEST1 "$nest1Humidity\n");
 print (NEST1 "$uptime\n");
 print (NEST1 "Nest 1 Temp and Humidity\n");

open (NEST2, '>', '/home/pi/scripts/nest_2.log') || die $!;
 print (NEST2 "$nest2Temp\n");
 print (NEST2 "$nest2Humidity\n");
 print (NEST2 "$uptime\n");
 print (NEST2 "Nest 2 Temp and Humidity\n");

Heating and Cooling states are converted to either a 1 or 0 for graphing. Temperature is in degrees C, but being an American, I have no frame of reference to what C feels like, so I convert it to Fahrenheit. I also make sure all of the values are mulitplied by 100 so I can graph with 2 decimal point accuracy.

MRTG Config File

######################################################################
# System: Nest Learning Thermostats
# Description: Nest Thermostats
# Contact: Kyle Platts
######################################################################

### Nest Status >> Descr: 'NestUpstairsTemp/Humidity'

Options[nest_1]: gauge,growright,nolegend,noinfo,absolute
LegendI[nest_1]: Temperature
LegendO[nest_1]: Humidity
ShortLegend[nest_1]: F
YTicsFactor[nest_1]: 0.01
Factor[nest_1]: 0.01
YLegend[nest_1]: Temp/Humidity
Target[nest_1]: `cat /home/pi/scripts/nest_1.log`
MaxBytes[nest_1]: 10000
YTics[nest_1]: 5
Title[nest_1]: Nest Upstairs Readings
PageTop[nest_1]: <h1>Nest Upstairs Readings</h1>
 <div id="sysdetails">
 <table>
 <tr>
 <td>System:</td>
 <td>Nest Thermostat</td>
 </tr>
 <tr>
 <td>Maintainer:</td>
 <td>Kyle Platts</td>
 </tr>
 <tr>
 <td>Description:</td>
 <td>Upstairs Temperature & Humidity</td>
 </tr>
 </table>
 </div>

### Nest Status >> Descr: 'NestUpstairsTemp/Humidity'

Options[nest_2]: gauge,growright,nolegend,noinfo,absolute
LegendI[nest_2]: Temperature
LegendO[nest_2]: Humidity
ShortLegend[nest_2]: F
YTicsFactor[nest_2]: 0.01
Factor[nest_2]: 0.01
YLegend[nest_2]: Temp/Humidity
Target[nest_2]: `cat /home/pi/scripts/nest_2.log`
MaxBytes[nest_2]: 10000
YTics[nest_2]: 5
Title[nest_2]: Nest Downstairs Readings
PageTop[nest_2]: <h1>Nest Downstairs Readings</h1>
 <div id="sysdetails">
 <table>
 <tr>
 <td>System:</td>
 <td>Nest Thermostat</td>
 </tr>
 <tr>
 <td>Maintainer:</td>
 <td>Kyle Platts</td>
 </tr>
 <tr>
 <td>Description:</td>
 <td>Downstairs Temperature & Humidity</td>
 </tr>
 </table>
 </div>

### Nest Status >> Descr: 'Nest Battery Voltages'

Options[nest_bat]: gauge,growright,nolegend,noinfo,absolute
LegendI[nest_bat]: Upstairs
LegendO[nest_bat]: Downstairs
ShortLegend[nest_bat]: V
YTicsFactor[nest_bat]: 0.001
Factor[nest_bat]: 0.001
YLegend[nest_bat]: Volts
MaxBytes[nest_bat]: 4000
Target[nest_bat]: `cat /home/pi/scripts/nest_bat.log`
YTics[nest_bat]: 5
Title[nest_bat]: Nest Battery Levels
PageTop[nest_bat]: <h1>Nest Battery Levels</h1>
 <div id="sysdetails">
 <table>
 <tr>
 <td>System:</td>
 <td>Nest Thermostat</td>
 </tr>
 <tr>
 <td>Maintainer:</td>
 <td>Kyle Platts</td>
 </tr>
 <tr>
 <td>Description:</td>
 <td>Nest Battery Levels</td>
 </tr>
 </table>
 </div>

### Nest Status >> Descr: 'Nest On/Off State'

Options[nest]: gauge,growright,nopercent,noinfo,absolute
LegendI[nest]: Upstairs
LegendO[nest]: Downstairs
ShortLegend[nest]: &nbsp;
YLegend[nest]: HVAC On/Off
Target[nest]: `cat /home/pi/scripts/nest.log`
MaxBytes[nest]: 1
YTics[nest]: 2
Title[nest]: Nest HVAC Status
PageTop[nest]: <h1>Nest HVAC Status</h1>
 <div id="sysdetails">
 <table>
 <tr>
 <td>System:</td>
 <td>Nest Thermostats </td>
 </tr>
 <tr>
 <td>Maintainer:</td>
 <td>Kyle Platts</td>
 </tr>
 <tr>
 <td>Description:</td>
 <td>Nest HVAC Status</td>
 </tr>
 </table>
 </div>

In the MRTG config file, I use Factor and Y-ticks Factor to gain 2 decimal point granularity in my graphs. Since these are static values read each time, there is no need do math and figure out the “rate” of a digital output of the thermostat. You will see the use of gauge, nopercent and absolute on the states.
I do maintain percentages on humidity though, since that is what is calculated by the thermostat.

MRTG Log Files

These are the files that actually contain the data parsed from the web and and are read by MRTG for graphing.  Each file needs 4 lines. The first line is Value 1 (In) for MRTG. Line 2 is Value 2 (Out). Line 3 contains the Uptime if you choose to display it. Line 4 contains the graph title.

Nest Temperature and Humidity File

7466
3700
24 days, 3:54
Nest 1 Temp and Humidity

This indicates that right now, my upstairs Nest is 74.66 degrees F with 37% humidity.

Graphs

Nest Thermostat

Nest Thermostat

My wife got me a Nest thermostat (www.nest.com) for Christmas. I had it installed later that day.

For those of you who don’t know, the Nest is a learning thermostat with wireless capabilities. It has the capability to learn a schedule for you, lower the temp when you are away, and allow access remotely.

First Impression

It is slick! Setup was a breeze. Once on the wireless network, I created my Nest account and I was good to go. I loaded up the client on my iPad and I can now control my Nest from anywhere in the world.

Reports

The Nest has a daily energy use report so you can see what is happening in your house. It keeps track of the local weather too, so it knows if it was a warm or cold day.

20130128-154621.jpg

At a Glance

This is my favorite screen. It shows you all the data in one screen: inside temperature, outdoor temperature and humidity.

20130128-154832.jpg

Nest Installed

Here are a pair of pictures of the Nest installed. You can see my Dallas 1-Wire sensor hanging out underneath it.

20130128-155344.jpg

20130128-155353.jpg

Second Impression

I liked the Nest so much, I had to buy a second one for downstairs to run the hydronic floor. With my Arduino monitor, I could see that the difference in air temp would cause the floor to turn on. With 2 Nest’s, I am able to run the same schedule on both, so when the air temp drops overnight, the floor doesn’t run unnecessarily.

Monitoring the Nest

With the use of Scott Baker’s Nest.py program (Python API for Nest) and a little bit of Perl code, I can monitor the Nest’s in MRTG. The Nest does not have a published API, and it does not repsond to SNMP queries, so you have to query the web to get data, rather than query the device sitting on your local network. I hope Nest will open this up to developers at some point.

Here is the data flow.

Nest Thermostat — internet — Nest Web Portal
Raspberry Pi MRTG Server — python code — perl parse code — Nest Web Portal