Household Energy monitoring & publishing

General tips, questions and answers about going green in your home and business. Achieve a more environmentally friendly lifestyle!

Household Energy monitoring & publishing

Postby PeterReefman » Thu Apr 26, 2012 1:07 am

Hi all

A post to let you know of some progress I'm making in getting my house's energy consumption and solar PV production data collected and published onto the web.

After a lot of false starts, I think things are finally starting to come together. I'm using a device (Current Cost) that can have multiple clamp-based monitors sending data streams to a main console.

I currently have two streams, which are - overall household electricity consumption; and PV production (from two separate PV systems)

The console uses a new hardware device called a "Brain" which is basically a tiny server running it's own operating system. These devices are mainly used for file backups (like photos, etc) that can be accessed remotely from a web browser. My head is not around it totally yet, but it's working well enough.

Data is sent from the Current Cost console to the Brain device is in two forms. A Live (6 second interval) stream, which is okay but something I haven't been able to publish usefully yet; and a daily CSV file of all the 6 second data. This CSV file is my main interest right now.

But there is a problem with the CSV file - it gets VERY big very quickly. Even only two streams becomes 1MB per day. The system can have 10 streams going which is what I want, so the size will quickly eat up my web hosting space - and I just don't need that much detail.

I've set up a couple of web scripts that get all the data out and compress it into 30 minute blocks of consumption, this brings the size down to much less (20kb per day) - and still has the resolution of data I want to work with, which matches Smart Meter pulses.

I've now got a web page that shows the data. It's still very much in development and quite clunky - but you can hopefully see what I'm trying to do with the data, as well as see how our electricity use is going.

Things that I'm still in the process of doing:
* Install another 8 monitors, so that consumption can be further broken down to things like lighting; heating/cooling; hot water boosting; cooking; refrigeration; Audio/visual gear; dishwashing; and clothes washing.
* Streamline and further automate the publishing of the CSV file. Right now I need to manually FTP it to my website server before manually running the scripts. My plan is to have a scheduled task to this this automatically every 30 minutes.
* Vastly improve the web page that displays the data, and make it available to be dropped into other web pages.
* Share the code. Like many here, I'm pretty passionate about doing meaningful actions, and am in the middle of conducting a bulk purchase of the Current Cost/Brain devices on behalf of my local sustainability group, with a plan of allowing others to use the system as well.

Here's the display page. http://www.energised.com.au/brain.asp

Any feedback would be more than welcome.

Cheers,
Peter
PeterReefman
Solar Fanatic
Solar Fanatic
 
Posts: 32
Joined: Mon Jan 25, 2010 11:57 am
Location: Portland, Victoria

Re: Household Energy monitoring & publishing

Postby Gordon-Loomberah » Thu Apr 26, 2012 7:55 am

Nice work Peter :)

PeterReefman wrote:But there is a problem with the CSV file - it gets VERY big very quickly.


Tell me about it ;) I'm logging 4 steams at 1 sec intervals ATM, and will be adding more in the future when I separate out wind power out and add hydro and maybe some water pressure and system component temperature monitoring. Since the spreadsheet software is limited to 65K lines and there are 86400 seconds in a day, I also save a 5 sec interval log, which I import into Open Office manually (but might automate it one day), and produce detailed daily graphs, quite a few of which I've posted in the forums. I don't need to put it online, as I can access the monitoring laptop, which also has my weather station connected, remotely when away.

Looking at your graph for today I see there is a constant 100W load at night (.05kWh/30 mins), do you not have any refrigerator running? Also, after 15 30 min periods:
Thu 26 Apr 2012
Electricity Produced = 0 kWh
Electricity Consumed = 0 kWh

surely electricity consumed will be 0.75kWh?
http://gunagulla.com Loomberah weather and astronomy including live solar radiation intensity and UV + Gunagulla aquaponics, organic eggs and cherries
User avatar
Gordon-Loomberah
Community Moderator
 
Posts: 5748
Joined: Thu Sep 24, 2009 9:41 pm
Location: Loomberah NSW Australia

Re: Household Energy monitoring & publishing

Postby PeterReefman » Thu Apr 26, 2012 8:43 am

Thanks Gordon - wow your archive file must be in the GBs by now.

Re the night-time fridge, yes I agree I was expecting to see more of a spike. The fridge must be doing a cycle in each 30min block.

And yep there's certainly still some issues with data collected compared to totals. The device actually has it's time-stamp screwed up 10 hours which I need to adjust for, and I wouldn't be surprised if I's still missing something there.

Thanks for the feedback - and hey it'd be GREAT to see your data live online one day. Sounds great.

Cheers
PeterReefman
Solar Fanatic
Solar Fanatic
 
Posts: 32
Joined: Mon Jan 25, 2010 11:57 am
Location: Portland, Victoria

Re: Household Energy monitoring & publishing

Postby Gordon-Loomberah » Thu Apr 26, 2012 1:02 pm

I realised the fridge could well be hidden in the half hour blocks while out on a 3 hour mountain bike ride this morning :)

Here's what my gross plot for the day so far looks like, I also do graphs showing battery voltage and net kW, and another with volts and net amps for the day period, to show battery charging with more detail.
20120426gross.gif
Click to de-fuzz
http://gunagulla.com Loomberah weather and astronomy including live solar radiation intensity and UV + Gunagulla aquaponics, organic eggs and cherries
User avatar
Gordon-Loomberah
Community Moderator
 
Posts: 5748
Joined: Thu Sep 24, 2009 9:41 pm
Location: Loomberah NSW Australia

Re: Household Energy monitoring & publishing

Postby PeterReefman » Thu Apr 26, 2012 7:43 pm

Excellent chart Gordon, and wow great figures on the consumption! How much solar/wind capacity do you have?
PeterReefman
Solar Fanatic
Solar Fanatic
 
Posts: 32
Joined: Mon Jan 25, 2010 11:57 am
Location: Portland, Victoria

Re: Household Energy monitoring & publishing

Postby Gordon-Loomberah » Thu Apr 26, 2012 9:34 pm

Re consumption, just wait until I get the welder and cut-off saw out ;) In cold weather (3C this morning) we let the house cool down overnight, so the fridge doesnt use much, and the freezer barely comes on at all, I turn them down overnight and up (colder) during the day, to match PV power generation and ease the load on the batteries. The other fridge that we keep eggs in is uaually turned off overnight and kept cool with ice.
Power consumption is quite a bit higher when I am preserving tomatoes, cherries, apples etc which I grow here, or when I have a major construction project underway with lots of cutting and welding steel.

Array 1 is 2.19kW and array 2 is 945W, both a mix of various power and age panels. Array 2 has a 200W WTG on it ATM, but generally produces minimal power, as it isn't a windy place.
http://gunagulla.com Loomberah weather and astronomy including live solar radiation intensity and UV + Gunagulla aquaponics, organic eggs and cherries
User avatar
Gordon-Loomberah
Community Moderator
 
Posts: 5748
Joined: Thu Sep 24, 2009 9:41 pm
Location: Loomberah NSW Australia

Re: Household Energy monitoring & publishing

Postby PeterReefman » Wed May 23, 2012 7:57 pm

Hi Gordon (and all)

Okay there's been a little progress on my system - it can now show reasonably well where our power consumption goes.

The processing is a little heavy which makes the page take too long to load. I'm working on that (and a lot of other things...)

http://www.energised.com.au/brain.asp

By the way - yes our tariffs are high. 28c from Origin with an extra 5c for 100% wind which is credited to the Codrington Wind Farm about 30km from here.

And the "Benefits over an average house" takes into account that an average house spends about $4.30 on energy per day.

Cheers,
Peter
PeterReefman
Solar Fanatic
Solar Fanatic
 
Posts: 32
Joined: Mon Jan 25, 2010 11:57 am
Location: Portland, Victoria

Re: Household Energy monitoring & publishing

Postby Gordon-Loomberah » Wed May 23, 2012 8:21 pm

Peter, that link is not working for me:

>
Server not found
Firefox can't find the server at http://www.energised.com.au.

EDIT: after several more tries it eventually loaded :)
http://gunagulla.com Loomberah weather and astronomy including live solar radiation intensity and UV + Gunagulla aquaponics, organic eggs and cherries
User avatar
Gordon-Loomberah
Community Moderator
 
Posts: 5748
Joined: Thu Sep 24, 2009 9:41 pm
Location: Loomberah NSW Australia

Re: Household Energy monitoring & publishing

Postby PeterReefman » Wed May 23, 2012 9:01 pm

Ah cool - the server is local, and we're about to be hit by a storm. Perhaps it's going offline at times (I'd hope they have a UPS!)
PeterReefman
Solar Fanatic
Solar Fanatic
 
Posts: 32
Joined: Mon Jan 25, 2010 11:57 am
Location: Portland, Victoria


Return to Living Green

Who is online

Users browsing this forum: No registered users and 1 guest

new solar power specials