Setting Featured image on WordPress Posts in BULK

I’m in the middle of changing another site to a new theme. The problem being the front page uses the feature image from each post to build up the display. None of the posts have a feature image set. It uses a facebooktowordpress plugin (heavily customised the original no longer pull the images, and really couldn’t handle pulling what was needed).

With this plugin each post on facebook is posted to the blog, the first image (this need changing to all images) on the post is downloaded to the wordpress server and then the source of the post uses the local image. But at no point did I ever anticipate needing featured images to be set.

So here’s the code I’ve put together to:

  • Pull a list of posts without a thumbnail/featured image set.
  • Pull the contents of each post in the list and look for img src
  • Download the image
  • Add the image to the media library against the current post
  • Pull the ID of the attachment and set it as the featured image

This is meant again to run from the command line NOT as a plugin.

 $limit = 20;

 if( php_sapi_name() !== 'cli' ) {
 die("Meant to be run from command line");

 function find_wordpress_base_path() {
 $dir = dirname(__FILE__);
 do {
 //it is possible to check for other files here
 if( file_exists($dir."/wp-config.php") ) {
 return $dir;
 } while( $dir = realpath("$dir/..") );
 return null;

 define( 'BASE_PATH', find_wordpress_base_path()."/" );
 define('WP_USE_THEMES', false);
 global $wp, $wp_query, $wp_the_query, $wp_rewrite, $wp_did_header;
 require(BASE_PATH . 'wp-load.php');
 echo "Site URL: " . get_site_url() . "\n\r";
 echo "Base: " . find_wordpress_base_path()."/\n\r";
 echo "Upload DIR: " . wp_upload_dir()['path'] . "\n\r";
 echo "Posts: " . wp_count_posts()->publish."\n\r";

 $query = array (
 'posts_per_page' => -1,
 'post_type' => 'post',
 'meta_key' => '_thumbnail_id',
 'meta_compare' => 'EXISTS',
 $my_query = new WP_Query($query);
 $posts_with_thumbs = $my_query->post_count;
 echo "Posts with thumbnail_id: " . $posts_with_thumbs . "\n\r";

 $query = array (
 'posts_per_page' => -1,
 'post_type' => 'post',
 'meta_key' => '_thumbnail_id',
 'meta_compare' => 'NOT EXISTS',
 $my_query = new WP_Query($query);
 $posts_without_thumbs = $my_query->post_count;
 echo "Posts without thumbnail_id: " . $posts_without_thumbs . "\n\r";

 $counter = 0;
 while(( $my_query->have_posts() ) and ($counter < $limit)) {
 echo "\n\r\n\r";
 echo "Post ID: " . $my_query->post->ID . "\n\r";
 echo "Post Title: " . $my_query->post->post_title . "\n\r";
 $content = $my_query->post->post_content;

 if ( strpos($content, 'img src') !== false ) {
 $re = "/<img.*?src='([^\"]*)'.*\/>/i";
 preg_match_all($re, $content, $matches);
 $sub = $matches[1][0];
 echo "Image URL: " . $sub . "\n\r";
 if ( substr( $sub,0,11 ) === "/wp-content" ) {
 $sub = get_site_url() . $sub;
 echo "Real URL: " . $sub . "\n\r";

 if ( $sub == "" ) {
 if (strpos($content, 'youtube') !== false) {
 $re = "/\[[{embed\}]]([^\"]*)\[\/embed\]/i";
 preg_match_all($re, $content, $matches);
 $sub = $matches[1][0];
 echo "Youtube Detected. URL: " . $sub . "\n\r";
 $re = "/watch\?v=([^\"]*)\[\/embed\]/i";
 preg_match_all($re, $content, $matches);
 $video_id = $matches[1][0];
 $sub = "". $video_id . "/hqdefault.jpg";
 echo "Youtube Thumbnail: " . $sub . "\n\r";
 $sub_youtube = true;

 echo "Content: " . $content . "\n\r";

 if ( ($sub_image === true) || ($sub_youtube === true) ) {
echo "Image or Youtube detected!\n\r";
 if ($sub_image === true) {
echo "Image!!\n\r";
 $media = media_sideload_image($sub, $my_query->post->ID, $my_query->post->post_title);
 } elseif ($sub_youtube === true) {
echo "Youtube!!\n\r";
 // Download file to temp location
 $tmp = download_url( $sub );
 // Set variables for storage
 // fix file filename for query strings
 preg_match('/[^\?]+\.(jpg|JPG|jpe|JPE|jpeg|JPEG|gif|GIF|png|PNG)/', $sub, $matches);
// $file_array['name'] = basename($matches[0]);
 $file_array['name'] = $video_id . ".jpg";
 $file_array['tmp_name'] = $tmp;
 // If error storing temporarily, unlink
 if ( is_wp_error( $tmp ) ) {
 $file_array['tmp_name'] = '';
 // do the validation and storage stuff
 $media = media_handle_sideload( $file_array, $my_query->post->ID, "YouTube: " . $my_query->post->post_title );
 // If error storing permanently, unlink
 if ( is_wp_error($media) ) [email protected]($file_array['tmp_name']);}

 if(!empty($media) && !is_wp_error($media)){
 echo "File Downloaded!\n\r";
 $args = array(
 'post_type' => 'attachment',
 'posts_per_page' => 1,
 'post_status' => 'any',
 'post_parent' => $my_query->post->ID,

 $attachments = new WP_Query($args);
 while( $attachments->have_posts() ) {
 echo "Attachment ID: " . $attachments->post->ID . "\n\r";
 set_post_thumbnail($my_query->post->ID, $attachments->post->ID);
 if ($sub_image) {
 $atturl = wp_get_attachment_url($attachments->post->ID);
 $atturl = preg_replace("(^https?:)", "", $atturl );
 echo "Attachment URL: " . $atturl . "\n\r";
 if ($sub_changed) {
 $sub = str_replace(get_site_url(), "", $sub);
 $newcontent = str_replace($sub,$atturl,$content);

 if ($newcontent != $content){
 $update_post = array(
 'ID' => $my_query->post->ID,
 'post_content' => $newcontent,
 echo "\n\r";
 echo "With Thumbs: " . $posts_with_thumbs . " and " . $posts_without_thumbs . " Without.\n\r";
 echo "Updated: " . $updated . " of " . $counter . "\n\r";

BE WARNED this is designed to make changes to your wordpress posts. The usual about taking backups is a MUST. both of the database and your wordpress www folder.

  • The “Limit” is set low, you’ll need to adjust this as needed.
  • Within the script there is

You will need to change this to

[ embed\ ]

WITHOUT the spaces, wordpress decided I was trying to embed something so I can’t paste it without a slight change.

  • I’ve tried to keep it as generic as possible, so there shouldn’t be any mention of my site in forced links or searches. When it searches for an image it also looks to see if it’s local to the site using /wp-content instead of a full url, it should get around this problem using get_site_url() but you could force your domain or a different path if needed here.
  • It also looks for any youtube content in the post and tries to pull the relevant image. I just wish it would pull one with a play button on it (I dont want to use CSS to get around this)
  • It handles the youtube download differently (this was made over a few days fixing problems as they come up) but I think it’s needed to give each image a unique filename.
  • You will have to run several times probably increasing the limit each time, on 2nd run it again checks all the previous ones that it couldn’t get an image for, so you should end up with 0 updated of x where x is probably all your posts without an image. I’ll be updating it to use a/(random of a few) image relevant to the site so everything has a featured image.

WordPress Posts without Featured Thumbnail Count

I have another script I’ve been building up to download the first  embedded image in a post, add it to the media library, attach it to the post and set it as featured image.

Here’s some quick code to query how many posts do and do not have a featured image

 if( php_sapi_name() !== 'cli' ) {
 die("Meant to be run from command line");

 function find_wordpress_base_path() {
 $dir = dirname(__FILE__);
 do {
 //it is possible to check for other files here
 if( file_exists($dir."/wp-config.php") ) {
 return $dir;
 } while( $dir = realpath("$dir/..") );
 return null;

 define( 'BASE_PATH', find_wordpress_base_path()."/" );
 define('WP_USE_THEMES', false);
 global $wp, $wp_query, $wp_the_query, $wp_rewrite, $wp_did_header;
 require(BASE_PATH . 'wp-load.php');
 echo "Base: " . find_wordpress_base_path()."/\n\r";
 echo "Upload DIR: " . wp_upload_dir()['path'] . "\n\r";
 echo "Posts: " . wp_count_posts()->publish."\n\r";

 $query = array (
 'posts_per_page' => -1,
 'post_type' => 'post',
 'meta_key' => '_thumbnail_id',
 'meta_compare' => 'EXISTS',
 $my_query = new WP_Query($query);
 $posts_with_thumb = $my_query->post_count;
 echo "Posts with thumbnail_id: " . $posts_with_thumb . "\n\r";

 $query = array (
 'posts_per_page' => -1,
 'post_type' => 'post',
 'meta_key' => '_thumbnail_id',
 'meta_compare' => 'NOT EXISTS',
 $my_query = new WP_Query($query);
 $posts_without_thumb = $my_query->post_count;
 echo "Posts without thumbnail_id: " . $posts_without_thumb . "\n\r";

It’s made to run from the terminal NOT as a wordpress plugin, just place it in the root wordpress folder and run with php <filename>

It will output something like

Base: /var/www/{folder}/
Upload DIR: /var/www/{folder}/wp-content/uploads
Posts: 8547
Posts with thumbnail_id: 6824
Posts without thumbnail_id: 1723

Took a few seconds to return, it’s probably not the best way of doing it, but worked for what I wanted. I’d like to give credit for each bit of code but really no idea what I got the different bits from.

GlusterFS woes

If your looking for the gluster error ‘brick2.mount_dir not present’ 
jump to the end

Time for another post 🙂

I’ve been using DigitalOcean for some time now, and I’m still tweaking my setup. Once thing I really hope they sort soon is proper private lan between your own droplets, for now we just have to use a vpn between them.

Being responsible for a new website can give a lot of headaches, especially when you have to try to guess just how popular it will be. So about a year ago I setup a new droplet to host the new site, testing it was going well and I increased the droplet before we launched to handle a spike. Sadly I underestimated just how busy it would get, based on the numbers I was given I think I was about right but unfortunately those numbers were way off.

But each failure it just another learning curve 🙂 so as quickly as possible that was fixed, then the site got to normal volume so we scaled it back down (yes it’s a whole cost exercise especially when your paying for it). Then we had the lead up to Christmas, in an attempt to not repeat the problems at launch, I changed the whole configuration so that I could (if needed) take a server down while staying partly operational. This kind of worked and was needed when some brightspark promoted the site a day early and we hadn’t scaled back up!

Come the new year I decided it was time to seriously sort the infrastructure for the site. It now has an online store and it’s important it keep running, it’s not just a blog anymore. So I put in place the following setup (working around various obstacles).


  • All websites name servers are pointing to cloudflare and they handle the first web connection. It works really well on their free tier, and changes (adding new servers) are pretty quick to take effect.

DigitalOcean Droplets:

  • 1x Server running as a load balancer.
  • 2x Servers running as webservers.
  • 1x Server running as database server.
  • 1x Server running as email (not quite running).
  • At the same time as making this setup I decided to ditch apache and move to nginx, so loadbalancer and webservers are running that.


  • 4x Nginx (loadbalancer and webservers, and installed on database server for stats).
  • 2x Syncthing (webservers) to keep the www folders in sync.
  • 1x MySQL Server.
  • 4x OpenVPN (connections between loadbalancer and webservers, webservers and database).
  • 1x Redis Server (for session data, I tried nginx load balancing options but it still screwed up if I had to take one of the web servers out for maintenance, so installed Redis on the Database server.

As this progressed I dropped the VPN between loadbalancer and webservers and just use HTTPS/SSL instead. Syncthing already has it’s own SSL built-in so I could leave that over the semi-private LAN. but I really would like to change MySQL to be encrypted and drop the vpn from that too, but find info on doing this for wordpress seems to be non-existent at the moment.

Roll forward a few months, this has been working but still has areas to resolve. Such as syncthing: yes it keeps the folders in sync and is actually really good that I can also store them on another system easily. but it doesn’t listen to the OS for changes to files. Instead it polls every x seconds. Although there’s nothing much changing, updating plugins became a problem if you click the update button it downloads but then nginx sends your next request to another server and now the isn’t there so wordpress throws an error.

My whole reason for running syncthing was I wanted the files to be available on each server independently. So if Server A goes down it doesn’t matter Server B has all the files locally anyway. NFS would still give me a single point of failure. On looking into resolving this though I remembers GlusterFS. I’d played with it a long time ago, but dropped it as a solution (can’t remember what I was doing or why it wasn’t working). Now it’s time to try it again. downside I’m back to needing VPN’s and OpenVPN isn’t the easiest to quickly add a new server.

So I’ve done the following:

  • Added a new server just for the files (I don’t like gluster being in a 2 replica incase there’s a problem, there should be a majority who thinks they are holding the correct file).
  • Swapped out OpenVPN for Tinc, I have to say one of the best decisions. yes there are downsides, it creates a mesh (only doable with OpenVPN by running quagga for manually forcing routes) but I have no idea which Server is actually connected to which Servers. There’s no VPN status and I can’t see how much traffic has gone between 2 particular servers (iptables helps but it’s not 100%)
  • Added another new server for nagios and central logging.

There were a load of changes within a few weeks of each other, but I now have a setup I’m confident I can scale more quickly than ever before. Yes it has single points (load balancer, mysql) but I know if the load balancer has a problem it’s pretty static so can be wiped and redeployed quickly, as well as it will take a few minutes to open the webservers to the world and let cloudflare hit them directly. So MySQL is the real problem and I’ll be addressing that one soon enough.


So now onto today’s problem 🙂

I’ve had gluster running a few weeks, and I have our testing website (for theme changes etc) setup on our webservers behind the loadbalancer. The last few days I’ve need to do more extensive testing than just changing bits in a theme, so I’ve decided to split the tester site onto it’s own droplet (still behind the loadbalancer and with VPN to the databases). I thought I may as well make use of Gluster here too (yes it would be in a 2 replica setup itself and the fileserver. I don’t like that idea). So I brought up a new server and configured it: new users, firewall rules, tinc, nginx, php, etc.

I added gluster and copied the /etc/hosts entries over from the other servers. All looked good. I gluster peer probe ServerX and it worked, gluster peer status and I could see it fine. but on trying to add a new volume:

gluster volume create xxx-yyy-zzz replica 2 transport tcp FILESERVER:/GLUSTER/xxx.yyy-zzz TESTSERVER:/GLUSTER/xxx.yyy-zzz force

I was getting the error:

volume create: xxx-yyy-zzz: failed: Commit failed on localhost. Please check the log file for more details.

Checking the logs on both servers would show (maybe slight variation):

[2015-07-28 16:00:41.612907] E [glusterd-hooks.c:328:glusterd_hooks_run_hooks] 0-management: Failed to open dir /var/lib/glusterd/hooks/1/create/pre, due to No such file or directory
[2015-07-28 16:00:41.614499] E [glusterd-volume-ops.c:1811:glusterd_op_create_volume] 0-management: brick2.mount_dir not present
[2015-07-28 16:00:41.614587] E [glusterd-syncop.c:1288:gd_commit_op_phase] 0-management: Commit of operation 'Volume Create' failed on localhost

I tried a series of things to fix it:

I thought maybe the /GLUSTER/xxx-yyy-zzz needed to be created (I already made /GLUSTER) – Nope

  • I detached the peer and reattached – No.
  • I reboot the file server and test server – No.
  • I detached, reboot, reattached – No.
  • I tried creating the volume with just the test server and no replica – No.
  • I tried creating the volume on just the fileserver with no replica – Yes.

So the problem is point to the new system, but it’s a brand new system. They’re peers and connected.

  • I tried uninstalling and reinstalling gluster – No.
  • I tried uninstalling, purging and reinstalling – No.
  • I tried uninstalling, purging, manually deleting the /var/lib/gluster (probably a mistake that I didn’t detach first :() and reinstalling – No.
  • I have no idea why this wont WORK!!!!!

Let’s go further back, check the VPN, ping the servers.

  • Ping fileserver from testserver – Yes.
  • Ping testserver from fileserver – Yes/Hang on that’s the wrong IP!! Yes I’d copied an entry from webserverB into /etc/hosts, update the name but missed the IP address. Idiot! correct that. Ping – OK.
  • Try gluster again – Yes.

So if you’re having problems and seeing brickX.mount_dir not present make sure your DNS between servers is correct.

I don’t really know how the peer probe worked, but I think I must have done that from a servers who’s hosts was correct

Hyperion with Sunrise and Sunset

You may have gotten here from my hyperion with nagios writeup, this doesn’t follow on from that and is separate, but maybe of interest.

The basic idea: I’ve now got LED’s in my room and would like them to come on before I go to bed so I can see without falling over. The ones on the stair I just leave running, but like hell am I going to sleep with such a bright LED (I probably could, I can sleep in the day but I thought it would be a better idea for them to come on ready).

I could have just set this up on a basic cron and picked a time early enough to account for summer/winter before I go to bed. but where’s the fun in that. I know my PI can work out when the sunrise/sunset is. so it can’t be that difficult to set something up.

After a little bit of searching I come across sunwait you will need this or a similar program. I wont cover installing sunwait on the PI here. just the config I use with hyperion.

First I need a script that sunwait will call and tell it what it needs to do. Here’s my

case "$1" in
        /usr/bin/hyperion-remote -p 50 -e "Knight rider"
        /usr/bin/hyperion-remote -p 50 -e "Little Chaser Blue"
        echo "Usage: $0 {sunrise|sunset}"
        exit 1
exit 0

Don’t forget to make this script executable ‘chmod +x’

This is basically told to either run sunset or sunrise and will then call hyperion-remote passing the relevant priority and effect. (Little Chaser Blue is a copy and customisation of Knight Rider)

Then I added the following to /etc/crontab

0 02   * * *   root    sunwait -p sun up 51.xxxxN 3.xxxxW ; /root/ sunrise
12 02   * * *   root    sunwait -p sun down 51.xxxxN 3.xxxxW ; /root/ sunset

This basically run’s sunwait which will wait until the sun is either coming up or going down at the specified co-ordinates before running the bit after ;

The important bit to get your head around if you must have cron run this at a time well before the sun will rise or set. midnight and midday seems like a good safe bet.

I know I’ve skipped over the actual installation of sunwait and more details on hyperion and running the scripts to check it works, but it’s 1am and I just want to save this 🙂 So if you’ve got this far and are still confused, comment below and I’ll expand on the relevant bits.

Hyperion LED’s & Nagios (Part 3)

Hopefully you’d read Part 2. If not you’ll need to or this wont work.

So in this part we’re going to setup the nagios stuff to set off the alert LED’s. As a little bit of background my nagios installation is on a completely separate PI to the hyperion LED’s, but I have install hyperion on this pi to make use of hyperion-remote. Yes I was being lazy I could have used other methods instead of installing the whole thing.

First my nagios installation is in ‘/usr/local/nagios’, I’m not going to go through the commands to cd and edit, if you’ve installed and configured nagios I’ll assume you can do them 🙂

This is my /usr/local/nagios/libexec/

case $STATE in
   EFFECT="Red Alert"
   EFFECT="Yellow Alert"
   EFFECT="Green Alert"
hyperion-remote -a osmc-l:19444 -d $DURATION -p 10 -e "$EFFECT" &
hyperion-remote -a rasp-light:19444 -d $DURATION -p 10 -e "$EFFECT" &
hyperion-remote -a webcam-pi:19444 -d $DURATION -p 10 -e "$EFFECT" &

For the nagios saavy amongst you, you’ll see I account for CRITICAL, WARNING & OK. Yes I do need to add DOWN, UNREACHABLE & UP for the host alerts

The DURATION sets how long in ms hyperion will run this effect for (best worked out in conjunction with the speed, freq & step from the hyperion config. I’ve got this just right to cut off the alert after (I think) 4 fades. I force the priority ‘-p 10’ to 10, anything else I do with hyperion generally has a priority 50, 100 or 1000 so these will take over.

The last 3 lines are 1 each for my hyperion installs, you will need to change the name’s or replace them with the IP addresses dependant upon your network configuration.

Don’t forget to make the script executable, and you can test it with ‘./ OK’

With the above tested and working, I’ve added the following to my nagios command.cfg

# 'notify-host-by-hyperion' command definition
define command{
        command_name    notify-host-by-hyperion
        command_line    /usr/local/nagios/libexec/ "$HOSTSTATE$"
# 'notify-service-by-hyperion' command definition
define command{
        command_name    notify-service-by-hyperion
        command_line    /usr/local/nagios/libexec/ "$SERVICESTATE$"

Then added the following to contacts.cfg

define contact{
        contact_name                    nagios-hyperion
        alias                           Nagios Hyperion
        service_notification_period     24x7
        host_notification_period        24x7
        service_notification_options    w,u,c,r,f
        host_notification_options       d,u,r,f,s
        service_notification_commands   notify-service-by-hyperion
        host_notification_commands      notify-host-by-hyperion
define contactgroup{
        contactgroup_name       nagioshyperion
        alias                   Nagios Hyperion Notifications
        members                 nagios-hyperion

For my installation I’ve then added

contact_groups                  nagioshyperion

To my template’s. You could add this to each service/host.

Within my setup I’ve stopped using email alerts, so changing the contacts to hyperion was fine. Within the templates I then have the notify_interval setup for 15 minutes. This means it will fire an alert to hyperion every 15 minutes. If you use email on your system too, you may not want to do this. an alternative could be changing the duration above, so that the red and yellow alerts are constant and the green run’s for a limited time before clearing down.

I did contemplate using event filters instead of contacts, so I could have the emails turned back on at some point, but decided against it as I would have to check before sending a green alert that it’s not already in green or I’d just end up with green all the time.

After all of the above make sure you restart nagios for the new config to take effect.

As a side note, I was sat watching TV this evening all of a sudden my room was yellow and I thought WTF. I do have hyperion setup in my room to start the LED’s at sunset but it was still light out and shouldn’t have fired. Then it clicked NAGIOS. and yes hey presto nagios had throw this site into warning status as there were updates available. I can see it getting annoying if my ISP drops out and I end up with alerts for hours. but on the whole I’m really happy it works, all I need now is a red alert klaxon 🙂

If your interest in setting up hyperion at sunrise/sunset I’ll be writing that one up separately.

Hyperion LED’s & Nagios (Part 2)

Maybe you read Part 1, maybe you skipped it 🙁

Either way Part 2 is going to cover creating a new effect for Nagios. I wont say it’s the best bit of python programming I’ve done (Yes I copied and changed another effect as a starting point). The main thing I wanted was an alert status (yes Star Trek does come into it).

There’s 4 files to this effect:

  1. – The actual python program
  2. alert-green.json – etting a green colour (yes it’s spelt COLOUR not color!!!!!)
  3. alert-yellow.json
  4. alert-red.json

My hyperion is installed to ‘/opt/hyperion’ so first thing is to go to the effects directory

cd /opt/hyperion/effects

Then create the file


Paste the following into it and save the file

import hyperion
import time
import colorsys
import numpy as np
# Get the rotation time
colorfrom =     hyperion.args.get('colorfrom', (0,0,0))
colorto =       hyperion.args.get('colorto', (0,255,0))
speed   =       float(hyperion.args.get('speed', 0.05))
freq    =       float(hyperion.args.get('frequency', 2.5))
step    =       float(hyperion.args.get('step', 30.0))
ledCount =      hyperion.ledCount
# Setup the colors
colorfromnp     =       np.array(colorfrom)
colortonp       =       np.array(colorto)
#define the lerp calc
def lerp(a, b, t):
        x = a*(1 - t) + b*t
        x = np.around(x)
        x = x.tolist()
        return x
# Start the write data loop
while not hyperion.abort():
        for i in range(0,int(step)):
                n = i / step
                thiscolor = lerp(colorfromnp, colortonp, n)
                colorLedsData = ledCount * bytearray((int(thiscolor[0]), int(thiscolor[1]), int(thiscolor[2])))
        for i in range(1,int(step)):
                n = i / step
                thiscolor = lerp(colortonp, colorfromnp, n)
                colorLedsData = ledCount * bytearray((int(thiscolor[0]), int(thiscolor[1]), int(thiscolor[2])))
        colorLedsData = ledCount * bytearray(colorfrom)

I do use a numpy array, I can’t remember installing it but may have done. You can test if you have it installed

import numpy as np

Then we create the alert-green.json

nano alert-green.json

And Paste

        "name" : "Green Alert",
        "script" : "",
        "args" :
                "colorfrom" : [0,0,0],
                "colorto" : [0,255,0],
                "speed" : 0.05,
                "freq" : 2.5,
                "step" : 30.0


        "name" : "Yellow Alert",
        "script" : "",
        "args" :
                "colorfrom" : [0,0,0],
                "colorto" : [255,255,0],
                "speed" : 0.05,
                "freq" : 2.5,
                "step" : 30.0


        "name" : "Red Alert",
        "script" : "",
        "args" :
                "colorfrom" : [0,0,0],
                "colorto" : [255,0,0],
                "speed" : 0.05,
                "freq" : 2.5,
                "step" : 30.0

Finally restart hyperion to pickup the new effects

service hyperion restart

That’s it, you should now be able to run these effects. From the terminal:

hyperion-remote -e "Red Alert"

Should start the led’s glowing Red. If not I’d start with ‘hyperion-remote -l’ to check the effects are listed, then move on to working out what’s missing (see above about numpy array).

A few notes on this effect:

  • Originally I programmed it forced to black, but it made sense to change this to a colorfrom value for future (if you want it green and fading to red for example).
  • The speed is how quickly to progress through the fade.
  • The step is how many colours it goes through to get from the colorfrom to the colorto
  • Working out the speed and the step together are important for a nice fade (not jerky).
  • The Freq in how long it stays on black before running the fade again.

This completes the hyperion side of the setup. I now have this effect installed and running on 3 PI’s and yes when there’s a problem they ALL go to yellow/red alert

See Part 3 for the nagios setup

Hyperion LED’s & Nagios (Part 1)

Part 1 is more Background story on my use of WS2801 WS2812b and Hyperion with the Raspberry Pi. Skip to Part 2 for the techy bit.

I’ve been using Hyperion for a while. I setup light behind the TV first off (using sticky tape) WS2801 and RaspBMC. This work brilliantly and I loved it. Spent hours tweaking the config so the LED’s were picking up the correct colour to the screen.

With all that working and a length of LED’s left over I decided to run some up the stairs. They sit just under the banister lip so you can’t see them, just the light on the stairs. I set these to Rainbow swirl and let it. They’re been running for months, occasionally changing the effect to show off what they can do.

Then disaster struck, the power adapter stopped working. Have to be honest I didn’t really notice until I was going to bed at 2am and almost fell over. They’ve been there giving off light (possibly a bit bright if anything) and I just got used to being able to see in the middle of the night without any other lights.

Anyway I digress, ordering a new power adapter I went searching for more LED’s (yes you can’t have enough of them once you’ve been playing). I decided that I’d really like to run some in my bedroom, the effects are cool and there would be plenty of light I wont need to use the main light with them on.

So I looking at where I originally bought my WS2801’s and nothing 🙁 so off to google, the obvious thing was I was going to have a hard time sourcing them in the UK. but why they’re great. So off I went to the hyperion git site for info and found there’s newer versions WS2811 and WS2812b. ah that may help, another search and I found someone selling a load on ebay. So I bought all he had 3xreels of WS1812b’s.

They turned up and I connected them up to try them as directed by hyperion. It was at this point I read the important bits RPi2 isn’t working yet and there maybe a problem with the PI communicating with them due to the voltage. I really thought I was going to have to make another little circuit to (buffer?) get them working. As a last ditched attempt it was mentioned try removing the resistor and try running them direct from python. I did both at the same time (not the best decision for troubleshooting. But to my surprise they worked.

So I killed the python program and restart hyperion, yep they’re working.

So off I went to stick them to the ceiling (they have sticky tape on the back). Done. If only I’d thought about connecting them before I stuck them up. I now had to work up in the air joining the cables. Not to worry I’ve done worse.

So I go to get what I need, by the time I got back up they’ve come down 🙁 bloody gravity! Now you’d think at this point I’d connect them up and sort out attaching them later Nope! (didn’t even enter my head) I was now on track to get them to stay up. Enter ‘SuperGlue’, applied little dots along the strip and stuck them up (yes I glued my fingers to the ceiling too). Finally they’re up and staying there. Oh I should have connected them when they were down!

‘Bugger it, where’s my screw driver’ I connected them up, put a power connector on the end and powered them and the PI.

Then installed hyperion on yet another pi. and it all worked like magic.

Have a look at the video, there’s no light other than the TV and it’s dark outside, but the room is really bright.


Click here for Part 2

Weewx+Raspberry PI+HDMI+PyGame

I’ve been using wview with my WH1080 weather station for some time (actually 2 of them). My main setup has been using my server, and every now and again the WH1080 would seem to lock up and nothing could get data out of it. The solution was to drop it’s power, on reboot it would all start working again.

However wview also seemed to introduce lockups of it’s own and the only solution there was to reboot the server (not ideal). So when it came to setting up a second weather station (in a remote location) I needed something a bit more stable and started looking at alternatives. I was doing this on a Raspberry PI and found wview. After installing it sometime last year it seemed pretty stable (although the WH1080 still manages to lockup).

Back to my house and I’ve finally had enough of missing weather data. One thing I really liked with wview was the ability to pull the archive data if the weather station had been running when the machine hadn’t (providing the USB hadn’t locked up), I really recommend looking at wview if your starting out.

I’ve already covered setting up weewx on a Raspberry PI and I’m not going to post about my exact configuration here. Instead I’m going to share my Python code for displaying the various graphs and gauges straight to the TV. At the moment I have my weather PI connected to the TV via HDMI. This may change later and then I’ll have to adjust the code to pull the images to another pi before displaying them (I have a similar project for displaying webcams already).

So a few things before the code.

  1. It’s my first real attempt at using classes, so my code will be more than a little scattered.
  2. You need to have python, pygame, wview, (and for this exact code Bootstrap for wview but you could just change the file paths to the Standard guages and graphs), mysql and wview configured for mysql.
  3. I have this started using an init script (added below).
  4. This runs the python program as root, I need to find a way to run this as a normal user (but that will affect point 5&6).
  5. This program checks mysql is able to be connected to and restarts mysql of not.
  6. It also checks the freshness of the index.html file (not the best way but a quick way) to make sure wview is running keeping the files upto date. If not it reboot the PI, this causes the weather station to reboot so if the USB locks up the whole system resets fixing it.

So now onto the python code 

import os
import time
import pygame
import MySQLdb as mdb
import signal
import sys
imglocation = "/var/www/weewx/Bootstrap"
class pyscreen :
   screen = None;
   def __init__(self):
       "Initializes a new pygame screen using the framebuffer"
       disp_no = os.getenv("DISPLAY")
       if disp_no:
           print "I'm running under X display = {0}".format(disp_no)
       # Check which framebuffer drivers are available.
       drivers = ['fbcon', 'directfb', 'svgalib']
       found = False
       for driver in drivers:
           # Make sure that SDL_VIDEODRIVER is set
           if not os.getenv('SDL_VIDEODRIVER'):
               os.putenv('SDL_VIDEODRIVER', driver)
           except pygame.error:
               print 'Driver: {0} failed.'.format(driver)
           found = True
       if not found:
           raise Exception('No suitable video driver found!')
       size = (pygame.display.Info().current_w, pygame.display.Info().current_h)
       print "Framebuffer size: %d x %d" % (size[0], size[1])
       self.screen = pygame.display.set_mode(size, pygame.FULLSCREEN)
       # Clear the screen to start
       self.screen.fill((0, 0, 0))
       # Initialise font support
       # Render the screen
   def __del__(self):
       "Destructor to make sure pygame shuts down, etc."
   def size(self):
       size = (pygame.display.Info().current_w, pygame.display.Info().current_h)
       return size
   def fill(self, colour):
       if self.screen.get_at((0,0)) != colour:
           self.screen.fill((0, 0, 0))
   def image(self, img, locX, locY, sizX, sizY):
           if (( "week" not in img) and (os.stat(img).st_mtime > time.time() - 600)):
               # 600 = 10 mins.
               image = pygame.image.load(img)
               image = pygame.transform.scale(image, (sizX, sizY))
               self.screen.blit(image, (locX, locY))
           elif(( "week" in img) and (os.stat(img).st_mtime > time.time() - 7200)):
               # 7200 = 2 hours.
               image = pygame.image.load(img)
               image = pygame.transform.scale(image, (sizX, sizY))
               self.screen.blit(image, (locX, locY))
               pygame.draw.rect(self.screen, (255, 0, 0), (locX, locY, sizX, sizY), 0)
       except pygame.error, message:
           pygame.draw.rect(self.screen, (255, 0, 0), (locX, locY, sizX, sizY), 0)
   def text_object(self, msg, font):
       black = (0, 0, 0)
       textSurface = font.render(msg, True, black)
       return textSurface, textSurface.get_rect()
   def error(self, msg):
       largeText = pygame.font.Font('freesansbold.ttf', 115)
       TextSurf, TextRect = screeny.text_object(msg, largeText)
       size = screeny.size = ((pygame.display.Info().current_w/2),(pygame.display.Info().current_h/2))
       self.screen.blit(TextSurf, TextRect)
class fileman:
   global imglocation
   def __init__(self):
       "Init for fileman. Nothing to do atm."
   def __del__(self):
       "Destructor for fileman. Nothing to do again."
   def total_files(self):
       for file in os.listdir(imglocation):
           if file.endswith(".png"):
       return count
class sqly:
   def __init__(self):
       "do nothing"
   def test(self):
           con = mdb.connect('localhost', 'weewx', 'weewx', 'weather')
           cur = con.cursor()
           cur.execute("SELECT VERSION()")
           ver = cur.fetchone()
           return 0
       except mdb.Error, e:
           return 1
            "do nothing"
   def restart(self):
       #wait 30 secs and try the connection again.
       if not mysql_con.test():
               os.system("service mysql start")
               "do nothing"
def sigterm_handler(_signo, _stack_frame):
    "When sysvinit send the TERM signal, cleanup before exiting"
    print("[" + get_now() + "] received signal {}, exiting...".format(_signo))
signal.signal(signal.SIGTERM, sigterm_handler)
def reboot():
    "check if we've been reboot in the last 30 mins"
    uptimef = open("/proc/uptime", "r")
    uptimestr =
    uptimelst = uptimestr.split()
    if float(uptimelst[0]) < 1800:
        "We've reboot in the last 30 mins, ignoring"
            "do nothing"
if __name__ == "__main__":
        screeny = pyscreen()
        filey = fileman()
        screeny.fill((0, 0, 255))
        size = screeny.size()
        border = 7
        sizewquarter = ((size[0]-(border*5))/4)
        sizehthird = ((size[1]-(border*4))/3)
        print("Total files : %d" % (filey.total_files()))
        while 1:
            mysql_con = sqly()
            if mysql_con.test():
                screeny.fill((250, 0, 0))
                screeny.error("Database Offline. Restarting!")
            elif (os.stat("/var/www/weewx/Bootstrap/index.html").st_mtime < time.time() - 600):
                screeny.fill((254, 0, 0))
                screeny.error("NOT Updating. Reboot Imminent!")
                screeny.fill((0, 0, 255))
                screeny.image("/var/www/weewx/Bootstrap/barometerGauge.png", (border*1), (border*1), sizewquarter, sizehthird)
                screeny.image("/var/www/weewx/Bootstrap/outTempGauge.png", ((border*2)+(sizewquarter*1)), (border*1), sizewquarter, sizehthird)
                screeny.image("/var/www/weewx/Bootstrap/windDirGauge.png", ((border*3)+(sizewquarter*2)), (border*1), sizewquarter, sizehthird)
                screeny.image("/var/www/weewx/Bootstrap/windSpeedGauge.png", ((border*4)+(sizewquarter*3)), (border*1), sizewquarter, sizehthird)
                screeny.image("/var/www/weewx/Bootstrap/big_images/weekbarometer-Bootstrap.png", (border*1), ((border*2)+(sizehthird*1)), ((sizewquarter*2)+(border*1)), sizehthird)
                screeny.image("/var/www/weewx/Bootstrap/big_images/weektempchill-Bootstrap.png", (border*1), ((border*3)+(sizehthird*2)), ((sizewquarter*2)+(border*1)), sizehthird)
                screeny.image("/var/www/weewx/Bootstrap/big_images/weekwinddir-Bootstrap.png", ((border*3)+(sizewquarter*2)), ((border*2)+(sizehthird*1)), ((sizewquarter*2)+(border*1)), sizehthird)
                screeny.image("/var/www/weewx/Bootstrap/big_images/weekwind-Bootstrap.png", ((border*3)+(sizewquarter*2)), ((border*3)+(sizehthird*2)), ((sizewquarter*2)+(border*1)), sizehthird)
    except KeyboardInterrupt:
        "We've got an interupt"

and now the init.d code

# init script for displayweewx
# Provides:          displayweewx
# Required-Start:    $syslog $network
# Required-Stop:     $syslog $network
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: init script to display weewx charts via HDMI output
# Description:       The python script queries mysql and file ages, so does not rely on mysql as a backup way to kick it.
. /lib/lsb/init-functions
test -f $DAEMON || exit 0
case "$1" in
        start-stop-daemon --start --background \
            --pidfile $PIDFILE --make-pidfile --startas /bin/bash \
            -- -c "exec stdbuf -oL -eL $DAEMON $DAEMONARGS > $LOGFILE 2>&1"
        log_end_msg $?
        start-stop-daemon --stop --pidfile $PIDFILE
        log_end_msg $?
        rm -f $PIDFILE
        $0 stop
        $0 start
        start-stop-daemon --status --pidfile $PIDFILE
        log_end_msg $?
        echo "Usage: $0 {start|stop|restart|status}"
        exit 2
exit 0

After all of this we get the following on the TV

From far far away.
From far far away.

Todays WordPress Adventures

Well it had to happen at some point, today I had a nice email from DigitalOcean saying they’ve disabled networking on one of my servers. This was because it’s ip address had been reported to RBL’s by several other servers.

Looking at the logs they included I was beating the s**t out of others wp-admin login pages. Now I know I wasn’t doing it personally, it was the first time in a long time I was in bed early and this seemed to start at 2am.

Luckily I could access my Droplet using the Console page, so after login I sat thinking ‘um…..’ where exactly do you start. The server normally has quite a bit of traffic so the logs are always cluttered. Needle in a haystack springs to mind.

I decided to run htop and see if the server was doing much without any traffic coming in. Oh yes /usr/bin/host is eating resources. So do I kill it or not. I decided not to at this point. Without networking I’m not doing anymore harm, and leaving it running may help find out what’s calling it.

It was a good call. I can’t give details of everything I did, I spent a hour hours checking through stuff. I do remember checking lsof and finding a link between a process id for host and a file within wpallimport’s uploads directory. So I had a look in there, followed by some further searching of google. 1 file in particular .sd0 seems to bring back results and this seems to be what’s caused it.

To get my server running again, I disabled the entire site within apache that was affected (luckily not a major site) and reboot the server. Once I was happy there were no cronjobs or anything calling on this script I mailed DigitalOcean and asked them to re-enable networking. They’re pretty speedy and within 15 mins had done it. A further reboot and my servers back online minus the one site I’ve disabled.

I expect the cleanup for this is going to take weeks of checking files, against backups while keeping as much as possible online.

I’m pretty confident I know what’s caused it, an out of date wordpress install with an out of date wpallimport install. It really goes to show that you have to check old stuff and keep it upto date.

The most annoying thing for me is that WordPress has a multisite option (which I use on 2 installs) and this allows me to keep plugins and everything upto date easily of sub-sites that are barely used. but it doesn’t extend to multiple domains which would really allow wordpress to be used across all my domains from one central console and then everything would be kept upto date in one go.

I know there’s a plugin for multisite domains, but I feel this is more of a hack of the wordpress system rather than wordpress properly designed to function with this in mind. I don’t want to install it and encounter many more problems.

It’s very bad admin on my part not having kept this site upto date, I’ll be the first to admit that but it’s easy to forget about installs you don’t use regularly. There must be some kind of nagios plugin to alert me to out of date plugins/versions for wordpress so I’ll be looking for that later in the week 🙂

WooCommerce New Bulk Action Part 2

Ok so in Part 1 I said I wasn’t going to use the email function so removed it from the code I used from

Clearly I need to think ahead a bit more. Over the last few weeks I’ve had a ton of sales, and I’m preparing everything to be ready to send them out as soon as I receive them from the supplier. Unfortunately our supplier contact me and advised there’s a few days delay on one product and about 10 days delay on another.

So I now need to mass mail people telling them.

I did look at using elasticemail to mass mail, but there’s a whole thing in their templates gearing towards marketing mails and opt out information. This isn’t a marketing mail it’s part of the transaction, I could live with giving it some opt out links. but what I really need is to be able to mail merge the order information into the mail.

So I quickly gave up on using elastcmail’s campaign function. I still use them as my mail relay and they’ve been very good, and support has been top notch for the few questions I’ve had.

Anyway, I thought back to the code for adding bulk actions and remembered there was an email function. So I checked out the code again, and yep it should be able to do what I need.

I decided to do things a little differently though.

First I create 2 new order status using JetPack (you can code this but it’s easier to just type and submit in jetpack) :

I’m going to be bulk assigning every order I want emailed to stock-email, anything that fails (wp-mail does fail) will go to stock-email-fail. This is the bit that wasn’t in the orginal code.

I also want to pass the orderid and name into the mail function. I could just have the mail function go get them, but it already queries the email address in the change_order_status function so I may as well keep it there.

Here’s my new Code.

//added for woocommerce bulk actions.
add_action('admin_footer-edit.php', 'custom_bulk_admin_footer');
function custom_bulk_admin_footer() {
 global $post_type;
 if($post_type == 'shop_order') {
 <script type="text/javascript">
 jQuery(document).ready(function() {
 //printing status
 jQuery('<option>').val('printing').text('<?php _e('Mark as Printing')?>').appendTo("select[name='action']");
 jQuery('<option>').val('printing').text('<?php _e('Mark as Printing')?>').appendTo("select[name='action2']");

 //stock status
 jQuery('<option>').val('stock').text('<?php _e('Mark as Awaiting Stock')?>').appendTo("select[name='action']");
 jQuery('<option>').val('stock').text('<?php _e('Mark as Awaiting Stock')?>').appendTo("select[name='action2']");

 //stock status
 jQuery('<option>').val('stock-emailed').text('<?php _e('Mark as Awaiting Stock Emailed')?>').appendTo("select[name='action']");
 jQuery('<option>').val('stock-emailed').text('<?php _e('Mark as Awaiting Stock Emailed')?>').appendTo("select[name='action2']");
add_action('load-edit.php', 'custom_bulk_action');
function custom_bulk_action() {
 global $typenow;
 $post_type = $typenow;

 if($post_type == 'shop_order') {
 $wp_list_table = _get_list_table('WP_Posts_List_Table');
 $action = $wp_list_table->current_action();
 $allowed_actions = array("stock","printing","stock-emailed");
 if(!in_array($action, $allowed_actions)) return;

 if(isset($_REQUEST['post'])) {
 $orderids = array_map('intval', $_REQUEST['post']);

 switch($action) {
 case "printing":
 foreach( $orderids as $orderis ) {
 change_order_status($orderid, $action);
 case "stock":
 foreach( $orderids as $orderid ) {
 change_order_status($orderid, $action);
 case "stock-emailed":
 foreach( $orderids as $orderid ) {
 change_order_status($orderid, $action);
 default: return;

 $sendback = admin_url( "edit.php?post_type=$post_type&success=1" );
function change_order_status($orderid, $action) {
 $order = new WC_Order($orderid);
 if(($action=='printing') && ($order->status!='printing')) {
 $order->update_status('printing', '');
 if(($action=='stock') && ($order->status!='stock')) {
 $order->update_status('stock', '');
 if(($action=='stock-emailed') && ($order->status!='stock-emailed')) {
 $email = get_post_meta( $orderid, '_billing_email' )[0];
 $name = get_post_meta( $orderid, '_billing_first_name')[0];
 $name = ucwords($name);
 if (send_this($email,$orderid,$name)) {
 $order->update_status('stock-emailed', '');
 } else {
 $order->update_status('stock-email-fail', '');
add_action('admin_notices', 'custom_bulk_admin_notices');
function custom_bulk_admin_notices() {
 global $post_type, $pagenow;
 if( $post_type == 'shop_order' && isset($_GET['success']) ) {
 echo '<div class="updated"><p>The orders have been successfully update!</p></div>';
function send_this($email,$orderid,$name) {

 $headers = 'From: Shop <[email protected]>' . "\r\n";
 $headers .= "MIME-Version: 1.0\r\n";
 $headers .= "Content-Type: text/html; charset=ISO-8859-1\r\n";

 $subject = 'Order Update';

 $message = '

 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
 Order Update: #' . $orderid . '
 <body leftmargin="0" marginwidth="0" topmargin="0" marginheight="0" offset="0">
 <div style=" background-color: #f5f5f5; width:100%; -webkit-text-size-adjust:none !important; margin:0; padding: 70px 0 70px 0;">
 <table border="0" cellpadding="0" cellspacing="0" height="100%" width="100%">
 <td align="center" valign="top">
 <div id="template_header_image">
 <table border="0" cellpadding="0" cellspacing="0" max-width="600" id="template_container" style=" box-shadow:0 0 0 3px rgba(0,0,0,0.025) !impo$
 <td align="center" valign="top">
 <!-- Header -->
 <table border="0" cellpadding="0" cellspacing="0" width="100%" id="template_header" style=" background-color: #557da1; color: #ffffff;$
 <h1 style=" color: #ffffff; margin:0; padding: 28px 24px; text-shadow: 0 1px 0 #7797b4; display:block; font-family:Arial; font$
 Order: #' . $orderid . ' Update
 <!-- End Header -->
 <td align="center" valign="top">
 <!-- Body -->
 <table border="0" cellpadding="0" cellspacing="0" width="600" id="template_body">
 <td valign="top" style=" background-color: #fdfdfd; border-radius:6px !important;">
 <!-- Content -->
 <table border="0" cellpadding="20" cellspacing="0" width="100%">
 <td valign="top">
 <div style=" color: #737373; font-family:Arial; font-size:14px; line-height:150%; text-align:left;">
 Hi ' . $name . ',
 Thank you 


 return wp_mail( $email, $subject, $message, $headers );

I haven’t include the full email above, just a snip so you can see where I put the orderid and name. The actual template I’ve used is one off woocommerce itself to keep the look of the emails the same as an order progress.

To get the mails flowing just select a bunch of orders and bulk change them to ‘Mark as Awaiting Stock Emailed’. This happily looped through 250 at a time on my system. After doing over 3,000 I only had 14 moved automatically to the failed status, and rerunning them didn’t cause any problems.

The one issue I did run into was the name of the failed status, originally I had it set as stock-emailed-fail. But there’s a limit on the number of characters you can use and it came into the system as stock-emailed-fai. I managed to loose 3 orders in the first test batch, as they were now assigned a status that didn’t exist. So I had to find them in the database and set them to the proper status. If you keep your slugs short, you shouldn’t see this issue. but make sure you also type the exactly the same in the code as they appear in jetpack.

I take no responsibility for any lost orders as a result of using this code.