Rob’s Blog

One of the best presents I received for Christmas this year was an Arduino starter kit. It’s been about 20 years since I did anything involving electronics; back in school I was fascinated with the subject, and the Arduino has suddenly reignited my interest.

The starter kit contains quite an interesting array of sensors (LDR, piezzo vibration sensor, thermistor), but a very limited number of basic components (such as resistors) in order to hook them up. This unfortunately has reduced what I can build for the moment to flashing LEDs. The first project I tackled was a nice colour-shifting effect with a tri-colour LED. Quite pointless, you might agree, but certainly not unimaginably pointless. I needed more pointlessness…

The unimaginably pointless project I decided on was more of a software project than an electronics one. Here’s the circuit I built:

I’ve attached the anode of an LED to the Arduino’s digital pin 13, and it’s cathode to ground. The Arduino has a built in resistor on pin 13, so this circuit is really the “Hello, World” of electronics. Not particularly complicated, I’m sure you’ll agree.

So what will this solitary LED do for us? Well my unimaginably pointless idea was to convert stuff into Morse code. There are several examples of this on the web already. For the most part, I’ve blatantly copied them, adding a few little modifications and enhancements along the way.

First up, I’ll create a Morse library for the Arduino board. This is based heavily on code from two sources:

1. David A. Mellis’s example code for creating a library
2. Mark VandeWettering’s excellent morse implementation

The Morse Code library looks like this:

/*
  MorseCode.cpp - Morse code library
  Created by Rob Poyntz 2nd Jan 2010
  Based on code by:
    1) David Mellis: http://www.arduino.cc/en/Hacking/LibraryTutorial
    2) Mark VandeWettering: http://brainwagon.org/2009/11/14/another-try-at-an-arduino-based-morse-beacon/
*/
#include "MorseCode.h"

struct t_mtab { char c, pat; };
struct t_mtab morsetab[] = {
   {'.', 106}, {',', 115}, {'?', 76}, {'/', 41},
   {'A', 6},   {'B', 17},  {'C', 21}, {'D', 9},
   {'E', 2},   {'F', 20},  {'G', 11}, {'H', 16},
   {'I', 4},   {'J', 30},  {'K', 13}, {'L', 18},
   {'M', 7},   {'N', 5},   {'O', 15}, {'P', 22},
   {'Q', 27},  {'R', 10},  {'S', 8},  {'T', 3},
   {'U', 12},  {'V', 24},  {'W', 14}, {'X', 25},
   {'Y', 29},  {'Z', 19},  {'1', 62}, {'2', 60},
   {'3', 56},  {'4', 48},  {'5', 32}, {'6', 33},
   {'7', 35},  {'8', 39},  {'9', 47}, {'0', 63}};

#define N_MORSE  (sizeof(morsetab) / sizeof(morsetab[0]))
#define SPEED  (12)
#define DOTLEN  (1200 / SPEED)
#define DASHLEN  (3 * (1200 / SPEED))
#define LOWERCASE_A (97)
#define LOWERCASE_Z (122)
#define CASE_SHIFT (32)

MorseCode::MorseCode(int pin)
{
  _pin = pin;
  _Unknown = '?';
  pinMode(_pin, OUTPUT);
}

void MorseCode::dash()
{
  digitalWrite(_pin, HIGH);
  delay(DASHLEN);
  digitalWrite(_pin, LOW);
  delay(DOTLEN);
}

void MorseCode::dot()
{
  digitalWrite(_pin, HIGH);
  delay(DOTLEN);
    digitalWrite(_pin, LOW);
  delay(DOTLEN);
}

void MorseCode::Signal(char c)
{
  int i;

  if (c == ' ')
  {
    Serial.print(c);
    delay(7 * DOTLEN);
    return;
  }

  if ((byte(c) >= LOWERCASE_A) && (byte(c) < = LOWERCASE_Z))
    c = char(byte(c) - CASE_SHIFT);

  // Loop through the Morse Code table, and try to find the requested character
  for (i = 0; i < N_MORSE; i++)
  {
    if (morsetab[i].c == c)
    {
      unsigned char p = morsetab[i].pat;

      while (p != 1)
      {
        if (p & 1)
          dash();
        else
          dot();
        p = p / 2;
      }
      delay(2 * DOTLEN);
      return;
    }
  }

  /* If we can't find the requested character, signal an 'unknown' */
  Signal(_Unknown);
}

And the header file:

/*
  MorseCode.h - Morse code library
  Created by Rob Poyntz 2nd Jan 2010
  Based on code by:
    1) David Mellis: http://www.arduino.cc/en/Hacking/LibraryTutorial
    2) Mark VandeWettering: http://brainwagon.org/2009/11/14/another-try-at-an-arduino-based-morse-beacon/
*/
#ifndef MorseCode_h
#define MorseCode_h

#include "WProgram.h"

class MorseCode
{
  public:
    MorseCode(int pin);
    void Signal(char c);

  private:
    int _pin;
    char _Unknown;
    void dot();
    void dash();
};

#endif

Mark’s project turned a hard-coded string into Morse code, but this seemed a little inflexible. I decided that I’d read content from the serial port and convert that – it’s pretty trivial to acheive. Here’s the Arduino sketch I used:

#include <MorseCode.h>

const int OutputPin = 13;

MorseCode morse(OutputPin);

void setup()
{
  Serial.begin(9600);
}

void loop()
{
  char c;
  if (Serial.available())
  {
    c = Serial.read();
    morse.Signal(c);
  }
}

And finally, finding stuff to turn into Morse code… I wrote a little bash script to grab the title from the latest headline in Slashdot’s RSS feed:


wget -O - http://rss.slashdot.org/Slashdot/slashdot 2>/dev/null | \
sed -r -e '/<title>/!d' -e '/Slashdot/d' -e 's/<title>(.*)<\/title>/\1/' | head -n 1 | tee /dev/ttyUSB0


Brilliant – converting readable text into an annoying flashing light

Sitting in a tree in our front yard:

Incidentally, I just noticed that WordPress has a category called “uncategorized”.

A few of my clients have businesses which rely on entirely on their website, and therefore on the database that powers it. What would happen to them if their database was lost or corrupted? Most likely, it would be a complete disaster for them. Of course, we could ask the web hosts to perform daily database backups of the database, but that usually adds significant cost to the price of hosting (one hosting company quoted us $100 per backup!).

Perhaps an easier way to implement an offset backup would be to dump the database, compress it and email it offsite somewhere. You’d need an email account with plenty of storage, obviously. Something like GMail’s 7GB of storage perhaps? Since you’re sending possibly sensitive data offsite, you might want to encrypt the data as well. Here’s a simply shell script which will do all of this:

#!/bin/bash
#

mysqldump -u USERNAME --password=DATABASE_PASSWORD DATABASE_NAME | gzip | gpg -a -c -o - --no-use-agent --passphrase 'YOUR_ENCRYPTION_PASSPHRASE' | uuencode "Database Backup `date`.gz.asc" | mail -s "Database Backup `date`" YOUR_EMAIL_ADDRESS

Let’s break this down a little to explain:

mysqldump -u USERNAME --password=DATABASE_PASSWORD DATABASE_NAME

This dumps the complete database “DATABASE_NAME” as SQL.

gzip

This compresses your database dump (no point using up your email storage unnecessarily)

gpg -a -c -o - --no-use-agent --passphrase 'YOUR_ENCRYPTION_PASSWORD'

This uses GPG’s symmetric encryption to keep your compressed database dump safe from prying eyes.

uuencode "Database Backup `date`.gz.asc"

(Possibly a bit “belt ‘n’ braces” since we’ve specified the “-a” option with GPG, but) uuencode the encrypted, compressed database backup so it’s suitable for emailing as an attachment. You might need to install the sharutils package before this program is available.

mail -s "Database Backup DB `date`" YOUR_EMAIL_ADDRESS

Finally, send the database backup to your nominated email account.

To automate the process, simply set up a cron job to run this script every n hours, where n is a number of your choosing according to a) the number of transactions on your database, b) the likely impact of losing data, and c) the amount of your storage at your disposal.

There you go: a basic strategy for automatically creating, encrypting and storing database backups on a remote server.

I’ve been using my simple MySQLi wrapper class for some time now, and feel that it’s been reasonably well fine-tuned. It’s been used on several live projects, and is working extremely well. Most importantly, it has helped to simplify my code enormously. It allows me to execute parameterised SQL statements with a single line of code, and return the results as an object or array.

A little while ago I also started investigating PHP Data Objects (PDO). PDO provides much the same functionality as MySQLi (with the advantage of being relatively database independent), and also provides a really nice set of additional functions such as fetching a result set into an existing object. I’ve created a wrapper with an identical interface as the earlier MySQLi version, which encapsulates PDO functionality to a MySQL database. This is a drop-in replacement to the MySQLi version, and has allowed me to benchmark the differences in performance. In the process of checking out PDO, I’ve learned a couple of interesting things:

1) Unrelated to databases, but PHP does allows properties to be added to objects dynamically at runtime. You may have already known this, but it was news to me. Previously I’ve been using the magic __get and __set functions. Instead, I could have just created a new instance of StdClass, and added my properties to it as so:

  $obj = new StdClass();
  $obj->myProperty = 17;
  $obj->anotherProperty = array('Blue', 1.712);

Kris Jordan thinks that using __get and __set is around 30 times slower than using dynamic properties on StdClass.

2) The PDO wrapper class seems to be quicker by a significant margin. I’ve timed the execution time for various pages (by differencing the results of a call to “microtime()” at the start and end of each script) and the PDO version is consistently quicker. For some sites, PDO reduces script execution time by nearly 50%.

To help simplify usage, I’ve bundled everything together, defined an interface that exposes the relevant access methods, and added a few factory methods to an abstract base class:

  interface DFData
  {
    function Close();
    function StartTransaction();
    function CommitTransaction();
    function RollbackTransaction();

    function FetchRow($sql);
    function FetchRowAssoc($sql);
    function FetchSingleObj($sql);
    function FetchObj($sql);
    function ExecuteSQL($sql);
  }

  abstract class DF_DataObject implements DFData
  {
    public static function PDOInstance($username, $password, $database, $servername = 'localhost') {...}
    public static function MySQLiInstance($username, $password, $database, $servername = 'localhost') {...}
  }

I haven’t been playing with this for long, but so far it’s proven to be robust, reliable, and in the case of the PDO version, quick.

Once again, for those that have missed the earlier posts, the wrapper class allows to you eliminate SQL injection attacks by providing a really easy way to execute parameterised queries:

  $db = DF_DataObject::PDOInstance('username', 'password', 'databasename');

  // Insert statements
  $db->ExecuteSQL('INSERT INTO Fruit (Name, Colour) VALUES (?, ?)', 'ss', 'Banana', 'Yellow');
  $db->ExecuteSQL('INSERT INTO Fruit (Name, Colour) VALUES (?, ?)', 'ss', 'Apple', 'Green');
  $db->ExecuteSQL('INSERT INTO Fruit (Name, Colour) VALUES (?, ?)', 'ss', 'Lemon', 'Yellow');
  $insertID = $db->ExecuteSQL('INSERT INTO Fruit (Name, Colour) VALUES (?, ?)', 'ss', 'Orange', 'Yellow');

  // Update statements
  $db->ExecuteSQL('UPDATE Fruit SET Colour = ? WHERE ID = ?', 'si', 'Orange', $insertID);

  // Select statements
  $obj = $db->FetchSingleObj('SELECT Name, Colour FROM Fruit WHERE Colour = ?', 's', 'Green');
  /*
     $obj->Name = 'Apple'
     $obj->Colour = 'Green'
  */

  while ($fruit = $db->FetchObj('SELECT Name, Colour FROM Fruit WHERE Colour = ?', 's', 'Yellow'))
    echo $fruit->Name;

  /*
    Banana
    Lemon
  */

  $arr = $db->FetchRowAssoc('SELECT Name, Colour FROM Fruit WHERE Colour = ?', 's', 'Orange');
  /*
    $arr = array('Name' => 'Orange', 'Colour' => 'Orange');
  */

Source code is here

Everybody loves Internet Explorer, especially web developers. Rather than being a boring, predictable browser that does what you expect, there’s always a little quirk to keep you on your toes. There’s always some intriguing idiosyncrasy that ensures your day is not wasted by actually writing productive code, but instead spent trawling the statistical noise of Google in search of enlightenment.

Today I was informed by a customer that they couldn’t download a PDF document from a website I’d created for them. The PDF document was being accessed via the site’s administrative section, and was being downloaded via SSL. Of course, the predictably boring Firefox, Chrome, Safari and Operas of the world had no problems downloading the document, but Internet Explorer said this:

Internet Explorer cannot download [filename] from [website.com]

Internet Explorer was not able to open this Internet site. The requested site is either unavailable or cannot be found. Please try again later.

The reason for this annoyance is that on the server, PHP automatically sets a “Cache-Control” HTTP header with a value of “Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0″ if you’re using sessions. This is usually a good thing, and ensures you always get an un-cached version of your dynamically generated pages, however in the case of downloading attachments, things go a little bit wrong. Here’s what happens:

1. Client makes a request for a page over SSL
2. Server says: You can’t cache this response
3. Server starts sending a PDF document
4. Client tries to store response in a temporary file until the whole PDF document has been downloaded
5. Internet Explorer remembers that it’s been asked not to cache the file. Since we’re using SSL, it must be really important, so throws the error message above.
6. All other browsers ignore the “cache-control” header and temporarily store the document anyway.

A few sites are claiming that this is a bug, but KB316431 says this behaviour is by design. The purist in me admits (rather grudgingly) that Microsoft might be right here, and they have implemented things correctly. This doesn’t please my client however, who still can’t download his PDF.

So we need a solution. The obvious answer is to get rid of the “no-cache” part of our headers. If you’re using PHP sessions however, it’s not that simple. Just adding:

< ?php
  header("Cache-Control: private, max-age=1, pre-check=1");
?>

won’t work, as PHP’s session handling automatically adds it’s own “cache-control” header, and removes any others. The answer is as follows:

< ?php
  // Get rid of anything in the output buffer
  ob_clean(); 

  // Determine the size of the download
  $filesize = filesize($filename);

  // Close and save any session variables
  session_write_close();

  // Write some headers, including a modified "cache-control"
  header("Cache-Control: private, max-age=1, pre-check=1");
  header("Pragma: none");
  header("content-type: application/x-pdf");
  header("content-disposition: attachment; filename=\"downloaded-file.pdf\"");
  header("content-length: $filesize");

  // Send the file contents to the output buffer
  readfile($filename);

  // And we're done
  exit(0);
?>

The key here is to close the user session, preventing your “cache-control” header from being over-written by PHP automatically.

À bientôt!

Update
After requests from Juanma and Manic, below is a modified version of the plugin which demonstrates custom navigation:

All source code is completed untested on any browser except FF3.5, and comes without warranty of any kind. Source code is contained within this file

I’ve been a huge fan of jQuery for a few years now, and it is incorporated in most of my online projects. I’ve never created a plugin for jQuery though, so I thought that maybe I should try my hand at it. My attempt builds on some code I wrote quite a while ago to transition a sequence of images (or any HTML elements, for that matter) by fading out one image, and fading in another. There are plenty of other plugins which achieve the same effect (and are probably better), but feel free to experiment with this one.

Demo:

There’s really not much source code to this plugin:

(function ($) {
  $.fn.fadeTransition = function(options) {
    var options = $.extend({pauseTime: 5000, transitionTime: 2000}, options);

    Trans = function(obj) {
      var timer = null;
      var current = 0;
      var els = $("> *", obj).css("display", "none").css("left", "0").css("top", "0").css("position", "absolute");
      $(obj).css("position", "relative");
      $(els[current]).css("display", "block");

      function transition() {
        var next = (current + 1) % els.length | 0;
        $(els[current]).fadeOut(options.transitionTime);
        $(els[next]).fadeIn(options.transitionTime);
        current = next;
        cue();
      };

      function cue() {
        if ($("> *", obj).length < 2) return false;
        if (timer) clearTimeout(timer);
        timer = setTimeout(transition, options.pauseTime);
      };

      cue();
    };

    return this.each(function() {
      var t = new Trans(this);
    });
  }
})(jQuery);

Usage is also quite straightforward. Include jQuery, and the plugin source, and use this:

    $(document).ready(function() { $(".container").fadeTransition(); });

There are a couple of options to configure the animation speeds:

  $(".container").fadeTransition({pauseTime: 5000, transitionTime: 2000});

pauseTime is the number of milliseconds each child of .container will be displayed for.
transitionTime is the number of milliseconds the fade in / fade out animation will last for.

For the live server:
wget -O- http://live.server/index.php | hexdump -C | head -n1

resulted in:

00000000 ef bb bf ef bb bf 3c 3f 78 6d 6c 20 76 65 72 73 |......< ?xml vers |

Aha! The dreaded double byte order mark! UTF-8 files don't need a byte order mark, but somehow at least two had slipped into my source files on the live server, and were causing IE to get all confused. But which files? You can't just view the files, because a byte order marker is not displayed in a text editor. I managed to track down the offending source files using the following script:

for i in `find ./ -type f -name '*.php'`; do hexdump -C $i | head -n1 | grep -i 'ef bb bf' && echo $i; done

This nice bit of bash script finds all files with a “.php” extension, does a hex dump of each and searches for the UTF-8 BOM. If the BOM is found, the first line of the hex dump is displayed and the filename printed below.

My father contacted me recently with a problem with his computer. Whenever he tried to open the windows command prompt (either by clicking on the “command prompt” icon in his start menu, or by typing “cmd” in Start | Run), the window would appear for a brief second on the screen, then close again almost immediately.

It turned out that cmd.exe wasn’t the only application affected. The computer also refused to run regedit, reged32, netstat and others. Added to this, my father reported that his AVG antivirus was having trouble updating, which he presumed was simply because their update server was busy. All of this clearly implies there was at least one piece of malware on his computer, which has somehow slipped past AVG, and disabled it’s update process.

After some Googling, I eventually found a clue on the Avira Antivirus forums. By taking a copy of regedit.exe, and renaming it to something arbitrary, we were able to get the program to run again. The malware had inserted a registry entry into HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Drivers32, and it was being loaded up as a driver on system startup. In this case, the “driver” file being loaded was called “nnpug.wbn“, although I’m sure this varies depending on the whim^* of the malware. Deleting this registry value and rebooting had a dramatic effect – all of the reported problems were now gone: cmd.exe and it’s friends would run again, and AVG could now perform updates as normal. We didn’t delete the “nnpug.wbn” file, because we wanted to let the antivirus software identify it, so we could determine what damage had been done.

AVG was updated, and after a full scan it had not detected that our rogue file was a virus! We ended up installing the Avira antivirus program, and that did thankfully pick up the malware and identify it: Trojan-PWS.Delf.AH. There is very little information about this malware on the web at the time of writing, although the one site that had a brief description gave us the worst possible news: the malware harvests usernames and passwords, and hands them back to a remote machine. #$@&*!

So, where does this leave my father’s computer? He’s normally extremely careful about what sites he visits, doesn’t generally open email that isn’t from a known sender, and keeps his machine patched and up-to-date. Yet somehow, this devious malware still managed to install itself as a system driver. Had it not disabled cmd.exe – he would never have even known it was there! Are there any other files lurking on his machine that are as-yet undetected? He’s reluctant to do a complete reinstall of Windows because of the hassle – but surely it must be worth it for the peace of mind?

It’s a pretty sad state of affairs when you need to run two different anti-virus solutions, plus Ad-Aware and SpyBlock just to keep your computer free from nasties, and your personal information personal. This post was never meant to be an anti-Microsoft rant, but really – the world of personal computing has gone insane! Why should we have to install software that makes our computers perform like their running through treacle? Processors shouldn’t be using 1/4 of their capabilities scanning every file that is opened, downloaded, or saved, scanning for registry or system changes just to ensure you’re not infected by some crapware. There must be a better way to safely browse the internet…

There is. Linux (or Mac, if you’re that way inclined).

^*I know you’re not supposed to anthropomorphise computers because they don’t like it.

I stumbled across a brilliant bug today. I’ve been building an eCommerce site for a client, and had written a whole bunch of code that works flawlessly on my development server. However, as soon as I deployed to the production server, things went a bit Pete Tong.

Object of class __PHP_Incomplete_Class could not be converted to string

You can Google this phrase and find a million (approx) results for this, and it’s generally caused by people storing objects in a session variable, but not including the class definition before the session_start() statement. However, I was not doing this. I’m doing:

  define("SESSION_CART", "cart");
  $_SESSION[SESSION_CART] = serialize($cart);

After many hours of trying everything I could think of, tearing (what’s left) of my hair out, and generally throwing a programming tantrum, I finally resorted to the last ditch solution: renaming the variable to something stupid. Using this code fixed the problem:

  define("SESSION_CART", "underpants");
  $_SESSION[SESSION_CART] = serialize($cart);

So why should the name of a session variable make so much of a difference? I can only presume that the issue is caused by some third party shopping cart add-on software that’s hijacking the production server’s PHP configuration.

The latest Linux NVidia drivers (180.29) finally contain a fix to a long-standing bug that affects 2D rendering performance in Firefox. Until this release, many javascript-powered visual effects (such as the Javascript image transition on this site) have caused horrendous performance in Firefox. A big thank you to NVidia for finally fixing this!

To install the drivers on Ubuntu:

1. Download the drivers from the NVidia site to your home directory
2. Log out of your Gnome / KDE session
3. Start a virtual terminal by pressing Ctrl+Alt+F1, and log in to your account
4. Remove any old NVidia packages you might have with
   sudo apt-get remove nvidia*
5. Stop Gnome with
   sudo /etc/init.d/gdm stop
   Could someone please provide the equivalent command for KDE?
6. Switch to run level 3 with
   sudo init 3
7. Install the drivers with
   sudo sh ./NVIDIA-Linux-x86-180.29-pkg1.run
   Just follow the prompts (and basically press enter a lot)
8. Once the installation has completed, restart your machine with:
   sudo shutdown -r now

Once you’ve rebooted, you should be running the latest drivers in all their 2D performance glory!