Rob's Blog

Another day, another curious problem to be solved…

Today I found that a recently developed project wasn’t rendering too well in (every web developers favourite) Internet Explorer. There were a few tell-tale signs that pointed to IE not understanding the doc-type declaration at the beginning of the file. The first few lines of the generated file were:


< ?xml version="1.0" ?>
< !DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">



Well, there’s nothing obviously wrong with that. As is usually the case, Firefox rendered everything correctly. The HTML output was fully XHTML strict compliant, and the CSS validated too. To add to my confusion, the exact same code hosted on my development server was rendering correctly in all browsers. This gave me a chance to compare the output of the two installations. Here’s what I found:

For my development server:
wget -O- http://dev.server/index.php | hexdump -C | head -n1

gave the following output:

00000000 3c 3f 78 6d 6c 20 76 65 72 73 69 6f 6e 3d 22 31 |< ?xml version="1|

For the live server:
wget -O- http://live.server/index.php | hexdump -C | head -n1

resulted in:

00000000 ef bb bf ef bb bf 3c 3f 78 6d 6c 20 76 65 72 73 |......< ?xml vers |

Aha! The dreaded double byte order mark! UTF-8 files don't need a byte order mark, but somehow at least two had slipped into my source files on the live server, and were causing IE to get all confused. But which files? You can't just view the files, because a byte order marker is not displayed in a text editor. I managed to track down the offending source files using the following script:

for i in `find ./ -type f -name '*.php'`; do hexdump -C $i | head -n1 | grep -i 'ef bb bf' && echo $i; done

This nice bit of bash script finds all files with a “.php” extension, does a hex dump of each and searches for the UTF-8 BOM. If the BOM is found, the first line of the hex dump is displayed and the filename printed below.

My father contacted me recently with a problem with his computer. Whenever he tried to open the windows command prompt (either by clicking on the “command prompt” icon in his start menu, or by typing “cmd” in Start | Run), the window would appear for a brief second on the screen, then close again almost immediately.

It turned out that cmd.exe wasn’t the only application affected. The computer also refused to run regedit, reged32, netstat and others. Added to this, my father reported that his AVG antivirus was having trouble updating, which he presumed was simply because their update server was busy. All of this clearly implies there was at least one piece of malware on his computer, which has somehow slipped past AVG, and disabled it’s update process.

After some Googling, I eventually found a clue on the Avira Antivirus forums. By taking a copy of regedit.exe, and renaming it to something arbitrary, we were able to get the program to run again. The malware had inserted a registry entry into HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Drivers32, and it was being loaded up as a driver on system startup. In this case, the “driver” file being loaded was called “nnpug.wbn“, although I’m sure this varies depending on the whim^* of the malware. Deleting this registry value and rebooting had a dramatic effect – all of the reported problems were now gone: cmd.exe and it’s friends would run again, and AVG could now perform updates as normal. We didn’t delete the “nnpug.wbn” file, because we wanted to let the antivirus software identify it, so we could determine what damage had been done.

AVG was updated, and after a full scan it had not detected that our rogue file was a virus! We ended up installing the Avira antivirus program, and that did thankfully pick up the malware and identify it: Trojan-PWS.Delf.AH. There is very little information about this malware on the web at the time of writing, although the one site that had a brief description gave us the worst possible news: the malware harvests usernames and passwords, and hands them back to a remote machine. #$@&*!

So, where does this leave my father’s computer? He’s normally extremely careful about what sites he visits, doesn’t generally open email that isn’t from a known sender, and keeps his machine patched and up-to-date. Yet somehow, this devious malware still managed to install itself as a system driver. Had it not disabled cmd.exe – he would never have even known it was there! Are there any other files lurking on his machine that are as-yet undetected? He’s reluctant to do a complete reinstall of Windows because of the hassle – but surely it must be worth it for the peace of mind?

It’s a pretty sad state of affairs when you need to run two different anti-virus solutions, plus Ad-Aware and SpyBlock just to keep your computer free from nasties, and your personal information personal. This post was never meant to be an anti-Microsoft rant, but really – the world of personal computing has gone insane! Why should we have to install software that makes our computers perform like their running through treacle? Processors shouldn’t be using 1/4 of their capabilities scanning every file that is opened, downloaded, or saved, scanning for registry or system changes just to ensure you’re not infected by some crapware. There must be a better way to safely browse the internet…

There is. Linux (or Mac, if you’re that way inclined).

^*I know you’re not supposed to anthropomorphise computers because they don’t like it.

I stumbled across a brilliant bug today. I’ve been building an eCommerce site for a client, and had written a whole bunch of code that works flawlessly on my development server. However, as soon as I deployed to the production server, things went a bit Pete Tong.

Object of class __PHP_Incomplete_Class could not be converted to string

You can Google this phrase and find a million (approx) results for this, and it’s generally caused by people storing objects in a session variable, but not including the class definition before the session_start() statement. However, I was not doing this. I’m doing:

  define("SESSION_CART", "cart");
  $_SESSION[SESSION_CART] = serialize($cart);

After many hours of trying everything I could think of, tearing (what’s left) of my hair out, and generally throwing a programming tantrum, I finally resorted to the last ditch solution: renaming the variable to something stupid. Using this code fixed the problem:

  define("SESSION_CART", "underpants");
  $_SESSION[SESSION_CART] = serialize($cart);

So why should the name of a session variable make so much of a difference? I can only presume that the issue is caused by some third party shopping cart add-on software that’s hijacking the production server’s PHP configuration.

The latest Linux NVidia drivers (180.29) finally contain a fix to a long-standing bug that affects 2D rendering performance in Firefox. Until this release, many javascript-powered visual effects (such as the Javascript image transition on this site) have caused horrendous performance in Firefox. A big thank you to NVidia for finally fixing this!

To install the drivers on Ubuntu:

1. Download the drivers from the NVidia site to your home directory
2. Log out of your Gnome / KDE session
3. Start a virtual terminal by pressing Ctrl+Alt+F1, and log in to your account
4. Remove any old NVidia packages you might have with
   sudo apt-get remove nvidia*
5. Stop Gnome with
   sudo /etc/init.d/gdm stop
   Could someone please provide the equivalent command for KDE?
6. Switch to run level 3 with
   sudo init 3
7. Install the drivers with
   sudo sh ./NVIDIA-Linux-x86-180.29-pkg1.run
   Just follow the prompts (and basically press enter a lot)
8. Once the installation has completed, restart your machine with:
   sudo shutdown -r now

Once you’ve rebooted, you should be running the latest drivers in all their 2D performance glory!

Here’s a regular expression to match the extension in a filename:

/\.(?!(.*\.)).*$/

It uses a negative look ahead to ensure that things all go smoothly when you have a filenames such as:

// Many dots in filename
/directory/folder/subdir/file-v1.2.3.zip

or

// Many dots in filename, relative paths and an extension with more than 3 characters in
./../folder/file-v1.2.3.gzip

Using this regular expression, a simple “ChangeFileExtension” routine in PHP might look like:

function ChangeFileExt($filename, $newExtension)
{
  if ($filename != "")
    return preg_replace("/\.(?!(.*\.)).*$/", ".$newExtension", $filename);
  else
    return "";
}

Edit: Just noticed that Nash has another nice regex to do this. See here

Here’s a quick and easy way to change stylesheets on the fly using a little javascript and jQuery:

function changeCSS(newStylesheetLocation) {
  var css = document.createElement("link");
  css.href = newStylesheetLocation;
  css.rel = "stylesheet";
  css.type = "text/css";
  $("link").remove();
  $("head").get(0).appendChild(css);
}

and

Change Stylesheet

This assumes that:

• You’ve only got one stylesheet in the head section of your document
• You’ve used a link tag to include your stylesheet

If you’ve used inline stylesheets, @import or have multiple link tags, you’ll need to adjust the javascript function to suit your needs.

Following my previous post on MySQLi parameterised queries, yet another major site has fallen victim to an automated SQL injection attack…

I’ve made some further changes to the class I presented back in that post. A quick recap of the functionality I presented originally:

1) Execution of prepared SQL statements that do not return a result set:

  $sql = "UPDATE tblCustomers SET LastLogin = NOW() WHERE UserID = ?";
  $db->ExecuteSQL($sql, "i", $userid);

replaced

  $sql = "UPDATE tblCustomers SET LastLogin = NOW() WHERE UserID = ?";
  if ($query = $db->prepare($sql))
  {
    $query->bind_param("i", $userid);
    $query->execute();
    $query->close();
  }

2) Fetch a single row using a prepared query:

  $sql = "SELECT Username, UNIX_TIMESTAMP(LastLogin) AS LastLogin FROM tblUsers WHERE UserID = ? LIMIT 1";
  list($username, $lastlogin) = $db->FetchRow($sql, "i", $userid);

replaced

  $sql = "SELECT Username, UNIX_TIMESTAMP(LastLogin) FROM tblUsers WHERE UserID = ? LIMIT 1";
  if ($query= $db->prepare($sql))
  {
    $query->bind_param("i", $userid);
    $query->execute();
    $query->bind_result($username, $lastlogin);
    $query->fetch();
    $query->close();
  }

3) Fetch multiple rows as objects using a prepared query:

  $sql = "SELECT UserID, Username FROM tblUsers WHERE LastLogin >= FROM_UNIXTIME(?)";
  $users = $db->FetchAll($sql, "i", $lastweek);
  foreach ($users as $user)
    echo "$user->UserID: $user->Username";

replaced

  $sql = "SELECT UserID, Username FROM tblUsers WHERE LastLogin >= FROM_UNIXTIME(?)";
  if ($query = $db->prepare($sql))
  {
    $query->bind_parem("i", $lastweek);
    $query->execute();
    $query->bind_result($userid, $username);
    while ($query->fetch())
      echo "$userid: $username";
    $query->close();
  }

This last item works extremely well for small result sets, however for large sets of data (eg if we had a very active website with, say 1,000,000 users who logged in within the last week), we’d quickly run into problems. To get around such situations, I’ve added an additional method to my extended MySQLi class:

4) Iterate over multiple rows in a result set from a prepared query:

  $sql = "SELECT UserID, Username FROM tblUsers WHERE LastLogin >= FROM_UNIXTIME(?)";
  while ($user = $db->FetchObj($sql, "i", $lastweek))
    echo "$user->UserID: $user->Username";

This method iterates over the result set and returns an object which exposes properties with the same names as the fields selected from the database. When the last record in the result set has been returned, clean-up code is automatically and transparently called to close the result set.

I’ve been using this code extensively over the last few weeks, and have not found any problems so far. YMMV, but feel free to test it out and drop me a line if you find any bugs, or have any suggestions for improvements.

Download the source code here

Update:
One user has commented that he kept getting “bind_param failed” messages when using table joins. This is most likely because he was selecting two fields with the same name eg:

SELECT A.UserID, B.UserID FROM tblUsers A INNER JOIN tblUsers B ON A.UserID = B.UserID

In this case, the two identical field names cannot be resolved, and causes the code to raise an error. To fix this, give your field names an alias where required:

SELECT A.UserID, B.UserID AS UserID_B FROM tblUsers A INNER JOIN tblUsers B ON A.UserID = B.UserID

Killer apps are everywhere these days. Just type “killer app x” into Google (where x could be almost anything computer related – some suggestions are internet, web, desktop, linux, windows) are you’ll find plenty of things claiming to be killer apps. Today I want to talk about a killer app for your desktop. No – this is not a belated article about Compiz Fusion, nor is it anything to do with the desktop presented to you by your operating system. This is about the desktop on which your keyboard rests.

If, like me, you have two computers running at once, you’ll probably also have two keyboards and two mice with which to control them. I currently run Windows and Ubuntu boxes, and have struggled for over a year with a cramped work environment containing two meeces^* and keyboards. Sure, you could buy a cheap KVM switch of some sort, but having to press a button whenever I want to swap from one machine to the other? No thanks.

Thankfully, I stumbled across this article on the Tombuntu website a few days ago. I initially dismissed the idea of using Synergy thinking that it would be a *nix thing only, but looking more closely at their website I saw it was cross platform. “Can’t hurt to try,” I thought, so I installed it and had it running as a host on my Ubuntu box and client on my Windows machine.

The verdict? Nothing short of brilliant. It just works, with no fuss or hassle. There are some nice little touches too – for example Synergy smoothly scales the mouse position at the interface of two monitors of different resolutions. The clipboard synchronisation is a god-send (I no longer have to save things in text files and copy them from machine to machine!).

It’s not completely flawless though. I’ve found a few little issues (for some reason Gnome-Do seems to trap my mouse in the Linux monitor, and very occasionally, there’s a small lag between moving the cursor from the host to the client), however these things I can certainly live with. What Synergy has done for me is quite brilliant – the seamless communication between a host running on Linux, and a client running on Windows has enabled me to completely ditch a keyboard and mouse.

Where I once had a keyboard and mouse, I can now clutter my desktop with old CD’s, bit’s of paper and random desktop knick-naks.

^* I know, it’s not really a word.

Synopsis

Presenting a simple extension for MySQLi that provides convenience functions similar to mysql->fetch_array() and mysql->fetch_assoc() for prepared MySQL statements.

The Problem

It’s the middle of 2008, and web servers everywhere are still being hacked with simple SQL injection attacks. If you’re a web developer, and you don’t know what an SQL injection attack is, you should not write a single line of code until you have read and fully understood this. Allowing an SQL injection attack to occur in your code is just plain lazy, and you can do better.

Since about 50% of all web servers are powered by Apache, it’s a fairly safe bet that the majority of the dynamic sites they host run PHP and MySQL. These free tools make it easy to produce interactive, database-driven websites even for non-programmers, but unfortunately make it too easy for anyone (developer or not) to generate code which opens up your site to SQL injection possibilities.

Before the advent of MySQLi and prepared statements, MySQL statements in PHP queries to a MySQL database would look like:

/*
 * Example 1
 */
$sql ="SELECT ID, Firstname FROM Users WHERE Username = 'myusername' AND Password = SHA1(CONCAT('mypasswordsalt', 'mypassword'))";

(By the way, you’re not storing passwords in plain text, are you? You are adding a salt and storing the hashed password? If not, this explains what you are risking.)

So in PHP, your code might look something like:

/*
 * Example 2
 */
  $username = $_POST["username"];
  $password = $_POST["password"];
  $sql = sprintf("SELECT ID, Firstname FROM Users WHERE Username = `%s` AND Password = SHA1(CONCAT(Salt, `%s`))", mysql_real_escape_string($username, $password);
  $query = mysql_query($sql);
  // etc

Read the rest of this entry »

After being in the web development game for all these years, it was about time I had a site of my own. So for all your web development needs: CodeSense

I particularly like the 404 page myself )