Jasny · web development

Turning links like www.example.com and http://twitter.com into clickable links. Sounds like an easy task, right? We’ll there are a few problems that might arise, especially if the text is already HTML formatted.

This function first takes out all potential dangers, by extracting links and tags and replacing them with a placeholder. It than extracts all URLs and replaces them with a placeholder, storing the full HTML link. At the end it replaces all placeholders with the links and tags.
Read the rest of this post »

A backdoor provides access to an application bypassing the normal authentication process. There are many ways to do this. Some are more secure than others.

Why do you need a backdoor?

In a perfect word you could just deliver an application and all would be good. However in the real world there are unforeseen issues which need to be solved. This means that you as a developer will need access to the application. To reproduce the problem, you usually want to run the application logged in as the user that spotted the issue.

Another use of the backdoor is in a situation where you want to allow a user, that has already been authenticated, to bypassing further authentication. For example if you have a (web hosting) control panel where the user is already logged in, you can allow him to directly access the dashboard of the application without have to enter his password again. This requires a backdoor, since you don’t know his (unencrypted) password.
Read the rest of this post »

Since a few months I’ve done away with using the Gnome main menu. Instead I use Gnome Do. I removed the bottom toolbar long ago, because always use alt-tab.

I’m not using the top toolbar much either. It was just taking up valuable screen space. I contains only the notification area and a logout button. I was looking at a way to remove it completely. The answer came in the Compiz widget layer. By placing it on the widget layer, fullsize windows actually fill the full screen, but the notification area is still available for applications who need it.

To move Gnome panel to the Widget layer, open ‘CompizConfig Settings Manager’ and enable ‘Widget Layer’. Go to tab ‘Behaviour’ and add the following text for the ‘Widget Windows’ field:

(class=Gnome-panel & type=Dock)

The desktop will now be completely clean:

With we can display the widget layer, where the panel is found:

PS. The widgets you see on the widget layer are screenlets. Ubuntu has the screenlets package in the universe repository.

I’ve seen a lot of methods used to take a website temporarily off-line for maintenance. Most involve a using PHP to disable the site or renaming the index file. There is however a far better method of doing this, by placing the following in the vhost file or in an .htaccess file in the document root:

Header always set Retry-After "Thu, 18 Jun 2009 08:00:00 +0200"
Redirect 503 /

This way you are sure no part of the site is used. Also by returning a 503 http response, search-engine crawlers will not reindex your site right at the moment it is down. You can use ‘ErrorDocument’ to place a different text than the apache default.

Associated websites often share user information, so a visitor only has to register once and can use that username and password for all sites. A good example for this is Google. You can use you google account for GMail, Blogger, iGoogle, google code, etc. This is nice, but it would be even nicer if logging in for GMail would mean I’m also logged in for the other websites. For that you need to implement single sign-on (SSO).

There are many single sign-on applications and protocols. Most of these are fairly complex. Applications often come with full user management solutions. This makes them difficult to integrate. Most solutions also don’t work well with AJAX, because redirection is used to let the visitor log in at the SSO server.

I’ve written a simple single sign-on solution (400 lines of code), which works by linking sessions. This solutions works for normal websites as well as AJAX sites.
Read the rest of this post »

Simple replacement
String replacement is often used as a way to apply templating. You might replace “%a:test” with “~~test~~” using the regexp: %a:(\w+), replacing it with “~~$1~~”.

Trying to escape
The only problem now, is that I can’t use “%a:” any more within my string. This could be solved by allowing escaping using the backslash. In the regexp we can use a negative lookbehind to see if the character before the % isn’t a backslash: (?<!\\)%a:(\w+).

Escaping the escaping
Now we’re close, however now it’s not possible to use “\%:a” anywhere. We need to be able to escape the backslash as well. We could state the problem as needing to match %a if there isn’t an uneven number of backslashes in front of it. Checking for an uneven number in a negative lookbehind isn’t possible unfortunately, so we need to get the backslashes into the match. We can say: match 0 or more pairs of backslashes, followed by “%a:”, if there is no backslash in front of it. This results in the regexp:
(?<!\\)((?:\\{2})*+)%a:(\w+), replacing it for “$1~~$2~~”.

To finish up
To only thing is that \% and \\ will still be displayed as that. This can simply be solved with a str_replace.

Data stored in a database is often also useful for statistical purposes. If you own a web-shop you want to be able to create a report about turnover. You can get statistical information by using GROUP BY, eg.

SELECT DATE_FORMAT(invoice.date, '%M') AS `month`, COUNT(*) AS `invoice_count`, SUM(`invoice`.`amount`) AS `turnover`
FROM `invoice`
WHERE `date` BETWEEN '2008-01-01' AND '2008-12-31'
GROUP BY MONTH(`invoice`.`date`)

month     invoice_count  turnover
January   84             9532.26
February  141            20857.61
March     91             10922.71
April     112            15044.48
May       101            9676.60 
June      137            12860.88
July      281            34291.20
August    191            26377.66
September 103            16324.78
October   99             12873.23

If you are selling a wide variety of products, you might like to see the turnover for each product category. You could do this with a simple GROUP BY as:

SELECT DATE_FORMAT(`invoice`.`date`, '%M') AS `month`, `category`.`description` AS `category`, COUNT(*) AS `product_count`, SUM(`invoice_product`.`amount`) AS `turnover`
FROM `invoice` INNER JOIN `invoice_product` ON `invoice`.`id` = `invoice_product`.`invoice_id` LEFT JOIN `product` ON `invoice_product`.`product_id` = `product`.`id` LEFT JOIN `category` ON `product`.`category_id` = `category`.`id`
WHERE `date` BETWEEN '2008-01-01' AND '2008-12-31' 
GROUP BY MONTH(`invoice`.`date`), `category`.`id`

month     category   product_count  turnover   
January   Hardware   62             4821.31   
January   Software   51             4419.41   
January   Cables     12             291.54   
February  Hardware   71             8408.93   
February  Software   101            11726.36   
February  Cables     17             312.32   
February  Other      2              410.00   
March     Hardware   21             2371.58   
March     Software   81             8238.81    
March     Cables     13             312.32   
... 

This would give you each category in a different row, ordered by month. Though this contains all the information the format is far from nice. Instead you would like to have 1 row per month with each category as a column as the information about the invoices as well.

SELECT DATE_FORMAT(`invoice`.`date`, '%M') AS `month`, COUNT(DISTINCT `invoice`.`id`) AS `product_count`, COUNT(*) AS `invoice_count`, SUM(`invoice_product`.`amount`) AS `turnover`,
  SUM(`product`.`category_id`=1) AS `hardware_count`, SUM(IF(`product`.`category_id`=1, `invoice_product`.`amount`, 0)) AS `hardware_turnover`,
  SUM(`product`.`category_id`=2) AS `software_count`, SUM(IF(`product`.`category_id`=2, `invoice_product`.`amount`, 0)) AS `software_turnover`,
  SUM(`product`.`category_id`=3) AS `cables_count`, SUM(IF(`product`.`category_id`=3, `invoice_product`.`amount`, 0)) AS `cables_turnover`,
  SUM(`product`.`category_id`=4) AS `other_count`, SUM(IF(`product`.`category_id`=4, `invoice_product`.`amount`, 0)) AS `other_turnover`
FROM `invoice` INNER JOIN `invoice_product` ON `invoice`.`id` = `invoice_product`.`invoice_id` LEFT JOIN `product` ON `invoice_product`.`product_id` = `product`.`id` LEFT JOIN `category` ON `product`.`category_id` = `category`.`id`
WHERE `date` BETWEEN '2008-01-01' AND '2008-12-31'
GROUP BY MONTH(`invoice`.`date`), `category`.`id`

month     invoice_count  turnover    hardware_count  hardware_turnover  software_count  software_turnover  cables_count  cables_turnover  other_count  other_turnover
January   84             9532.26     62              4821.31            51              4419.41            12            291.54           0            0
February  141            20857.61    71              8408.93            101             11726.36           17            312.32           2            410.00
March     91             10922.71    21              2371.58            81              8238.81            13            312.32           0            0
...

The big downside of this method is that you need to modify the query if a category is added. This can be solved though by dynamically creating the query in a PHP script.

If you want to do advanced statistical you should have a look at OLAP cubes. Pentaho is an open-source reporting app which supports MySQL. http://mondrian.pentaho.org/

All data is fictional. The SQL queries are untested.

How to get a file extension in PHP:

1

$ext = pathinfo($file_name, PATHINFO_EXTENSION);

$ext = pathinfo($file_name, PATHINFO_EXTENSION);

How to get a file extension in Perl:

1

my $ext = ($file_name =~ m/([^.]+)$/)[0];

my $ext = ($file_name =~ m/([^.]+)$/)[0];

How to get a file extension in Ruby:

1

ext = File.extname(file_name)

ext = File.extname(file_name)

How to get a file extension in Bash:

ext=${file_name##*.}
name=${file_name%.*}

How to get a file extension in Python (thanks to Jensen):

import os
ext = os.path.splitext(file_name)[1]

How to get a file extension in JavaScript:

1

var ext = /\.(\w+)$/.exec(file_name)[1]

var ext = /\.(\w+)$/.exec(file_name)[1]

—
Got more? Please post a comment.

It’s time for a new article in the ‘How I PHP’ series, which are about methods I use, but aren’t commonly known. This time it is about an Apache module ‘mod_xsendfile’.

Normally when want to let a user download a file, you simply stick it in a dir under the document root and let Apache do the rest.

However in some cases that is not good enough. You might need to do some authenticate first or you need to lookup the actual file name. In that case you would use PHP, which would result in a script looking like this:

1
2
3
4
5
6
7
8
9

authenticate(); # authenticate and authorize, redirect/exit if failed
$file = determine_file();
 
if (!file_exists($file)) trigger_error("File '$file' doesn't exist.", E_USER_ERROR);
 
header("Content-type: application/octet-stream");
header('Content-Disposition: attachment; filename="' . basename($file) . '"');
header("Content-Length: ". filesize($file));
readfile($file);

authenticate(); # authenticate and authorize, redirect/exit if failed
$file = determine_file();

if (!file_exists($file)) trigger_error("File '$file' doesn't exist.", E_USER_ERROR);

header("Content-type: application/octet-stream");
header('Content-Disposition: attachment; filename="' . basename($file) . '"');
header("Content-Length: ". filesize($file));
readfile($file);

This means PHP has to read in the file, which goes through the output buffer, is flushed to Apache and processed before send to client. In this small I didn’t specify any other headers Apache normally sends like last-modified. If I want to actually make the caching based on last-modified work, I need to check the if-modified-since request header, check the mtime of the file and send a 304 result header. (I’m to lazy right now to write a code example for that, sorry)

Wouldn’t it be nicer to tell Apache, please send that file, and be done with it. Well, you can. When you enable mod_xsendfile in Apache, you can send an X-SendFile header, which is processed by Apache.

1
2
3
4
5
6

authenticate(); # authenticate and authorize, redirect/exit if failed
$file = determine_file();
 
header("X-Sendfile: $somefile");
header("Content-type: application/octet-stream");
header('Content-Disposition: attachment; filename="' . basename($file) . '"');

authenticate(); # authenticate and authorize, redirect/exit if failed
$file = determine_file();

header("X-Sendfile: $somefile");
header("Content-type: application/octet-stream");
header('Content-Disposition: attachment; filename="' . basename($file) . '"');

Note that this technique was copied from Lighttp, so if you’re using that it will also work.

The normal way to save a state, like the fact that a user is logged in, is to use sessions. Session work really nice, you can save all kind of data on the server (in a temp file) which is identified by a hash which is known on the client in a cookie or by url rewriting.

However if you only need to save a small bit of login info, like a username and timestamp, using sessions is a bit of an overkill. Also, if your system would grow beyond to a size where you need load balancing over multiple servers, having a solution with sessions would be somewhat of a burden. You would need to store them in a centralized place, like a DB server.

Another way to do this, is to create an authentication hash. The idea is fairly simple: Join the information you want to known into a single string and append an md5 key of all the info + a secret word. On each request, check if the hash is available and correct, otherwise redirect the user to a login form. You can use the hash the same way PHP uses the session hash, using cookies or URL rewriting.