I’ve been separately planning to make a barbeque sauce and a taco sauce. Putting the chaining principle to work, I realised that they both have (or at least can have) a common base – tomato sauce (aka ketchup). So rather than make each from scratch, I figured I’d make a tomato sauce first, then add to it as necessary to make the barbeque and taco sauces. In doing so, I was able to make the three sauces (tomato, bbq and taco), with less effort than it would have been to make just two sauces (bbq and taco) separately.

The flip side of this is that you need to ensure the base is good, otherwise you could end up spoiling both sauces. A ketchup-style sauce is made by reducing a large quantity of tomato then flavouring with various spices, a bit of vinegar and some sugar. There is a lot of latitude in this for varying the added flavourings to a taste you like. But I’ve found the recipe below produces both a very enjoyable tomato ketchup, and makes a great base other tomato-based sauces, like the Kansas City barbeque sauce and the taco sauce below.

Tomato Sauce (Ketchup)

A rich, flavoursome tomato sauce. Feel free to use whatever tomatoes you have at hand (canned, vine ripened, cherry, etc.). A mix of half canned and half fresh gives a good balance between a fresh taste and a ketchupy consistency.

• 1.5kg tomatoes (roughly chopped)
• 1/8th cup extra virgin olive oil
• 1/2 brown onion (chopped)
• 1 clove of garlic (minced)
• 1/4 cup brown sugar
• 200ml white-wine vinegar
• 1/2 tsp paprika
• 1 tsp cloves
• 1 tsp allspice
• 1 tsp cracked black pepper
• 1 tsp salt

Heat the oil in a large pan over a medium-high heat. Add onion and garlic and cook until starting to brown, 4-5 mins. Add the tomatoes. Cover, bring to a gentle boil, then reduce the heat and simmer until the tomatoes are very tender (approx 1hr). Cool slightly, transfer into a blender and process until smooth. Return to a clean pan and stir in the vinegar and sugar. Add in the spices and seasoning. Then continue to simmer until it has reduced down to the consistency of a standard ketchup (1-2hrs). Pass through a sieve to remove any remaining pulp. Decant into sterilized bottles and store in the fridge. Makes 3 cups of sauce.

Kansas City Barbeque Sauce

An intense, fiery sauce that’s not for the faint-hearted.

• 1 cup tomato sauce (recipe above)
• 1/4 cup water
• 1/4 cup vinegar
• 1/4 cup brown sugar
• 3 tbsp olive oil
• 2 tbsp smoked paprika
• 1 tbsp chilli
• 2 cloves garlic (minced)

• 1 cup tomato sauce (recipe above)
• 1/4 cup water
• 1/4 tsp chilli powder
• 1 tsp cumin
• 1 tbsp onion (chopped)
• 1 tbsp white vinegar
• 1/2 clove garlic (minced)
• 1/4 tsp salt
• 1/4 tsp paprika
• 1/4 tsp sugar
• 1/4 tsp chilli flakes

Blend ingredients together until smooth. Transfer to a pan and simmer for 15 mins. Cool and store in a sterilized bottle.

Overview

Sorbets despite their sophisticated appearance, have a notably unsophisticated base. At its most basic, a sorbet is little more than frozen sugar water. While tricks like adding alcohol or egg whites help make a smoother sorbet, a very acceptable result can be achieved without this extra complexity.

A typical ratio of water to sugar seems to be three units of water per unit of sugar. The water is heated to dissolve the sugar, and then cooled before freezing.

The only other thing that must be added is the flavor-giving ingredients themselves. The simple, neutral base makes it possible to concoct a wide-range of flavours. If these ingredients contain water or sugar themselves, it may be necessary to adjust the ratio used in the base.

The real magic in the sorbet is in how it is frozen. Rather than putting it in the freezer and forgetting about it, the sorbet should be stirred well while it is freezing. The purpose of this is to break up the large ice-crystals which form, resulting in a smoother sorbet. (At the other extreme, where large ice-crystals are actively sought, is the Italian dessert, granita). I find the easiest and most effective way to do this is to freeze the sorbet until it is firm, but not fully frozen, then blend it for 15s or so. The result is a very smooth, thick but runny sorbet which can be re-tubbed and returned to the freezer to freeze completely.

Here are the recipes for the sorbets I have made.

Dark Chocolate Sorbet

A rich, decadent dark chocolate sorbet balancing bitterness and sweetness.

• 1 cup sugar
• 3/4 cup cocoa powder
• pinch of salt
• 170g dark chocolate (70% cocoa is ideal)
• 1/2 tsp vanilla extract

In a large pan, combine the the sugar, cocoa powder, salt and 1 1/2 cups of water. Bring to the boil, while stirring. Remove from the heat after 45s. Then add the chocolate and stir until fully melted. Add the vanilla extract and 3/4 cup of water. Blend the mixture for 15s. Allow to cool, then transfer to a tub and place in the freezer until firm, but not fully frozen. Remove from the freezer, blend until smooth and then return to the freezer to finish freezing. When ready to eat, remove from the freezer and sit for 10 mins to soften before serving.

Mixed Berry and Watermelon Sorbet

A light, refreshing sorbet.

• 2 cups watermelon, diced
• 2 cups mixed berries
• 1/2 cup sugar
• 1 tbsp fresh mint, finely chopped

In a large pan, combine the the sugar with 1 cup of water. Heat, stirring until the sugar has fully dissolved. Remove from the heat and allow to cool slightly. Tip mixture into a blender and add the watermelon and berries. Blend until smooth (2-3mins), then sieve into a tub to remove the pips. Allow to cool, then transfer to a tub and place in the freezer. Stir after 1 hour, adding the mint as you do so. Return to the freezer until firm, but not fully frozen. Remove from the freezer, blend until smooth and then return to the freezer to finish freezing. When ready to eat, remove from the freezer and sit for 10 mins to soften before serving.

Lemon, Lime and Bitters Sorbet

A delicious twist on a traditional lemon sorbet.

• Juice of two limes
• Juice of four lemons
• 1/2 cup white wine
• 2 tbsp Angostura bitters
• 2/3 cup sugar

Combine the lime juice, lemon juice, wine, bitters, sugar and 1/2 cup of water in a small pan. Heat, stirring until the sugar has fully dissolved. Then remove from the heat and allow to cool slightly. Transfer to a tub and place in the freezer until firm, but not fully frozen. Remove from the freezer, blend until smooth and then return to the freezer to finish freezing. When ready to eat, remove from the freezer and sit for 10 mins to soften before serving.

Overview

The essential ingredient in any mustard is, of course, the mustard seed. It comes in two basic forms: wholegrain (i.e. just the seed), and powder (i.e. ground seeds). The seeds come in several colours, namely the milder yellow seed and the more potent brown and black seeds. A wholegrain mustard will often mix yellow and brown seeds for balance and visual appeal.

To make the mustard proper, the seed is soaked in a cool or cold liquid, such as water, which kicks off a chemical reaction (between the myrosinase enzyme and glucosilinates present in the seeds) and produces the ‘heat’ of the mustard. The longer the reaction runs the hotter the resulting mustard will be. Vinegar and salt are usually used to control the reaction, and hence heat, and to enhance the flavour.

For wholegrain mustard, the heat-generating reaction will generally take 2-3 days to reach a suitable level and, after blending, the mustard may take several days (and sometimes upto several months) to ‘mature’ to its final taste. With mustard powder, the heat-generating reaction is a lot faster, taking a matter of hours. The powerfully anti-bacterial mustard seed when combined with vinegar and salt will be very resiliant and long-lasting. Note, though, that the addition of short-lived flavour ingredients such as fresh herbs will reduce the shelf-life of the mustard.

Flavours can be introduced in the liquid used, for instance swapping the vinegar for a flavoured variety (white wine, apple cider) or an alcohol (such as whisky or beer) with enough vinegar (or other acid) added to attain and balance the desired heat. Alternatively, or additionally, flavours can be mixed in at the blending stage, for instance adding herbs, seasoning and spices. A sweetener, such as sugar or honey, is often added as well. The combinations which can be made are practically (and perhaps actually) limitless.

Here’s the recipe I used, along with the three variants I made. As I had ran out of black mustard seeds by the time I got to the red wine mustard, I used 4 tbsp yellow mustard seeds instead.

Wholegrain Mustard

A basic wholegrain mustard, from which endless variations can be produced.

• 2 tbsp mustard seeds (yellow)
• 2 tbsp mustard seeds (black)
• 100ml white wine vinegar
• 1 tsp sugar
• 1/2 tsp salt

Mix the mustard seeds and vinegar in a non-metallic bowl. Cover with cling-film and leave to stand for 2-3 days (3 days for a hotter mustard). Check occasionally and add extra vinegar if it has all been absorbed.

Then, drain any excess liquid and set aside. In a mortar and pestle (or blender), blend the seeds into a paste. Blend in the salt and sugar, along with some of the reserved liquid (if required) to get the right consistency.

Decant the resulting mustard into a clean, sterilised jar. Cap and leave to mature for 2 days prior to use.

Dark Ale and Honey Mustard

Replace the white wine vinegar with 100ml of your favourite dark ale, mixed with 2 tsp of white vinegar. Replace the sugar with 1 tsp of honey.

Spicy Whisky Mustard

Replace the white wine vinegar with a mixture of 50ml whisky, 50ml white wine vinegar and 1/2 tsp chilli flakes. Replace the sugar with 1 tsp honey and 1 tsp nutmeg.

Red wine, Rosemary and Thyme Mustard

Replace the white wine vinegar with a mixture of 50ml water, 50ml red wine vinegar, 2 tsp dried thyme and 2 tsp dried rosemary. Use brown sugar in place of normal sugar.

Content:

What topics are they covering?

According to their own tag line: design, business, experience, simplicity, the web, culture, and more.

What topics are they ignoring?

Despite being an online business, they don’t seem to write much about (or do) much in the way of traditional SEO / link building, etc.

What voice/style do they post in?

Posts are generally informal, with a cut-the-crap attitude.

How often are they posting?

Once or twice a week on average.

What level are they pitching their posts at (beginners, advanced etc)

Their posts are typically high level, avoiding nitty gritty technical details. Even the technical articles tackle whatever specific technical issue at a high level.

Reader Engagement:

What topics generate most conversation?

It seems to be those that are somewhat political in nature, or relate to significant new product features.

What styles of posts seem to connect with readers best?

Posts of a technical nature seem to produce the most in-depth comments.

What questions are readers asking in comments?

Asking for further details on the topics discussed.

What complaints do you see readers making in comments?

Complaints about product features / changes required. Also complaints about nonsensical topics / lines of argument.

What tools/mediums is the site using (eg: are they using Twitter, forums etc)

Twitter widget showing latest tweets. RSS feed and stats counter. Comments. Newsletter signup. Site search.

Design:

What first impression does their design give?

Clean, colourful.

What have they done well? What have they done poorly?

Good emphasis / de-emphasis / elimination of page elements. Somewhat confusing ‘mini-posts’ which visually appear to belong to the major post above, but which I don’t think actually do. RSS feed blends with ad block – I wasn’t sure if it was part of the ad, or part of the site, until I hovered over it to see the URL.

What Options do they give readers to subscribe?

Newsletter, RSS, link to follow on twitter.

Monetization:

what advertisers are targeting this blog?

IT ads.

what type of affiliate programs are they promoting?

None that I can see, but promote their own web apps and books quite prominently.

Traffic:
3,412 on Alexa for the entire 37signals domain, with 16k sites linking in. Traffic has decreased sightly (by around 10%) over the past 3 months.

SEO:

Who is linking to this blog? (use the link:www.blogurl.com command in Google to find out)

Lots of diverse sites, especially tech / design blogs.

What does their source code reveal about how they’ve set up their site

Very clean HTML source, with defined meta tags, etc.

If they have an open or unlocked stats package what can you learn from their stats? What pages are popular? Where does their incoming traffic come from?

Not available.

I’ve read many books, some good and some a waste of time. If you’re looking for a great book, here is my pick of the very best (predominantly web-related) business books of the year:

1. The Lean Startup - Eric Ries introduces the lean startup methodology, a way to bring to web businesses the massive success companies manufacturing companies such as Toyota have had with lean manufacturing. Ries covers achieving competitive advantage through rapid product iterations and the use of ‘validated learning’. I consider this a must-read for anyone running a web-business. I’m already working to implement, and starting to benefit from, the many techniques it covers.

2. Evil Plans - Cartoonist Hugh MacLeod’s book is a brilliantly easy read. While not an in-depth how-to book, it is simply packed full of motivation. I challenge anyone to read it and not be inspired to start blogging, cartooning or ‘doing something you love’.

3. Unlikely Destinations: The Lonely Planet Story - I struck gold with this one while trawling through a stack of (incomprehensible to me) Vietnamese books at a book stand in Ho Chi Minh airport. It is the story of how Tony and Maureen Wheeler started the Lonely Planet travel book behemoth, almost by accident while trying to fund their round-the-world travels. Part travel diary, part business book, it makes for great reading.

4. Making it All Work - In this follow-up to his wildly successful book, Getting Things Done, David Allen expands on and generalises on the principles of his work/life management system. Definitely worth a read for anyone trying to implement their own GTD system.

5. Great Ikea! - While the furniture giant may now be in the ‘cashing in’ phase of the business life-cycle, Elen Lewis’ book documents how the company rose to dominate the world of flat-pack goods by focusing tirelessly on delivering brilliantly designed products at the lowest prices possible.

6. Delivering Happiness - In delivering happiness, Zappos CEO Tony Hsieh tells of his path to Zappos and how this lead to the online shoe retailer focusing explicitly on the happiness of its customers, employees and partners as a business goal more important than profit. By prioritising happiness in this way, Zappos was rapidly able to transform itself into a billion-dollar organisation with aims much higher than just selling shoes.

7. The Art of Learning - Not strictly a business book, but highly relevant none the less. The Art of Learning documents Josh Waitzkin’s life as a childhood chess prodigy and subsequent reinvention as world champion in the combat form of Tai chi chuan. With the tagline ‘an inner journey to optimal performance’, Josh delivers a lucid description of the techniques he uses to achieve mastery in both highly competitive and complex disciplines. The lessons have clear applications to the world of business.

8. Rework - Written by the guys at 37signals (responsible for a number of highly successful web applications such as basecamp), this book is a collection of unconventional tips for building web-businesses. With tips such as ‘underdo the competition’ and ‘welcome obscurity’, it is full of insight into building better web applications with less stress and less overheads.

9. The Facebook Effect - This is the inside story into the phenomenon that is Facebook (what, never heard of it?). From the early beginnings to managing a company with hundreds of millions of users, the book covers it all in depth. If you’ve seen the ‘social network’ movie, I’ll just say this – the book is better.

10. Founders at Work - In founders at work, Jessica Livingstone (co-founder of startup incubator Y-combinator) brings an impressive collection of interviews with founders of some of the biggest and most successful companies and products around: Apple, Yahoo!, Gmail, Hotmail, Research in Motion, Craigslist, del.icio.us and many more. The interviews discuss the early days of these startup ventures, what they did, and ultimately how they managed to achieve success with the odds stacked against them.

If you are at all interested in running a business, particularly a web-based one, I highly recommend taking a look at whichever of these caught your eye. They won’t disappoint.

New Design Launched – I got up at 4am to put the finishing touches to a complete redesign of the site. By the time I had to leave for my day job, it was up and running. I think the design improves significantly on the previous version, but you can judge for yourself at www.calcatraz.com (oh, and let me know what you think using the feedback form…).

User Accounts Launched - On my lunch break I forewent the traditional pub lunch, and took my laptop down to a cafe where I nutted out the last of the code for handling user accounts. Then immediately after work I headed back to the cafe and pushed the code live. So you can now sign up for a free Calcatraz Account which gives you permanent access to your full calculation history and touch-typing history (on the Calculator and Typist apps respectively).

Melbourne Lean Startup Meeting - Calcatraz then took its first steps into the wider world at last night’s lean startup meeting. The night featured awesome talks from the NineFold, RomeToRio and Kaggle guys. It was also great to talk to people doing some really interesting stuff in the startup space, including Peter, Daryl, Jai from UnBaa (a social decision making site), and Anita from TechSeeder. Cheers guys!

So yeah, quite a day overall! P.S. Here’s the new design:

To establish a baseline, I looked at the fastest page load time possible on my local host. This was around 12ms avg for a minimal html page (containing the word ‘test’ only) and around 15-17ms for a minimal php page. Note that I cleared the cache after each request. My aim then is to get the api working on localhost at something approaching these times. Any additional delay in the production environment would presumably be down to dns / latency / shared hosting, which I can deal with separately.

I then tested the api for its response time. A simple calculation such as 3*3 gave a response time of 250-320ms. Not bad, but not as good as I would like. As a start, I aimed to reduce it below 50ms. This should be done with improvements which benefit all calculations, rather than being achieved with caching of previously seen answers (which I want to have as a separate speedup).

Output Buffering

Using webgrind, I profiled the call. The call took 406ms this time (presumably slowed by the profiling code). It showed that the slowest individual function call (i.e. self-cost) was the single database call, taking 147ms. This call is used to simply log the calculation made – it isn’t actually used in the production of the calculation. As a result, it can be moved to the end of the code and executed after the output has been sent by the user. That way, though the time to run the php script hasn’t improved, the time until the user gets their answer has.

I did this, using the following PHP code to send the output buffer contents to the user and close the connection:

// Initialise content buffering (using compression)
ob_start(‘ob_gzhandler’);

… code to generate user output goes here …

// close the http connection and send the data to the user
echo str_pad(”, 1024); // pad so IE and Chrome will render short answers
header(“Connection: close”);
header(“Content-Length: “.ob_get_length());
ob_end_flush(); // send the results to the user
ob_flush();
flush();

… code to process behind the scenes goes here …

The result of this change was that the response time dropped from 250-320ms to around 110ms – more than halving the response time.

File Inclusion

Going back to the webgrind profile output, I found that the next largest bottleneck was a bunch of file inclusions I was doing at the start of the code. These include library functions which I store with one file per function. The time taken to include the files seems to vary between about 30 and 60ms.

Originally the files were included using the include_once() function. I changed this to the include() function to remove the (unnecessary) check for a previous inclusion. This seemed to slightly reduce the time to load to 25-30ms. While I will probably inline the functions at a later time, having them separate makes development easier. So until I sort out a solution I’ll leave it as it is.

Function Caching

The next target for improvement was a function which consumed around 20ms per page load. Reviewing the function revealed that while caching was in use on the function return values, not all return paths were being cached – essentially the function was traversing a tree structure and caching the nodes but not the leaves. By adding caching I was able to cut the time used by the function to around 8ms. With the changes so far, the time for the api call has now dropped from 250-320ms to around 65-90ms. As an added bonus, I was able to make use of PHP static variables to replace the previous, more clumsy, caching method (I learned how to do this while studying the Drupal codebase). And as a further bonus I was able to remove two of the library functions being included (as discussed above), very slightly addressing that problem.

Dead Code Elimination

While it might seem excessive, I continued to try and further reduce the time towards my 50ms goal. I took a look at one frequently called function which was using around 10ms. It was being called from two locations. The vast majority of the calls were coming from a function which, on inspection, turned out not to actually need to call it. Removing this call all but eliminated the 10ms usage.

File Inclusion Revisited

By this point, the file inclusion issue below has become the clear bottleneck, accounting for over a third of the total runtime of the API call. So I decided to bite the bullet and tackle it. I realised that Toyota-style lean principles suggest that I should combine the many files into fewer as the multiple calls do not add value to the user and are therefore ‘waste’. While it benefits me as a developer, there are other ways I can achieve the same effect. A short-term approximation is to use the ‘fold all’ feature of notepad++ to collapse the functions into a browseable list. If I need something more closely resembling explorer’s view of the files (e.g. so I can work on two or more functions in a tabbed view, I can easily write something which will let me do it. I combined the functions into four larger library files (corresponding to the four directories the function files were stored in), and changed the code to include just these. Immediately the time cost of loading the functions dropped form around 70ms to around 7ms. I could take it further and combine these four files, but I think it would complicate development more than it would benefit runtime.

Conclusion

Through a series of optimisations I was able to reduce the time the server takes to feed the user a response to a simple API query from 250-320ms to only 45ms. This was done exclusively through optimisations which simplified the codebase, or just moved things around in a way that will have minimal impact on ease of development. While other potential optimisations exist to reduce the time requirement further, they will require more involved development work and so are not going to be carried out at this time.

I’ve installed this combination on several web servers in the past and it has always proved a bit more time-consuming and painful than it really should be. To save myself, and hopefully others, from similar pain in the future, here are the steps I used this time round. Note that these instructions apply to installing XDebug and Webgrind for use with XAMPP on Windows.

1. Create a php file under your localhost webroot containing the command:

<?php phpinfo() ?>

2. Open the file in a browser – it should display a load of configuration about your php setup.

3. Select and copy all the content of that page

4. Paste it into the textbox on this page: http://www.xdebug.org/find-binary.php

5. Click ‘Analyse my phpinfo() output

6. This will generate some output like:

Summary
Xdebug installed: no
Server API: Apache 2.0 Handler
Windows: yes – Compiler: MS VC6 – Architecture: x86
Zend Server: no
PHP Version: 5.3.5
Zend API nr: 220090626
PHP API nr: 20090626
Debug Build: no
Thread Safe Build: yes
Configuration File Path: C:\Windows
Configuration File: C:\Program Files\xampp\php\php.ini
Extensions directory: C:\Program

Instructions

Download php_xdebug-2.1.2-5.3-vc6.dll
Move the downloaded file to C:\Program
Edit C:\Program Files\xampp\php\php.ini and add the line
zend_extension = C:\Program\php_xdebug-2.1.2-5.3-vc6.dll
Restart the webserver

7. Before following these instructions, note that they are wrong! The extension dir path has been truncated. You will need to locate your extension dir and update the instructions accordingly. For instance, my extension dir is at: C:\Program Files\xampp\php\ext

This gives the revised instructions:

Download php_xdebug-2.1.2-5.3-vc6.dll
Move the downloaded file to C:\Program Files\xampp\php\ext\
Edit C:\Program Files\xampp\php\php.ini and add the line
zend_extension = “C:\Program Files\xampp\php\ext\php_xdebug-2.1.2-5.3-vc6.dll”
Restart the webserver

8. Follow these instructions, but to save a bit of time you don’t yet need to restart the webserver. Also keep php.ini open.

9. Add the following to php.ini to run XDebug every time a page is requested:
xdebug.profiler_enable = 1

10. Still in php.ini, Change the line:
xdebug.profiler_output_name = “xdebug_profile.%R::%u”
To:
xdebug.profiler_output_name = “xdebug_profile.%R.%u”

This is because windows can’t handle the colons in the file name, and won’t write the files otherwise. This took me a while to figure out.

11. Now restart the Apache web server

12. It should be working. To test, load (in a browser) the localhost page you want to profile. If all is well, XDebug will produce a file named xdebug_profile.xxxx.xxxx or similar in the XDebug output directory. (For me the directory is “C:\Program Files\xampp\tmp”, but you find / set the output dir from the php.ini line which looks like: xdebug.profiler_output_dir = “C:\Program Files\xampp\tmp”).

13. Install webgrind:

* Download webgrind from https://github.com/jokkedk/webgrind (use the ‘downloads’ button).
* Extract webgrind to a folder accessible to the webserver.
* Start using webgrind by browsing to the URL of your webgrind install, e.g. http://localhost/webgrind.

If all has gone well you should now be able to use webgrind to view the profile information for the page you loaded in step 12.

I’ve been working on an API for the new version of Calcatraz (both API and new version coming soon). As part of this, I’ve been trying to configure the API to use nice URLs. This has caused me to bump up against a tricky problem in Apache’s mod_rewrite module.

I want to be able to accept URLs containing a calculation such as:

http://www.calcatraz.com/api/calc/3*3.txt

And redirect it to a script which can extract the calculation (3*3) and the required format (in this case plain text ‘txt’).

The Basics

For the case above it can be done without problem, you just use a .htaccess redirect rule such as:

RewriteRule calc/ api.php [L]

which calls api.php to serve the page. Then in api.php you parse the $_SERVER['REQUEST_URI'] variable – which contains the original url – and extract the various parts. Alternatively you can get a bit more complex in mod_rewrite and do the redirect and parsing in one, with something like:

RewriteRule calc/(.+)\.(txt|xml)$ api.php?calc=$1&format=$2 [L]

The Problem

So far, so good. But this breaks down when the calculation contains characters with special meaning in URLs, such as ‘/’, ‘?’, etc. My initial thought was to require that the calculation is url-encoded before inclusion in the API URL. This works for the likes of ‘?’. Encoding a slash is a different matter. The encoded URL for 3/3 would be:

http://www.calcatraz.com/api/calc/3%2F3.txt

Here the %2F is the urlencoded form of ‘/’.

This seems like it should be passed through to the api.php script, but it isn’t. Instead a 404 not found page is found. It turns out that this is a known bug in Apache / mod_rewrite.

No matter what I tried, there seems to be no way to redirect an API url containing a %2F to a particular script.

The Solution(s)

1) What does work is to perform a double url-encoding of the calculation before placing it in an API URL. So the api call to get the answer for 3/3 would be:

http://www.calcatraz.com/api/calc/3%252f3.txt

This works, redirecting to the script and allowing us to extract the calculation and get its original form by performing a single url-decoding (as Apache / mod_rewrite performs one level of decoding implicitly). The main problem with it is that the URLs are pretty messy, and the user has to be asked to perform a double url-encoding. It also makes the URLs less obviously hackable / readable.

2) We can remove the need for the user to perform a double url-encoding by replacing it with a single base-64 encoding. This slightly tidies up the client-side API calls, but it means the URLs are completely inscrutable - the user can’t easily play around with them at all.

3) Since the slash will actually work without any encoding (it is only other characters as ‘?’ which must be encoded), we could create  a custom version of urlencode which encodes everything normally encoded, other than slashes. This would yield the most readable URLs, but significantly complicates things for the user.

4) Switch web servers. As discovered in this ServerFault discussion I’ve been having on the issue, while it affects Apache, it doesn’t affect some other servers such as nginx.

5) Update: Okay, thanks to the guys over at ServerFault, I now have the answer. By default Apache decodes encoded slashes (the problem). This behaviour can be disabled by adding the following to Apache’s httpd.conf file (or the relevant vhost configuration):

AllowEncodedSlashes On

Conclusion

I haven’t tested it on my public web server yet (only my dev server), but all being well Solution 5 is going to be the winner. Look out for the API, coming soon along with nice URLs, and a new version of Calcatraz calculator.

Side Note

Interestingly, it seems that WordPress also falls prey to this issue. I originally posted this article with the term %2F in the title. WordPress took this title and url-decoded it to produce the URL “slug” for the post – which now contained a slash. When the slug is used in the permalink of the page title it is URL encoded back up, and clicking on the link then requests a URL containing a %2F. Because wordpress makes use of redirects, then as seen before, the redirect fails and a 404 error is returned. So it was not possible to actually view the post!