Content:

What topics are they covering?

According to their own tag line: design, business, experience, simplicity, the web, culture, and more.

What topics are they ignoring?

Despite being an online business, they don’t seem to write much about (or do) much in the way of traditional SEO / link building, etc.

What voice/style do they post in?

Posts are generally informal, with a cut-the-crap attitude.

How often are they posting?

Once or twice a week on average.

What level are they pitching their posts at (beginners, advanced etc)

Their posts are typically high level, avoiding nitty gritty technical details. Even the technical articles tackle whatever specific technical issue at a high level.

Reader Engagement:

What topics generate most conversation?

It seems to be those that are somewhat political in nature, or relate to significant new product features.

What styles of posts seem to connect with readers best?

Posts of a technical nature seem to produce the most in-depth comments.

What questions are readers asking in comments?

Asking for further details on the topics discussed.

What complaints do you see readers making in comments?

Complaints about product features / changes required. Also complaints about nonsensical topics / lines of argument.

What tools/mediums is the site using (eg: are they using Twitter, forums etc)

Twitter widget showing latest tweets. RSS feed and stats counter. Comments. Newsletter signup. Site search.

Design:

What first impression does their design give?

Clean, colourful.

What have they done well? What have they done poorly?

Good emphasis / de-emphasis / elimination of page elements. Somewhat confusing ‘mini-posts’ which visually appear to belong to the major post above, but which I don’t think actually do. RSS feed blends with ad block – I wasn’t sure if it was part of the ad, or part of the site, until I hovered over it to see the URL.

What Options do they give readers to subscribe?

Newsletter, RSS, link to follow on twitter.

Monetization:

what advertisers are targeting this blog?

IT ads.

what type of affiliate programs are they promoting?

None that I can see, but promote their own web apps and books quite prominently.

Traffic:
3,412 on Alexa for the entire 37signals domain, with 16k sites linking in. Traffic has decreased sightly (by around 10%) over the past 3 months.

SEO:

Who is linking to this blog? (use the link:www.blogurl.com command in Google to find out)

Lots of diverse sites, especially tech / design blogs.

What does their source code reveal about how they’ve set up their site

Very clean HTML source, with defined meta tags, etc.

If they have an open or unlocked stats package what can you learn from their stats? What pages are popular? Where does their incoming traffic come from?

Not available.

To establish a baseline, I looked at the fastest page load time possible on my local host. This was around 12ms avg for a minimal html page (containing the word ‘test’ only) and around 15-17ms for a minimal php page. Note that I cleared the cache after each request. My aim then is to get the api working on localhost at something approaching these times. Any additional delay in the production environment would presumably be down to dns / latency / shared hosting, which I can deal with separately.

I then tested the api for its response time. A simple calculation such as 3*3 gave a response time of 250-320ms. Not bad, but not as good as I would like. As a start, I aimed to reduce it below 50ms. This should be done with improvements which benefit all calculations, rather than being achieved with caching of previously seen answers (which I want to have as a separate speedup).

Output Buffering

Using webgrind, I profiled the call. The call took 406ms this time (presumably slowed by the profiling code). It showed that the slowest individual function call (i.e. self-cost) was the single database call, taking 147ms. This call is used to simply log the calculation made – it isn’t actually used in the production of the calculation. As a result, it can be moved to the end of the code and executed after the output has been sent by the user. That way, though the time to run the php script hasn’t improved, the time until the user gets their answer has.

I did this, using the following PHP code to send the output buffer contents to the user and close the connection:

// Initialise content buffering (using compression)
ob_start(‘ob_gzhandler’);

… code to generate user output goes here …

// close the http connection and send the data to the user
echo str_pad(”, 1024); // pad so IE and Chrome will render short answers
header(“Connection: close”);
header(“Content-Length: “.ob_get_length());
ob_end_flush(); // send the results to the user
ob_flush();
flush();

… code to process behind the scenes goes here …

The result of this change was that the response time dropped from 250-320ms to around 110ms – more than halving the response time.

File Inclusion

Going back to the webgrind profile output, I found that the next largest bottleneck was a bunch of file inclusions I was doing at the start of the code. These include library functions which I store with one file per function. The time taken to include the files seems to vary between about 30 and 60ms.

Originally the files were included using the include_once() function. I changed this to the include() function to remove the (unnecessary) check for a previous inclusion. This seemed to slightly reduce the time to load to 25-30ms. While I will probably inline the functions at a later time, having them separate makes development easier. So until I sort out a solution I’ll leave it as it is.

Function Caching

The next target for improvement was a function which consumed around 20ms per page load. Reviewing the function revealed that while caching was in use on the function return values, not all return paths were being cached – essentially the function was traversing a tree structure and caching the nodes but not the leaves. By adding caching I was able to cut the time used by the function to around 8ms. With the changes so far, the time for the api call has now dropped from 250-320ms to around 65-90ms. As an added bonus, I was able to make use of PHP static variables to replace the previous, more clumsy, caching method (I learned how to do this while studying the Drupal codebase). And as a further bonus I was able to remove two of the library functions being included (as discussed above), very slightly addressing that problem.

Dead Code Elimination

While it might seem excessive, I continued to try and further reduce the time towards my 50ms goal. I took a look at one frequently called function which was using around 10ms. It was being called from two locations. The vast majority of the calls were coming from a function which, on inspection, turned out not to actually need to call it. Removing this call all but eliminated the 10ms usage.

File Inclusion Revisited

By this point, the file inclusion issue below has become the clear bottleneck, accounting for over a third of the total runtime of the API call. So I decided to bite the bullet and tackle it. I realised that Toyota-style lean principles suggest that I should combine the many files into fewer as the multiple calls do not add value to the user and are therefore ‘waste’. While it benefits me as a developer, there are other ways I can achieve the same effect. A short-term approximation is to use the ‘fold all’ feature of notepad++ to collapse the functions into a browseable list. If I need something more closely resembling explorer’s view of the files (e.g. so I can work on two or more functions in a tabbed view, I can easily write something which will let me do it. I combined the functions into four larger library files (corresponding to the four directories the function files were stored in), and changed the code to include just these. Immediately the time cost of loading the functions dropped form around 70ms to around 7ms. I could take it further and combine these four files, but I think it would complicate development more than it would benefit runtime.

Conclusion

Through a series of optimisations I was able to reduce the time the server takes to feed the user a response to a simple API query from 250-320ms to only 45ms. This was done exclusively through optimisations which simplified the codebase, or just moved things around in a way that will have minimal impact on ease of development. While other potential optimisations exist to reduce the time requirement further, they will require more involved development work and so are not going to be carried out at this time.

I’ve installed this combination on several web servers in the past and it has always proved a bit more time-consuming and painful than it really should be. To save myself, and hopefully others, from similar pain in the future, here are the steps I used this time round. Note that these instructions apply to installing XDebug and Webgrind for use with XAMPP on Windows.

1. Create a php file under your localhost webroot containing the command:

<?php phpinfo() ?>

2. Open the file in a browser – it should display a load of configuration about your php setup.

3. Select and copy all the content of that page

4. Paste it into the textbox on this page: http://www.xdebug.org/find-binary.php

5. Click ‘Analyse my phpinfo() output

6. This will generate some output like:

Summary
Xdebug installed: no
Server API: Apache 2.0 Handler
Windows: yes – Compiler: MS VC6 – Architecture: x86
Zend Server: no
PHP Version: 5.3.5
Zend API nr: 220090626
PHP API nr: 20090626
Debug Build: no
Thread Safe Build: yes
Configuration File Path: C:\Windows
Configuration File: C:\Program Files\xampp\php\php.ini
Extensions directory: C:\Program

Instructions

Download php_xdebug-2.1.2-5.3-vc6.dll
Move the downloaded file to C:\Program
Edit C:\Program Files\xampp\php\php.ini and add the line
zend_extension = C:\Program\php_xdebug-2.1.2-5.3-vc6.dll
Restart the webserver

7. Before following these instructions, note that they are wrong! The extension dir path has been truncated. You will need to locate your extension dir and update the instructions accordingly. For instance, my extension dir is at: C:\Program Files\xampp\php\ext

This gives the revised instructions:

Download php_xdebug-2.1.2-5.3-vc6.dll
Move the downloaded file to C:\Program Files\xampp\php\ext\
Edit C:\Program Files\xampp\php\php.ini and add the line
zend_extension = “C:\Program Files\xampp\php\ext\php_xdebug-2.1.2-5.3-vc6.dll”
Restart the webserver

8. Follow these instructions, but to save a bit of time you don’t yet need to restart the webserver. Also keep php.ini open.

9. Add the following to php.ini to run XDebug every time a page is requested:
xdebug.profiler_enable = 1

10. Still in php.ini, Change the line:
xdebug.profiler_output_name = “xdebug_profile.%R::%u”
To:
xdebug.profiler_output_name = “xdebug_profile.%R.%u”

This is because windows can’t handle the colons in the file name, and won’t write the files otherwise. This took me a while to figure out.

11. Now restart the Apache web server

12. It should be working. To test, load (in a browser) the localhost page you want to profile. If all is well, XDebug will produce a file named xdebug_profile.xxxx.xxxx or similar in the XDebug output directory. (For me the directory is “C:\Program Files\xampp\tmp”, but you find / set the output dir from the php.ini line which looks like: xdebug.profiler_output_dir = “C:\Program Files\xampp\tmp”).

13. Install webgrind:

* Download webgrind from https://github.com/jokkedk/webgrind (use the ‘downloads’ button).
* Extract webgrind to a folder accessible to the webserver.
* Start using webgrind by browsing to the URL of your webgrind install, e.g. http://localhost/webgrind.

If all has gone well you should now be able to use webgrind to view the profile information for the page you loaded in step 12.

I’ve been working on an API for the new version of Calcatraz (both API and new version coming soon). As part of this, I’ve been trying to configure the API to use nice URLs. This has caused me to bump up against a tricky problem in Apache’s mod_rewrite module.

I want to be able to accept URLs containing a calculation such as:

http://www.calcatraz.com/api/calc/3*3.txt

And redirect it to a script which can extract the calculation (3*3) and the required format (in this case plain text ‘txt’).

The Basics

For the case above it can be done without problem, you just use a .htaccess redirect rule such as:

RewriteRule calc/ api.php [L]

which calls api.php to serve the page. Then in api.php you parse the $_SERVER['REQUEST_URI'] variable – which contains the original url – and extract the various parts. Alternatively you can get a bit more complex in mod_rewrite and do the redirect and parsing in one, with something like:

RewriteRule calc/(.+)\.(txt|xml)$ api.php?calc=$1&format=$2 [L]

The Problem

So far, so good. But this breaks down when the calculation contains characters with special meaning in URLs, such as ‘/’, ‘?’, etc. My initial thought was to require that the calculation is url-encoded before inclusion in the API URL. This works for the likes of ‘?’. Encoding a slash is a different matter. The encoded URL for 3/3 would be:

http://www.calcatraz.com/api/calc/3%2F3.txt

Here the %2F is the urlencoded form of ‘/’.

This seems like it should be passed through to the api.php script, but it isn’t. Instead a 404 not found page is found. It turns out that this is a known bug in Apache / mod_rewrite.

No matter what I tried, there seems to be no way to redirect an API url containing a %2F to a particular script.

The Solution(s)

1) What does work is to perform a double url-encoding of the calculation before placing it in an API URL. So the api call to get the answer for 3/3 would be:

http://www.calcatraz.com/api/calc/3%252f3.txt

This works, redirecting to the script and allowing us to extract the calculation and get its original form by performing a single url-decoding (as Apache / mod_rewrite performs one level of decoding implicitly). The main problem with it is that the URLs are pretty messy, and the user has to be asked to perform a double url-encoding. It also makes the URLs less obviously hackable / readable.

2) We can remove the need for the user to perform a double url-encoding by replacing it with a single base-64 encoding. This slightly tidies up the client-side API calls, but it means the URLs are completely inscrutable - the user can’t easily play around with them at all.

3) Since the slash will actually work without any encoding (it is only other characters as ‘?’ which must be encoded), we could create  a custom version of urlencode which encodes everything normally encoded, other than slashes. This would yield the most readable URLs, but significantly complicates things for the user.

4) Switch web servers. As discovered in this ServerFault discussion I’ve been having on the issue, while it affects Apache, it doesn’t affect some other servers such as nginx.

5) Update: Okay, thanks to the guys over at ServerFault, I now have the answer. By default Apache decodes encoded slashes (the problem). This behaviour can be disabled by adding the following to Apache’s httpd.conf file (or the relevant vhost configuration):

AllowEncodedSlashes On

Conclusion

I haven’t tested it on my public web server yet (only my dev server), but all being well Solution 5 is going to be the winner. Look out for the API, coming soon along with nice URLs, and a new version of Calcatraz calculator.

Side Note

Interestingly, it seems that WordPress also falls prey to this issue. I originally posted this article with the term %2F in the title. WordPress took this title and url-decoded it to produce the URL “slug” for the post – which now contained a slash. When the slug is used in the permalink of the page title it is URL encoded back up, and clicking on the link then requests a URL containing a %2F. Because wordpress makes use of redirects, then as seen before, the redirect fails and a 404 error is returned. So it was not possible to actually view the post!

I first got on Google and looked for a solution. The best (as in most brief) solution I found could be distilled down to this:

(strlen($str)>10)?substr_replace($str,'...',10):$str;

(strlen($str)>10)?substr($str,0,10).'...':'';

preg_replace('/(.{10}).*/', '\\1...', $str);

This is shorter still, only uses the variable name once and is a single function call making it easier to spot the start and end. It’s not perfectly descriptive, but the second argument in particular does hint at it’s function. I’d be happy using this inline if it was only to be used a couple of times (and the surrounding code was otherwise quite straightforward).

div#elemid { display: inline-block; }

However, it doesn’t work in IE. It took me a while to figure out how to get it working, so here’s the solution:

div#elemid { display: inline-block; *display: inline; zoom: 1; }

It works for the latest versions of IE, Firefox and Chrome (and possibly others – I haven’t tested yet).

Hopefully that’ll save someone some time.

table { border: 1px solid red; }

div { border: 1px solid green; }

This will draw a border round each div and table on your page and let you visually see which are really necessary. It also works better than looking at each table / div in turn as often each seems necessary; it is only when you look at the big picture that you see that two or more can be combined.

As an illustration, here are the before and after screenshots when I applied this to the Calcatraz homepage. The excess tables / divs can be seen in the before shot (click on the image to see it full size). Notice how I’ve been able to two of them in the after shot, leading to shorter, cleaner html, with more of the presentation being handled in CSS where it should be.

Before:

After:

At the end of this post are the notes I made while trying to figure it out. They’re probably not of any use to anyone, but I though I’d share them. Before that, though, I want to outline the process I went though as hopefully this will be a bit more useful.

Step 1: Identify the main source of value you provide

Step 3: Identify time costs

The  other three also incur the time cost involved in typing in the calculation. This is then followed by the time cost in waiting for a response. Hopefully all these costs should be minimal, but it would be wrong to ignore them.

By analysing web logs, etc, it should be possible to get exact values for this time cost in most cases. The only thing that can’t be exactly pinpointed is the time it takes for the user to find the site as this may include, e.g. searching on a different site such as Google.

Step 4: Identify the user-derived benefit and costs

Assuming the user takes the action required to produce the value, three things can happen. Either the user gets the value they were after, they don’t get the value, or they get a wrong answer which may actually cost them (in that they may act on incorrect information).

In the case that the right answer is provided, we know that the user has received some value. While we don’t know how much there is one assumption we can make – it should be greater than the cost the time cost the user expected to incur. Otherwise the user wouldn’t have take the action (assuming rationality). We can figure out the expected time cost in a way similar to the actual time cost calculated in step 3. This just involves estimated averages across all sites (such as average site response time) rather than using data specific to my site. I also made an assumption that a user would expect a return of at least 10% greater than the time invested to make it worthwhile.

In the case that the calculator returns no answer (where I haven’t yet implemented the calculation asked), then the user gets no benefit, but also no real cost other than the time invested.

The final case is that the user gets given the wrong answer. This could potentially be have a large cost to the user if they act on the incorrect information. While hard to quantify, one assumption I felt confident making is that a rational user would cap their potential losses in relation to their potential gain. For this I assumed that the user would cap losses at, at most, 1o times their potential gain.

Step 5: Write out value expressions for each outcome

Step 6: Derive a lower-bound on each expression solely in terms of the gain

We know alpha is the actual time cost the user incurs. This is related to a lower-bound on the gain, G, by the user’s expected time cost (call it alpha_e). With a bit of work (see the notes below for details), we can rewrite the value expressions so that they are expressed in terms of the lower-bound on G. So essentially, we have value expressions written solely in terms of (a lower-bound on) G.

Without getting into to much detail, we can basically total up the value provided to a user when they perform a set of calculations. This will give us a value score for that set in terms of G. While we don’t know what G actually is, it doesn’t matter – as we will be using the value calculation simply to compare one set of calculations to another (separated in time, by user, by calculator version, etc.) we can just drop G entirely.

Conclusion

Now we have an expression for the lower-bound on the value provided by any set of calculations and resulting outcomes. While not perfect (an exact score would be better than a lower-bound), I think it might be the best that can be done without figuring out what user’s are actually doing with their calculation results (and I think that’s their own business). It will still be very useful to be able to compare the lower-bounds as these are likely to be reasonably well correlated with the actual values. Also as the aim is to increase value over time we can be fairly confident that this is occurring if we can see that the lower-bound on the value is increasing over time.

My Rambling Notes (or Stop Here and Don’t Read On)

For the time being I’m restricting the value calculation to the core calculation service. There are four basic outcomes which can occur when someone interacts with main calculator page:

If it is possible to assign a reasonable measure of user value to each item, then it will be possible to combine them into a reasonable total measure of the value provided by the calculator in a given time period, per user, etc.

In general, there will be a fixed time cost involved in performing any calculation (regardless of the result). For the time being, call this alpha. The cost when no calculation is performed will be somewhat less than this. At a rough estimate it will be alpha / 2.

Should a calculation be performed correctly there will be some gain G for the user.

If a calculation is not answered, then this gain G will not be realised. However, should the calculation be incorrectly answered, not only will the gain not be realised, but the user may be significantly affected (e.g. by entering into a bad financial transaction based on the incorrect result). While we can’t say exactly what magnitude of loss this might be, it seems reasonable to expect that any rational user would cap their potential losses at 10 times the possible gain. At the very least, at that risks beyond that level, they could be expected to verify the calculation results with other sources. So the maximum loss can, I think, be taken to be -10*G.

Combining these we can get value expressions for each of the possible outcomes:

This will be more useful if we can express it in a single quantity. To do so we need to look at the relationship between alpha and G.

There is a quantity related to alpha which will be useful to us. This is the time cost the user expects to pay, assuming a reasonably responsive system. Depending on the performance of our system, alpha may be greater or smaller than this. We’ll call this quantity alpha_e.

A rational user would not carry out actions which did not at least repay the expected initial investment plus a bit of ‘profit’ to make it worthwhile and absorb associated risks. Thus we can expect G to have a lower-bound of perhaps 1.1 * alpha_e.

That’s a bit better, but still not quite there. We can get the rest of the way by looking at what exactly alpha and alpha_e are. These, as noted above, are the time cost of performing a calculation and the expected time cost of performing one on an averagely responsive system respectively. They have the same formula:

time_to_input_calc + time_to_perform_calc + time_to_reach_calculator / num_calcs_per_user

The only difference is that alpha_e represents the user’s expectation of these values while alpha represents the actual values.

We can assign approximate values to these for alpha_e. Something like this, perhaps:

alpha_e = time_to_input_calc_e + time_to_perform_calc_e + time_to_reach_calculator_e / num_calcs_per_user_e

= 10 + 3 + 10 / 5

= 15 seconds

This gives a lower-bound on G of 16.5 seconds.

Now, we can perform a similar calculation for alpha. I don’t know exactly, but the result will probably be something similar. Assuming for the time being that it is, we can then say that G >= 1.1 * alpha. We could easily adjust the multiplicative factor should analysis of the data show that alpha and alpha_e show that they differ.

But is relating alpha to a lower-bound on G any use? It’s not precise, but it gives a possible approach.

Totalling up the value for a set of calculations (based on the expressions above) we will get something in terms of G and alpha, e.g.

value = a * G – b * alpha

If we can get an upper-bound on b * alpha in terms of G, then we’ll get a lower-bound on the value in terms of G:

Setting G = 1.1 * alpha, we get our upper-bound on b * alpha:

b * alpha < (b / 1.1) * G

This gives us a lower-bound on the value provided as:

value >= (a – (b / 1.1)) * G

So that is our lower-bound on the value provided to the user by a set of calculations. The a and b variables can easily be calculated by totalling the values of G and alpha for each calculation in the set. The 1.1 factor can be assumed constant (or calculated from the actual data observed). And G is essentially irrelevant to us. As all value calculations will contain a multiple of G, we can simply drop it – it contains no useful information.

So we have, pulling everything together, finally:

value >= (a – (b / c))

where

a = answered_correctly – 10 * answered_incorrectly

b = 0.5 * no_calculation + answered_correctly + answered_incorrectly + not_answered

c = 1.1 * 15 / (time_to_input_calc + time_to_perform_calc + time_to_reach_calculator / num_calcs_per_user)

While not an exact measure, (which wouldn’t really be feasible), it does give us a lower-bound on the value provided (given the assumptions made above). This is almost as useful as we can now operate with this as our measure in value and strive for a constantly increasing lower-bound on the value provided. While the actual value may be somewhere higher than this, improvements in the lower-bound will almost certainly result in the actual value provided being increased (if not perfectly correlated, this should at least happen on average over time).

I aim to make Calcatraz into a great web application. To do so effectively it is important to understand the qualities a great web application exhibits. So what are they? Here are a few things I have thought of:

Punctual

A web application should be quick: quick to load, quick to respond to user input, etc. However, I don’t think that calling the required quality ‘speed’ would be entirely accurate. There are instances when things can be too fast – an error message not displayed for long enough, for example. In general, events should happen at the appropriate time. If the user does not need a time lag, then there should not be one. I think the correct word is ‘punctual’ – the app should do things at, rather than before or after, the right time.

Example: Not an web app as such, but Google Chrome is exemplary in this area – compare it to Firefox or IE to see the difference.

Minimal

A web application should be minimal. Not in the sense of leaving out important features, but more in the sense of containing no unnecessary parts. Anything which does not perform a necessary function should usually be removed. Where several parts perform similar functions they should be generated by the same code.

Example: The 37 Signals guys do this really well. See BaseCamp for instance.

Magical

There is a quote ‘any sufficiently advanced technology will be indistinguishable from magic’. I think this applies to great web apps. Components of the application should be worked on and crafted to such an extent that it is no longer possible for the user to readily comprehend the underlying process. Alternatively it should be surprising – as in it offers something obvious but that no one even conceived as being possible.

Example: PHPAnywhere – coding is suddenly mobile.

Intuitive

Use of the application should be easy. Even complex parts of the application should be straight-forward to use without prior knowledge. This requires not just simplicity, but an understanding of human psychology. By understanding the mind and it’s quirks, an experience can be tailored to it.

Unlimited

A great web app will be unlimited in the sense that users do not hit upon limits while making use of it. That doesn’t mean the app can do anything in the world, but it does mean that it can do anything in its domain. This requires general mechanisms for extending the application upon its lines of functionality. An example is Google Analytics, while it doesn’t capture everything you need to know about your app, it can be readily extended to do so.

Example: Twitter – while the app is incredibly simple in itself, it is being put to an incredibly wide range of uses.

Responsive

An application should be the product of user feedback. While it should not necessarily address every bit of feedback (as good for one user may be bad for another), it should be shaped and moulded into the form and function which delivers the most value to the most users. It should continue to evolve over time with changing needs.

Predictive

As well as just responding to user requirements, a great application will predicted future requirements. New functionality will appear before a user realises they need it. Only genuinely useful improvements will be made – those that enhance and improve the user’s experience.

Available

The application will be on-hand wherever the user requires it. It should be unobtrusive, but always available and easily invoked.