10 Years of W3Counter

W3Counter is turning 10 years old.

Of the dozens of sites I put together in 2004 and 2005 from my college dorm, W3Counter’s the only one still around aside from this blog. It’s rather popular still — with around 70,000 users acquired completely organically and billions of page views being tracked — despite the explosion of competitors in the now enormous analytics industry. I’m sometimes asked why Google Analytics didn’t kill it — I think that has a lot to do with the fact that “easy” and “fun” are rarely used to describe GA, but I hear them used to describe W3Counter all the time. There’s a high school teacher that has his entire class sign up for accounts every year after creating their own webpages — they get a kick out of seeing the counters, and it’s easy enough to set up that it takes most people just a minute or two.

Every 2-3 years I’ve given the site some needed attention to keep up with the times. The architecture’s gone through several major revisions — from running on a single VPS to several servers, to a short stint running in Amazon’s Elastic Compute Cloud, then back to bare metal servers at SoftLayer. 2014’s users demand two things, at least if I care to keep some portion of them upgrading for the paid perks — real-time dashboards, and a mobile site.

Tracking Active Visitors in Real-Time


Handling even a few thousand concurrent visitors pinging the service every few seconds with their current activity to run these dashboards necessitates something a little more efficient than the way W3Counter was originally programmed — a traditional PHP MVC application, with the Symfony framework on one or more frontend servers, talking to one or more backend database servers that store data about page loads from which the reports are generated. All that framework overhead and database communication can take 40-50ms to finish.

The code that collects data for the dashboards cuts that down to under one millisecond. It’s a node.js application, which bypasses all the overhead of the PHP framework. Redis, an in-memory key-value store is the first thing that code talks to. Redis caches information I’d otherwise need a database lookup to get. While MySQL has its own query cache, using it would still require a round-trip to the remote database server, and having the spare RAM on the database servers to dedicate to that.

The information on what all those active visitors are doing still sits in a MySQL database, but an entirely in-memory one on the front-end servers using the MEMORY storage engine. Unlike the database servers, my front-end web servers typically use less than 25% of their available RAM — they’re CPU-limited rather than memory-limited in their capacity — so keeping the ephemeral data for these dashboards there is just using otherwise unused resources.

By keeping the code efficient and only storing data in memory, one relatively low-end server can serve several thousand concurrent active visitors with negligible load. The only problem I actually ran into was the raw number of TCP connections. They came in faster than nginx was accepting them — I needed to increase its worker_rlimit_nofile and worker_connections parameters to handle more. There were still more than the OS was happy to handle, complaining through the messages log that I was under a SYN flood attack — no, it’s not an attack — until I increased the available local port range and SYN backlog size. I quickly became friends with the sysctl command.

I’m a fan of dashboards myself. W3Counter’s Pulse Dashboard for Improvely runs in a rotation on a screen in my living room along with dashboards from Geckoboard, New Relic and my self-built home automation system for controlling lights, locks and temperature in my home. I’ll have to write about building that some day.

Mobile Web Design

I was also, a bit reluctantly, forced by increasing mobile use to finally learn responsive web design. I still haven’t got it quite perfect, but W3Counter’s now usable on any screen size between your 4″ smart phone and 30″ monitor without half the screen being whitespace, like some vertical equivalent of the black bars on a 4:3 TV watching a letterboxed movie.


Various reports switch from multi-column to single-column layouts at smaller screen sizes, and menus disappear into a touch-friendly button-activated dropdown at the top of the page.

Most of the credit goes to Bootstrap 3’s responsive grid and “collapsible” plugin, along with a smattering of hand-written media queries that even work on IE8 thanks to Respond.js. Nobody’s running IE8 on their phones, but without that library, it shows the mobile version of the site which doesn’t look so great on a 20″ monitor.

A Minimal Website

I also redesigned the marketing site for W3Counter to match the new look — responsive, mostly flat aesthetic, using the same two shades of blue (#c1deef and #e1f3fd) as I’ve used with W3Counter since around 2006.

I found that most pages of the old site were rarely if ever viewed, so why have them at all? I trimmed it down to just 3 — a homepage with a giant image showing off the reports and signup button, the monthly browser & OS market share report that gets quoted in news at least a few times a month, and a signup form.

So far so good — I published the new design three days ago. Only one user wrote in with a note that they didn’t like something I changed. The daily signup rate has doubled. Several of them upgraded to paid accounts within minutes of signing up. 2014 should be another good year for W3Counter.

  • http://saharahacker.com/ Sam the Sahara Hacker

    Haha welcome to the future W3Counter! It was getting pretty rusty 😉
    I’ve known you for while (from Improvely) but had no idea you were behind W3Counter. All that recent work I suppose means revenue is coming strong still yes?

  • Aleks Li

    download it?

  • http://hostcollate.com Hostcollate

    Good. Yandex Metrica have a really cool tool – webvizor (record mouse movements on the screen) – helps to identify bots.

  • Pingback: Eric()

  • Pingback: Luther()

  • Pingback: Orlando()