Comments on: Thoughts on W3Counter page view limits

By: Dan

Dan — Sun, 20 May 2007 10:38:41 +0000

Some follow-up to this issue: I did some testing today on performance against larger log tables. While W3Counter 3 had several reports that entered multiple-second query times to generate, not one showed up in my slow query log using W3Counter 4. Looks like I’ve knocked out all the query-related bottlenecks.

That means I’m now more free to raise the log size, and I plan to do so. Stats for free accounts will still run off the last 25,000 page views, but I’ll be at least doubling that for premium accounts. I’ll also be increasing the daily page view limits; I’ll have chosen the new values by the time the new site goes up tomorrow night.

Increasing log size does still increase the time it takes for nightly maintenance when the older entries are deleted to maintain the table size, but with a bit of artificial delay between queries (thanks to SLEEP()), that won’t be a serious issue.

By: Dan

Dan — Thu, 17 May 2007 05:34:23 +0000

Yep, that’s certainly doable, and not very hard. The design is set up to do that eventually if it’s necessary. But right now there just aren’t enough paying accounts. It takes a lot of $5 accounts to pay for beefy servers.

Maybe this is the excuse I’ve been looking for to play around with Amazon Elastic Compute Cloud. I’ll look into it this weekend, and thanks for bringing it up.

By: Mike

Mike — Thu, 17 May 2007 04:32:31 +0000

It seems to me you have an application that could very easily scale to multiple servers (as long as the paid accounts can support the cost of course!).

The stats from siteX don’t really need to live beside those from siteY. They could be completely autonomous. Sure you are probably interested in global stats and averages, but you don’t need those reports to be 100% fresh.

Why not just replicate your app across N servers? You would need to maintain multiple copies of the code and database, but so does all the distributed software in the world

The servers wouldn’t even need to communicate with each other in a special way. They each just act as the database, file and application server for the accounts they host.

By: Dan

Dan — Thu, 17 May 2007 02:35:16 +0000

This has to run on a computer, and computational power is still quite limited in 2007. Even with 4 cores, 4GB RAM, it’s still just a computer, with one hard disk. How many sites of that size, that need to do multiple database queries on every page view, do you run on one server? W3Counter has to handle thousands of sites without blinking.

By: Max

Max — Thu, 17 May 2007 01:24:19 +0000

So even if someone pays for your largest “Premium” package, they can’t surpass 10,000 pageviews per day? I’m guessing 200,000 is out of the question