OpenCalais Class and WordPress Plugin Updated

My OpenCalais PHP class and WP Calais Auto Tagger WordPress plugin have been updated. The class now supports the latest API endpoint, all entity types and social tags. The WordPress plugin was completely rewritten to use that new class and to use jQuery for all its DOM manipulation.

Spam Filter For Your Twitter Followers

I’m tired of checking the profile of everyone that follows me on Twitter to decide whether to follow them back. At the same time, I don’t want to follow everyone, or I’ll add spammers, and I do want to know what most of the people that follow me have to say.

I created Follow Ham this morning to solve that problem. It looks up who’s following you that you’re not following back, computes a “spamminess” score for each of those people, then recommends which you should follow and which you should block. Check it out and let me know if you run into any problems — consider it in beta status with only me having used it so far.

If you’re interested in the machine learning techniques behind the spam classification, read the About Follow Ham page.

Step by Step of Launching a New Service

It’s been almost three months since I decided to create and sell an ad tracking service (in progress), and I’m still probably a few weeks away from being able to find out if anyone wants to use it. There’s so much involved in launching a new service beyond just writing the code:

• Plan and Develop the Software
  • Determine what data will need to be collected and stored.
  • Design the database, then convert the schema to SQL/DDL statements.
  • Decide on the major units of modularity for the software, lay out the application structure.
  • Code the model layer classes which retrieve, represent, modify and store the data,
    along with stubs for the controller that are just intelligent enough to start recording
    ad clicks and conversions. Doing this and verifying the queries and the code were accurate
    took over a month (this is the core of what the software does, so it has to be right).
  • Complete the controller code with a basic skeletal UI.
  • Design a proper interface and write complete user interfaces to the reports.
  • Code the CRUD and authentication stuff — logging in, adding campaigns, editing
    ads, importing and exporting data, etc.
• Plan and Design the Website
  • Find a designer to create a logo so I can set up a splash page for the service
    and finish the UI design.
  • Mock up the website layout and pages needed.
  • Explore galleries and directories of other sites to identify what kind of design
    I think will work for this site.
  • Write up a design brief and find a designer to do the work. I ended up doing this
    with a contest at 99Designs for about $950.
    A week and a half later, get the finished design in Photoshop format.
  • Slice and code the design in HTML and CSS, with days spent hacking it for
    Internet Explorer 6 (still not quite there!).
• Write the Website Copy
  • Spend a lot of time identifying benefits to focus on.
  • Write, write and rewrite feature pages.
  • Create supporting graphics, screenshots, etc.
  • Produce a promotional video for the service, highlighting key benefits and features.
  • Decide on initial plans and pricing, considering competition and target customers.
• Prepare Support and Documentation
  • Spend days recording screencasts of how to use the service to track
    advertising, integrate with various sites, import and export data, etc.
  • Convert screencasts to Flash videos, write up the HTML pages to embed them.
  • Write tutorials, glossary, etc.
  • Sign up with ZenDesk for ticket management,
    help desk/knowledgebase, forums, feedback forms, etc.
  • Set up e-mail hosting with Rackspace Email (formerly MailTrust), since my own
    spam filters suck. Make DNS changes for the e-mail hosting and help desk.
• CYA & Legal Stuff
  • Write a Terms of Service agreement to protect myself as much as possible.
    Very important, businesses are going to be using this service to make material financial decisions.
  • Write a Privacy Policy with the help of the Direct Marketing Association’s generator.
  • Come up with a backup and recovery plan, and write code to create offiste copies of all
    user data each night.
  • Document all the server software needed, configuration changes to make, etc. in order to
    set up the service on a different server in case the first should crash/die/disappear/etc. Guesstimate
    capacity until I need to upgrade.
• Plan the Marketing Campaign
  • Sign up with an e-mail marketing company to manage mailing lists, autoresponders, track e-mail
    opens and bounces, etc. Set up the lists, confirmation and reply mail text.
  • Write a 12 page guide to optimizing AdWords Quality Score to give away for free to
    e-mail newsletter signups.
  • Contact affiliate networks about support for recurring commissions. Figure out what’s
    reasonable to pay to affiliates without destroying any profit margin. Pay an overly high
    setup fee to open the new account.
  • Compile a list of bloggers that reach target customers and are willing to write a
    sponsored review of the service when it’s available.
  • Start a Twitter account specific to the service and start following people that talk about
    advertising and tracking every day.
  • Start an online marketing blog to establish the service as an authority in the field and
    attract search engine traffic to the site, potentially resulting in new customers. Spend a day
    finding and installing MySQL libraries for PHP 5.2, which is not yet officially supported
    by Red Hat Enterprise Linux, so that WordPress will run.
  • Buy SSL certificate not only to secure the signup/payment forms, but to have a “secure site”
    badge to show on those pages to increase conversions.
  • Prewrite e-mails for the autoresponder (drip campaign, automatically sent on schedule).
  • Start writing a press release and comparing wire services.
• Getting Ready to Launch
  • Decide how to handle payments, upgrades, downgrades, overage fees. Look at invoicing
    services for the overage fees beyond the monthly subscriptions.
  • Set up Authorize.net’s Customer Information Manager for PCIDSS-compliant storage of
    payment data for the monthly billing. Write the code to automatically charge customers each
    month. Test test test.
  • Set up PayPal subscription links, IPN script to handle updating the database.
  • Write and test a signup/payment form.

I’m probably 80% through all this work… hope it’s worth it. Using w3roi, I’ve saved myself about
$200 a month by eliminating unprofitable ads, keywords, times of day, countries, etc. But I think I’ve
invested so much time and money into this, it’ll take a long time for that to add up.

Delicious: Proof that Symfony is a scalable framework

Back almost a year ago, top social bookmarking site del.icio.us, now just delicious, launched a preview of its version 2.0. At the time they announced it was built on the Symfony PHP framework, what I used for W3Counter. With almost a year going by, and rumors of scaling issues, I worried that perhaps Symfony wasn’t really ready for such a massively popular service.

Yesterday I got confirmation from one of delicious’s engineers that the recently released new site was indeed built on Symfony. That’s great news for the framework, which was already used to create Yahoo! Bookmarks, proving it’s enterprise-ready and able to scale up to millions of active users. Watch the delicious blog for a promised update on “what we learned and how we made certain decisions”.

Review Site Plugin Update Coming

FeedLines has sold 7 copies so far, Award Winning Hosts is racking in the ever-rising web host affiliate commissions, and the plugin that powers its ratings is one of this blog’s most popular pages.

It only makes sense that I should eventually do two things: Update the aging plugin, and charge for copies of it. Amidst other projects, I’ve started rewriting it from scratch, using AWH as the test blog again. Within the next few days I should have something easier to install, nicer to look at, and more compatible with WP 2.5+ than the original.

The top plugin users to date have been affiliate marketers building sites like AWH, so I expect there might be an actual market here, unlike most plugins targeted to average bloggers. Even if not a single person buys it, though, the work will pay for itself since AWH needs an update anyway.

Open Calais PHP Class Updated

I’ve updated my Open Calais PHP Class with the entity types added in Calais’ last update. It now matches a bunch of new entertainment-related entities including movies, music and sports event names.

You can drop the new opencalais.php into the folder with either plugin for WordPress and they’ll be able to add the new types of entities as tags automatically. I’ll be merging and updating the plugins some time in the future as well. I just used them to tag all the articles on Website Goodies, which is getting its first update in a few years as I move it onto WordPress and add a few more goodies.

FeedBurner Stops Reporting Reach

It looks like FeedBurner‘s API has moved in with Google at Google Code. Without an e-mail or a blog post, they made a few changes that broke W3Counter’s slurping of feed stats. Notably, they are no longer reporting the “reach” metric for some feeds, and it’s disappeared from the API reference… perhaps it was harder to track than they thought.

My First Ray Tracer

Quite tangential to everything else I’ve been doing recently, I wrote my first ray tracer in C++ today. It can render arbitrary triangle meshes defined in SMF format. Next weekend I’ll teach myself shading.

Blog Stats Plugin for WordPress

In addition to a visual overhaul, WordPress 2.5 added several new plugin hooks, including a widgetized dashboard. The new Blog Stats Plugin for WordPress adds blog stats to the dashboard, and makes integrating the tracking code into a theme simple.

It’s also the first use of W3Counter’s “visitor labeling” I previewed in October (spooky visitor labeling). By using the plugin instead of inserting the tracking code manually, the names of any previous commentors on your blog is picked up from the cookie WordPress sets and passed on to W3Counter.

Creating a dashboard widget was surprisingly easy. Of course, most of the code isn’t in the plugin — I just fill the widget’s box with an iframe that pulls up the stats from a specially formatted w3counter.com page. Some JavaScript trickery passes in the width of the box on your screen — if it’s narrow, you get two columns of stats, but if your window is approximately 800 pixels or wider, you get three.

If you’re using W3Counter on your blog, and have upgraded to WordPress 2.5, download it now.

Auto-Tagging Content with Open Calais

Calais is a free web service from Clearforest, a Reuters company, that can perform semantic analysis on any English text. It uses natural language processing to extract concepts and relationships from the text. It’s been around for a few months, but there’s been very little developer activity around it, and even fewer completed applications using the technology.

Not finding any other work to build on, I wrote my own PHP class for extracting tags from content with Open Calais’ API. You can get the source and read more here. This class takes a block of text or HTML, sends it to Open Calais or parsing, and extracts all of the entities (things like peoples’ names, companies, technologies, etc.). It returns a multidimensional array organized by entity type.

There’s more Open Calais can do, but I hope this class contributes something to those PHP developers that’d like to start using it but had no place to start with the lightweight documentation and eerily quiet official forums. I plan on putting this class to work as an auto-tagging plugin for WordPress posts. I still need some time to figure out how to integrate that into the new authoring interface of WP 2.5, which this blog is now running on.