Nilex WebWatch

Contents:

News

2006-02-07 : Yup, WebWatch has been down for a few days - server CPU pretty much caught fire :-(

2004-07-31 : 300th WebWatch added. Estelle requests the 300th URL to be added to WebWatch :-)

2003-06-21 : Sorry, WebWatch has suffered an outage for about a week. First I had a major hard-drive failure and took a few days fixing it, then I failed to notice a permissions-related problem with one of webwatch's stats files. It's now running again, and the machine has 2 drives which will be mirrored soon, so single-drive failures shouldn't take down the whole machine again.

Bonus treat! : It's not a useable working example yet, but take a look at this example of a possible web interface for WebWatch. What do you reckon, any good? Never know, I might get time to make it WORK eventually :-) ... in the meantime, PLEASE give me your feedback - it's the only thing that keeps me motivated to work on WebWatch ;-)

2002-12-05 : Noticed a redirection bug. If you WebWatch, for example, http://2shortplanks.com/xmas/, which redirects you to http://www.perladvent.org/2002/, which includes images like "/images/5.png", the WebWatch mail will try to get them from 2shortplanks.com/images/5.png when they should come from www.perladvent.org/images/5.png. I was wondering why quite a few WebWatch mails included broken images. I now know. I ought to work out a nice way to handle redirection, I s'pose.

I have big plans for a new program architecture and a web interface for adding/removing/reviewing your WebWatches, and possibly a majordomo-ish email add/remove mechanism.

Converted this page to xhtml 1.0

2002-11-28 : I notice my old arch-enemy, www.netmind.com, no longer seems to be resolvable. You can no longer pay for what WebWatch is doing for free :-/

I s'pose I ought to do a web interface for WebWatch sometime. Anyone wanna help?

2002-04-23 : OK... A potential solution to the "Today's date is ..." problem. I'd appreciate feedback from all beta-testers as to whether they think this is a good or a bad idea. If it sounds workable, I'll implement it soon. *** THIS IS NOT IMPLEMENTED YET! *** It goes like this: If a changed "word" is a digit from 1 to 31, or "1st" to "31st", or one of the words "Mon", "Tue", "Wed", or "Monday", "Tuesday", "Wednesday", etc, it will still be highlighted, but will NOT trigger a mailing - the document will require some other change elsewhere in order to count.

This ought to stop the mails when the only web-page change is "today's date", EXCEPT once a month when the month changes, which is probably fair. It ought to be OK for share prices, which are really unlikely to be shown as "24", more likely "24.0" or whatever. The only potential problem I can think of is if you WANTED an update-email when something like "number of comments: 12" has changed - is that likely to upset anyone? Is there anything else I might have forgotten? Does anyone have a problem with dates in the format "05/06/02" or "2002-04-23" or are there any other date formats (or, good grief, time formats?) I need to take into account which CAN'T be confused with things like share prices that some WebWatchers are trying to track?

I'd appreciate your feedback. Remember, that's all I ask in exchange for your free WebWatch membership :-)

2002-04-04 : Spel-chequed this page!

2002-04-04 : Was shown (by g0tai) Web Secretary, which seems to be a similar sort of thing to WebWatch. In my opinion, the highlighting algorithm and tags are worse, and it does some really surreal things with whitespace, and it has a few other faults, but it has already implemented a couple of things on my "To Do" list, and of course it's an option if you were desperate to run your own "WebWatch" sort of thing rather than using mine! :-)

2002-03-24 : The phrase "WebWatch Vortex Of Doom" invented by one of my subscribers on his blog at wettone.com, though the vortex itself was arguably started by another at Viking On Tour. Of course, by mentioning this here, I've now started the TRIPLE WebWatch Vortex Of Doom - Help! I have created a monster! :-)

2002-03-17 : Fixed it so whitespace changes don't count as changes. Previously, "this example text" and "this example text" would look different, and get highlighted (the source has more spaces in the first example). Furthermore, it used to make the WORDS look like they'd changed, not just the spaces.

2002-03-03 : Changed the email format - it's now one single text/html, not a multipart/mixed text/plain followed by the text/html. This means MS outhouse and outhouse express ought to now show the page itself rather than treating it as an attachment. Also allowed me to add colour-tweaks so the WebWatch message is never black text on a black background.

2002-03-03 : After adding a few tweaks to the above, I can now also view the WebWatch emails on my BlackBerry wireless email gadget, and it probably also works better on other text browsers that can't show the highlighting colour.

2002-03-01 : Congratulations to Poppy for being the owner of WebWatch's 100th URL!

2002-01-18 : After a brief adverty-thing in the EH spodmag, I got my first beta-tester: Subby!

2002-01-15 : First vaguely usable WebWatch ready just in time

Why Am I Getting Emailed This Page?

Sorry, you probably didn't ask to be watching this page, so if you've just got this mailed to you, it's arguably unsolicited junkmail! I thought it was an ideal way to get info to my beta-testers though, so please forgive me. If you don't want this page, I'm happy to remove your subscription to it, but if you don't want beta-tester's news, please ask yourself whether you want to help me beta-test or not :-p

The Main Idea

Basically, I used to use mindit.netmind.com quite a lot. Theory is, you give them a load of URLs, and they check the pages once a day for you, and mail you when a page has changed. They send you an HTML mail including the web page itself, with the changes highlighted.

All fine, until a survey a few months ago where they asked worrying questions like "If mindit were to become a commercial service, how much would you be willing to pay?", to which I answered something like "Nothing, I'd write my own if I couldn't find another free one!"

Well, 15th Jan 2002, after a few weeks of warning, they decided to start charging, and just in time, I had my first prototype "WebWatch" working. It's just fetching web pages, doing something very much like "diff", and sending html mails, right? Nothing particularly difficult, nothing particularly patentable, plenty of "prior art", so hey, why not?

I have no pretty web GUI or anything yet, but it fairly reliably does the fetching, it does the highlighting of changes (better than mindit did, IMHO), and mails 'em out. It has a few quirks (EG if 90% of a page has changed, it'll kindly highlight everything except the occasional "a", "the", "if" that was on the old page!), it's not great with "this page last updated at 16:42", it's not very configurable, but it's fair for a first-draft "version 1.0", and I'm looking for spods to help me test it.

If you have a load, or even just a couple of pages that you ought to check regularly but can never be bothered, please mail me giving me a list of URLs and the email address(es) you want me to send them to when they change. Then please provide feedback, advice, bug reports, etc.

Plans

This stuff is maintained by Nick Waterman - Email Me Valid XHTML 1.0! Valid CSS!