using Verity and ColdFusion MX

Spent all day yesterday battling last minute emergencies for a site that’s about to launch. One of the things ‘problems’ was the speed, or actually the lack thereof, of full text searches using Verity and ColdFusion MX (documentation here). The site I’m working on has 9 full text collections (each one representing a different area of the site) and includes multiple forms allowing for full text search. One of these forms actually searches all 9 collections and returns results. In my testing, I couldn’t get any better than approximately 250ms per search, which meant that the search would take a mininum of 2.250 seconds, which isn’t acceptable at this point. I did optimize (multiple times) the Verity collections, many thanks to Geoff Bowers for this great post on ‘Verity Optimisation‘ (even if they do spell it wrong over there), as well as for his help offline. Geoff mentioned that my best bet would be to use K2, but it looks like we’re not going to be able to move to that before launch. Couple interesting things came out of an internal discussion about using K2 within CFMX (mainly from Ray Camden, thanks Ray!):

a) K2 reads in the collections at startup and apparently will not see updates to its’ collections until after restarted. Anyone have any official documentation that says that? (update: I do see this note: “Macromedia does not recommend running K2 Server as a Windows service. You must stop the service before you modify or delete collections registered with K2 Server. You must then remember to restart the service. You must also verify that the vdkHome information in your k2server.ini file is uncommented-that is, it has no leading pound (#) signs-and points to the correct location of the common directory.“)

b) because of a. above, you should probably restart K2 using cfexecute after the collection has been updated or at a set time/schedule using a Scheduled Task

c) because of b. above, there is a handy function ‘IsK2ServerOnline()‘ that you can use to determine whether or not the server is online, and if not, gracefully handle the error condition.

Small Pieces Loosely Joined: A Unified Theory of the Web

I finished Small Pieces Loosely Joined: A Unified Theory of the Web [amazon], [allconsuming], [official site] tonight. I don’t know how to review the book, the quality wasn’t at all like Emergence or The Future of Ideas, I guess it’s a different kind of book though, not as academic. Emergence reads like a well researched, factual, essay as evidenced by the small print and lengthy bibiolography. Small Pieces Loosely Joined reads like marketing prose, commenting on philosophy, society, and even sometimes religion, using larger print and pointing mainly at websites, rather than other books. Give it a glance at a local bookstore sometime before you buy it, you might find that reading the dustcover is all you need to read. Interestingly, this book was number 2 on the list of Top 100 Most Frequently Mentioned Books of 2002.

Now with 50% less caffeine!