Spent all day yesterday battling last minute emergencies for a site that’s about to launch. One of the things ‘problems’ was the speed, or actually the lack thereof, of full text searches using Verity and ColdFusion MX (documentation here). The site I’m working on has 9 full text collections (each one representing a different area of the site) and includes multiple forms allowing for full text search. One of these forms actually searches all 9 collections and returns results. In my testing, I couldn’t get any better than approximately 250ms per search, which meant that the search would take a mininum of 2.250 seconds, which isn’t acceptable at this point. I did optimize (multiple times) the Verity collections, many thanks to Geoff Bowers for this great post on ‘Verity Optimisation‘ (even if they do spell it wrong over there), as well as for his help offline. Geoff mentioned that my best bet would be to use K2, but it looks like we’re not going to be able to move to that before launch. Couple interesting things came out of an internal discussion about using K2 within CFMX (mainly from Ray Camden, thanks Ray!):
a) K2 reads in the collections at startup and apparently will not see updates to its’ collections until after restarted. Anyone have any official documentation that says that? (update: I do see this note: “Macromedia does not recommend running K2 Server as a Windows service. You must stop the service before you modify or delete collections registered with K2 Server. You must then remember to restart the service. You must also verify that the vdkHome information in your k2server.ini file is uncommented-that is, it has no leading pound (#) signs-and points to the correct location of the common directory.“)
b) because of a. above, you should probably restart K2 using cfexecute after the collection has been updated or at a set time/schedule using a Scheduled Task
c) because of b. above, there is a handy function ‘IsK2ServerOnline()‘ that you can use to determine whether or not the server is online, and if not, gracefully handle the error condition.
That sounds… awful. I don’t know how the situation is compuared to CF 4.5 days, but I had all sorts of trouble with Verity performance while running the Action XChange… I had to optimize all the time to get anywhere near acceptable performance.
Incidentally, I played with using Lucene as a replacement and seem to remember it being at least 2 orders of magnitude faster… it was a long time ago though, so I could be wrong.
Surely someone out there has written a CFX wrapper around Lucene?
I’m still ticked that I have to use a older version of Redhat to make Verity work at ALL under MX.
It’s actually not Macromedia’s or Redhat’s issue. It’s the Verity folks who apparently have not yet updated their app for newer kernels..
As for the 2.25 seconds doing the search:
Too long? Glad we don’t have your clients… 🙂 Those folks must really hate the web if they can’t wait 2.25 seconds for a search.
hey, at least you don’t have to write the code yourself, and at 2 seconds its probably still faster than the SQL search you’d write if you did.