Friday, November 4, 2011

How reliable is Roistr?

One question asked of us is how reliable Roistr is under stress. It works fine with the limited demonstrations, for example, but can it handle something closer to real-world work?

Lately, we've been preparing more detailed demonstrations. They both involve linking someone's social media to a list of a) book reviews and b) movie reviews (note: the latter is just for internal use right now). Although limited in scope, the data sets are more like real world data sets: the book reviews cover over 14,000 books; the movies over 150,000. So far, the engine has been working solidly and has produced results without a single blip on it.

The problem we do have is that it's not so fast in producing results - the 14,000+ books took 30 minutes; but calculating each vector is not a trivial operation: retrieving each vector takes many millions of floating point operations. We would like to reduce the time taken for this but this is where Roistr offers real value over existing methods: things like keyword matches are quicker but they aren't as good. Mimicking human performance takes a lot of effort.

We're happier to put up with slower results because they're more accurate and the important thing is for relevance to be maximised.

If you would like a demonstration or field test of Roistr, talk to me, Alan Salmoni (email link) and I'll see what we can organise for you.