What is Roistr built on? I can talk a little bit about some of the stuff we work on. There is a lot of proprietary code but a fair bit is built upon various open source libraries.
A vast amount is Python which is a wonderful language for rapid prototyping. Ally it with the numpy and scipy libraries, and it's pretty fast at hard-core number crunching which is a lot of what Roistr does. A part of the analysis we do uses NLTK (the Natural Language ToolKit) and another part uses Gensim. Both come highly recommended.
The server is the Tornado server released as open source by Facebook and written by FriendFeed.
This software represents awesome work so thank you to all those who have released it.