When the compute requirements for a Big-Data problem are large enough, they will have to be provisioned from different sources, which makes the task of getting the data to the processing nodes challenging. We present a massively distributed evolutionary algorithm that runs on a compute grid of hundreds of thousands of CPUs, and is capable of utilizing sub-samples of the data in order to discover patterns and classifications in a variety of big-data problems. We demonstrate some of the powerful fault-tolerant, asynchronous features of the system's federated hub-and-spoke architecture and present a case-study on a biotech application in time-series prediction. We finish off with a discussion on how and where Deep Learning can augment this approach.
Babak Hodjat is co-founder and chief scientist of Sentient Technologies, responsible for the core technology behind the world’s largest distributed artificial intelligence system. Babak is a serial entrepreneur, having started a number of Silicon Valley companies as main inventor and technologist. Prior to co-founding Sentient Technologies, Babak was senior director of engineering at Sybase iAnywhere, where he led mobile solutions engineering. Prior to Sybase, Babak was co-founder, CTO and board member of Dejima Inc., acquired by Sybase in April 2004. Babak is the primary inventor of Dejima's patented, agent-oriented technology applied to intelligent interfaces for mobile and enterprise computing – the technology behind Apple’s Siri. Babak is a published scholar in the fields of Artificial Life, Agent-Oriented Software Engineering, and Distributed Artificial Intelligence, and has 25 granted or pending patents to his name. Babak holds a PhD in Machine Intelligence from Kyushu University, in Fukuoka, Japan.