Deep Learning is the hottest trend now in AI and Machine Learning. We review the popular software for Deep Learning, including Caffe, Cuda-convnet, Deeplearning4j, Pylearn2, Theano, and Torch.
By Ran Bi.
Deep Learning is now of the hottest trends in Artificial Intelligence and Machine Learning, with daily reports of amazing new achievements, like doing better than humans on IQ test.
In 2015 KDnuggets Software Poll, a new category for Deep Learning Tools was added, with most popular tools in that poll listed below.
- Pylearn2 (55 users)
- Theano (50)
- Caffe (29)
- Torch (27)
- Cuda-convnet (17)
- Deeplearning4j (12)
- Other Deep Learning Tools (106)
I haven’t used all of them, so this is a brief summary of these popular tools based on their homepages and tutorials.
Theano & Pylearn2:
Theano and Pylearn2 are both developed at University of Montreal with most developers in the LISA group led by Yoshua Bengio. Theano is a Python library, and you can also consider it as a mathematical expression compiler. It is good for making algorithms from scratch. Here is an intuitive example of Theano training.
If we want to use standard algorithms, we can write Pylearn2 plugins as Theano expressions, and Theano will optimize and stabilize the expressions. It includes all things needed for multilayer perceptron/RBM/Stacked Denoting Autoencoder/ConvNets. Here is a quick start tutorial to walk you through some basic ideas on Pylearn2.
Caffe is developed by the Berkeley Vision and Learning Center, created by Yangqing Jia and led by Evan Shelhamer. It is a fast and readable implementation of ConvNets in C++. As shown on its official page, Caffe can process over 60M images per day with a single NVIDIA K40 GPU with AlexNet. It can be used like a toolkit for image classification, while not for other deep learning application such as text or speech.
Torch & OverFeat:
Torch is written in Lua, and used at NYU, Facebook AI lab and Google DeepMind. It claims to provide a MATLAB-like environment for machine learning algorithms. Why did they choose Lua/LuaJIT instead of the more popular Python? They said in Torch7 paper that “Lua is easily to be integrated with C so within a few hours’ work, any C or C++ library can become a Lua library.” With Lua written in pure ANSI C, it can be easily compiled for arbitrary targets.
OverFeat is a feature extractor trained on the ImageNet dataset with Torch7 and also easy to start with.
Cuda: Read more of this post