Visualizing & Understanding Recurrent Neural Networks with Andrej Karpathy, OpenAI

Original
Andrej Karpathy is a Research Scientist working on deep learning, generative models and reinforcement learning at OpenAI. He studied at Stanford University, focusing on deep learning and its applications in computer vision and natural language processing (NLP). In particular, his more recent work focused on image captioning, recurrent neural network language models and reinforcement learning. 

His work and personal expertise has received a lot of media attention, including features in WiredThe EconomistPopular Mechanics and Bloomberg, to name a few. In particular, a fun project he created that uses convolutional neural networks to distinguish what makes a "good" selfie was picked up worldwide, causing discussion on AI, computer science and neural networks to appear everywhere from Elle to NBC News.

At the 2016 RE•WORK Deep Learning Summit in San Francisco, Andrej held a presentation on 'Recurrent Network Language Models for Dense Image Captioning'. View his presentation and slides in the video below.

Recurrent Network Language Models for Dense Image Captioning

Recurrent Neural Networks (RNNs), and specifically a variant with Long Short-Term Memory (LSTM), are enjoying renewed interest as a result of successful applications in a wide range of machine learning problems that involve sequential data. However, while LSTMs provide exceptional results in practice, the source of their performance and their limitations remain rather poorly understood.

Using character-level language models as an interpretable testbed, we aim to bridge this gap by providing a comprehensive analysis of their representations, predictions and error types. In particular, our experiments reveal the existence of interpretable cells that keep track of long-range dependencies such as line lengths, quotes and brackets. Moreover, an extensive analysis with finite horizon n-gram models suggest that these dependencies are actively discovered and utilized by the networks. Finally, we provide detailed error analysis that suggests areas for further study. 

Watch more video presentations on the RE•WORK video hub and our YouTube channel.

See the full events list here for events focused on AI, Deep Learning and Machine Intelligence taking place in London, Amsterdam, Boston, San Francisco, New York, Hong Kong and Singapore. 

Machine Learning Deep Learning AI Deep Learning Summit Text Analysis NLP Deep Learning Algorithms


0 Comments

    As Featured In

    Original
    Original
    Original
    Original
    Original
    Original

    Partners & Attendees

    Intel.001
    Nvidia.001
    Ibm watson health 3.001
    Acc1.001
    Rbc research.001
    Twentybn.001
    Mit tech review.001
    Kd nuggets.001
    Facebook.001
    Maluuba 2017.001
    Graphcoreai.001
    Forbes.001
    This website uses cookies to ensure you get the best experience. Learn more