On-device Neural Networks for Natural Language Processing
Deep neural networks reach state-of-the-art performance for wide range of Natural Language Processing, Computer Vision and Speech applications. Yet, one of the biggest challenges is running these complex networks on devices with tiny memory footprint and low computational capacity such as mobile phones, smart watches and Internet of Things. In this talk, I will introduce novel on-device Self-Governing Neural Networks (SGNNs), which learn compact projection vectors with local sensitive hashing. The key advantage of SGNNs over existing work is that they surmount the need for pre-trained word embeddings and complex networks with huge parameters. I will showcase results from extensive evaluations on wide range of natural language tasks such as dialog act classification and user intent prediction. Our findings show that SGNNs are effective at capturing low-dimensional semantic text representations, while maintaining high accuracy and improving over state-of-the-art results.
Dr. Zornitsa Kozareva is a Manager at Google, leading and managing the Natural Language Understanding group and efforts in Google Apps Intelligence. Prior to that, Dr. Kozareva was Manager of Amazon’s AWS Deep Learning group that built and launched the Natural Language Processing and Dialog services Amazon Comprehend and Amazon Lex. Dr. Kozareva was as Senior Manager at Yahoo! leading the Query Processing group that powered Mobile Search and Advertisement. From 2009 to 2014, Dr. Kozareva wore an academic hat as Research Professor at the University of Southern California CS Department with affiliation to Information Sciences Institute, where she spearheaded research funded by DARPA and IARPA on learning to read, interpreting metaphors and building knowledge bases from the Web. Dr. Kozareva regularly serves as Area Chair and PC of top tier NLP and ML conferences. Dr. Kozareva has organized four SemEval scientific challenges and has published over 80 research papers. Dr. Kozareva is a recipient of the John Atanasoff Award given by the President of Republic of Bulgaria in 2016 for her contributions and impact in science, education and industry; the Yahoo! Labs Excellence Award in 2014 and the RANLP Young Scientist Award in 2011.