skip to primary navigationskip to content

Microsoft Research Cambridge- Isabelle Lorge, Theoretical and Applied Linguistics

Isabelle Lorge   
Theoretical and Applied Linguistics
External Partner: Microsoft Research Cambridge
Project title: Understanding Conversation to improve automated conversation recognition with Machine Learning

 

Summary:

The purpose of the internship was to build a conversation system using advances in neural networks and deep learning. The idea was to examine small sets of data but also analyse larger sets of data in a more systematic way to gain insights about context-specific features and how an automated system could handle them. Ideally, the network would also be able to ‘ground’ the conversation with the help of a physical/visual context and, additionally, to represent these features in memory so as to be able to deal with long-term dependencies within the discourse. The internship was part of the bigger project aiming first at production of ‘toy’ conversational data which could be used to train the network in a few basic features before testing it on more complex training and test sets. As such, there was an extended team of both interns and Microsoft Research employees working together to achieve this.

 

I was hired to provide advice on the project as both a psycholinguist and a pragmatician, to gather some pilot data from speakers by running a few ‘real-life’ instances of the type of game we were working on and get insight from the language used. My role was to help identify the main features which would be challenging for an automatic system to imitate in a human-like way and classify them, mostly features related to context and speaker communicative intentions (which are less formal and less predictible than purely semantic and syntactical characteristics of speech), which is where my knowledge of pragmatic theories came into play. An additional goal was to find a system for labeling utterances along a number of ways so as to teach the neural network to predict the use of these features and produce them appropriately and, ideally, in a probabilistically sensitive way.

I was able to guide the team in which linguistic theoretical background they should be looking into and what were the most recent developments in the field, as well as identifying for them what would be the main challenges in automated conversation systems, and why. This was a tricky exercise in being able to convey very specialized and/or technical ideas from one field to an audience which was not specialized, and also had very diverse backgrounds (sociology, neuroscience, engineering, computer science etc.), but also a very good one in terms of teamwork dynamics. My only regret is the internship ended up not being a ‘finished’ achievement of a specific project, but more like continuous ‘consulting’ work.

This was an incredible opportunity for me. I have been interested in acquiring these types of skills and knowledge for a long time, mostly self-teaching myself. First, writing the letter and going through the interview process, and then spending the three months in a working environment taught me a lot, since this was more or less my first job experience. But as a first experience it was as rewarding as it could have been: I was working full-time on subjects closely related to my phD research, but with that computational/deep learning twist and the need to at least familiarize myself with related concepts and some programming, which I was willing and keen (and hopefully fast enough) to learn. In addition, I was working with a team of brilliant minds, which made me feel like being part of something, both challenged and valued at the same time. They truly listened to my suggestions and integrated me in the decision process for the project.

This was a truly life-changing experience which should make a tremendous difference both in my ability to more confidently take my PhD research towards interdisciplinary directions and being able to handle the technical aspects of it, but also in terms of career opportunities, as it was for me a step that will probably open many doors which would have been closed for someone with my background. Without this added experience, the range of careers I could hope to apply for after the end of my PhD would have been much more limited, but I am now hoping to be able to enter a field which I could not have considered before, and with confidence that I will be able to bring added value to a team. It very much made me realize that this was the kind of research I was interested in and would like to engage in on a daily basis.