Working with Visionaries on the Frontlines of Scientific Progress Worldwide
Nuance Foundation Grants

Department of Computer Science and Operations Research

 

The Department of Computer Science and Operations Research (DIRO) was one of the 1st university-level computer science departments in North America. The DIRO is part of the Faculty of Arts and Science and has been training computer scientists for 5 decades now: our graduates have gone on to change the worlds of technology, business, academics, medicine, transportation, and communication.

Project: Scaling Up Deep Learning

Yoshua Bengio, Aaron Courville

Although deep learning methods have yielded revolutionary results in many areas (including speech recognition, object recognition, and language modeling), the best results are almost always achieved with the largest models one could train in a reasonable time. To obtain much better models will require advances allowing us to train models that have a lot more capacity (and that can be trained to extract that additional information from very large datasets). This project is focused on methods aimed at scaling up deep learning, on three fronts: better numerical optimization (in particular to deal better with saddle points, which may end up being the major stumbling block for current training procedures), conditional computation (where, for each example, only a subset of the model is activated), and distributed training (across many computing nodes).

Programs & Grants

100 years ago nobody would have imagined that it may make sense to talk to machines. Today, in the days of speech recognition and speech synthesis to be found in cars, computers, phones and many other devices this is already normal. But it doesn't stop there.

Learn More