Acknowledgments
This book would not have been possible without the contributions of many people.
We would like to thank those who commented on our proposal for the book
and helped plan its contents and organization: Guillaume Alain, Kyunghyun Cho,
Çağlar Gülçehre, David Krueger, Hugo Larochelle, Razvan Pascanu and Thomas
Rohée.
We would like to thank the people who offered feedback on the content of the
book itself. Some offered feedback on many chapters: Martín Abadi, Guillaume
Alain, Ion Androutsopoulos, Fred Bertsch, Olexa Bilaniuk, Ufuk Can Biçici, Matko
Bošnjak, John Boersma, Greg Brockman, Pierre Luc Carrier, Sarath Chandar,
Pawel Chilinski, Mark Daoust, Oleg Dashevskii, Laurent Dinh, Stephan Dreseitl,
Jim Fan, Miao Fan, Meire Fortunato, Frédéric Francis, Nando de Freitas, Çağlar
Gülçehre, Jurgen Van Gael, Javier Alonso García, Jonathan Hunt, Gopi Jeyaram,
Chingiz Kabytayev, Lukasz Kaiser, Varun Kanade, Akiel Khan, John King, Diederik
P. Kingma, Yann LeCun, Rudolf Mathey, Matías Mattamala, Abhinav Maurya,
Kevin Murphy, Oleg Mürk, Roman Novak, Augustus Q. Odena, Simon Pavlik,
Karl Pichotta, Kari Pulli, Tapani Raiko, Anurag Ranjan, Johannes Roith, Halis
Sak, César Salgado, Grigory Sapunov, Mike Schuster, Julian Serban, Nir Shabat,
Ken Shirriff, Scott Stanley, David Sussillo, Ilya Sutskever, Carles Gelada Sáez,
Graham Taylor, Valentin Tolmer, An Tran, Shubhendu Trivedi, Alexey Umnov,
Vincent Vanhoucke, Marco Visentini-Scarzanella, David Warde-Farley, Dustin
Webb, Kelvin Xu, Wei Xue, Li Yao, Zygmunt Zając and Ozan Çağlayan.
We would also like to thank those who provided us with useful feedback on
individual chapters:
•
Chapter 1, Introduction: Yusuf Akgul, Sebastien Bratieres, Samira Ebrahimi,
Charlie Gorichanaz, Brendan Loudermilk, Eric Morris, Cosmin Pârvulescu
and Alfredo Solano.
•
Chapter 2, Linear Algebra: Amjad Almahairi, Nikola Banić, Kevin Bennett,
viii
CONTENTS
Philippe Castonguay, Oscar Chang, Eric Fosler-Lussier, Andrey Khalyavin,
Sergey Oreshkov, István Petrás, Dennis Prangle, Thomas Rohée, Colby
Toland, Massimiliano Tomassoli, Alessandro Vitale and Bob Welland.
•
Chapter 3, Probability and Information Theory: John Philip Anderson, Kai
Arulkumaran, Vincent Dumoulin, Rui Fa, Stephan Gouws, Artem Oboturov,
Antti Rasmus, Andre Simpelo, Alexey Surkov and Volker Tresp.
•
Chapter 4, Numerical Computation: Tran Lam An, Ian Fischer, and Hu
Yuhuang.
•
Chapter 5, Machine Learning Basics: Dzmitry Bahdanau, Nikhil Garg,
Makoto Otsuka, Bob Pepin, Philip Popien, Emmanuel Rayner, Kee-Bong
Song, Zheng Sun and Andy Wu.
•
Chapter 6, Deep Feedforward Networks: Uriel Berdugo, Fabrizio Bottarel,
Elizabeth Burl, Ishan Durugkar, Jeff Hlywa, Jong Wook Kim, David Krueger
and Aditya Kumar Praharaj.
•
Chapter 7, Regularization for Deep Learning: Inkyu Lee, Sunil Mohan and
Joshua Salisbury.
•
Chapter 8, Optimization for Training Deep Models: Marcel Ackermann,
Rowel Atienza, Andrew Brock, Tegan Maharaj, James Martens and Klaus
Strobl.
•
Chapter 9, Convolutional Networks: Martín Arjovsky, Eugene Brevdo, Eric
Jensen, Asifullah Khan, Mehdi Mirza, Alex Paino, Eddie Pierce, Marjorie
Sayer, Ryan Stout and Wentao Wu.
•
Chapter 10, Sequence Modeling: Recurrent and Recursive Nets: Gökçen
Eraslan, Steven Hickson, Razvan Pascanu, Lorenzo von Ritter, Rui Rodrigues,
Mihaela Rosca, Dmitriy Serdyuk, Dongyu Shi and Kaiyu Yang.
• Chapter 11, Practical methodology: Daniel Beckstein.
• Chapter 12, Applications: George Dahl and Ribana Roscher.
• Chapter 15, Representation Learning: Kunal Ghosh.
•
Chapter 16, Structured Probabilistic Models for Deep Learning: Minh Lê
and Anton Varfolom.
• Chapter 18, Confronting the Partition Function: Sam Bowman.
ix
CONTENTS
•
Chapter 20, Deep Generative Models: Nicolas Chapados, Daniel Galvez,
Wenming Ma, Fady Medhat, Shakir Mohamed and Grégoire Montavon.
• Bibliography: Leslie N. Smith.
We also want to thank those who allowed us to reproduce images, figures or
data from their publications. We indicate their contributions in the figure captions
throughout the text.
We would like to thank Ian’s wife Daniela Flori Goodfellow for patiently
supporting Ian during the writing of the book as well as for help with proofreading.
We would like to thank the Google Brain team for providing an intellectual
environment where Ian could devote a tremendous amount of time to writing this
book and receive feedback and guidance from colleagues. We would especially like
to thank Ian’s former manager, Greg Corrado, and his current manager, Samy
Bengio, for their support of this project. Finally, we would like to thank Geoffrey
Hinton for encouragement when writing was difficult.
x
