Graduate Summer School: Deep Learning, Feature Learning

July 9 - 27, 2012

Schedule

All times in this Schedule are Pacific Time (PT)

Monday, July 9, 2012

Morning Session

8:00 - 8:45 Check-In/Light Breakfast (Hosted by IPAM)
8:45 - 9:00 Welcome and Opening Remarks
9:00 - 10:00
10:00 - 10:30 Break
10:30 - 11:30
Geoffrey Hinton (University of Toronto)

PART 2: Using backpropagation for fine-tuning a generative model to be better at discrimination

11:30 - 12:00 Break
12:00 - 1:00 Joint talk: James Bergstra (Harvard), Clement Farabet (NYU)
1:00 - 2:30 Lunch (on your own)

Afternoon Session

2:30 - 3:30
Yann LeCun (New York University)

Deep Learning, Graphical Models, Energy-Based Models, Structured Prediction
PDF Presentation

3:30 - 4:00 Break
4:00 - 5:00
Yann LeCun (New York University)

Deep Learning, Graphical Models, EnergyBased Models, Structured Prediction (Part 2)
PDF Presentation

5:00 - 6:30 Reception (Location: IPAM Lobby)

Tuesday, July 10, 2012

Morning Session

8:00 - 9:00 Continental Breakfast
9:00 - 10:00
Andrew Ng (Stanford University)

Deep Learning, Self-Taught Learning and Unsupervised Feature Learning (Part 1 Slides1-68; Part 2 Slides 69-109)
PDF Presentation

10:00 - 10:30 Break
10:30 - 11:30
Andrew Ng (Stanford University)

Advanced topics + Research philosophy / Neural Networks: Representation
PDF Presentation

11:30 - 12:00 Break
12:00 - 1:00
Andrew Ng (Stanford University)

Non-linear hypotheses

1:00 - 2:30 Lunch (on your own)

Afternoon Session

2:30 - 3:30
Andrew Ng (Stanford University)

Non-linear hypotheses (Part 2)

3:30 - 4:00 Break
4:00 - 5:00 Joint talk: James Bergstra (Harvard), Clement Farabet (NYU)

Wednesday, July 11, 2012

Morning Session

8:00 - 9:00 Continental Breakfast
9:00 - 10:00
Yann LeCun (New York University)

Learning Hierarchies of Invariant Features (Parts 1 & 2)
PDF Presentation

10:00 - 10:30 Break
10:30 - 11:30
Yann LeCun (New York University)

Deep Learning, Graphical Models, EnergyBased Models, Structured Prediction (Part 3)

11:30 - 12:00 Break
12:00 - 1:00 Joint talk: James Bergstra (Harvard), Clement Farabet (NYU)
1:00 - 2:30 Lunch (on your own)

Afternoon Session

2:30 - 3:30
Geoffrey Hinton (University of Toronto)

PART 3: Some applications of deep learning (Slides 1-38)
PDF Presentation

3:30 - 4:00 Break
4:00 - 5:00
Geoffrey Hinton (University of Toronto)

PART 4: A computational principle that explains sex, the brain, and sparse coding (Slides 39-92)
PDF Presentation


Thursday, July 12, 2012

Morning Session

8:00 - 9:00 Continental Breakfast
9:00 - 10:00
Rob Fergus (New York University)

Deep Learning Methods for Vision
PDF Presentation

10:00 - 10:30 Break
10:30 - 11:30
Rob Fergus (New York University)

Deep Learning Methods for Vision (Part 2)

11:30 - 12:00 Break
12:00 - 1:00
Geoffrey Hinton (University of Toronto)

 
PDF Presentation

1:00 - 2:30 Lunch (on your own)

Afternoon Session

2:30 - 3:30
Alan Yuille (University of California, Los Angeles (UCLA))

Part 1

3:30 - 4:00 Break
4:00 - 5:00 Joint talk: James Bergstra (Harvard), Clement Farabet (NYU)

Friday, July 13, 2012

Morning Session

8:00 - 9:00 Continental Breakfast
9:00 - 10:00
Graham Taylor (University of Guelph)

 
PDF Presentation

10:00 - 10:30 Break
10:30 - 11:30
Graham Taylor (University of Guelph)

 
PDF Presentation

11:30 - 12:00 Break
12:00 - 1:00
Alan Yuille (University of California, Los Angeles (UCLA))

Part 2

1:00 - 2:30 Lunch (on your own)

Afternoon Session

2:30 - 3:30
Alan Yuille (University of California, Los Angeles (UCLA))

Part 3

3:30 - 4:00 Break
4:00 - 5:00
Yann LeCun (New York University)

Deep Learning, Graphical Models, EnergyBased Models, Structured Prediction (Part 4)


Monday, July 16, 2012

Morning Session

8:00 - 9:00 Continental Breakfast
9:00 - 10:00
Yoshua Bengio (University of Montreal)

Part 1

10:00 - 10:30 Break
10:30 - 11:30
Yoshua Bengio (University of Montreal)

Part 2

11:30 - 12:00 Break
12:00 - 1:00
Stephen Wright (University of Wisconsin-Madison)

Some Relevant Topics in Optimization (Part 1) - (Slides Cover Parts 1 & 2)
PDF Presentation

1:00 - 2:30 Lunch (on your own)

Afternoon Session

2:30 - 3:30
Stephen Wright (University of Wisconsin-Madison)

Some Relevant Topics in Optimization (Part 2)

3:30 - 4:00 Break
4:00 - 5:00
Arthur Szlam (New York University)

A tutorial on sparse modeling.
PDF Presentation

5:00 - 6:30 Reception (Location: IPAM Lobby)

Tuesday, July 17, 2012

Morning Session

8:00 - 9:00 Continental Breakfast
9:00 - 10:00
Stephen Wright (University of Wisconsin-Madison)

Sparse and Regularized Optimization Part 1 (Slides Cover Parts 1 & 2)
PDF Presentation

10:00 - 10:30 Break
10:30 - 11:30
Stephen Wright (University of Wisconsin-Madison)

Sparse and Regularized Optimization Part 2
PDF Presentation

11:30 - 12:00 Break
12:00 - 1:00
Arthur Szlam (New York University)

 

1:00 - 2:30 Lunch (on your own)

Afternoon Session

2:30 - 3:30
Yoshua Bengio (University of Montreal)

Part 3

3:30 - 4:00 Break
4:00 - 5:00
Yoshua Bengio (University of Montreal)

Part 4


Wednesday, July 18, 2012

Morning Session

8:00 - 9:00 Continental Breakfast
9:00 - 10:00
Stéphane Mallat (École Polytechnique)

Scattering Invariant Deep Networks for Classification
PDF Presentation

10:00 - 10:30 Break
10:30 - 11:30
Stéphane Mallat (École Polytechnique)

Scattering Invariant Deep Networks for Classification (Part 2)

11:30 - 12:00 Break
12:00 - 1:00
Kai Yu (Baidu Inc.)

Part 1

1:00 - 2:30 Lunch (on your own)

Afternoon Session

2:30 - 3:30
Kai Yu (Baidu Inc.)

Part 2

3:30 - 4:00 Break
4:00 - 5:00
Jorge Nocedal (Northwestern University)

Tutorial on Optimization methods for machine learning
PDF Presentation


Thursday, July 19, 2012

Morning Session

8:00 - 9:00 Continental Breakfast
9:00 - 10:00
Jorge Nocedal (Northwestern University)

Tutorial on Optimization methods for machine learning (Part 2)

10:00 - 10:30 Break
10:30 - 11:30
Jorge Nocedal (Northwestern University)

Tutorial on Optimization methods for machine learning (Part 3)
PDF Presentation

11:30 - 2:00 Lunch (on your own)

Afternoon Session

2:00 - 3:30
Stéphane Mallat (École Polytechnique)

Scattering Invariant Deep Networks for Classification (Part 3)

3:30 - 4:00 Break
4:00 - 5:00
Jason Morton (Pennsylvania State University)

An Algebraic Perspective on Deep Learning
PDF Presentation


Friday, July 20, 2012

Morning Session

8:00 - 9:00 Continental Breakfast
9:00 - 10:00
Jason Morton (Pennsylvania State University)

An Algebraic Perspective on Deep Learning (Part 2)
PDF Presentation

10:00 - 10:30 Break
10:30 - 11:30
Jason Morton (Pennsylvania State University)

An Algebraic Perspective on Deep Learning (Part 3)
PDF Presentation

11:30 - 12:00 Break
12:00 - 1:00
Yoshua Bengio (University of Montreal)

 

1:00 - 2:30 Lunch (on your own)

Afternoon Session

2:30 - 3:30
Stanley Osher (University of California, Los Angeles (UCLA))

 

3:30 - 4:00 Break
4:00 - 5:00
Stanley Osher (University of California, Los Angeles (UCLA))

 


Monday, July 23, 2012

Morning Session

8:00 - 9:00 Continental Breakfast
9:00 - 10:00
Ruslan Salakhutdinov (University of Toronto)

 
PDF Presentation

10:00 - 10:30 Break
10:30 - 11:30
Ruslan Salakhutdinov (University of Toronto)

 
PDF Presentation

11:30 - 12:00 Break
12:00 - 1:00
Marc'Aurelio Ranzato (Google Inc.)

Deep Gated MRF's - part 1
PDF Presentation

1:00 - 2:30 Lunch (on your own)

Afternoon Session

2:30 - 3:30
Marc'Aurelio Ranzato (Google Inc.)

Deep Gated MRF's - part 2
PDF Presentation

3:30 - 4:00 Break
4:00 - 5:00
Jason Weston (Google Research)

Part 1

5:00 - 6:30 Reception (Location: IPAM Lobby)
6:30 - 7:30
Brian Milch (Google Inc.)

From Text to Concepts at Google


Tuesday, July 24, 2012

Morning Session

8:00 - 9:00 Continental Breakfast
9:00 - 10:00
Jason Weston (Google Research)

Part 2

10:00 - 10:30 Break
10:30 - 11:30
Jason Weston (Google Research)

Part 3

11:30 - 12:00 Break
12:00 - 1:00
Ruslan Salakhutdinov (University of Toronto)

 
PDF Presentation

1:00 - 2:30 Lunch (on your own)

Afternoon Session

2:30 - 3:30
Bruno Olshausen (University of California, Berkeley (UC Berkeley))

From natural scene statistics to models of neural coding and representation (part 1)
PDF Presentation

3:30 - 4:00 Break
4:00 - 5:00
Marc'Aurelio Ranzato (Google Inc.)

Large Scale Deep Learning
PDF Presentation


Wednesday, July 25, 2012

Morning Session

8:00 - 9:00 Continental Breakfast
9:00 - 10:00
Thomas Serre (Brown University)

Deep learning in the visual cortex - Part 1
PDF Presentation

10:00 - 10:30 Break
10:30 - 11:30
Thomas Serre (Brown University)

Deep learning in the visual cortex - Part 2
PDF Presentation

11:30 - 12:00 Break
12:00 - 1:00
Roland Memisevic (Johann Wolfgang Goethe-Universität Frankfurt)

Multiview Feature Learning - Part 1
PDF Presentation

1:00 - 2:30 Lunch (on your own)

Afternoon Session

2:30 - 3:30
Roland Memisevic (Johann Wolfgang Goethe-Universität Frankfurt)

Multiview Feature Learning - Part 2
PDF Presentation

3:30 - 4:00 Break
4:00 - 5:00
Bruno Olshausen (University of California, Berkeley (UC Berkeley))

From natural scene statistics to models of neural coding and representation (part 2)
PDF Presentation


Thursday, July 26, 2012

Morning Session

8:00 - 9:00 Continental Breakfast
9:00 - 10:00
Iain Murray (University of Edinburgh)

Introduction to MCMC for deep learning
PDF Presentation

10:00 - 10:30 Break
10:30 - 11:30
Iain Murray (University of Edinburgh)

Density estimation
PDF Presentation

11:30 - 12:00 Break
12:00 - 1:00
Nando de Freitas (University of British Columbia)

Part 1

1:00 - 2:30 Lunch (on your own)

Afternoon Session

2:30 - 3:30
Nando de Freitas (University of British Columbia)

Part 2

3:30 - 4:00 Break
4:00 - 5:00
Thomas Serre (Brown University)

Deep learning in the visual cortex - Part 3
PDF Presentation


Friday, July 27, 2012

Morning Session

8:00 - 9:00 Continental Breakfast
9:00 - 10:00
Nando de Freitas (University of British Columbia)

An Informal Mathematical Tour of Feature Learning
PDF Presentation

10:00 - 10:30 Break
10:30 - 11:30
Nando de Freitas (University of British Columbia)

Part 3

12:00 - 1:30 Lunch (on your own)

Afternoon Session

1:30 - 2:30 Panel Discussion: Nando de Freitas, Iain Murray, Bruno Olshausen, Ruslan Salakhutdinov, Roland Memisevic
2:30 - 3:00 Closing Remarks by Russ Caflisch (IPAM Director)