Course10610621192 lectures
Applied Natural Language Processing
Prof. Ramaseshan R · Chennai Mathematical Institute
Lectures
Week 1
- Introduction
- Operations on a Corpus
- Probability and NLP
- Vector Space models
- Sequence Learning
- Machine Translation
- Preprocessing
- Statistical Properties of Words - Part 01
- Statistical Properties of Words - Part 02
- Statistical Properties of Words - Part 03
Week 2
- Vector Space Models for NLP
- Document Similarity - Demo, Inverted index, Exercise
- Vector Representation of words
- Contextual understanding of text
- Co-occurence matrix, n-grams
- Collocations, Dense word Vectors
- SVD, Dimensionality reduction, Demo
- Query Processing
- Topic Modeling
Week 3
- Examples for word prediction
- Introduction to Probability in the context of NLP
- Joint and conditional probabilities, independence with examples
- The definition of probabilistic language model
- Chain rule and Markov assumption
- Generative Models
- Bigram and Trigram Language models -peeking indide the model building
- Out of vocabulary words and curse of dimensionality
- Exercise
- Naive-Bayes, classification
Week 4
- Machine learning, perceptron, linearly separable
- Linear Models for Claassification
- Biological Neural Network
- Perceptron
- Perceptron Learning
- Logical XOR
- Activation Functions
- Gradient Descent
Week 5
- Feedforward and Backpropagation Neural Network
- Why Word2Vec?
- What are CBOW and Skip-Gram Models?
- One word learning architecture
- Forward pass for Word2Vec
- Matrix Operations Explained
- CBOW and Skip Gram Models
Week 6
- Building Skip-gram model using Python
- Reduction of complexity - sub-sampling, negative sampling
- Binay tree, Hierarchical softmax
- Mapping the output layer to Softmax
- Updating the weights using hierarchical softmax
- Discussion on the results obtained from word2vec
- Recap and Introduction
- ANN as a LM and its limitations
- Sequence Learning and its applications
Week 7
- Introuduction to Recurrent Neural Network
- Unrolled RNN
- RNN - Based Language Model
- BPTT - Forward Pass
- BPTT - Derivatives for W,V and U
- BPTT - Exploding and vanishing gradient
- LSTM
- Truncated BPTT
- GRU
Week 8
- Introduction and Historical Approaches to Machine Translation
- What is SMT?
- Noisy Channel Model, Bayes Rule, Language Model
- Translation Model, Alignment Variables
- Alignments again!
- IBM Model 1
- IBM Model 2
Week 9
- Introduction to Phrase-based translation
- Symmetrization of alignments
- Extraction of Phrases
- Learning/estimating the phrase probabilities using another Symmetrization example
- Introduction to evaluation of Machine Translation
- BLEU - "A short Discussion of the seminal paper"
- BLEU Demo using NLTK and other metrics
Week 10
- Encoder-Decoder model for Neural Machine Translation
- RNN Based Machine Translation
- Recap and Connecting Bloom Taxonomy with Machine Learning
- Introduction to Attention based Translation
- Research Paper discussion on "Neural machine translation by jointly learning to align and translate"
- Typical NMT architecture architecture and models for multi-language translation
- Beam Search
- Variants of Gradient Descend
Week 11
- Introduction to Conversation Modeling
- A few examples in Conversation Modeling
- Some ideas to Implement IR-based Conversation Modeling
- Discussion of some ideas in Question Answering
Week 12
Syllabus and materials
At NPTEL, as the course publishes them.
Credits and licence
- Institution
- Chennai Mathematical Institute
- Department
- Computer Science and Engineering
- Teacher
- Prof. Ramaseshan R
- Level
- Undergraduate
- Language
- English
- Subjects
- Computer science, Engineering
- Source
- NPTEL
- Licence
- CC BY-NC-SA, as NPTEL states it. Non-commercial use only. NPTEL's terms
- Attribution
- NPTEL states: "Distributed under Creative Commons Attribution-ShareAlike / CC BY - NC - SA", beside a BY-NC-SA badge. Coordinated by IIT Madras.
- Who can watch
- NPTEL gives no rating. Invisible Network shows it to Tweens profiles and older, judged from what the source says about it. How courses are placed
More like this
NPTEL106106131
Course
Design and analysis of algorithms
56 lectures · Source's player
NPTEL106106145
Course
Programming, Data Structures And Algorithms Using Python
45 lectures · Source's player
NPTEL106106136
Course
Model Checking
55 lectures · Video
NPTEL106106931
Course
Introduction To Haskell Programming
47 lectures · Video
NPTEL106106137
Course
Introduction To Haskell Programming
1 lecture · Source's player
TED
Talk
The future we're building -- and boring
Source's player
TED
Talk
The thrilling potential of SixthSense technology
Source's player
TED
Talk
Why don't perpetual motion machines ever work?
Source's player
Course
History 7b Spring 2006 UC Berkeley (History 7B US History: from Civil War to Present)
39 lectures · Audio
Course
Science Events Video
84 lectures · Video
Course
Nuclear Engineering 101 001 Fall 2009 UC Berkeley Nuclear Reactions and Radiation (Nuclear Engineering 101, 001|Fall 2009|UC Berkeley (Nuclear Reactions and Radiation))
29 lectures · Audio
Course
Matrix Methods in Data Analysis, Signal Processing, and Machine Learning
34 lectures · Video
Course
Matrix Methods in Data Analysis, Signal Processing, and Machine Learning
34 lectures · Video