Motivation
Joint model
- Reduce error propagation
- Allow information exchange between task
Challenge
- Joint learning
- Search
Solutions
![](../asset/Joint Models for NLP/20190117094814.png)
Statistical Models
###Graph-Based Methods
Traditional solution
• Score each candidate, select the highest-scored output
• Search-space typically exponential
Joint Label Structure
Joint Learning, Joint Search
Joint Segmentation and POS tagging
Joint Parsing and NER
Reranking
Separate Learning, Joint Search
Joint Segmentation and POS Tagging
![](../asset/Joint Models for NLP/20190117101631.png)
Joint Parsing and SRL
Rerank k-best parse trees from a probabilistic parser using an SRL system.
Joint Modeling (Multi task)
Separate Learning, Joint Search
Joint Entity and Sentiment
Optimize the joint objective function which is defined as a linear combination of the potentials from different predictors with a parameter λ to balance the contribution of these two components: opinion entity identification and opinion relation extraction.
![](../asset/Joint Models for NLP/20190117102354.png)
Joint Modeling (Single task)
Joint Learning, Joint Search
Joint Segmentation and POS Tagging
The decoding algorithm for the joint word segmentor and POS tagger, agendas[i] stores the best sequences that end at i
Joint Entity Relation Extraction
Beam Search
###Transition-Based Methods
Transition-based Dependency Parsing
![](../asset/Joint Models for NLP/20190117103723.png)
A Learning+Search Framework
• Advantages
• Low computation complexity
• Arbitrary non-local features
• Learning-guided-search
• State-of-the-art accuracies and speeds
• Constituent parsing
• Dependency parsing
• Word Segmentation
• CCG parsing
• Enable joint models
• Address complex search space and use joint features, which have been difficult for traditional models
Joint Segmentation, Tagging and Normalization
Joint Segmentation, POS-tagging and Constituent Parsing
Traditional: word-based Chinese parsing
This: character-based Chinese parsing
Chinese words have syntactic structures.
Joint POS tagging and Dependency Parsing
Joint Entity and Relation Extraction
Deep Learning Models
Neural Transition-based Models
Joint Learning, Joint Search
Joint Entity and Relation Extraction
![](../asset/Joint Models for NLP/20190117113620.png)
Joint Parsing and SRL
Joint Word Segmentation, POS Tagging and Dependency Parsing
![](../asset/Joint Models for NLP/20190117140629.png)
![](../asset/Joint Models for NLP/20190117140757.png)
Joint Extraction of Entities and Relations
Neural Graph-based Models (Multi-task Learning)
Joint Learning, Separate Search
![](../asset/Joint Models for NLP/20190117141801.png)
Cross Task
Identifying beneficial task relations
![](../asset/Joint Models for NLP/20190117142724.png)