日韩性视频-久久久蜜桃-www中文字幕-在线中文字幕av-亚洲欧美一区二区三区四区-撸久久-香蕉视频一区-久久无码精品丰满人妻-国产高潮av-激情福利社-日韩av网址大全-国产精品久久999-日本五十路在线-性欧美在线-久久99精品波多结衣一区-男女午夜免费视频-黑人极品ⅴideos精品欧美棵-人人妻人人澡人人爽精品欧美一区-日韩一区在线看-欧美a级在线免费观看

歡迎訪問 生活随笔!

生活随笔

當前位置: 首頁 > 编程资源 > 编程问答 >内容正文

编程问答

迁移学习( Transfer Learning )

發布時間:2024/1/1 编程问答 33 豆豆
生活随笔 收集整理的這篇文章主要介紹了 迁移学习( Transfer Learning ) 小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

轉載于:?http://apex.sjtu.edu.cn/apex_wiki/Transfer Learning

遷移學習( Transfer Learning )

薛貴榮

???????在傳統的機器學習的框架下,學習的任務就是在給定充分訓練數據的基礎上來學習一個分類模型;然后利用這個學習到的模型來對測試文檔進行分類與預測。然而,我們看到機器學習算法在當前的Web挖掘研究中存在著一個關鍵的問題:一些新出現的領域中的大量訓練數據非常難得到。我們看到Web應用領域的發展非常快速。大量新的領域不斷涌現,從傳統的新聞,到網頁,到圖片,再到博客、播客等等。傳統的機器學習需要對每個領域都標定大量訓練數據,這將會耗費大量的人力與物力。而沒有大量的標注數據,會使得很多與學習相關研究與應用無法開展。其次,傳統的機器學習假設訓練數據與測試數據服從相同的數據分布。然而,在許多情況下,這種同分布假設并不滿足。通常可能發生的情況如訓練數據過期。這往往需要我們去重新標注大量的訓練數據以滿足我們訓練的需要,但標注新數據是非常昂貴的,需要大量的人力與物力。從另外一個角度上看,如果我們有了大量的、在不同分布下的訓練數據,完全丟棄這些數據也是非常浪費的。如何合理的利用這些數據就是遷移學習主要解決的問題。遷移學習可以從現有的數據中遷移知識,用來幫助將來的學習。遷移學習(Transfer Learning)的目標是將從一個環境中學到的知識用來幫助新環境中的學習任務。因此,遷移學習不會像傳統機器學習那樣作同分布假設。

???????我們在遷移學習方面的工作目前可以分為以下三個部分:同構空間下基于實例的遷移學習,同構空間下基于特征的遷移學習與異構空間下的遷移學習。我們的研究指出,基于實例的遷移學習有更強的知識遷移能力,基于特征的遷移學習具有更廣泛的知識遷移能力,而異構空間的遷移具有廣泛的學習與擴展能力。這幾種方法各有千秋。

1.同構空間下基于實例的遷移學習

????????基于實例的遷移學習的基本思想是,盡管輔助訓練數據和源訓練數據或多或少會有些不同,但是輔助訓練數據中應該還是會存在一部分比較適合用來訓練一個有效的分類模型,并且適應測試數據。于是,我們的目標就是從輔助訓練數據中找出那些適合測試數據的實例,并將這些實例遷移到源訓練數據的學習中去。在基于實例的遷移學習方面,我們推廣了傳統的AdaBoost算法,提出一種具有遷移能力的boosting算法:Tradaboosting [9],使之具有遷移學習的能力,從而能夠最大限度的利用輔助訓練數據來幫助目標的分類。我們的關鍵想法是,利用boosting的技術來過濾掉輔助數據中那些與源訓練數據最不像的數據。其中,boosting的作用是建立一種自動調整權重的機制,于是重要的輔助訓練數據的權重將會增加,不重要的輔助訓練數據的權重將會減小。調整權重之后,這些帶權重的輔助訓練數據將會作為額外的訓練數據,與源訓練數據一起從來提高分類模型的可靠度。

????????基于實例的遷移學習只能發生在源數據與輔助數據非常相近的情況下。但是,當源數據和輔助數據差別比較大的時候,基于實例的遷移學習算法往往很難找到可以遷移的知識。但是我們發現,即便有時源數據與目標數據在實例層面上并沒有共享一些公共的知識,它們可能會在特征層面上有一些交集。因此我們研究了基于特征的遷移學習,它討論的是如何利用特征層面上公共的知識進行學習的問題。

2.同構空間下基于特征的遷移學習

????????在基于特征的遷移學習研究方面,我們提出了多種學習的算法,如CoCC算法[7],TPLSA算法[4],譜分析算法[2]與自學習算法[3]等。其中利用互聚類算法產生一個公共的特征表示,從而幫助學習算法。我們的基本思想是使用互聚類算法同時對源數據與輔助數據進行聚類,得到一個共同的特征表示,這個新的特征表示優于只基于源數據的特征表示。通過把源數據表示在這個新的空間里,以實現遷移學習。應用這個思想,我們提出了基于特征的有監督遷移學習與基于特征的無監督遷移學習。

2.1 基于特征的有監督遷移學習

????????我們在基于特征的有監督遷移學習方面的工作是基于互聚類的跨領域分類[7],這個工作考慮的問題是:當給定一個新的、不同的領域,標注數據及其稀少時,如何利用原有領域中含有的大量標注數據進行遷移學習的問題。在基于互聚類的跨領域分類這個工作中,我們為跨領域分類問題定義了一個統一的信息論形式化公式,其中基于互聚類的分類問題的轉化成對目標函數的最優化問題。在我們提出的模型中,目標函數被定義為源數據實例,公共特征空間與輔助數據實例間互信息的損失。

2.2 基于特征的無監督遷移學習:自學習聚類

????????我們提出的自學習聚類算法[3]屬于基于特征的無監督遷移學習方面的工作。這里我們考慮的問題是:現實中可能有標記的輔助數據都難以得到,在這種情況下如何利用大量無標記數據輔助數據進行遷移學習的問題。自學習聚類 的基本思想是通過同時對源數據與輔助數據進行聚類得到一個共同的特征表示,而這個新的特征表示由于基于大量的輔助數據,所以會優于僅基于源數據而產生的特征表示,從而對聚類產生幫助。

???????上面提出的兩種學習策略(基于特征的有監督遷移學習與無監督遷移學習)解決的都是源數據與輔助數據在同一特征空間內的基于特征的遷移學習問題。當源數據與輔助數據所在的特征空間中不同時,我們還研究了跨特征空間的基于特征的遷移學習,它也屬于基于特征的遷移學習的一種。

3 異構空間下的遷移學習:翻譯學習

????????我們提出的翻譯學習[1][5]致力于解決源數據與測試數據分別屬于兩個不同的特征空間下的情況。在[1]中,我們使用大量容易得到的標注過文本數據去幫助僅有少量標注的圖像分類的問題,如上圖所示。我們的方法基于使用那些用有兩個視角的數據來構建溝通兩個特征空間的橋梁。雖然這些多視角數據可能不一定能夠用來做分類用的訓練數據,但是,它們可以用來構建翻譯器。通過這個翻譯器,我們把近鄰算法和特征翻譯結合在一起,將輔助數據翻譯到源數據特征空間里去,用一個統一的語言模型進行學習與分類。

?

List of Conferences and Workshops Where Transfer Learning Paper Appear
From:?http://www.cse.ust.hk/~sinnopan/conferenceTL.htm?

?

List of Conferences and Workshops Where Transfer Learning Paper AppearThis webpage will be updated regularly.

Main Conferences

Machine Learning and Artificial Intelligence Conferences
AAAI? 2008

Transfer Learning via Dimensionality Reduction??[Link]??[Bibtex]

Transferring Localization Models across Space??[Link]??[Bibtex]
Transferring Localization Models over Time??[Link]??[Bibtex]

Transferring Multi-device Localization Models using Latent Multi-task Learning?[Link]??[Bibtex]

Text Categorization with Knowledge Transfer from Heterogeneous Data Sources?[Link]??[Bibtex]

Zero-data Learning of New Tasks??[Link]??[Bibtex]

2007

Transferring Naive Bayes Classifiers for Text Classification??[Link]??[Bibtex]

Mapping and Revising Markov Logic Networks for Transfer Learning??[Link]?[Bibtex]

Measuring the Level of Transfer Learning by an AP Physics Problem-Solver?[Link]??[Bibtex]

2006 Using Homomorphisms to Transfer Options across Continuous Reinforcement Learning Domains??[Link]??[Bibtex]

Value-Function-Based Transfer for Reinforcement Learning Using Structure Mapping??[Link]??[Bibtex]

IJCAI

2009

Transfer Learning Using Task-Level Features with Application to Information Retrieval??[Link]??[Bibtex]

Transfer Learning from Minimal Target Data by Mapping across Relational Domains??[Link]??[Bibtex]

Domain Adaptation via Transfer Component Analysis??[Link]??[Bibtex]

Knowledge Transfer on Hybrid Graph??[Link]??[Bibtex]

Manifold Alignment without Correspondence??[Link]??[Bibtex]

Robust Distance Metric Learning with Auxiliary Knowledge??[Link]??[Bibtex]

Can Movies and Books Collaborate? Cross-Domain Collaborative Filtering for Sparsity Reduction??[Link]??[Bibtex]

Exponential Family Sparse Coding with Application to Self-taught Learning ?[Link]??[Bibtex]

2007

Learning and Transferring Action Schemas??[Link]??[Bibtex]

General Game Learning Using Knowledge Transfer??[Link]??[Bibtex]

Building Portable Options: Skill Transfer in Reinforcement Learning??[Link]?[Bibtex]

Transfer Learning in Real-Time Strategy Games Using Hybrid CBR/RL??[Link]?[Bibtex]

An Experts Algorithm for Transfer Learning??[Link]??[Bibtex]

Transferring Learned Control-Knowledge between Planners??[Link]??[Bibtex]

Effective Control Knowledge Transfer through Learning Skill and Representation Hierarchies??[Link]??[Bibtex]

Efficient Bayesian Task-Level Transfer Learning??[Link]??[Bibtex]

ICML

2009

Deep Transfer via Second-Order Markov Logic??[Link]??[Bibtex]

Feature Hashing for Large Scale Multitask Learning??[Link]??[Bibtex]

A Convex Formulation for Learning Shared Structures from Multiple Tasks?[Link]??[Bibtex]

EigenTransfer: A Unified Framework for Transfer Learning??[Link]??[Bibtex]

Domain Adaptation from Multiple Sources via Auxiliary Classifiers??[Link]?[Bibtex]

Transfer Learning for Collaborative Filtering via a Rating-Matrix Generative Model??[Link]??[Bibtex]

2008

Bayesian Multiple Instance Learning: Automatic Feature Selection and Inductive Transfer??[Link]??[Bibtex]

Multi-Task Learning for HIV Therapy Screening??[Link]??[Bibtex]

Self-taught Clustering??[Link]??[Bibtex]

Manifold Alignment using Procrustes Analysis??[Link]??[Bibtex]

Automatic Discovery and Transfer of MAXQ Hierarchies??[Link]??[Bibtex]

Transfer of Samples in Batch Reinforcement Learning??[Link]??[Bibtex]

Hierarchical Kernel Stick-Breaking Process for Multi-Task Image Analysis??[Link]?[Bibtex]

Multi-Task Compressive Sensing with Dirichlet Process Priors??[Link]??[Bibtex]

A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning??[Link]??[Bibtex]

2007

Boosting for Transfer Learning??[Link]??[Bibtex]

Self-taught Learning: Transfer Learning from Unlabeled Data??[Link]??[Bibtex]

Robust Multi-Task Learning with t-Processes??[Link]??[Bibtex]

Multi-Task Learning for Sequential Data via iHMMs and the Nested Dirichlet Process??[Link]??[Bibtex]

Cross-Domain Transfer for Reinforcement Learning??[Link]??[Bibtex]

Learning a Meta-Level Prior for Feature Relevance from Multiple Related Tasks?[Link]??[Bibtex]

Multi-Task Reinforcement Learning: A Hierarchical Bayesian Approach??[Link]?[Bibtex]

The Matrix Stick-Breaking Process for Flexible Multi-Task Learning??[Link]?[Bibtex]

Asymptotic Bayesian Generalization Error When Training and Test Distributions Are Different??[Link]??[Bibtex]

Discriminative Learning for Differing Training and Test Distributions??[Link]?[Bibtex]

2006

Autonomous Shaping: Knowledge Transfer in Reinforcement Learning??[Link]?[Bibtex]
Constructing Informative Priors using Transfer Learning??[Link]??[Bibtex]
NIPS 2008

Clustered Multi-Task Learning: A Convex Formulation??[Link]??[Bibtex]

Multi-task Gaussian Process Learning of Robot Inverse Dynamics??[Link]?[Bibtex]

Transfer Learning by Distribution Matching for Targeted Advertising??[Link]?[Bibtex]

Translated Learning: Transfer Learning across Different Feature Spaces??[Link]?[Bibtex]

An empirical Analysis of Domain Adaptation Algorithms for Genomic Sequence Analysis??[Link]??[Bibtex]

Domain Adaptation with Multiple Sources??[Link]??[Bibtex]

2007

Learning Bounds for Domain Adaptation??[Link]??[Bibtex]

Transfer Learning using Kolmogorov Complexity: Basic Theory and Empirical Evaluations??[Link]??[Bibtex]

A Spectral Regularization Framework for Multi-Task Structure Learning??[Link]?[Bibtex]

Multi-task Gaussian Process Prediction??[Link]??[Bibtex]

Semi-Supervised Multitask Learning??[Link]??[Bibtex]

Gaussian Process Models for Link Analysis and Transfer Learning??[Link]??[Bibtex]

Multi-Task Learning via Conic Programming??[Link]??[Bibtex]

Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation??[Link]??[Bibtex]

2006

Correcting Sample Selection Bias by Unlabeled Data??[Link]??[Bibtex]

Dirichlet-Enhanced Spam Filtering based on Biased Samples??[Link]??[Bibtex]

Analysis of Representations for Domain Adaptation??[Link]??[Bibtex]

Multi-Task Feature Learning??[Link]??[Bibtex]

AISTAT

2009

A Hierarchical Nonparametric Bayesian Approach to Statistical Language Model Domain Adaptation??[Link]??[Bibtex]

2007 Kernel Multi-task Learning using Task-specific Features??[Link]??[Bibtex]
Inductive Transfer for Bayesian Network Structure Learning??[Link]??[Bibtex]
ECML/PKDD

2009

Relaxed Transfer of Different Classes via Spectral Partition??[Link]??[Bibtex]
Feature Selection by Transfer Learning with Linear Regularized Models??[Link]?[Bibtex]
Semi-Supervised Multi-Task Regression??[Link]??[Bibtex]
2008

Actively Transfer Domain Knowledge??[Link]??[Bibtex]

An Algorithm for Transfer Learning in a Heterogeneous Environment??[Link]?[Bibtex]
Transferred Dimensionality Reduction??[Link]??[Bibtex]
Modeling Transfer Relationships between Learning Tasks for Improved Inductive Transfer??[Link]??[Bibtex]
Kernel-Based Inductive Transfer??[Link]??[Bibtex]
2007 Graph-Based Domain Mapping for Transfer Learning in General Games??[Link]?[Bibtex]
Bridged Refinement for Transfer Learning??[Link]??[Bibtex]
Transfer Learning in Reinforcement Learning Problems Through Partial Policy Recycling??[Link]??[Bibtex]
Domain Adaptation of Conditional Probability Models via Feature Subsetting?[Link]??[Bibtex]
2006 Skill Acquisition via Transfer Learning and Advice Taking??[Link]??[Bibtex]

COLT?

2009 Online Multi-task Learning with Hard Constraints??[Link]??[Bibtex]
Taking Advantage of Sparsity in Multi-Task Learning??[Link]??[Bibtex]
Domain Adaptation: Learning Bounds and Algorithms??[Link]??[Bibtex]
2008 Learning coordinate gradients with multi-task kernels??[Link]??[Bibtex]
Linear Algorithms for Online Multitask Classification??[Link]??[Bibtex]
2007 Multitask Learning with Expert Advice??[Link]??[Bibtex]
2006 Online Multitask Learning??[Link]??[Bibtex]

UAI?

2009 Bayesian Multitask Learning with Latent Hierarchies??[Link]??[Bibtex]
Multi-Task Feature Learning Via Efficient L2,1-Norm Minimization??[Link]?[Bibtex]
2008 Convex Point Estimation using Undirected Bayesian Transfer Hierarchies??[Link]?[Bibtex]
Data Mining Conferences?
KDD? 2009

Cross Domain Distribution Adaptation via Kernel Mapping??[Link]??[Bibtex]

Extracting Discriminative Concepts for Domain Adaptation in Text Mining??[Link]?[Bibtex]

2008

Spectral domain-transfer learning??[Link]??[Bibtex]

Knowledge transfer via multiple model local structure mapping??[Link]??[Bibtex]
2007

Co-clustering based Classification for Out-of-domain Documents??[Link]??[Bibtex]

2006

Reverse Testing: An Efficient Framework to Select Amongst Classifiers under Sample Selection Bias??[Link]??[Bibtex]

ICDM?

2008 Unsupervised Cross-domain Learning by Interaction Information Co-clustering?[Link]??[Bibtex]
Using Wikipedia for Co-clustering Based Cross-domain Text Classification??[Link]?[Bibtex]
SDM 2008 Type-Independent Correction of Sample Selection Bias via Structural Discovery and Re-balancing??[Link]??[Bibtex]
Direct Density Ratio Estimation for Large-scale Covariate Shift Adaptation?[Link]??[Bibtex]
2007 On Sample Selection Bias and Its Efficient Correction via Model Averaging and Unlabeled Examples??[Link]??[Bibtex]
Probabilistic Joint Feature Selection for Multi-task Learning??[Link]??[Bibtex]
Application Conferences?
SIGIR? 2009 Mining Employment Market via Text Block Detection and Adaptive Cross-Domain Information Extraction??[Link]??[Bibtex]
Knowledge transformation for cross-domain sentiment classification??[Link]?[Bibtex]?
2008 Topic-bridged PLSA for cross-domain text classification??[Link]??[Bibtex]
2007 Cross-Lingual Query Suggestion Using Query Logs of Different Languages?[Link]??[Bibtex]
2006 Tackling Concept Drift by Temporal Inductive Transfer??[Link]??[Bibtex]
Constructing Informative Prior Distributions from Domain Knowledge in Text Classification??[Link]??[Bibtex]
Building Bridges for Web Query Classification??[Link]??[Bibtex]
WWW? 2009

Latent Space Domain Transfer between High Dimensional Overlapping Distributions??[Link]??[Bibtex]

2008

Can Chinese web pages be classified with English data source???[Link]??[Bibtex]

ACL? 2009

Transfer Learning, Feature Selection and Word Sense Disambiguation??[Link]?[Bibtex]

Graph Ranking for Sentiment Transfer??[Link]??[Bibtex]

Multi-Task Transfer Learning for Weakly-Supervised Relation Extraction??[Link]?[Bibtex]

Cross-Domain Dependency Parsing Using a Deep Linguistic Grammar??[Link]?[Bibtex]

Heterogeneous Transfer Learning for Image Clustering via the SocialWeb??[Link]?[Bibtex]

2008

Exploiting Feature Hierarchy for Transfer Learning in Named Entity Recognition?[Link]??[Bibtex]

Multi-domain Sentiment Classification??[Link]??[Bibtex]
Active Sample Selection for Named Entity Transliteration??[Link]??[Bibtex]
Mining Wiki Resources for Multilingual Named Entity Recognition??[Link]??[Bibtex]
Multi-Task Active Learning for Linguistic Annotations??[Link]??[Bibtexs]
2007

Domain Adaptation with Active Learning for Word Sense Disambiguation??[Link]?[Bibtex]

Frustratingly Easy Domain Adaptation??[Link]??[Bibtex]
Instance Weighting for Domain Adaptation in NLP??[Link]??[Bibtex]
Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification??[Link]??[Bibtex]
Self-Training for Enhancement and Domain Adaptation of Statistical Parsers Trained on Small Datasets??[Link]??[Bibtex]
2006 Estimating Class Priors in Domain Adaptation for Word Sense Disambiguation?[Link]??[Bibtex]
Simultaneous English-Japanese Spoken Language Translation Based on Incremental Dependency Parsing and Transfer??[Link]??[Bibtex]
CVPR

2009

Domain Transfer SVM for Video Concept Detection??[Link]??[Bibtex]
Boosted Multi-Task Learning for Face Verification With Applications to Web Image and Video Search??[Link]??[Bibtex]

2008

Transfer Learning for Image Classification with Sparse Prototype Representations??[Link]??[Bibtex]

Workshops

NIPS 2005 Workshop – Inductive Transfer: 10 Years Later
NIPS 2005 Workshop – Interclass Transfer
NIPS 2006 Workshop – Learning when test and training inputs have different distributions
AAAI 2008 Workshop – Transfer Learning for Complex Tasks

?

本文引用地址:?http://www.sciencenet.cn/m/user_content.aspx?id=268960

總結

以上是生活随笔為你收集整理的迁移学习( Transfer Learning )的全部內容,希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯,歡迎將生活随笔推薦給好友。