机器学习中的转导是什么？

转导模型一般用于基于RNN的序列预测问题，比如自然语言处理。

接下来会围绕以下三点展开：

转导的一般定义以及一些特定领域的研究。
机器学习中的转导是什么？
序列预测问题中的转导意味着什么？

1 什么是转导？

何为转导？我们先从字典中的定义开始。

transduce: to convert (something, such as energy or a message) into another form essentially sense organs transduce physical energy into a nervous signal

— Merriam-Webster Dictionary (online), 2017

这是一个在电子与信号处理领域常见的术语。“换能器”，一种能将声音转换为看能量的元件，反之亦然。

All signal processing begins with an input transducer. The input transducer takes the input signal and converts it to an electrical signal. In signal-processing applications, the transducer can take many forms. A common example of an input transducer is a microphone.

— Digital Signal Processing Demystified, 1997

在生物学中，特别是遗传学，转导指的是讲一种微生物的遗传物质转移给另一种微生物的过程。

transduction: the action or process of transducing; especially : the transfer of genetic material from one microorganism to another by a viral agent (such as a bacteriophage)

— Merriam-Webster Dictionary (online), 2017

所以，一般地讲，我们可以将转导看作是将信号转换为另一种形式。

在将声波转换为电信号的系统中，这种信号处理的描述是最贴切的。每种声音都能在特定采样等级被特定电信号代表。

信号处理中的换能器

2 转导学习（Transduction Learning）

转导或转导学习在统计学习中指的是在特定领域中根据给定样本预测特定样本。它与归纳学习(Inductive Learning)、演绎学习(Deductive Learning)等其他学习类型形成对比。

Induction, deriving the function from the given data. Deduction, deriving the values of the given function for points of interest. Transduction, deriving the values of the unknown function for points of interest from the given data.

— Page 169, The Nature of Statistical Learning Theory, 1995

归纳、演绎、与转导的关系

这是监督学习中一个有趣的框架，在这个框架中，“从数据中逼近映射函数并利用它进行预测”这一经典问题被认为比要求的更困难。取而代之的是，直接从领域中的真实样本进行具体预测。不需要函数近似。

The model of estimating the value of a function at a given point of interest describes a new concept of inference: moving from the particular to the particular. We call this type of inference transductive inference. Note that this concept of inference appears when one would like to get the best result from a restricted amount of information.

— Page 169, The Nature of Statistical Learning Theory, 1995

一个转导算法的典型例子就是K临近算法(k-Nearest Neighbors algorithm)，它不对训练数据进行建模，而在每次预测时都直接使用训练数据。

Transduction is naturally related to a set of algorithms known as instance-based, or case-based learning. Perhaps, the most well-known algorithm in this class is k-nearest neighbour algorithm.

— Learning by Transduction, 1998

3 语言学中的转导

在自然语言处理领域，转导已经被应用。例如，有一种叫做“转导语法”的概念指的是一种将一个语言的实例转化为两一种语言的规则。

A transduction grammar describes a structurally correlated pair of languages. It generates sentence pairs, rather than sentences. The language-1 sentence is (intended to be) a translation of the language-2 sentence.

— Page 460, Handbook of Natural Language Processing, 2000.

计算理论中还有一个 “有穷状态转换器”（FST，finite-state transducter）的概念，在谈到将一组符号映射到另一组符号的翻译任务时，我们会引用这个概念。需要注意的是，每个输入产生一个输出。

A finite state transducer consists of a number of states. When transitioning between states an input symbol is consumed and an output symbol is emitted.

— Page 294, Statistical Machine Translation, 2010.

4 序列预测中的转导

约阿夫·戈德伯格在他的语言处理神经网络教材中将转换器(transducer)定义为一种用于NLP任务的特定网络模型。狭义上，转换器是指每提供一个输入时间步，就输出一个时间步的模型。这与语言学的用法相对应，特别是与有限状态转换器相对应。

编码器-解码器模型结构中的解码器允许为给定的输入序列提供不同数量的输出，这打破了定义中的 “每个输入一个输出”。

RNN转换器训练图

更加一般地讲，转导被用于自然语言处理的序列预测任务，尤其是翻译任务。

例如，爱德华·格蕾芬斯特将转导描述为输入字符串到输出字符串的映射。

Many natural language processing (NLP) tasks can be viewed as transduction problems, that is learning to convert one string into another. Machine translation is a prototypical example of transduction and recent results indicate that Deep RNNs have the ability to encode long source strings and produce coherent translations

— Learning to Transduce with Unbounded Memory, 2015.

他们进一步提供了一系列具体的NLP任务，让这个广泛的定义更加具体。

String transduction is central to many applications in NLP, from name transliteration and spelling correction, to inflectional morphology and machine translation

阿莱克斯·格拉维斯也使用转导作为转换的同义词，并提供了一份符合该定义的 NLP 任务示例清单，非常有用。

Many machine learning tasks can be expressed as the transformation—or transduction—of input sequences into output sequences: speech recognition, machine translation, protein secondary structure prediction and text-to-speech to name but a few.

— Sequence Transduction with Recurrent Neural Networks, 2012.

总结

至此，我们能够列出转导自然语言处理的任务如下：

转写：根据源语言的例子，转写出目标语言的单词
拼写纠正：在单词拼写错误是给出正确拼写
屈折词形态学：根据源序列和上下文产生新序列
机器翻译：根据给定的源语言例子，生成目标语言的语句
语音识别：根据给定的音频生成文本序列
蛋白质二级结构预测：根据给定的氨基酸序列预测3D结构
语音合成：根据给定文本生成音频

最后，除了指代广泛的NLP问题和RNN序列预测问题外，一些新方法也明确命名为转导。纳夫迪普·贾伊特利等人将他们新的RNN序列到序列预测方法命名为“神经转换器”(Neural Transducer)。

we present a Neural Transducer, a more general class of sequence-to-sequence learning models. Neural Transducer can produce chunks of outputs (possibly of zero length) as blocks of inputs arrive – thus satisfying the condition of being “online”. The model generates outputs for each block by using a transducer RNN that implements a sequence-to-sequence model.

— A Neural Transducer, 2016

https://machinelearningmastery.com/transduction-in-machine-learning/