Transformation invariance in pattern recognition - tangent distance and tangent propagation

Overview & Synopsis

In pattern recognition, statistical modeling, or regression, the amount of data is the most critical factor affecting the performance. If the amount of data and computational resources are near infinite, many algorithmes will probably converge to the optimal solution. When this is not the case, one has to introduce regularizers and a-priori knowledge to supplement the available data in order to boost the performance. Invariance (or known dependance) with respect to transformation of the input is a frequent occurrence of such an a-priori knowledge. In this chapter, we introduce the concept of tangent vectors, which compactly represent the essence of these transformation invariances, and two classes of algorithms, “Tangent distance” and ‘Tangent propagation”, which make use of these invariances to improve performance.