A minimax framework for classification with applications to images and high dimensional data

Qiang Cheng, Hongbo Zhou, Jie Cheng, Huiqing Li

Research output: Contribution to journalArticleResearchpeer-review

Abstract

This paper introduces a minimax framework for multiclass classification, which is applicable to general data including, in particular, imagery and other types of high-dimensional data. The framework consists of estimating a representation model that minimizes the fitting errors under a class of distortions of interest to an application, and deriving subsequently categorical information based on the estimated model. A variety of commonly used regression models, including lasso, elastic net and ridge regression, can be regarded as special cases that correspond to specific classes of distortions. Optimal decision rules are derived for this classification framework. By using kernel techniques the framework can account for nonlinearity in the input space. To demonstrate the power of the framework we consider a class of signal-dependent distortions and build a new family of classifiers as new special cases. This family of new methods-minimax classification with generalized multiplicative distortions-often outperforms the state-of-the-art classification methods such as the support vector machine in accuracy. Extensive experimental results on images, gene expressions and other types of data verify the effectiveness of the proposed framework.

Original languageEnglish
Article number6824834
Pages (from-to)2117-2130
Number of pages14
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume36
Issue number11
DOIs
StatePublished - 1 Nov 2014

Fingerprint

High-dimensional Data
Minimax
Gene expression
Elastic Net
Minimax Methods
Support vector machines
Multi-class Classification
Ridge Regression
Lasso
Classifiers
Decision Rules
Categorical
Gene Expression
Framework
Support Vector Machine
Multiplicative
Regression Model
Classifier
Nonlinearity
kernel

Keywords

  • Bayesian optimal decision
  • generalized multiplicative distortion
  • high dimensional data
  • kernel
  • minimax optimization
  • Multiclass classification

Cite this

@article{975d3738b41d402290d7167afa33b1fb,
title = "A minimax framework for classification with applications to images and high dimensional data",
abstract = "This paper introduces a minimax framework for multiclass classification, which is applicable to general data including, in particular, imagery and other types of high-dimensional data. The framework consists of estimating a representation model that minimizes the fitting errors under a class of distortions of interest to an application, and deriving subsequently categorical information based on the estimated model. A variety of commonly used regression models, including lasso, elastic net and ridge regression, can be regarded as special cases that correspond to specific classes of distortions. Optimal decision rules are derived for this classification framework. By using kernel techniques the framework can account for nonlinearity in the input space. To demonstrate the power of the framework we consider a class of signal-dependent distortions and build a new family of classifiers as new special cases. This family of new methods-minimax classification with generalized multiplicative distortions-often outperforms the state-of-the-art classification methods such as the support vector machine in accuracy. Extensive experimental results on images, gene expressions and other types of data verify the effectiveness of the proposed framework.",
keywords = "Bayesian optimal decision, generalized multiplicative distortion, high dimensional data, kernel, minimax optimization, Multiclass classification",
author = "Qiang Cheng and Hongbo Zhou and Jie Cheng and Huiqing Li",
year = "2014",
month = "11",
day = "1",
doi = "10.1109/TPAMI.2014.2327978",
language = "English",
volume = "36",
pages = "2117--2130",
journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence",
issn = "0162-8828",
publisher = "IEEE Computer Society",
number = "11",

}

A minimax framework for classification with applications to images and high dimensional data. / Cheng, Qiang; Zhou, Hongbo; Cheng, Jie; Li, Huiqing.

In: IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 36, No. 11, 6824834, 01.11.2014, p. 2117-2130.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - A minimax framework for classification with applications to images and high dimensional data

AU - Cheng, Qiang

AU - Zhou, Hongbo

AU - Cheng, Jie

AU - Li, Huiqing

PY - 2014/11/1

Y1 - 2014/11/1

N2 - This paper introduces a minimax framework for multiclass classification, which is applicable to general data including, in particular, imagery and other types of high-dimensional data. The framework consists of estimating a representation model that minimizes the fitting errors under a class of distortions of interest to an application, and deriving subsequently categorical information based on the estimated model. A variety of commonly used regression models, including lasso, elastic net and ridge regression, can be regarded as special cases that correspond to specific classes of distortions. Optimal decision rules are derived for this classification framework. By using kernel techniques the framework can account for nonlinearity in the input space. To demonstrate the power of the framework we consider a class of signal-dependent distortions and build a new family of classifiers as new special cases. This family of new methods-minimax classification with generalized multiplicative distortions-often outperforms the state-of-the-art classification methods such as the support vector machine in accuracy. Extensive experimental results on images, gene expressions and other types of data verify the effectiveness of the proposed framework.

AB - This paper introduces a minimax framework for multiclass classification, which is applicable to general data including, in particular, imagery and other types of high-dimensional data. The framework consists of estimating a representation model that minimizes the fitting errors under a class of distortions of interest to an application, and deriving subsequently categorical information based on the estimated model. A variety of commonly used regression models, including lasso, elastic net and ridge regression, can be regarded as special cases that correspond to specific classes of distortions. Optimal decision rules are derived for this classification framework. By using kernel techniques the framework can account for nonlinearity in the input space. To demonstrate the power of the framework we consider a class of signal-dependent distortions and build a new family of classifiers as new special cases. This family of new methods-minimax classification with generalized multiplicative distortions-often outperforms the state-of-the-art classification methods such as the support vector machine in accuracy. Extensive experimental results on images, gene expressions and other types of data verify the effectiveness of the proposed framework.

KW - Bayesian optimal decision

KW - generalized multiplicative distortion

KW - high dimensional data

KW - kernel

KW - minimax optimization

KW - Multiclass classification

UR - http://www.scopus.com/inward/record.url?scp=84907818752&partnerID=8YFLogxK

U2 - 10.1109/TPAMI.2014.2327978

DO - 10.1109/TPAMI.2014.2327978

M3 - Article

VL - 36

SP - 2117

EP - 2130

JO - IEEE Transactions on Pattern Analysis and Machine Intelligence

JF - IEEE Transactions on Pattern Analysis and Machine Intelligence

SN - 0162-8828

IS - 11

M1 - 6824834

ER -