Sparse Metric Learning via Smooth Optimization

Ying Yiming; Kaizhu  Huang; ICG Campbell

Sparse Metric Learning via Smooth Optimization

Ying Yiming
, Kaizhu Huang
, ICG Campbell

Intelligent Systems Laboratory

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

92 Citations (Scopus)

Abstract

In this paper we study the problem of learning a low-rank (sparse) distance matrix. We propose a novel metric learning model which can simultaneously conduct dimension reduction and learn a distance matrix. The sparse representation involves a mixed-norm regularization which is non-convex. We then show that it can be equivalently formulated as a convex saddle (min-max) problem. From this saddle representation, we develop an efficient smooth optimization approach [17] for sparse metric learning, although the learning model is based on a nondifferentiable loss function. Finally, we run experiments to validate the effectiveness and efficiency of our sparse metric learning model on various datasets.

Translated title of the contribution	Sparse Metric Learning via Smooth Optimization
Original language	English
Title of host publication	Advances in Neural Information Processing Systems 22: 23rd Annual Conference on Neural Information Processing Systems 2009. Proceedings of a meeting held 7-10 December 2009, Vancouver, British Columbia, Canada
Subtitle of host publication	NIPS 22
Editors	Yoshua Bengio, Dale Schuurmans, John D. Lafferty, Christopher K.I. Williams, Aron Culotta
Publisher	Curran Associates, Inc
Pages	2214 - 2222
Number of pages	8
Volume	22
ISBN (Electronic)	9781615679119
Publication status	Published - 2009

Publication series

Name	Neural Information Processing Systems (NIPS)

Bibliographical note

Name and Venue of Event: Vancouver, Canada
Conference Proceedings/Title of Journal: Advances in Neural Information Processing Systems Conference Organiser: NIPS Foundation

Keywords

metric
learning
sparse

Access to Document

Handle.net

Persistent link

Cite this

Yiming, Y., Huang, K., & Campbell, ICG. (2009). Sparse Metric Learning via Smooth Optimization. In Y. Bengio, D. Schuurmans, J. D. Lafferty, C. K. I. Williams, & A. Culotta (Eds.), Advances in Neural Information Processing Systems 22: 23rd Annual Conference on Neural Information Processing Systems 2009. Proceedings of a meeting held 7-10 December 2009, Vancouver, British Columbia, Canada: NIPS 22 (Vol. 22, pp. 2214 - 2222). (Neural Information Processing Systems (NIPS)). Curran Associates, Inc. http://books.nips.cc/papers/files/nips22/NIPS2009_0304.pdf

Yiming, Ying ; Huang, Kaizhu ; Campbell, ICG. / Sparse Metric Learning via Smooth Optimization. Advances in Neural Information Processing Systems 22: 23rd Annual Conference on Neural Information Processing Systems 2009. Proceedings of a meeting held 7-10 December 2009, Vancouver, British Columbia, Canada: NIPS 22. editor / Yoshua Bengio ; Dale Schuurmans ; John D. Lafferty ; Christopher K.I. Williams ; Aron Culotta. Vol. 22 Curran Associates, Inc, 2009. pp. 2214 - 2222 (Neural Information Processing Systems (NIPS)).

@inproceedings{a5390eb35e5e4961b788c1fcde9bc805,

title = "Sparse Metric Learning via Smooth Optimization",

abstract = "In this paper we study the problem of learning a low-rank (sparse) distance matrix. We propose a novel metric learning model which can simultaneously conduct dimension reduction and learn a distance matrix. The sparse representation involves a mixed-norm regularization which is non-convex. We then show that it can be equivalently formulated as a convex saddle (min-max) problem. From this saddle representation, we develop an efficient smooth optimization approach [17] for sparse metric learning, although the learning model is based on a nondifferentiable loss function. Finally, we run experiments to validate the effectiveness and efficiency of our sparse metric learning model on various datasets.",

keywords = "metric, learning, sparse",

author = "Ying Yiming and Kaizhu Huang and ICG Campbell",

note = "Name and Venue of Event: Vancouver, Canada Conference Proceedings/Title of Journal: Advances in Neural Information Processing Systems Conference Organiser: NIPS Foundation",

year = "2009",

language = "English",

volume = "22",

series = "Neural Information Processing Systems (NIPS)",

publisher = "Curran Associates, Inc",

pages = "2214 -- 2222",

editor = "Yoshua Bengio and Dale Schuurmans and Lafferty, \{John D.\} and Williams, \{Christopher K.I. \} and Aron Culotta",

booktitle = "Advances in Neural Information Processing Systems 22: 23rd Annual Conference on Neural Information Processing Systems 2009. Proceedings of a meeting held 7-10 December 2009, Vancouver, British Columbia, Canada",

}

Yiming, Y, Huang, K & Campbell, ICG 2009, Sparse Metric Learning via Smooth Optimization. in Y Bengio, D Schuurmans, JD Lafferty, CKI Williams & A Culotta (eds), Advances in Neural Information Processing Systems 22: 23rd Annual Conference on Neural Information Processing Systems 2009. Proceedings of a meeting held 7-10 December 2009, Vancouver, British Columbia, Canada: NIPS 22. vol. 22, Neural Information Processing Systems (NIPS), Curran Associates, Inc, pp. 2214 - 2222. <http://books.nips.cc/papers/files/nips22/NIPS2009_0304.pdf>

Sparse Metric Learning via Smooth Optimization. / Yiming, Ying; Huang, Kaizhu ; Campbell, ICG.
Advances in Neural Information Processing Systems 22: 23rd Annual Conference on Neural Information Processing Systems 2009. Proceedings of a meeting held 7-10 December 2009, Vancouver, British Columbia, Canada: NIPS 22. ed. / Yoshua Bengio; Dale Schuurmans; John D. Lafferty; Christopher K.I. Williams; Aron Culotta. Vol. 22 Curran Associates, Inc, 2009. p. 2214 - 2222 (Neural Information Processing Systems (NIPS)).

Research output: Chapter in Book/Report/Conference proceeding › Conference Contribution (Conference Proceeding)

TY - GEN

T1 - Sparse Metric Learning via Smooth Optimization

AU - Yiming, Ying

AU - Huang, Kaizhu

AU - Campbell, ICG

N1 - Name and Venue of Event: Vancouver, Canada Conference Proceedings/Title of Journal: Advances in Neural Information Processing Systems Conference Organiser: NIPS Foundation

PY - 2009

Y1 - 2009

N2 - In this paper we study the problem of learning a low-rank (sparse) distance matrix. We propose a novel metric learning model which can simultaneously conduct dimension reduction and learn a distance matrix. The sparse representation involves a mixed-norm regularization which is non-convex. We then show that it can be equivalently formulated as a convex saddle (min-max) problem. From this saddle representation, we develop an efficient smooth optimization approach [17] for sparse metric learning, although the learning model is based on a nondifferentiable loss function. Finally, we run experiments to validate the effectiveness and efficiency of our sparse metric learning model on various datasets.

AB - In this paper we study the problem of learning a low-rank (sparse) distance matrix. We propose a novel metric learning model which can simultaneously conduct dimension reduction and learn a distance matrix. The sparse representation involves a mixed-norm regularization which is non-convex. We then show that it can be equivalently formulated as a convex saddle (min-max) problem. From this saddle representation, we develop an efficient smooth optimization approach [17] for sparse metric learning, although the learning model is based on a nondifferentiable loss function. Finally, we run experiments to validate the effectiveness and efficiency of our sparse metric learning model on various datasets.

KW - metric

KW - learning

KW - sparse

M3 - Conference Contribution (Conference Proceeding)

VL - 22

T3 - Neural Information Processing Systems (NIPS)

SP - 2214

EP - 2222

BT - Advances in Neural Information Processing Systems 22: 23rd Annual Conference on Neural Information Processing Systems 2009. Proceedings of a meeting held 7-10 December 2009, Vancouver, British Columbia, Canada

A2 - Bengio, Yoshua

A2 - Schuurmans, Dale

A2 - Lafferty, John D.

A2 - Williams, Christopher K.I.

A2 - Culotta, Aron

PB - Curran Associates, Inc

ER -

Yiming Y, Huang K, Campbell ICG. Sparse Metric Learning via Smooth Optimization. In Bengio Y, Schuurmans D, Lafferty JD, Williams CKI, Culotta A, editors, Advances in Neural Information Processing Systems 22: 23rd Annual Conference on Neural Information Processing Systems 2009. Proceedings of a meeting held 7-10 December 2009, Vancouver, British Columbia, Canada: NIPS 22. Vol. 22. Curran Associates, Inc. 2009. p. 2214 - 2222. (Neural Information Processing Systems (NIPS)).

Sparse Metric Learning via Smooth Optimization

Abstract

Publication series

Bibliographical note

Keywords

Access to Document

Handle.net

Fingerprint

Cite this