# Optimal Mechanisms for Machine Learning: A Game-Theoretic Approach to Designing Machine Learning Competitions

Optimal Mechanisms for Machine Learning: A Game-Theoretic Approach to Designing Machine Learning Competitions - Descarga este documento en PDF. Documentación en PDF para descargar gratis. Disponible también para leer online.

competition, machine learning, game theory, mechanism design, optimal

Ajallooeian, Mohammad Mahdi

Supervisor and department: Szepesvári, Csaba Computing Science

Examining committee member and department: György, András Computing Science Jinag, Hai Electrical and Computer Engineering Schuurmans, Dale Computing Science Bulitko, Vadim Computing Science

Department: Department of Computing Science

Specialization:

Date accepted: 2013-01-31T12:18:00Z

Graduation date: 2013-06

Degree: Master of Science

Degree level: Master's

Abstract: In this thesis we consider problems where a self-interested entity, called the principal, has private access to some data that she wishes to use to solve a prediction problem by outsourcing the development of the predictor to some other parties. Assuming the principal, who needs the machine learning solution, and the potential providers of the solution are two independent, self-interested agents, which is the case for many real-world situations, this then becomes a game-theoretic problem. We propose mathematical models for variants of this problem by borrowing techniques from the literature of mechanism design and provide principled solutions. We consider experimental design when there are multiple self-interested agents involved in developing a solution for a machine learning problem. A first case is when there is a public competition, each agent offers a single solution and solutions are available off-the-shelf to the agents: there is no development cost included. The problem considered is to find a set of payment rules that guarantees to maximize the profit of the principal on expectation even when the developers are self-interested. The solution depends on the distribution of the skill-level of developers available, which is assumed to be known. To deal with our problem, the standard mechanism design techniques are revisited and extended in a number of ways. In particular, a general approach is given that allows the design of payment rules more generally, mechanisms when such payment rules must depend on some quantity that becomes known only after the mechanism is executed. This extension plays a key role in our solution to the machine learning payment-rule design problem, where data must be kept private otherwise the developers could submit -overfitting- predictors, yet the principal’s profit and thus the payment should depend on the performance of the predictor chosen on theprivate data. Then, we address other interesting variants of the problem and provide solutions : when a single developer can submit multiple solutions, and when the solution is to be developed in multiple stages, or when the development cost is non-zero.

Language: English

DOI: doi:10.7939-R3172B

Rights: Permission is hereby granted to the University of Alberta Libraries to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves all other publication and other rights in association with the copyright in the thesis and, except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatsoever without the author's prior written permission.

Autor: ** Ajallooeian, Mohammad Mahdi **

Fuente: https://era.library.ualberta.ca/

## Introducción

“The maximum principle for the fulfilling of wishes guides the
building up of the world by requiring that it be the best possible.”
– Kurt F.

Gödel
University of Alberta
Optimal Mechanisms for Machine Learning
A Game-Theoretic Approach to Designing Machine Learning
Competitions
by
Mohammad Mahdi Ajallooeian
A thesis submitted to the Faculty of Graduate Studies and Research
in partial fulfillment of the requirements for the degree of
Master of Science
Department of Computing Science
c
Mohammad
Mahdi Ajallooeian
Spring 2013
Edmonton, Alberta
Permission is hereby granted to the University of Alberta Libraries to reproduce
single copies of this thesis and to lend or sell such copies for private, scholarly or
scientific research purposes only.

Where the thesis is converted to, or otherwise
made available in digital form, the University of Alberta will advise potential users
of the thesis of these terms.
The author reserves all other publication and other rights in association with the
copyright in the thesis, and except as herein before provided, neither the thesis nor
any substantial portion thereof may be printed or otherwise reproduced in any
material form whatever without the author’s prior written permission.
Dedicated to Csaba.
A patient teacher and a good friend.
Abstract
In this thesis we consider problems where a self-interested entity, called the
principal, has private access to some data that she wishes to use to solve a prediction problem by outsourcing the development of the predictor to some other
parties.

Assuming the principal, who needs the machine learning solution, and
the potential providers of the solution are two independent, self-interested
agents, which is the case for many real-world situations, this then becomes
a game-theoretic problem.

We propose mathematical models for variants of
this problem by borrowing techniques from the literature of mechanism design and provide principled solutions.

We consider experimental design...