AUC Confidence Bounds for Performance Evaluation of Brain-Computer InterfaceReportar como inadecuado

AUC Confidence Bounds for Performance Evaluation of Brain-Computer Interface - Descarga este documento en PDF. Documentación en PDF para descargar gratis. Disponible también para leer online.

1 SPMC - Signal Processing and Multimedia Communications research group

Abstract : Currently most performance evaluation of Brain- Computer Interface BCI systems is simply reported in terms of accuracy. In this paper we propose a novel approach to evaluate the true performance of BCI systems based on Receiver Operating Characteristic ROC analysis, that removes the limitations of the accuracy performance measure. We demonstrate the need to provide, and particularly for small sample size, Confidence Interval CI bounds to indicate reliability of the BCI system performance. The ROC-based methodology makes it possible to calculate CI, shown as a contour at each any points of the ROC curve, with value of the lower bound of the Area Under the Curve AUC. We illustrate the usefulness of the methodology using the results the BCI Competition IV data set 3, dealing with the classification of wrist movements from four directions recorded using agnetoencephalogram MEG. Plotting the 95% CI contours overlayed on the ROC curves revealed some overlap with the chance level, thus revealing potential different interpretation from claims based on single accuracy value. The ROC-based methodology will also help to determine minimal sample size, an important requirement for future BCI studies and competitions.

keyword : Receiver Operating Characteristic ROC Area Under the Curve AUC Confidence Interval CI Performance Evaluation Brain-Computer Interface BCI

Autor: Brahim Hamadicharef -



Documentos relacionados