ROC Papers

Key papers

  • Comparing Three-class Diagnostic Tests by Three-way ROC Analysis
    Stephan Dreiseitl, Lucila Ohno-Machado, Michael Binder
    Medical Decision Making, Vol. 20, No. 3, 323-331 (2000)
    DOI: 10.1177/0272989X0002000309 
     
  • Maximum likelihood estimation of receiver operating characteristic (ROC) curves from continuously-distributed data
    C. E. Metz, B. A. Herman, and J. Shen
    Statistics Med., vol. 17, pp. 1033-1053, 1998.
     
  • Efficient confidence bounds for ROC curves
    H. Schafer
    Statistics Med., vol. 3, pp. 1551-1561, 1994
     
  • Three-Class ROC Analysis—A Decision Theoretic Approach Under the Ideal Observer Framework
    He, X. Metz, C.E. Tsui, B.M.W. Links, J.M. Frey, E.C.
    IEEE Transactions on Medical Imaging, May 2006,
    Vol. 25, No. 5 On page(s): 571- 581
    http://ieeexplore.ieee.org/iel5/42/34139/01626320.pdf
     
  • The Use of the Area Under the ROC Curve in the Evaluation of Machine Learning Algorithms
    A. P. Bradley
    Pattern Recognition, vol. 30(6), pp. 1145-1159, 1997.
    http://www.doc.ic.ac.uk/~xh1/Referece/ROC-analysis/The-use-of-the-area-u..-machine-learning-algorithms.pdf 
     
  • Signal detection theory and ROC analysis
    http://www.geo.wvu.edu/~jmiller/Geog694_files/Fielding_Bell1997.pdf 
     
  • The meaning and use of the area under a receiver operating characteristic (ROC) curve
    Hanley, J.A. and McNeil, B.J.
    Radiology 143(1982): 29-36.
     
  • Relationship between Brier score and area under the binormal ROC curve
     

New

Graphics

General 

Medical

Medical Imaging

Misc. Applications

  • Simultaneous Evolution of Feature Subset and Neural Classifier on High-Dimensional Data
    Jennifer Hallinan and Paul Jackway
    http://www.itee.uq.edu.au/~hallinan/publications/DICTA1999.pdf 
       

  • Tornado-WARNING PERFORMANCE IN THE PAST AND FUTURE A Perspective from Signal Detection Theory
    HAROLD E. BROOKS
    http://www.nssl.noaa.gov/users/brooks/public_html/papers/warnsdt.pdf 
     
  • Receiver Operating Characteristics for a Class of Three Channel Receivers
    R. Kenefic
    APRIL 1999 / VOLUME 35 / NUMBER 2 / ISSN 0018-9251 p 743 
    http://people.indtech.edu/rich_kenefic/AES35-2.pdf     
     
  • Receiver operating characteristic of a thresholded sum-squared coherence detector
    Kirlin, R.; Moore, D.;
    Acoustics, Speech, and Signal Processing [see also IEEE Transactions on Signal Processing], IEEE Transactions on , Volume: 26 , Issue: 4 , Aug 1978 Pages:369 - 371
     
     
  • Asymptotic receiver operating characteristics for envelope detection of a phase modulated sinusoid (Corresp.)
    Kenefic, R.; Weiner, D.;
    Information Theory, IEEE Transactions on , Volume: 29 , Issue: 1 , Jan 1983 Pages:168 - 171
     

Performance

Journals

Newly found

Book chapter

Papers 

  • In case ROC's and feature selection interest you, I have a technical report dealing with this at 
    ftp://svr-ftp.eng.cam.ac.uk/pub/reports/Scott_tr323.ps.gz  This shows that for cost-variable environments,
    the ROC curves of different feature sets  cross, and presents a novel algorithm for combining feature sets
    and classifiers to produce systems that are optimised across multiple cost domains. 
     
  • Biomedical applications of uncertainty modeling and analysis with fuzzy receiver operating characteristic methodology 
    M. DeLeo, G. Campbell , Nat. Inst. of Health, Bethesda, MD, USA
    3rd International Symposium on Uncertainty Modelling and Analysis March 17 - 20, 1995 College Park, Maryland 
     

  • Routine receiver operating characteristic analysis in mammography as a measure of radiologists' performance
    CC Goddard, FJ Gilbert, G Needham, and HE Deans 
    Br J Radiol 1998 71: 1012-1017.
    http://bjr.birjournals.org/cgi/reprint/71/850/1012.pdf 
     

  • A ROC analysis of different types of image compression methods for MRI data sets
    Authors ?
    http://www.mabuse.de/publications.mhtml 
     

  • Title ?
    Authors ?
    http://www.acb.org.uk/AnnClinBiochem/annals_pdf/July01/311.pdf 
     

  • A New Adaptive Long-Term Spectral Estimation Voice Activity Detector
    Javier Ramirez, Jose C. Segura, Carmen Benitez, Angel de la Torre, Antonio Rubio
    EUROSPEECH 2003, Geneva
    http://ceres.ugr.es/~rubio/vitae/papers/adaptivevad.pdf 
    Note Section 3.2 : VAD Comparison by means of the ROC Curves 
     

  • Objective Evaluation of Subjective Decisions (2003)
    Mel Siegel, Huadong Wu
    in Proceedings of SCIMA'2003 (International Workshop on Soft Computing Techniques in Instrumentation, Measurement and Related Applications Brigham Young University, Provo, Utah, USA, May 17, 2003) 
    http://www-2.cs.cmu.edu/~whd/publications/SCIMA-siegel.pdf 
     

  • A Parameter-Insensitive False Alarm Rate Detection Processor
    David M. Drumheller and Henry Lew
    http://www.dsto.defence.gov.au/corporate/reports/DSTO-TR-1153.pdf
     

  • A Decision Analysis Method for Evaluating Computer Intrusion Detection Systems (2004)
    Jacob W. Ulvila and John E. Gaffney
    Decision Analysis Vol. 1, No. 1, March 2004, pp. 35–50
    http://www.extenza-eps.com/extenza/loadPDF?objectIDValue=27624 
     

  • A Comparison of Machine Learning Methods for the Diagnosis of Pigmented Skin Lesions (2001)
    Stephan Dreiseitl, Lucila Ohno-Machado, Harald Kittler, Staal Vinterbo, Holger Billhardt, and Michael Binder
    Journal of Biomedical Informatics 34, 28–36 (2001)
    http://bioinformatics.cs.vt.edu/~easychair/DreiseitlEtAl_JBI_2001.pdf
     

  • Receiver-Operating-Characteristic Analysis Reveals Superiority of Scale-Dependent Wavelet and Spectral Measures for Assessing Cardiac Dysfunction
    Stefan Thurner, Markus C. Feurstein, Steven B. Lowen, and Malvin C. Teich
    VOLUME 81, NUMBER 25 PHYSICAL REVIEW LETTERS 21 DECEMBER 1998
    http://people.bu.edu/teich/pdfs/PRL-81-5688-1998.pdf  
     

  • A Comparison of Parametric and Nonparametric Approaches to ROC Analysis of Quantitative Diagnostic Tests
    KARIM 0. HAJIAN-TILAKI, JAMES A. HANLEY, LAWRENCE JOSEPH, and JEAN-PAUL COLLET
    http://umg.umdnj.edu/smdm/pdf/17-01-094.pdf 
     

  • Analysis of competing classifiers using components of variance of ROC accuracy measures (2002)
    Maloof, M.A., Beiden, S.V., & Wagner, R.F.
    Technical Report CS-02-01. Department of Computer Science, Georgetown University, Washington, DC.
    http://www.cs.georgetown.edu/~maloof/pubs/cstr-02-01.pdf 
     
  • On machine learning, ROC analysis, and statistical tests of significance (2002)
    Maloof, M.A.
    Proceedings of the Sixteenth International Conference on Pattern Recognition , 204-207, Los Alamitos, CA: IEEE Press.
    http://www.cs.georgetown.edu/~maloof/pubs/icpr02.pdf 
     

  • Receiver operating characteristic analysis: a general tool for DNA array data filtration and performance estimation
    http://fgf.bsd.uchicago.edu/publication/Khodarev1.pdf 
     

  • The geometry of ROC space: understanding machine learning metrics through ROC isometrics.
    P.A. Flach (Home page)
    Proc. 20th International Conference on Machine Learning (ICML'03), pages 194--201. AAAI Press, January 2003.
    http://www.cs.bris.ac.uk/Publications/Papers/1000704.pdf 
     

  • Deriving biased classifiers for better ROC performance (2002)
    H. Blockeel, and J. Struyf, 
    Informatica 26 (1), pp. 77-84, May, 2002.
    http://www.cs.kuleuven.ac.be/cgi-bin-dtai/publ_info.pl?id=37767 
     

  • Deriving biased classifiers for better ROC performance (2001)
    H. Blockeel, and J. Struyf,
    Information Society 2001 (Grobelnik, Marko and Mladenic, Dunja, eds.), pp. 124-127, 2001 
    http://www.cs.kuleuven.ac.be/cgi-bin-dtai/publ_info.pl?id=36366  
     

  • Decision support for data mining: An introduction to ROC analysis and its applications (2003)
    P. Flach, H. Blockeel, C. Ferri, J. Andez-Orallo, and J. Struyf
    Data Mining and Decision Support: Integration and Collaboration, vol. 745, 2003, pp.81-90
    http://www.cs.kuleuven.ac.be/cgi-bin-dtai/publ_info.pl?id=40941 
     

  • Improving accuracy and cost of two-class and multi-class probabilistic classifiers using ROC curves.
    Nicolas Lachiche and Peter Flach
    Proceedings of the Twentieth International Conference on Machine Learning (ICML'03), Washington, USA. 2003.
    http://hydria.u-strasbg.fr/~lachiche/icml03-155.pdf  
     

  • Receiver operating characteristics laboratory (ROCLAB): Software for developing decision strategies, the account for uncertainty
    DeLeo JM. 
    Proceedings of the Second International Symposium on Uncertainty Modelling and Analysis. San Diego: IEEE Computer Society Press, 1993.
     

  • Using Rule Sets to Maximize ROC Performance
    Tom Fawcett
    http://www.hpl.hp.com/personal/Tom_Fawcett/papers/ICDM-final.pdf 
     

  • Evolutionary Multi-Objective Feature Selection and ROC Analysis with Application to Industrial Machinery Fault diagnosis
    Christos Emmanouilidis
    EUROGEN 2001, Evolutionary Methods for Design, Optimisation and Control with Applications to Industrial Problems"
    Athens, Greece 19-21 September 2001. 
    http://www.cet.sunderland.ac.uk/~cs0cem/cem_eurogen2001.pdf 
     

  • A General Model for Finite-Sample Effects in Training and Testing of Competing Classifiers
    Sergey V. Beiden, Marcus A. Maloof and Robert F. Wagner
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 1, NO. 1, JULY 2003
    http://www.cs.georgetown.edu/~maloof/pubs/pami03.pdf  
     

  • Volume Under the ROC Surface for Multi-class Problems. Exact Computation and Evaluation of Approximations (2003)
    C. Ferri, J. Hernández-Orallo, M.A. Salido 
    http://citeseer.nj.nec.com/ferri03volume.html  
     

  • Analysis of competing classifiers using components of variance of ROC accuracy measures (2002)
    Maloof, M.A., Beiden, S.V., & Wagner, R.F.
    Technical Report CS-02-01. Department of Computer Science, Georgetown University, Washington, DC.
    http://www.cs.georgetown.edu/~maloof/pubs/cstr-02-01.pdf  
     
  • On Machine Learning, ROC Analysis, and Statistical Tests of Significance (2002)
    Marcus A. Maloof
    Proceedings of the Sixteenth International Conference on Pattern Recognition (ICPR-02)
    http://citeseer.nj.nec.com/maloof02machine.html 
     

  • Using Rule Sets to Maximize ROC Performance (2001)
    Tom Fawcett
    http://citeseer.nj.nec.com/fawcett01using.html 
     

  • Receiver operator characteristic analysis for intelligent medical systems - a new approach for finding confidence intervals (2000)
    Tilbury, J., Van-Eetvelt, P., Garibaldi, J., Curnow, J., and Ifeachor, E.C.
    IEEE Transactions Biomedical Engineering, 47 no. 7 pp. 952-963, 2000.
    http://www.tech.plym.ac.uk/spmc/people/jtilbury/pdf/ieee-be-00.pdf
     

  • Multiobjective Genetic Optimization of Diagnostic Classifiers with Implications for Generating Receiver Operating Characteristic Curves (1999)
    Matthew A. Kupinski and Mark A. Anastasio
    IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 18, NO. 8, AUGUST 1999
    http://citeseer.nj.nec.com/599025.html 
     

  • Analysis and visualization of classifier performance (1997)
    Foster Provost, Tom Fawcett, Ron Kohavi 
    http://citeseer.nj.nec.com/provost97analysis.html 
     

  • The Use of Receiver Operating Characteristic (ROC) Analysis to Evaluate Sequence Matching (1996)
    Michael Gribskov and Nina L. Robinson
    http://citeseer.nj.nec.com/gribskov96use.html  
     

  • Comparing the areas under two or more correlated ROC curves: a nonparametric approach (1988)
    DeLong ER, DeLong DM, Clarke-Pearson DL.
    Biometrics 1988; 44:837-845.
    http://radiology.rsnajnls.org/cgi/external_ref?access_num=3203132&link_type=MED 
     

  • A Comment on the ROC Curve and the Area Under it as Performance Measures
    Caren Marzban
    http://www.nhn.ou.edu/~marzban/roc.pdf 
     

  • An evaluation of methods for estimating the area under the Receiver Operating Characteristic (ROC) curve
    Centor, R.M. and Schwartz, J.S.
    Medical Decision Making 5 (1985) 149-156.
     
  • Signal detectability: The use of ROC curves and their analyses
    Centor, R.M.
    Medical Decision Making 11 (1991) 102-106. 
     
  • Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach
    DeLong, E.R., DeLong, D.M. and Clarke-Pearson, D.L.
    Biometrics 44 (1988) 837-845. 
     
  • An Introduction to the Bootstrap (1993)
    Efron, B. and Tibshirani, R.J.
    New York: Chapman and Hall, 1993.
     
  • Receiver operating (ROC) curves and non-normal data: an empirical study (1990)
    Goddard M.J. and Hinberg, I.
    Statistics in Medicine 9 (1990) 325-337.
     
  • A method of comparing the areas under receiver operating characteristic curves derived from the same cases
    Hanley, J.A. and McNeil, B.J.
    Radiology 148(1983): 839-843.
     
  • The robustness of the "binormal" assumptions used in fitting ROC curves (1988)
    Hanley, J.A.
    Medical Decision Making 8 (1988) 197-203.
      
  • Nonparametric estimation of degenerate ROC data sets used for comparison of imaging systems
    Rockette, H.E., Obuchowski, N.A. and Gur, D.
    Invest Radiology 25 (1990) 835-7.
     
  • A computer program for non-parametric receiver operating characteristic analysis
    Vida, S.
    Computer Methods and Programs in Biomedicine 40 (1993) 95-101.
     

 Interesting conferences  on the topic

2006 - SPMC / SoCCE / UoP