Can relative skill be determined from a photographic portfolio?
Date of Issue2013
Human Vision and Electronic Imaging (18th : 2013 : Burlingame, USA)
School of Computer Engineering
In this study, our primary aim is to determine empirically the role that skill plays in determining image aesthetics, and whether it can be deciphered from the ratings given by a diverse group of judges. To this end, we have collected and analyzed data from a large number of subjects (total 168) on a set of 221 of images taken by 33 photographers having different photographic skill and experience. We also experimented with the rating scales used by previous studies in this domain by introducing a binary rating system for collecting judges’ opinions. The study also demonstrates the use of Amazon Mechanical Turk as a crowd-sourcing platform in collecting scientific data and evaluating the skill of the judges participating in the experiment. We use a variety of performance and correlation metrics to evaluate the consistency of ratings across different rating scales and compare our findings. A novel feature of our study is an attempt to define a threshold based on the consistency of ratings when judges rate duplicate images. Our conclusion deviates from earlier findings and our own expectations, with ratings not being able to determine skill levels of photographers to a statistically significant level.
DRNTU::Engineering::Computer science and engineering
© 2013 SPIE-IS&T. This paper was published in Proceeding of SPIE, Human Vision and Electronic Imaging XVIII and is made available as an electronic reprint (preprint) with permission of (SPIE). The paper can be found at the following official DOI: [http://dx.doi.org/10.1117/12.2008065]. One print or electronic copy may be made for personal use only. Systematic or multiple reproduction, distribution to multiple locations via electronic or other means, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper is prohibited and is subject to penalties under law.