A sample of convenience was obtained in which 15 officers anonymously completed fitness reports on each other. Fitness report scales were examined to determine their quality based on the statistical considerations of a 'discrimination' and 'disagreement' index. The spread in scores (fitness marks) when a number of judges rated one individual was compared with that when an average judge rates a number of individuals. Generalizing from the study is limited by the size and nature of the sample. The study demonstrates a type of analysis that can be performed and the type of information that can be obtained by studies of this type.