All that glitters isn’t gold: The complexities of usage statistics as an assessment tool for digital libraries

Abstract

Purpose:Digital collection assessment has focused mainly on evaluating systems, metadata and usability. While use evaluation is discussed in the literature, there are no standard criteria and methods for how to perform assessment on use effectively. This paper asserts that use statistics have complexities that prohibit meaningful interpretation and assessment. The authors aim to discover the problems inherent in the assessment of digital collection use statistics and propose solutions to address such issues.

Design/methodology/approach: This paper identifies and demonstrates five inherent problems with use statistics that need to be addressed when doing assessment for digital collections using the statistics of assessment tools on local digital repositories. The authors then propose solutions to resolve the problems that present themselves upon such analysis.

Findings: The authors identified five problems with digital collection use statistics. Problem one is the difficulty of distinguishing different kinds of internet traffic. Problem two is the lack of direct correlation of a digital item to its multiple URLs, so statistics from external web analytics tools are not ideal. Problem three is the analytics tools’ inherent bias in statistics that are counted only in the positive way. Problem four is the different interaction between digital collections with search engine indexing. Problem five is the evaluator’s bias toward simple growing statistics over time for surmising a positive use assessment. Because of these problems, statistics on digital collections do not properly measure a digital library’s value.

Practical implications: Findings highlight problems with current use measures and offer improvements.

Originality/value: This paper identifies five problems that need to be addressed before a meaningful assessment of digital collection use statistics can take place. The paper ends with a call for evaluators to try to solve or mitigate the stated problems for their digital collections in their own evaluations.