Keyboard ALT + g to toggle grid overlay
Task-based analysis is a common and effective way to measure expertise levels of software users. However, such assessments typically require in-person laboratory studies, and inherently require knowledge of the user’s task. Today, there is no accepted method for assessing a user’s expertise levels outside of a lab, during a user’s own home or work environment activities. In this article, we explore the feasibility of software applications automatically inferring a user’s expertise levels, based on the user’s in-situ usage patterns. We outline the potential usage metrics that may be indicative of expertise levels, and then perform a study, where we capture such metrics, by installing logging software in the participants’ own workplace environments. We then invite those participants into a laboratory study, and perform a more traditional task-based assessment of expertise. Our analysis of the study examines if metrics captured in-situ, without any task knowledge, can be indicative of user expertise levels. The results show the existence of significant correlations between metrics calculated from in-situ usage logs, and task-based user expertise assessments from our laboratory study. We discuss the implications of the results, and how future software applications may be able to measure and leverage knowledge of the expertise of its users.
The Learning project aims to investigate advanced techniques for assisting users in learning complicated applications. We are interested in a range of investigations from the scientific study of the human learning process to prototyping novel interaction techniques for improving the general learning mechanisms that can be applied to all applications.