TY - JOUR

T1 - Three Contexts for Evaluating Organizational Usability

AU - Hertzum, Morten

PY - 2018

Y1 - 2018

N2 - Organizational usability is about the match between the user and the system, between the organization and the system, and between the environment and the system. While the first of these matches can, to a large extent, be evaluated in the lab, the two others cannot. Organizational usability must instead be evaluated in situ, that is, while the system is used for real work. We propose three contexts for such evaluation: pilot implementation, technochange, and design in use. Pilot implementation aims to inform the finalization of a system on the basis of testing it in the field prior to go-live. Technochange focuses on shaking down a system during go-live to realize the benefits it was developed to help achieve. Design in use is the tailoring performed by users after go-live to fit a system and its use to their local and emerging needs. For each evaluation context we describe its aim and scope, provide a brief example, and discuss the challenges it presents. To strengthen the focus on evaluation in the three contexts we propose the measurement of specified effects, combined with a sensitivity toward emergent effects. Incorporating effects in the evaluation of organizational usability makes for working systematically toward realizing benefits from the use of a system.

AB - Organizational usability is about the match between the user and the system, between the organization and the system, and between the environment and the system. While the first of these matches can, to a large extent, be evaluated in the lab, the two others cannot. Organizational usability must instead be evaluated in situ, that is, while the system is used for real work. We propose three contexts for such evaluation: pilot implementation, technochange, and design in use. Pilot implementation aims to inform the finalization of a system on the basis of testing it in the field prior to go-live. Technochange focuses on shaking down a system during go-live to realize the benefits it was developed to help achieve. Design in use is the tailoring performed by users after go-live to fit a system and its use to their local and emerging needs. For each evaluation context we describe its aim and scope, provide a brief example, and discuss the challenges it presents. To strengthen the focus on evaluation in the three contexts we propose the measurement of specified effects, combined with a sensitivity toward emergent effects. Incorporating effects in the evaluation of organizational usability makes for working systematically toward realizing benefits from the use of a system.

KW - Faculty of Humanities

KW - Usability

KW - usability evaluation

KW - organizational usability

KW - usage effects

KW - Pilot implementation

KW - technochange

KW - Design in use

M3 - Journal article

VL - 14

SP - 35

EP - 47

JO - Journal of Usability Studies

JF - Journal of Usability Studies

SN - 1931-3357

IS - 1

ER -