Abstract

Pilot implementation is a powerful approach to identifying and evaluating technical as well as organizational design and implementation issues before full-scale deployment. While pilot implementations are opportunities for learning, they often face a number of challenges that prevent or reduce learning. As a result, pilot implementations may fail to provide the evaluations needed to ensure a well-designed solution and a successful full-scale deployment. This is evident also for healthcare IT systems, an area that makes frequent use of pilot implementations.

First, we review the scarce literature on pilot implementation. We characterize and define pilot implementation by its position and role in the overall systems development life-cycle, and we relate this approach to the literature on, particularly, prototyping.

Second, based on this review, we propose a model for the overall system development process including prototyping as well as pilot implementation approaches.

Third, to investigate the practical benefits and challenges involved in pilot implementation, we analyze two cases. The first case concerns the pilot implementation of an electronic pregnancy record as part of the Danish national ehealth portal Sundhed.dk; this case is viewed from the perspective of the customer organization. The second case concerns the pilot implementation of a system supporting clinical pathways in newly established health centres in Copenhagen, Denmark; this case is viewed from the perspective of the vendor organization. We identify and discuss the major challenges that emerged in these two cases. In both cases, the challenges were large, and the pilot implementations failed in the sense that the projects were subsequently abandoned. Thus, the pilot implementations did not become opportunities for learning and for informing a later full-scale implementation. The pilot implementations were characterized by ambiguous scope, insufficient attention to the need for adapting work procedures to the new systems, failure to manage commitment to use the systems during the pilot phase, and a competing commercial agenda that tended to overrule the pilot implementations’ focus on learning.

Finally, we discuss general implications for the pilot implementation of healthcare IT systems and suggest further research directions.