Beyond Customer Needs Software Development: an Empirical Investigation of Its Forms and Individual-Level Causes

Most innovation projects fail to meet quality, time and cost targets (Markham and Lee, 2013). Poor performance levels in new product and service development are typically attributed to a number of syndromes, such as design loopbacks, firefighting, escalation of commitment to a negative course of action (Repenning, 2001; Keil and Mahring, 2010). One syndrome that has been found to cause excess development costs, long delays and terminated projects but has not been adequately investigated by existing research, is that of overdesign (Coman and Ronen, 2010). Overdesign refers to designing and developing new products or services beyond what is required by the specifications and/or the requirements of the customer or the market. This phenomenon of “biting off more than one can and should chew” has been named in many different ways, e.g., scope overload, featuritis, requirements creep (Shmueli et al., 2015; Rust et al., 2006). This terminological confusion is an early indication of the need for a rigorous definition and conceptualization of the phenomenon. A first objective of this study is to develop and test a taxonomy of overdesign that clarifies and captures the multifaceted nature of this phenomenon. Secondly, we comprehensively investigate the antecedents and performance consequences of overdesign. The few existing studies on this subject have mainly focused on individual and behavioral drivers of overdesign (Shmueli et al., 2015) in a software development context (Bjarnason et al., 2012). We complement these studies by hypothesizing and testing the effect of organizational, process and strategic factors (e.g., R&D and marketing incentive systems, timing of concept “freezing”, long-termism) on overdesign in multiple product categories. Our research is carried out in two stages. In the first qualitative stage, we conducted interviews with product developers from different companies, both to understand the nature and origins of overdesign and to develop a taxonomy of the different forms of the phenomenon. In the second quantitative stage, we collected data about innovation projects in different industries, using a survey distributed to innovative ventures identified through a review of crowdfunding platforms. These data are then analyzed by means of structural equation modeling (SEM). The use of the survey method to identify antecedents and performance consequences of overdesign complements prior studies that were mainly based on case studies and experiments. Preliminary results show that there are different and sometimes contrasting forms of overdesign, depending on the stage of the innovation process in which it occurs (early specification vs late design), and on the type of attributes it concerns (fit, taste, quality). We also find that the positive impact of behavioral factors (e.g., overconfidence and optimizing approach) on overdesign is magnified by the existence of organizational and process mechanisms, e.g., the use of buffering tactics to cope with unstable customer requirements, the low involvement of developers in early phases, the detailed specification of the new product upfront. These findings offer several contributions to research on new product development, technology and innovation management and user-driven innovation. On a managerial level, this study informs managers of the origins and dangers of overdesign and warns them of the risks brought by some elements of traditional innovation management frameworks, such as waterfall development and stage-gate. It suggests that principles and techniques from agile and lean product development might prevent and alleviate overdesign, although this claim requires further research.