Background: Business intelligence systems (BIS) hold promise for improving organisational decision-making in South Africa. Yet, the use of BIS has been associated with a number of challenges.

Objectives: The aim of the study was to identify post implementation factors that contribute to the success of BIS in South African organisations

Method: This study draws on the DeLone and McLean Model of Information Systems success and recent literature on business-intelligence (BI) to develop and test a BIS success model. A quantitative study was conducted in the form of a survey of 102 BI users to validate the BIS success model.

Results: Five interrelated factors of BIS success were confirmed. The study found that the quality of information has a strong influence on system use and user satisfaction. It was found that system quality is positively associated with user satisfaction. The results also indicated that service quality is negatively related to user satisfaction. The study also found that user satisfaction is positively associated with nett benefits of a BI system.

Conclusion: The study provides insights for both managers and practitioners on the factors to focus on when implementing BIS thereby minimising the adoption risks associated with BI failures.

Business-intelligence systems (BIS) have a potential to provide many benefits to an organisation. Business intelligence (BI) has been identified as a significant growth area due to its valuable functionality and its ability to add value (Woodside 2011). Furthermore, Gartner (2011) reports that worldwide expenditure in BIS was more than USD 10 billion in 2010 and was expected to continue to grow at a rate of approximately 8.1% annually. However, implementing new BIS is often a challenge (Benard & Atir 1993; Clavier, Lotriet & van Loggerenberg 2012; Olbrich, Poppelbuß & Niehaves 2012). Hence, understanding the drivers of and barriers to BIS success is very important. The success of BIS has not been fully investigated (Chaveesuk 2010; Yeoh & Koronios 2010). A search in the main electronic databases for journals, such as EbscoHost, Emerald, Google Scholar, Proquest and ScienceDirect, suggests that no academic research has validated the DeLone and McLean (2003) model of information-systems (IS) success in the context of BI in South Africa. This study seeks to address this gap. It is important to study the South African context because most reports on IS success are drawn from settings in industrialised countries and are focused on e-government and enterprise resource planning ERP system success (Heeks 2010). Furthermore, the results of these reports cannot easily be translated to the South-Africa context because South Africa is a developing economy. In most developing countries, a shortage of expertise, staff turnover and limitations in financial resources are cited as challenges to the implementation and utilisation of IS (Avgerou 2008).

This paper is organised as follows. The next section briefly reviews the existing literature on BI. Then the information-systems success model used in the study is highlighted. The model chosen for this study is the DeLone and McLean (2003) model. Next, the proposed BIS success model is presented, and the research methodology is discussed. The results of the study are then presented, followed by the discussion section. The conclusion is presented in the final section of the article.

BI is defined as a collection of technology and applications used to enhance decision-making (Wixom & Watson 2010). Turban et al. (2011:12) classify a BI system into four main components: (1) a data-warehouse (DW) environment, (2) business analytics, (3) business performance management (BPM) and (4) a user interface such as a dashboard. BI offers many benefits to the adopting organisation, for example increasing sales, reducing costs and providing new products and services (Hwang & Xu 2007). BI enables organisations to make well-informed business decisions and can thus be a source of competitive advantage (Ranjan 2009). Similarly, Vitt, Luckevich and Misner (2010) argue that the primary goal of BI is to help people make better decisions that improve a company’s performance and promote its competitive advantage in the marketplace. Ranjan (2009) asserts that BI reveals the following:

(1) the position of the firm in comparison to its competitors,

(2) changes in customer behaviour and spending patterns,

(3) the capabilities of the firm,

(4) market conditions, future trends, demographic and economic information,

(5) the social, regulatory, and political environment

(6) what the other firms in the market are doing. (p. 63)

Hwang and Xu (2007) further point out that return on investment in BI could be as high as 400%. However, international evidence suggests that BIS are expensive and have generally not been a success (Beal 2005; Legodi & Barry 2010). Some explanations for the low levels of success include the following: system quality; information quality; service quality; user satisfaction (Shin 2003), support for end users; accuracy, format and preciseness; fulfilment of end users’ needs; reduced effort by developers to produce information; user ability to produce information and better decision-making (Wixom & Watson 2010). BI is clearly very important to an organisation, but it is evident that a number of issues may limit its use and adoption. It is important therefore that we improve our understanding of the key factors that influence the success of BIS in South Africa. The next section of this paper describes the DeLone and McLean (2003) model of IS success, which is the foundational model chosen for this study.

The theoretical underpinning chosen for this study is the updated DeLone and McLean (2003) model of IS success. The DeLone and McLean (1992) model propose six major factors of IS success: (1) system quality, (2) information quality, (3) use, (4) user satisfaction, (5) individual impact and (6) organisational impact. Pitt, Watson and Kavan (1995) propose a modification of the DeLone and McLean (1992) model to include service quality. Seddon (1997) challenges the combination of a process and variance model. He (Seddon 1997:23) argues that the DeLone and McLean (1992) model is ‘confusing and mis-specified’. The DeLone and McLean (2003) information-systems success model address the weaknesses of the original model. The model consists of the following six factors: system quality, information quality, service quality, use or intention to use, user satisfaction and nett benefit. For the purposes of this study, we have chosen to assess BIS success using the updated DeLone and McLean (2003) model. One of the reasons for this choice was because it was identified as the single most cited IS success model in IS literature (Lowry, Karunga & Richardson 2007). Furthermore, the DeLone and McLean (2003) model framework has been used extensively in various empirical works on IS success. Empirical work has drawn on the updated DeLone and McLean (2003) model to examine the success of a student information system using student users (Rai, Lang & Welker 2002), tourism websites (Stockdale & Borovicka 2006; Wang & Liao 2008), knowledge-management systems (Wu & Wang 2006), e-government systems (Hussein, Abdul Karim & Selamat 2007), online learning systems (Lin 2007) and e-commerce systems (Wang & Liao 2008). Prior studies confirm the model’s usefulness in assessing different IS applications. However, in the South African BI context, the researchers found no study that has utilised the updated DeLone and McLean (2003) model to assess the success of local BI systems. The research model and the hypotheses are discussed next.

Figure 1 illustrates the research model developed for the study. The research model attempts to explain BIS success in a South African context. The research model posits that BIS success is represented by six factors, and each factor is in turn measured, using multiple variables adapted from current literature and input from BI experts.

FIGURE 1: Research model.

Information quality focuses on the characteristics of the information that is produced by the BIS (Petter, Delone & McLean 2008). This information is mainly in the form of reports. High-quality information is expected to lead to use and user satisfaction of the BIS. There is solid support for the association between information quality and system use and between information quality and user satisfaction (Halawi, McCarthy & Aronson 2007; Kositanurit, Ngwenyama & Osei-Bryson Kweku 2006; Livari 2005; Rai et al. 2002). Therefore, in this study, we propose the following hypotheses:

H₁₀: Information quality is not related to system use in a business-intelligence system.

H_1A: Information quality is related to system use in a business-intelligence system.

H₂₀: Information quality is not related to user satisfaction in a business-intelligence system.

H_2A: Information quality is related to user satisfaction in a business-intelligence system.

Petter et al. (2008) define system quality as follows:

… the desirable characteristics of an information system. For example: ease of use, system flexibility, system reliability, and ease of learning, as well as system features of intuitiveness, sophistication, flexibility, and response times. (p. 239)

Previous studies on the success of information systems have demonstrated the positive impact of system quality on user satisfaction and system use (DeLone & McLean 2003; Rai et al. 2002; Seddon 1997). Therefore, we propose the following hypotheses:

H₃₀: System quality is not related to system use in a business-intelligence system.

H_3A: System quality is related to system use in a business-intelligence system.

H₄₀: System quality is not related to user satisfaction in a business-intelligence system.

H_4A: System quality is related to user satisfaction in a business-intelligence system.

Service quality focuses on the level of support that BI users receive. Service quality is measured, based on responsiveness, accuracy, reliability, technical competence and empathy of the personnel (Petter et al. 2008). Like most studies on IS success, quality service is expected to have a positive influence on user satisfaction and system use (DeLone & McLean 2003; Rai et al. 2002). Therefore, we propose the following hypotheses:

H₅₀: Service quality is not related to system use in a business-intelligence system.

H_5A: Service quality is related to system use in a business-intelligence system.

H₆₀: Service quality is not related to user satisfaction in a business-intelligence system.

H_6A: Service quality is related to user satisfaction in a business-intelligence system.

System use focuses on the utilisation of the BIS by the users. System use is defined as follows:

… the degree and manner in which staff and customers utilise the capabilities of an information system. For example: amount of use, frequency of use, nature of use, appropriateness of use, extent of use, and purpose of use. (Petter et al. 2008:239)

Previous studies on the success of IS have demonstrated the positive impact of system use on user satisfaction and nett benefits (DeLone & McLean 2003; Rai et al. 2002). Therefore, we propose the following hypotheses:

H₉₀: System use is not related to nett benefits in a business-intelligence system.

H_9A: System use is related to nett benefits in a business-intelligence system.

Seddon (1997) defines user satisfaction as follows:

… the net feeling of pleasure or displeasure resulting from aggregating all the benefits that a person hopes to receive from interaction with the information system. Each user has a set of expected benefits or aspirations for the information system. To the extent that the system meets or fails to meet each of these aspirations, the user is more or less satisfied. (p. 246)

Previous studies on the success of IS have demonstrated the positive impact of user satisfaction on net benefits (DeLone & McLean 2003; Rai et al. 2002; Wang, Fan & Xu 2012). Therefore, we propose the following hypotheses:

H₇₀: User satisfaction is not related to system use in a business-intelligence system.

H_7A: User satisfaction is related to system use in a business-intelligence system.

H₈₀: User satisfaction is not related to nett benefits in a business-intelligence system.

H_8A: User satisfaction is related to nett benefits in a business-intelligence system.

In order to validate the research model, data were collected by means of an online questionnaire. The online questionnaire consisted of two main parts. The first part included demographic questions such as age, gender and industry. The second part consisted of 35 five-point Likert-scale questions aiming to assess the six factors of the proposed research model. The five-point Likert scale ranged from strongly agree to strongly disagree. The online questionnaire was developed from prior studies and adapted to suit the BI context. Table 1 below shows the items for each construct in the second section of the questionnaire.

Before the major study, the instrument was pilot tested by six professionals to ensure that the wording was understandable and that its length was appropriate. The final instrument was administered over a four-week period. Ethical clearance was obtained before conducting this study. The results of the data analysis used to test the hypotheses will be presented in the next section.

A total of 102 conveniently selected professionals responded to the survey. Most of the responding professionals were male (52%), roughly 60% of respondents were between 31 and 40 years of age, and 20% of respondents were between 41 and 50 years old. Over half of the respondents (55%) reported having more than five years of experience in their current role whilst 45% reported less than five years of experience. Approximately two thirds of the responding professionals reported that they use BIS for ad-hoc reporting. All participants participated voluntarily in this study.

The results of the reliability tests are shown in Table 2. The results show that the value of Cronbach’s alpha for all constructs is higher than 0.7. This suggests that the questionnaire and its constructs are suited for the study (Hair et al. 2006).

Convergent validity was established by examining the average variance extracted (AVE) scores. These were all well above the recommended 0.50 (Fornell & Larcker 1981), confirming that constructs explained above 50% of the variance in their underlying items.

After validating the measurement model, the hypotheses were tested by examining the structural model. Figure 2 shows the path coefficients, which indicate the strengths of the relationships between the independent and dependent variables.

FIGURE 2: Structural model test results.

The results indicate that information quality is positively related to both system use and user satisfaction. The null hypotheses (H₁₀, H₂₀) were thus rejected. Therefore, the alternative hypotheses (H_1A, H_2A) stand (0.092 and 0.011, respectively). The path coefficient between system quality and system use is –0.014 with a p-value of 0.624. The Kendall correlation coefficient between system use and system quality is 0.046 with a p-value of 0.568. System quality is thus not related to system use. Therefore, the null hypothesis cannot be rejected (H₃₀). Considering user satisfaction, however, the results indicate that system quality is positively related to user satisfaction. In this case, the null hypothesis was rejected. Therefore, the alternative hypothesis (H_4A) stands (0.108). The path coefficient between system use and service quality is 0.068 with a p-value of 0.188. The Kendall correlation coefficient between system use and service quality is 0.049 with a p-value of 0.574. Therefore, the null hypothesis cannot be rejected. Service quality, in turn, is not related to system use. For the sixth hypothesis, the path coefficient is −0.123 with a p-value of 0.051. The Kendall correlation coefficient between user satisfaction and service quality is −0.200 with a p-value of 0.019. Therefore, the null hypothesis can be rejected. User satisfaction is negatively related to service quality. The results indicate that, as service quality increases, user satisfaction decreases. The path coefficient between system use and nett benefits is −0.134 with a p-value of 0.495. The Kendall correlation coefficient between system usage and nett benefits is −0.028 with a p-value of 0.733. System use is not related to nett benefits. Therefore, the null hypothesis cannot be rejected. The results indicate that the path coefficient between user satisfaction and a nett benefit is 0.961 with a p-value of 0.000. The Kendall correlation coefficient between user satisfaction and nett benefits is 0.314 with a p- value of 0.000. Therefore, the null hypothesis can be rejected. User satisfaction is positively related to nett benefits. Overall, information quality exhibited a stronger effect than system quality and service quality in influencing system use and user satisfaction, respectively.

This study formulated and verified a BIS success model. Using the updated DeLone and McLean model as a theoretical framework, we constructed ten measures in information quality, eight measures in system quality, five measures in service quality, three measures in user satisfaction, three measures in system use and six measures in nett benefits. To our knowledge, this is the first study that extends the DeLone and McLean Model (2003) to a BI context in South Africa. Consistent with previous studies (DeLone & McLean 2003; Kositanurit et al. 2006; Halawi et al. 2007; Holsapple & Lee-Post 2006; Rai et al. 2002), our findings indicate that information quality is positively linked to system use and user satisfaction. The results are less clear with respect to the influence of system quality on system use and user satisfaction. System quality was not found to be related to system use. This finding is unexpected and inconsistent with previous studies where system quality is reported to have a significantly positive influence on system use (DeLone & McLean 2003; Fitzgerald & Russo 2005). The different types of applications investigated and the different contexts can explain differences amongst the findings of this study and the research in previous studies. Thus, whilst the present study examined BIS in South Africa, the DeLone and McLean (2003) study examined e-commerce in Western countries. Whilst the DeLone and McLean model is intended to be valid for all IS in general (DeLone & McLean 2003), they recommended that researchers should adapt the research model for specific domains to better address the characteristics of the latter (Dinter, Schieder & Gluchowski 2011). In this study, the success factors where adapted to suit the BI context. The results of the test of the effect of system quality on user satisfaction found that system quality is positively related to user satisfaction. As the quality of the system increases so does the user-satisfaction levels. The results of this study agree with the findings of previous studies (DeLone & McLean 2003; Halawi et al. 2007; Holsapple & Lee-Post 2006). The implication for management is that system quality plays an important role in the success of BI projects. Managers could make an effort to address system quality to increase the chances of success of the BIS as users are bound to be satisfied by a high-quality system. The empirical results of the study indicate that there is no significant relationship between service quality and system use. This finding suggests that the quality of service does not influence the use of the system. This is inconsistent with previous studies where service quality is reported to have a significant influence on system use (Caldeira & Ward 2002; Fitzgerald & Russo 2005).

A possible explanation for this is that service quality is outsourced to BI vendors who manage the service, and consequently, BI users do not view it as an integral part of the organisation and tend to take the availability of this service for granted. The results of this study indicate that service quality influences user satisfaction negatively, results which are inconsistent with the information-systems model of DeLone and McLean (2003). DeLone and McLean’s work suggest that service quality is positively related to user satisfaction. The different types of systems investigated and their different needs for respective contexts can explain differences amongst the findings of this study and DeLone and McLean’s. Hypothesis 7 investigated the relationship between user satisfaction and system use. The empirical results of the study indicate that user satisfaction does not influence system use. This is unexpected, given that one would have expected the levels of system use to increase as the levels of user satisfaction increase. A plausible explanation is that the model used in the present study did not include the intention-to-use factor, and it also did not include system-use and user-satisfaction association. However, this research is not alone in providing evidence of a non-significant relationship between user satisfaction and system use. Some previous studies (Ang & Soh 1997; Vlahos & Ferratt 1995) also found that user satisfaction did not directly influence system use. The influence of system use on the nett benefits of business-intelligence systems in South Africa was examined by hypotheses eight. The empirical results of the study indicate that there is no significant relationship between system use and nett benefits. These results were inconsistent with the theoretical arguments based on DeLone and McLean (2003). The results of this study also do not agree with the findings of other previous studies (Devaraj & Kohli 2003; Leclercq 2007; Zhu & Kraemer 2005). However, a study by Gelderman (1998) did not find a significant positive correlation between system and nett benefit. A plausible explanation for this difference is that the model for the present study focused only on system use and not on the intention to use a system. Furthermore, the model used in this study does not include the nett benefits to the system-use relationship but rather focuses on the relationship between system use and nett benefits. It was found that user satisfaction positively influences nett benefits, results that are consistent with the information-systems model of DeLone & McLean (2003) which suggested that user satisfaction is positively related to nett benefits. The results imply that, as levels of satisfaction increase, so do the nett benefits of the system. In this regard, the results of this study agree with the findings of other previous studies (Gelderman 1998; Law & Ngai 2007).

The study is one of only a few to formulate and test a BIS success model based on the DeLone and McLean’ s (2003) IS success model in a South African context. Therefore, this study contributes knowledge to information-systems literature in general and BI in particular. As a result, by using the DeLone and McLean’ s (2003) model as its basis, this study adds to the area of information-systems success by supporting the DeLone and McLean’ s (2003) model and by refining it to be more suited to BI success, specifically in the context of South Africa. Some of the factors identified through this study as important to BIS success corroborate previous literature. The different types of applications investigated and the different contexts can explain differences amongst the findings of this study and previous research. Thus, whilst the present study examined BIS in South Africa, the DeLone and McLean’ s (2003) IS success model examined e-commerce in Western countries. Whilst the DeLone and McLean’ s (2003) model is intended to be valid for all information systems in general, they recommend that researchers should adapt the research model for specific domains to better address the characteristics of the latter (Dinter et al. 2011). In this study, the success factors where adapted to suit the BI context. The model of BI-system success formulated and tested in this study will be of interest to organisations wishing to adopt BI systems. Two groups of people who could possibly be drawn from this study. Firstly, BI developers and BI Managers who will develop and implement a BIS would benefit. Secondly, the BI vendor industry would benefit from this study. The main practical implication of this study for BI developers and managers is that it would make them more aware of the factors that influence the success of the BIS they develop. With these factors in mind, they would then be able to incorporate into their development such factors as the ease of use, ease of learning, stability and security of their system. This research enables people in the BI vendor industry to understand the pitfalls of BI adoption and the reasons why adoption can be problematic. It could allow them to take a more proactive approach when new versions of solutions are been developed.

There are some limitations to this study. The first limitation of the study is that context in which the study was set was limited to organisations operating in South Africa. Consequently, caution needs to be taken when generalising the results of this study. A second limitation was that we tested only part of the updated DeLone and McLean (2003) model. Further research could consider incorporating and validating the entire updated DeLone and McLean (2003) model.

The objective of this study was to identify key factors influencing BIS success in South Africa and to test those factors using data from BIS users. We developed a conceptual model for BIS success in South Africa, based on the updated DeLone and McLean (2003) model. Survey data from 102 BI users in South Africa were used to test the BIS success model. This study reached several conclusions based on the empirical findings. Firstly, the results of the empirical analysis indicate that the quality of information has a strong influence on system use and user satisfaction. Secondly, the results indicate that system quality is positively associated with user satisfaction. Thirdly, service quality is negatively related to user satisfaction. Fourthly, nett benefits are positively associated with user satisfaction. Overall, the empirical results of the study provide moderate support for the proposed research model. Of the nine relationships tested, five were significant. In summary, this research sheds light on the key factors to be considered by organisations that are considering implementing or which have already adopted a BI system. Therefore, this study has improved our understanding of the factors influencing the success of BIS in South Africa. The findings also contribute to the growing body of literature on the measurement of the success of information systems in general. Future research should aim to confirm the specific findings of this study and further to investigate the interdependency of the success factors.

The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.

T.M. (University of KwaZulu-Natal) and M.M. (University of KwaZulu-Natal) contributed equally to the writing of this article.