The role of user satisfaction in implementing a Business Intelligence System

Background: Despite increasing importance of the use of Business Intelligence (BI) as a technology-driven process for giving decision support, the success or failure of BI has not been investigated fully in South Africa. BI is not well understood because of an absence of documented proof of its practice. Objectives: This study was intended to investigate BI and identify the moderating and mediating effects of user satisfaction on the relationship between system quality, information quality and service quality on the one hand and perceived net benefits on the other in South Africa. Methods: The quantitative methods approach was predominantly used in this study. A total of 211 responses were obtained from a random sample of 250 BI users throughout South Africa. A semi-structured online survey questionnaire was used to collect the data, and correlation and multiple regression analyses were used to analyse it. Results: It was found that user satisfaction mediates the relationship between perceived net benefits and system quality and service quality. It also moderates the effects of system quality and service quality on perceived net benefits. Information quality is not related with user satisfaction and perceived net benefits. Conclusion: The implication of the results is that system quality and user satisfaction should be enhanced and maintained to achieve perceived positive net benefits in order to make the BI system more effective and efficient.


Introduction
Business intelligence (BI) systems have become increasingly important over the past few decades and are, today, one of the top spending priority areas of most organisations (Davenport 2010;Foley & Manon 2010;Gartner 2009;Olszak & Batko 2012;Trninic, Durkovic & Rakovic 2011;Turban et al. 2011a) because decision makers need accurate and timely information in order to make effective decisions.For the purpose of this study, BI will be defined as a technology-driven process, which facilitates achievement of organisations' mission objectives through providing required information to the decision makers (Petrini & Pozzebon 2009;Turban et al. 2011b;Wixom & Watson 2001).According to Martin, Maladhy and Venkatesan (2011), a typical BI system consists of a data warehouse (Ponniah 2010); an extraction, transformation and loading tool; and a set of analytical tools.
A lot of research has been done on BI; however, with its rapid development, some researchers (e.g.Cidrin & Adamala 2011;Mohamad & Mohamed 2012;YellowFin 2012) argue that it still warrants further academic scrutiny.One of the reasons for this is that most of the available research does not apply to the developing countries because it has been done in developed countries (which have a different culture) (Avgerou 2008;Murugan, Magid & Uzoamaka 2000) and it focuses on technological and operational features rather than on human, managerial and strategic factors (Chaveesuk 2010).Ponelis and Britz (2011) assert that an absence of documented proof of BI practice in developing countries might imply that it is still at an early stage of adoption.
In South Africa, BI has been widely adopted by many organisations and attempts have been made to explore and examine the success factors of BI systems (Dawson & Van Belle 2013;Nkuna 2011), but these studies have been case studies, thus making the results not capable of being generalised to all business organisations in the country.Indeed, the fact that there have been case studies reporting penetration of BI in South Africa's socio-economic value chains is evident enough to show that something is happening; however, the success or failure of a BI system has not been well articulated in the country (Chaveesuk 2010;Yeoh & Koronios 2010).There is still a limited amount of empirical research that investigates the nature of end-user satisfaction with BI systems (Acheampong & Moyaid 2016;Boonsiritomachai, McGrath & Burgess 2014) and many of the firms in South Africa that have adopted BI are not yet utilising BI to the full potential.Therefore, this study was an attempt to narrow this knowledge gap, aiming at providing insight into those factors that organisations need to address to improve BI projects' chances of success and ensure an effective and efficient initiative.The study focuses on identifying the moderating and mediating effect of user satisfaction on the relationship between system quality, information quality and service quality on the one hand and perceived net benefits on the other in South Africa.

Literature review and theoretical framework
Several authors (e.g.DeLone & McLean 1992, 2003;Hayen, Rutashobya & Vetter 2007;Shin 2003) have found that system quality has an effect on the use of BI system and user satisfaction, and user satisfaction has an influence on individual and organisational impact, which together make up a firm's 'net benefits'.Studies indicate that in general service quality also has a positive influence on user satisfaction (Almutairi & Subramanian 2005;Coombs, Doherty & Loan-Clarke 2001;Gelderman 2002;Halawi, McCarthy & Aronson 2007;Livari 2005;McGill, Hobbs & Klobas 2003;Seddon & Kiew 1996;Wu & Wang 2006) and that when users are satisfied with a BI system it is hoped that they would derive positive benefits from using it (Gelderman 1998;Law & Ngai 2007).
Researchers (e.g. DeLone & McLean 1992;Doll & Torkzadeh 1988), by utilising, for example, the case study design, survey design or Delphi technique method have identified user satisfaction as one of the most extensively used single measure of BI success.Chiu et al. (2005) further explained that the user's level of satisfaction is based on the level to which the application meets the users' expectations.According to Smart (2009), individual impact affects the organisational impact of the BI system and they are both influenced by user satisfaction.This study adopted Seddon's (1997:246) definition of user satisfaction as 'the net feeling of pleasure or displeasure resulting from aggregating all the benefits that a person hopes to receive from interaction with the information system'.Perceived net benefits are the core dimension used to determine the diverse and multifaceted success of BI (Wixom & Watson 2001).
This study was based on three theoretical models, namely, technology acceptance model (TAM) (Davis 1989), task technology fit (TTF) (Goodhue & Thompson 1995) and social cognitive theory (SCT) (Bandura 1986).TAM posits that perceived usefulness of a BI system is directly impacted by perceived ease of its use and they both determine an individual's intention to use it with intention to use serving as a mediator of actual system use.The model assumes that when people form an intention to act, they will be free to act without limitation, and in practice, constraints such as limited ability, time, environmental or organisational limits, and unconscious habits will limit the freedom to act.In this study, this model was extended by introducing six major factors of information systems success, namely: (1) system quality, (2) information quality, (3) use, (4) user satisfaction, (5) individual impact and (6) organisational impact factors from the DeLone and Mclean model (1992).The TTF theory holds that IT affects individual performance if its capabilities match the concerned tasks and quality, compatibility, ease of use/training, systems reliability and relationship with users, etc., and affect the task-technology fit which in turn determines the system's effectiveness (Goodhue & Thompson 1995).SCT identifies human behaviour as an interaction of personal factors, behaviour, and the environment (Bandura 1986) (which involves the influences of a person's thoughts and actions).The second interaction between the person and the environment involves human beliefs and cognitive competencies that are developed and modified by social influences and structures within the environment.The third interaction between the environment and behaviour involves a person's behaviour determining the aspects of their environment (Jones 1989) and in turn their behaviour is modified by that environment.
This study hypothesised that in the BI system, user satisfaction moderates the effects of system quality, information quality and service quality on perceived net benefits and it mediates the relationship between these independent variables and net benefits.System quality as an essential factor in a successful data warehouse implementation (Seddon 1997) comprises five key dimensions: flexibility, reliability, response time, accessibility and integration (Nelson, Todd & Wixom 2005;Petter, DeLone & McLean 2008;Seddon, 1997).Information quality consists of four dimensions: accuracy, completeness, currency and format, which are, by implication, an essential asset for BI success for any organisation (Haley 1997;Hwang & Xu 2008;Nelson et al. 2005;Rudra & Yeo 1999;Shin 2003;Thomann & Wells 1999;Wixom & Watson 2001).Zhu and Kraemer (2005) identified four attributes with respect to information quality as follows: (1) origin has an effect on reliability of the data and the trust one can place in it referring to the origin relationship as data lineage, (2) correctness signifies that the data are free of errors, (3) completeness describes the coverage of the data and (4) objectivity refers to the lack of prejudice in the data.Service quality is defined as the quality of the support that BI system users receive from the IS department and IT support personnel (Pitt, Watson & Kavan 1995;Petter et al. 2008).Grover and Segars (1996), defined service quality based on the degree of disparity between what the customer expects in terms of service standard and perceived service performance (Culiberg & Rojsek, 2010;Parasuraman, Zeithaml & Berry 1988).
The conceptual framework diagram is shown in Figure 1.
Table 1 shows the items or concepts that were measured for a particular construct.

Data
A total of 211 businesses (with more than 100 full-time employees) and a maximum response of two BI users (from different departments) were used in this study.An anonymous and confidential online semi-structured questionnaire was used to collect data.

Profile of the participants
The majority (53.08%) of the respondents were aged between 31 and 40 years and most of them were male respondents (57.35%) (Singh 2004) and blacks (47.87%), followed by whites (35.07%).Some of the respondents (35.3%) were from the financial services field and accounting and many were operational staff (76.78%).The highest education level was mostly a bachelor's degree (49.29%), followed by a diploma (37.91%).

Normality of the data
Skewness of the constructs (ranging between -1.216 for system quality and -0.385 for information quality) showed that the data approximately followed a normal distribution (Hair et al. 2007).Kurtosis (ranging from 2.404 for net benefits to 4.283 for user satisfaction) was relatively high.See the relevant histograms in Figure 2.
By looking at the histograms and distribution curves of the constructs, and their skewness and kurtosis, it can be concluded that all constructs except information quality and service quality, were not normally distributed.However, although some of the variables did not appear to be normally distributed, it was not necessary to transform and normalise them for the reason that independent variables did not have to be normally distributed to obtain valid results.The skewness of net benefits of -0.777 and kurtosis of 2.404 indicated that the variable was more or less normally distributed and it did not require any transformation.The desired statistical analyses were applied.

Data analysis
In this study, correlational analysis and multiple regression were used to analyse the data.The dependent variable was perceived net benefits.System quality, information quality and service quality were the independent variables and user satisfaction was treated as a moderating or mediating variable.A moderator is a variable that alters the direction or strength of the relationship between an independent variable and a dependent variable.This implies that the effect of an independent variable on the dependent variable depends on the level of another variable referred to as a moderating variable, just like an interaction effect.A mediator helps to explain a relationship between an independent variable and a dependent variable.The multiple regression model used in this study was: where y i is the i th observation on the net benefit variable, x th is the i th observation on the j th independent variable, and b i is the effect of the j th independent variable on net benefit and e i is the error of the i th observation.
Statistical significance was checked by the F-test of the overall model fit and t tests of individual parameters.In order to test for the moderation effect of user satisfaction on the effect of the independent variables on the dependent variable, new variables were created by multiplying system quality, information quality and service quality by user satisfaction and including the interaction terms in the regression model.Two regression models were fitted; the first one included only the independent variables and user satisfaction.The second, apart from the independent variables and user satisfaction, also included the newly created interaction variables.Because in multiple regression, interaction variables are often highly correlated with the variables from which they are created, the independent variables and interaction variables were standardised to decrease the correlation.
In order to know whether adding the interactions led to a significant improvement in how well the regression performed, R 2 was examined to find out whether it increased.This increase was tested for significance using the F-test.
To test for mediation, again, two models were fitted.The first one was fitted with only system quality, information quality and service quality as independent variables, and the second one, apart from including these variables, also included user satisfaction.The mediation effect of user satisfaction was identified by examining whether the differences between the coefficients of the first model and those of the second model were statistically significant.The Stata Statistical Package was used to analyse the data.

Descriptive analysis
The mean scores of the constructs were 4.34, 3.95, 3.49 and 4.36 for system quality, information quality, service quality and user satisfaction, respectively.This implies that in general, the respondents were satisfied with the BI system.
The Kendall rank correlation coefficient was used to measure the correlations between the constructs as the sample was relatively small (Field, Miles & Field 2012;Yamane 1967).
Table 2 shows the results (with probabilities in the brackets).
The results indicate that there is a high positive correlation between system quality and user satisfaction (r = 0.34, p = 0.000 < 0.01), information quality and user satisfaction (r = 0.27, p = 0.001), information quality and perceived net benefits (r = 0.20, p = 0.006 < 0.01) and user satisfaction and perceived net benefits (r = 0.31, p = 0.000 < 0.01); and there is a high negative correlation between service quality and user satisfaction (r = -0.20,p = 0.019 < 0.05) and service quality and perceived net benefits (r = -0.21,p = 0.007 < 0.01).However, these correlations are close to zero, which implies that the relationships are weak.

Inferential analysis User satisfaction regression model
Table 3 indicates that there is no relationship between information quality and user satisfaction (t = -0.36,p = 0.716 > 0.05) but there is a negative relationship between user satisfaction and system quality (t = -3.18,p = 0.002 < 0.01) and service quality (t = -3.67,p = 0.000 < 0.01).
This implies that when system quality or service quality increases, user satisfaction decreases, holding other variables constant.The reason for this kind of relationship was not investigated in this study.

Moderating effect of user satisfaction
First, only information quality, system quality, service quality and user satisfaction were included in the regression model as independent variables (see Table 4).
The results indicate that all the independent variables except information quality (t = 0.53, p = 0.596 > 0.05) affect perceived net benefits positively at least at the 10% level of statistical significance.The adjusted R-square is 0.2167, which means that the regression model explains about 22% of the variation in the data.Table 5 shows the regression results when the interaction variables were added to the model.According to the results, it is only the effects of system quality and service quality (on perceived net benefits) that seem to be moderated by user satisfactions because the probability values corresponding to the interaction terms [and t values (2.26 and -4.84)] are, respectively, 0.025 (< 0.05) and 0.000 (< 0.01).The coefficient of the service quality interaction term is negative.This implies that the higher the value of the interaction, the lower the score of perceived net benefits will be, provided other factors are held constant.It also implies that the moderating effect of user satisfaction is to suppress or decrease the effect of service quality on perceived net benefits.The interaction between system quality and user satisfaction affects perceived net benefits positively, such that the higher the value of the interaction, the higher the score of perceived net benefits will be.This implies that the moderating effect of user satisfaction is to increase or enhance the effect of system quality on perceived net benefits.In other words, user satisfaction tends to suppress the effect of service quality on perceived net benefits but to enhance the effect of system quality on perceived net benefits.
Applying the F formula, we obtain: The F ratio calculated from the change in R-square between the first and second model is 9.0259.The corresponding theoretical F ratio (3, 202) (at the 1% level) obtained from the F table is 3.782.This means that the F ratio is statistically significant; so, the null hypothesis that the interaction terms in the model are not significant is rejected, and the alternative hypothesis that at least one interaction term is significant is accepted.Because the interaction terms of system quality and service quality are the only ones that are significant, this implies that user satisfaction moderates the effects of system quality and service quality on perceived net benefits.

Mediation effect of user satisfaction
To test user satisfaction as a mediator between the independent variables and perceived net benefits, at first, the regression model included only the independent variables (i.e., information quality, system quality and service quality).Then, both the independent variables and user satisfaction were included in the same regression model.This was done, as implied above, to find out whether user satisfaction would significantly decrease or increase the relationships between the independent variables and net benefits.When the regression model was fitted without user satisfaction, the results in Table 6 were obtained.
The results show that only system quality (t = 4.37, p = 0.000 < 0.01) affects perceived net benefits positively at the 1% level.If the level of system quality increases by one unit, the level of perceived net benefits increases by 1.609 units, holding other factors constant.Adjusted R-square is 0.0961, which implies that about 9% of the variation of the perceived net benefits is explained by the model.
Table 7 shows the results when user satisfaction was also included in the model.The regression results in Table 7 show that there is no relationship between information quality and perceived net benefits (t = 0.53, p = 0.596 > 0.05).In the regression model of Table 6 (before user satisfaction was included in the model), information quality was not significant.Also, when it was included in the model, still it was not significant.This means that inclusion of user satisfaction in the model did not change the coefficient of information quality.Therefore, it can be concluded that user satisfaction does not affect the relationship between information quality and perceived net benefits and is hence not a mediator between them.
According to the results, there is a relationship between system quality and user satisfaction and between service quality and user satisfaction.There is also a relationship between user satisfaction and perceived net benefits (t = 5.73, p = 0.000 < 0.01).It is interesting to note that inclusion of user satisfaction in the model dramatically changed the coefficients of system quality [from 1.609 (t = 4.37, p = 0.000 < 0.01) to 2.044 (t = 5.82, p = 0.000 < 0.01)] and service quality [from 0.093 (t = 0.25, p = 0.801) to 0.595 (t = 1.68, p = 0.095 < 0.1)].System quality is significant in the first regression model and also in the second regression model (but with a bigger t-value) at the 1% level.Service quality is not significant in the first regression model, but after including user satisfaction, it became significant in the second regression model (of Table 7) at the 10% level.Therefore, it can be concluded that user satisfaction has a mediation effect on the relationships between system quality and perceived net benefits and between service quality and perceived net benefits.
The regression results in Tables 5 and 7 disqualify the hypothesised BI model in Figure 1 but suggest the BI model shown in Figure 3 as the model applicable in South Africa.It has been found that user satisfaction does not have any moderating or mediating effect on the effect of information quality on perceived net benefits and information quality does not affect perceived net benefits.The only paths left are those shown in Figure 3.

Conclusion
It has been found that there is no relationship between information quality and perceived net benefits and user satisfaction in a BI system.System quality and service quality are negatively related to user satisfaction but positively related with perceived net benefits in a BI system.User satisfaction is positively related to perceived net benefits in a BI system.User satisfaction mediates the relationship between perceived net benefits and system quality and service quality and moderates the effect of system quality on perceived net benefits by enhancing it, whereas it moderates the effect of service quality on perceived net benefits by reversing it.
Self-reporting bias may have led the study participants to over-report those factors that seem to be more acceptable and under-rate those that seem to be less acceptable.Some of them may have been unwilling to disclose problems or negative aspects about their systems.

Recommendations
In South Africa, system quality and user satisfaction should be enhanced and maintained in order to achieve perceived positive net benefits and this would make the BI system more effective and efficient.A research study focussing on a specific industry could be very helpful in facilitating a clearer understanding of the relationships among the variables.

FIGURE 2 :
FIGURE 2: Histograms and curves for key study variables: (a) system quality; (b) information quality; (c) user satisfaction; (d) service quality; (e) net benefits.

TABLE 1 :
Adapted from Davis 1989 and DeLone and Mclean 1992 modelsFIGURE 1: Conceptual framework for successful Business Intelligence System implementation.Survey instrument constructs.
Davis 1989 and Derived fromDavis 1989 and DeLone and Mclean 1992 models

TABLE 3 :
Regression model for user satisfaction.

TABLE 4 :
Regression model for net benefits (without interaction terms).

TABLE 5 :
Regression model for net benefits (with interaction terms).