Academia.eduAcademia.edu

Figure 6 – uploaded by Alan Chamberlain

See full PDFdownloadDownload figure
Fig.6 Overall scores of BUS-11 per chatbots. Chatbots are numbered in an order different from the list presented in Appendix A as we did not ask permission or inform the service providers about the usage of these chatbots The BUS-11 is a flexible scale used to assess user satisfaction of CRM chatbots; the current version of the tool has yet to be tested outside the domain of CRM and it could be considered providing a solid basis to investigate and support the design with other types of chatbots (e.g. general or domain-specific

Figure 6 Overall scores of BUS-11 per chatbots. Chatbots are numbered in an order different from the list presented in Appendix A as we did not ask permission or inform the service providers about the usage of these chatbots The BUS-11 is a flexible scale used to assess user satisfaction of CRM chatbots; the current version of the tool has yet to be tested outside the domain of CRM and it could be considered providing a solid basis to investigate and support the design with other types of chatbots (e.g. general or domain-specific

Related Figures (11)

Table 1 The original (English) version of BUS 15. Each item is assessed on a 5-point Likert scale from 1 (“strongly disagree”) to 5 (“strongh agree”)
Table 1 The original (English) version of BUS 15. Each item is assessed on a 5-point Likert scale from 1 (“strongly disagree”) to 5 (“strongh agree”)
Table 2. Number of questionnaires (BUS-15) per each available lan- guage: English, German, Dutch, Spanish should be proficient in writing and reading English to take part in the study.
Table 2. Number of questionnaires (BUS-15) per each available lan- guage: English, German, Dutch, Spanish should be proficient in writing and reading English to take part in the study.
the RSMEA is equal to 0.065. The scale is strongly reliable with an overall Cronbach’s alpha of 0.90. Although the model appears robust (Fig. 3) the factors loading for items 4, 5, 7, and 8 are only acceptable, i.e. above but very close to 0.6. This might indicate that alternative models could be explored.
the RSMEA is equal to 0.065. The scale is strongly reliable with an overall Cronbach’s alpha of 0.90. Although the model appears robust (Fig. 3) the factors loading for items 4, 5, 7, and 8 are only acceptable, i.e. above but very close to 0.6. This might indicate that alternative models could be explored.
privacy and security (factor 4) and time response (fac- tor 5). BUS-11 has a better fit than BUS 15, and it main- tains the structure of original solutions while reducing the scale of four items (see Fig. 4). The overall reliabil- ity of the 11 items solution is relatively high (a=0.89) but the RSMEA on average is slightly over the expected threshold of 0.07. The difference in terms of items between the two alternative models (BUS-11 and BUS-9) is minimal with the same overall reliability, but the BUS-11 is a more complete solution as this provides insights on spe- cific and relevant characteristics of chatbots. Hence, the 11-item model (BUS-11) seems preferable to the shorter one (BUS-9). The result of the CFA using the designometric matrix shows that while the model was stable in terms of factor loading for the first item of the scale, a problem due to negative variance, and the model fit was inferior com- pared to the psychometric model (chi-square/df = 2.7; CIF = 0.86, RSMEA = 0.27; SRMR = 0.06). shows that while the model was stable in terms of factor
privacy and security (factor 4) and time response (fac- tor 5). BUS-11 has a better fit than BUS 15, and it main- tains the structure of original solutions while reducing the scale of four items (see Fig. 4). The overall reliabil- ity of the 11 items solution is relatively high (a=0.89) but the RSMEA on average is slightly over the expected threshold of 0.07. The difference in terms of items between the two alternative models (BUS-11 and BUS-9) is minimal with the same overall reliability, but the BUS-11 is a more complete solution as this provides insights on spe- cific and relevant characteristics of chatbots. Hence, the 11-item model (BUS-11) seems preferable to the shorter one (BUS-9). The result of the CFA using the designometric matrix shows that while the model was stable in terms of factor loading for the first item of the scale, a problem due to negative variance, and the model fit was inferior com- pared to the psychometric model (chi-square/df = 2.7; CIF = 0.86, RSMEA = 0.27; SRMR = 0.06). shows that while the model was stable in terms of factor
Table 3 Factor loading of BUS 15 with the original solution at 5 fac- tors proposed by [5]
Table 3 Factor loading of BUS 15 with the original solution at 5 fac- tors proposed by [5]
Fig. 3 Graphic representation of the factorial model of the BUS 15
Fig. 3 Graphic representation of the factorial model of the BUS 15
Fig.4 Graphical representation of the factorial model of the BUS-11 significant differences, except for chatbots 13, 18, and 19 (see Appendix C). Figure 6 shows the difference in the assessment of satisfaction among the chatbots.
Fig.4 Graphical representation of the factorial model of the BUS-11 significant differences, except for chatbots 13, 18, and 19 (see Appendix C). Figure 6 shows the difference in the assessment of satisfaction among the chatbots.
Table 4 Comparative analysis of the BUS-15 and two alternative models: BUS-11 and BUS-9
Table 4 Comparative analysis of the BUS-15 and two alternative models: BUS-11 and BUS-9
Fig.5 Graphical representation of the correlation between the overall scores of the BUS-11 and the UMUX-LITE offers a very reliable, short, and solid solution, but in such a case designers will lose relevant aspects to inform their decision making. Therefore, we recommend using BUS- 11 also to collect data about key aspects such as privacy, security, and time to respond. However, BUS-9 could be used with mock-ups or early-stage prototypes when chat- bots are not yet fully functional and specific aspects of the systems are still hard to judge for participants.
Fig.5 Graphical representation of the correlation between the overall scores of the BUS-11 and the UMUX-LITE offers a very reliable, short, and solid solution, but in such a case designers will lose relevant aspects to inform their decision making. Therefore, we recommend using BUS- 11 also to collect data about key aspects such as privacy, security, and time to respond. However, BUS-9 could be used with mock-ups or early-stage prototypes when chat- bots are not yet fully functional and specific aspects of the systems are still hard to judge for participants.
Table 5 The BUS-11 versions are reported in English, Dutch, German, and Spanish. Each item is associated with a 5-point Likert scale from strongly agree (1) to strongly disagree (5)
Table 5 The BUS-11 versions are reported in English, Dutch, German, and Spanish. Each item is associated with a 5-point Likert scale from strongly agree (1) to strongly disagree (5)
Related topics:
MarketingComputer ScienceSoftware EngineeringArtificial IntelligenceHuman Computer InteractionInformation TechnologyLanguages and LinguisticsInformaticsNatural Language ProcessingUsability
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved