1 Background and Research Objectives

1.1 Significance of Customer Satisfaction

In the past year, international giants in e-commerce has almost invariably expedited their market strategy globally. Internet shopping (IS) has extended customer’s physical limit. Customers from Beijing can taste the newly picked cherry from Chile, while ladies from Ukraine can enjoy the most up-to-date fashion in China. As smart phone becomes more affordable worldwide, it is a matter of a click to purchase products from anywhere in the world at any time. According to eMarketer, nearly one-quarter of the world’s total population uses smartphone [1]. The consumption urge for more product selection together with technology availability boost IS’ expansion.

Even in the booming stage of international IS, customer experience is and needs to be a core part in the strategic planning of IS websites rather than something nice to have. Compared to in-store shopping, it is much less controllable for online sellers to persuade a purchase. Online shopping offers more opportunities for interactive and personalized marketing [2], enabling customers to compare hundreds of alternatives at one time. The cost of retrieval or withdrawal when shopping online is very subtle compared with in-store shopping. Meanwhile, given the risk and uncertain nature of online shopping, online customer tends to be less rational. Emotions and feelings at the moment of clicking are more involved in the occasions of online shopping. From all the choices and browses online customers engaged, they look for value [3]. However, there is no practice yet which is able to anatomize the value clearly. On the contrary, value brings stronger competition among e-commerce websites than ever in the battle of winning customers. Hence, understanding and improving customer satisfaction is vital for players in B2C (business to customer) e-commerce market.

1.2 Conceptualizing Customer Satisfaction

Before thinking of how to measuring customer satisfaction, it is worthwhile to take one step back and seriously think of what customer satisfaction really is. Even if this topic has been deeply researched, there is no consensus in the definition of customer satisfaction [4]. Giese and Cote [5] define satisfaction as “response (emotional or affective) pertaining to a particular focus (product, consumption experience, etc.) determined at a particular time (immediately upon purchase, after consumption, based on accumulated experience).” Some research [6] defines it as “a cumulative evaluation of a customer’s purchase and consumption experience to-date.” There is also definition that “the important service attributes and measure customer’s perception of those attributes and overall customer satisfaction [7].” As we can see, satisfaction is so difficult to be explicitly defined. All of these definitions are entangled with capturing some traits of the state of satisfaction or what satisfaction produces. Besides, all of these definitions can hardly be used in corporate practice. As long as there is no agreement being made on the definition of customer satisfaction, debates over which attributes are the best reflections of customer satisfaction would not end. Thus, it seems more practical to focus on measuring the behavioral reflections of satisfaction rather than defining satisfaction. How do you show loyalty to a brand/product/service? Or what will you do if you are satisfied with a brand/product/service? It draws us back to the concern that to what extent it is likely to reflect customer satisfaction either by giving referral, buying more, not using competitors’ products, giving visually support, or other ways.

2 Past Research on Customer Satisfaction Measure

2.1 Net Promoter Score

A variety of IS customer satisfaction research has been discussing the construct of satisfaction, or metrics of satisfaction [8]. Even if there is no such a perfect measurement yet, the net promotor score (NPS) is one of the simplest as well as the most widely adopted customer satisfaction and loyalty measurement. NPS asks customer’s willingness to recommend a product or service to other people, evaluated on a scale from 0 to 10. Based on the score given, it divides all customers into three groups, naming detractors, passives, and promoters accordingly. The net promoter score is calculated by subtracting detractors from promoters. The net promoter score is used as the estimate for customer loyalty and business growth.

Compared with other traditional customer satisfaction measures, NPS asks fewer questions and is very easy to calculate. It reduces the barrier for customer satisfaction data collection greatly. Customer feels less bothered by one simple question than a set of questions. Many organizations conduct their NPS study on a monthly or weekly pace. Some even monitor their NPS performance in a real-time matter. NPS is very much explicit and self-explained. It diagnoses the health of business from the customer side, and notifies executives for adjustment or changes. Fred Reichheld [9] called NPS the ultimate question for the sustainable growth of business. With the persistent endeavor of Reichheld and his follower, nowadays, NPS has been adopted by many leading B2C companies and organizations of a variety of industries and business. An important outcome of this NPS phenomenon is that individual company or organization is able to compare their performance with competitors. More than that, NPS benchmarking forms a kind of standard in a sense that who satisfy customers better regardless of industries and business. That creates an atmosphere of customer first rather than profit first. It also allows dialogues between corporates from varied business in the scope of customer centered economy.

From the nation level, NPS has also made great contribution in the process of experience economy. In China, for instance, the government recently founded a special organization to build nation-wide customer NPS (C-NPS) management system [10]. Their research covers major domestic and foreign brands from FMCG, durable consumer goods, and service industries. The result is released publicly, aiming to promote customer satisfaction in regular corporate performance evaluation as well as to facilitate the so called customer centered culture in Chinese economy.

2.2 Criticism to NPS – from Intention to Implementation

In spite of its world-wide popularity, discussions and notions over NPS is not overwhelmingly positive. NPS has been challenged for its doubtful in predicting customer loyalty behaviors [11] and in helping make management decisions [12]. The underneath rationale of NPS lies in that referral intention is believed to be the very direct reflection of loyalty. Furthermore, NPS put emphasis on customer mind share. Customer mind share is reflected via the proportion changes of customer groups with different levels of referral intention. It claims that improving NPS can help companies to win bigger market share.

Along with more practice in NPS, it is challenged that the predictive validity of NPS for loyalty behavior. Some research compared NPS with traditional ACSI (American Customer Satisfaction Index) model in predicting company growth, and found no better performance with NPS [13]. Higher NPS score does not necessarily promise more positive comments for your service, or more purchase for your product, or even not more actual recommendatory behaviors for your website. NPS itself is unable to fix the gap between intention and actual behavior. Besides, using NPS alone is difficult to provide actionable plan for company executives or operation staffs. The user experience research sector in e-commerce companies are often put in a situation to explain to their product and merchandizing fellows about how the score is linked to website GMV (Gross Merchandise Volume) and what they can get from the score. For business with well-established profit model and steady sales growth, their NPS score may not have major changes over time given that the business is in a relatively stable stage [11]. At the same time, for newly developed e-commerce business, NPS score alone is not reliable enough to reflect customer satisfaction. Since the score can be very sensitive to major website events and has big variation across different customer groups depending on their familiarity to the website.

2.3 Continuous Purchase Intention and Satisfaction

In the realm of customer satisfaction research, continuous purchase intention (CPI), also called repeat purchase intention, is another attribute that has been thoroughly researched. Continuous purchase intention refers to the subjective probability that a customer will continue to purchase a product from the same online seller [14]. Compared with referral intention, CPI may have higher predictive influence on the share of wallet of business. Some studies use CPI as an equivalent metric for satisfaction. However, how CPI correlates with customer satisfaction and loyalty is not so obvious and direct as NPS. Customers with higher satisfaction level are very likely to shop again; whereas, customers’ continuous shopping may be due to various reasons rather than being satisfied or loyal. Although some empirical studies find indirect positive correlation existing between CPI and loyalty [15, 16] argue that CPI may not represent loyalty effectively. They extend loyalty into a relationship construct based on the different characters and various degrees of relationship strength, and claim that CPI correlates differently with different types of loyalty relationship.

More research discusses determinants or the construct of CPI [17], exploring ways to improve repeat purchase. Prior research has indicated a set of determinants of CPI, including but not limited to perceived usefulness, trust, satisfaction, switching cost, and perceived value. Research reveals that perceived value includes both utilitarian value and hedonic value. It is the main driver for customer to repeatedly purchase from the same store [18, 19]. Utilitarian value is proposed as a construct formed by product offerings, product information, monetary savings, and convenience. Hedonic value is proposed as a construct of six hedonic benefits, which are adventure, gratification, role, best deal, social, and idea. Other researches find that customers’ online shopping habit moderates the influence of those determinants’ influence on CPI [17]. For instance, the more experience a customer have with shopping in one particular store, the more likely she or he will come back and shop again. Despite of the presence of large amount of CPI research, applicable implications are rarely discussed.

2.4 Other Research Model on Measuring Customer Satisfaction

Given the shortages of NPS, many endeavors have been made in exploring a more precise measurement for customer satisfaction and loyalty and how the measure can promote business growth. In the aim of that, measures with more explicit predictive correlation with actual sales growth need to be introduced. A customer can be likely to recommend to other people without buying more himself. Also, a customer satisfaction measure system will work better than a single satisfaction score.

Interestingly, we notice that other research explored similar idea. In a study conducted by [20], researchers asked consumers “How do you show loyalty to the firms you do business with.” The top two answers were “spreading the word and telling others” and “buying more”, with a selection of 78 % and 69 % respectively. Further than that, Aite Group [12] started measuring referral performance score. In their metric, customers were divided on two dichotomous dimensions, referral (refer and don’t refer) and business growth (customers who increased consumption of the business and customers who do not increase consumption). The score was calculated by multiplying the percentage of customers that refer by the percentage of customers that grow their relationship. This is a worthy trying of combining both loyalty intention and loyalty behavior in one measure. The different customer segments sliced by referral intention and business growth provide valuable trace. It shows clearly how your customer groups with different referral and consumption pattern compose. It also calls attention to the inconsistency between behavior and intention, and what can be done to transform inconsistent customers to a better relationship stage with your business. However, depending on business features, customer segments sliced by business growth can be misleading and may not be appropriate for some business. Business relationship growth involves many variables, such as amount of spending, frequency of consumption, customer quality, length of doing business, and so no. Of some many variables, it brings out another problem that which is the proper one to pick out as customer’s behavioral loyalty contribution. Moreover, the metric tells only where you are, however, it is still a myth that where to go.

In the China Customer Satisfaction Index Model (C-CSI) [21], three metrics are considered, including total satisfaction (TS), factor satisfaction (FS), and continuous purchase intention (CPI). TS and CPI are both measured on a 5-point Likert scale, and calculate the percentage of the top 2 answers, very satisfied and satisfied, and willing to buy very much and willing to buy, respectively. FS considers both the satisfaction and importance of a set of sub-attributes of products, service, or images. Each satisfaction attribute is weighted accordingly. C-CSI score is the sum of the three metrics, with each given a specific weight. The formula is

$$ {\text{C}} - {\text{CSI }} = {\text{ TS}}*0. 4 { } + {\text{ FS}}*0. 4 { } + {\text{ CPI}}*0. 2 $$

As we can see, rather than composed by a single score, C-CSI adopts repeat purchase and satisfaction attributes into its measure system. Corporate performance can also be measured and compared on specific influence factors. Even though C-CSI still measures from a purely cognitive level, it extends satisfaction and loyalty to repeat purchase intention, which is helpful for companies to know how their market is going to change.

3 From Customer Satisfaction to Customer Experience

3.1 Taobao International and Tmall Global

When building customer centered experience testing system in an e-commerce setting, not only satisfaction and loyalty, but also practical implications for business growth should be considered. As discussed earlier, the practical value of customer satisfaction measurement is determined by the stage of business and market maturity. If this business factor is taken into account as early as possible, fewer barriers would be expected in either communicating results or facilitating further actions.

In our case, the business we service for is Taobao International and Tmall Global. Taobao International is a platform to connect Chinese domestic sellers and oversea individual buyers. On the contrary, Tmall Global is aimed to help Chinese domestic individual buyers to purchase oversea products easier. No matter either an import or an export direction, the two international B2C websites are both quite young, and are in a stage of introducing the marketplaces to more customers.

3.2 User Experience Pulse Tracking

Reviewing different endeavors in exploring a better customer satisfaction, although some look for a revolutionary replacement for NPS, evolutionary approaches are more commonly used. To continuously and systematically track customer satisfaction and loyalty for our websites, Taobao International and Tmall Global, we also take a more evolutionary approach. Since both websites are in early stage of business growth, it becomes the first and primary priority to building reputation and retain customers. This big issue, when translating into customer satisfaction tracking, lays priority on how do they like our websites, and how much likely they will buy again. Furthermore, while working with merchandizing and product fellows side by side, we have really made a lot of efforts on transforming the original customer feelings and thinking into interpretations that are meaningful to different stakeholders. For a useful tracking system, satisfaction metric alone is not expected to answer all “Why?” questions behind. It is vital to inform different stakeholders the good as well as the not good enough experience customers’ have with our websites. Considered all the factors above, our measure system is extended from customer satisfaction tracking to customer experience tracking. Three key metrics are used, which are NPS, CPI, and PSAT (product satisfaction), naming together User Experience Pulse Tracking (UEPT) (Fig. 1).

Fig. 1.
figure 1

User experience pulse tracking model

Full size image

3.2.1 NPS and Why?

NPS is the first and foremost metric in UEPT. First, referral intention has high correlation with loyalty and loyalty behavior according to prior research. Second, NPS is one of the most often used satisfaction and loyalty measure internationally. Many research entities and enterprises share data with the public. It allows comparison with similar products horizontally. Third, besides knowing the scores, we also want to know reasons behind. It is worthwhile to know what make them to recommend or not to recommend for explanatory and practical purposes. Based on traditional NPS practices and our website features, we generate 21 referral determinable factors. The 21 items list out the key user-faced touch point throughout a complete purchase experience, including product, price, promotion, payment, logistics, customer service, and website design. For customers who give a score of 7 or above, who are passives or promoters according to NPS system, we ask them to select reasons from the 21 items for making a referral to others. For customers who give a score of 6 or below, who are the detractors, we ask them to select the items that making them not recommending our website as well.

3.2.2 Continuous Purchase Intention

Given the practical importance in business growth, CPI is the second metric added into UEPT. A study [14] shows that an average customer must shop four times at least in a same online shop before the shop can make profit from the customer. High continuous purchase can bring increased gross profit rate for e-commerce websites, therefore, it leads to better ROI performance and higher profit [22]. Now some people may wonder why not asking customers’ repurchase behavior directly. One risk for measuring behavior directly is that repurchase behavior is often uncontrollable. Purchase behavior is often triggered by factors irrelevant with websites, many other random factors are involved at the same time. Every purchase is independent from the other, it is risky to predict one purchase by using the prior one. Whereas intention is relatively stable and is generally acknowledged to play a significant role in the prediction of future behavior. Besides, it is a memory burden to ask customers to recall how many times they have purchased. Plus that the reliability of recalling is doubtful. Self-reported data shall have higher reliability that collecting customers’ intention versus asking them to predict behavior in the future. It will be more practical to research respondents’ purchasing data if necessary.

Furthermore, high continuous purchase intention indicates more active website traffics and higher customer retention rate. For new websites to survive on a competitive battlefield, CPI tracking is a vital reference in regards to strategic operation planning for improving customer retention.

In UEPT, CPI asks customers on a 0 to 10 scale: “How likely is it that you are going to continue to purchase on our website?” Customers who give a score of 7 or above are considered the high CPI type, whereas customers score lower than 7 are considered the low CPI type. The question is open to all respondents, however, only those who have made one order at least on the website are taken into calculation.

3.2.3 Product Satisfaction

Product satisfaction (PSAT) is another index that have grown in popularity over the last couple years. There has been debates that whether PSAT or NPS is more important than the other. In fact, drivers of PSAT or NPS are not identical [23]. PSAT depends on your customers’ expectations on products and it emphases on the state of content of the user; whereas NPS asks customers if they believe your product is worth recommending to others. It indicates how customers rank your product compared with other competitors. There is known a strong correlation between the two metrics. Both are business-outcome metrics that are equally important to measure. In UEPT, PSAT asks respondents “How satisfied are you with our website based on your experience with it?” It is measured on a 5-point Likert scale, from very dissatisfied to very satisfied.

3.3 An Integrated System

In UEPT, the three scores are present as a complete customer experience tracking system, each index is reported independently. Some other satisfaction models apply generated metric by using statistical methods, such as ACSI and C-CSI. However, metrics produced by statistical modeling on the average are complex to explain. For inner stakeholders who do not have much patience with statistics, it adds extra burden on using the metrics. The three metrics in UEPT have all been applied maturely with plenty of successful cases, and are familiar to our stakeholders as well. Hence, it is much more effective and efficient to report the three metrics rather than creating a new one.

What is more, the three metrics together create rich and flexible customer dimensions. For example, when segmenting customer by using CPI and NPS together, it becomes possible to know that how much customers truly love us, whether it is verbal support only or it potentially increases sales. The extended customer segments also bring richer and more valuable information than the single dimensional segments of detractors/passives/promoters. It is valuable for marketing and merchandizing fellows to know the proportion of each of the segments and how the proportions change over time, and then seek solutions. For example, customers who do not recommend the website but are very likely to purchase again themselves may stay merely because they can hardly find alternatives. Customer satisfaction needs to be present to convert the mere repeat purchasers to committed purchasers [4].

In addition, it has also been found that generated metrics have reduced acuteness to website fluctuations since the numbers have been weighted. It is much easier to use NPS, PSAT with which industry benchmarking has been well established with public recognition, allowing quick comparison with other players. In general, NPS, CPI, and PSAT are strongly correlated, each of them contributes to satisfaction and loyalty. Moreover, using further referral factor questions, UEPT is able to help to identify the root causes for the up and down with customers’ attitude changes and improve (Fig. 2).

Fig. 2.
figure 2

Customer segments model

Full size image

4 Discussions

After reviewing various prior research on customer satisfaction and loyalty, there has not been any metric or model reached consensus. We adapted an evolutionary approach to develop the UEPT model. An innovative way was proposed to segment customers by using CPI and NPS, which gave us rich insight into customer attitude and root causes.

We also created a live system to continuously improve the questions used in referral factors in order to provide more meaningful information to business. The current 21 referral factors were reviewed frequently for refinement. We compared the selection of each referral factor for recommending or not recommending with their opening question feedbacks regarding to their complaints and what they believe we can make improvements to better serve them. We found factors and conditions that had been left out. For instance, if a new factor emerged as a frequently mentioned factor in their referral decision, which is not in the UEPT, we would add it to the questions.

Also, we think the referral factors need to be refined accordingly for different websites. The UEPT for Taobao International and Tmall Global are using the same referral factors for the purpose of keeping measurement consistent. The disadvantage is that the factors are less website specific and are less detail oriented. Besides, Taobao International and Tmall Global are facing different user groups and have different functions and services. It will be helpful to apply website customized referral factors. In general, how customer satisfaction and loyalty research can promote business growth requires persistent attempts.

For future research, we have not closely studied the relationship between NPS and PSAT. Occasionally, NPS and PSAT would change in opposed directions. There were no sufficient data to explain why customer are more willing to recommend us while they are less satisfied, or the other way around. Further research are needed to explore this area.