How Customer Experience Gets Overlooked in Analyst Reports
After the end of the spring analyst report season, Quadient is excited to be recognized as a leader by Aspire, Celent and IDC, and as a Dominant Vendor by Novarica. While we love the attention drawn to our leading strategy and capabilities, we are most proud about being called out by our customers as the CX leader in the Customer Communications Management (CCM) industry by several analyst firms. Let’s look at the process of Analyst Report creation to see how Customer Experience is overlooked.
When it comes to analyst reports, Customer Experience is generally relegated to the vendor summary portions of the report. In fact, most analyst reports do not score Customer Experience, in favour of exclusively reporting feature-based criteria. Analysts typically score three things; a spreadsheet of feature and revenue questions, three individual customer interviews, and a 60-90 minute overview presentation for twenty or more vendors.
In order to create a market report, analyst firms think about the past, present and future of the market. Each firm has a unique vision shaped by their vast experiences. This drives the creation and curation of the questionnaires that are distributed to the vendors. The number of questions for these reports in 2020 ranged from around 100 up to a gruelling 2,500 discrete questions, almost exclusively focusing on revenue and product capabilities. Answering yes to a variety of questions does not show potential clients how well those capabilities are integrated by the vendor. This could potentially cost a client millions in integration costs as they have to “connect the dots” themselves.
Along with the questionnaire, vendors are asked to submit between three and five customer references. This is one conversation per firm, while doesn’t allow the analysts to understand how well the offering serves multiple teams who collaborate on CCM delivery. With most reports covering over twenty vendors, there is not a lot of time to get into details with 60 to 100 interviews. Therefore, interoperability of features and other multi-department use cases are not explored. Some analyst firms, like Aspire and IDC, independently obtain thousands of CX data points. While useful, these are general NPS type of ratings and not a multi-departmental perspective of an implementation.
After the submission of the questionnaire, each vendor has 60-90 minutes to present to the analysts, showing the product, new features and an overview of the business and market. With over 20 vendors in several of the latest reports, it is also a considerable effort for the analyst firm to find all of the nuances in a market with several active niche areas. It is also not enough time for some of the disconnections in market offerings to reveal themselves.
This brings us back to customer experience. Customer experience scores are the best validation of a company and its offerings. Customer experience scores show how likely a customer is to recommend a solution, how satisfied they are with different aspects of the vendor in general and unfiltered open-ended commentary. As such, Quadient has the highest propensity to recommend in the industry at 97% on Gartner’s Peer Insights portal, with no critical reviews. Solutions from Opentext with multiple reviews range from 33% to 76% propensity to recommend. This is a sign of a wide portfolio that is not internally integrated, and what can happen when discrete questions are combined in a way that scores well but does not deliver the expected benefits to the customer.
While evaluating analyst reports is a great place to start the selection process for a new CCM system, it is not a substitute for a solid evaluation process. Make sure you demand that your current and prospective CCM vendors show you how integrated their portfolios are, because you have to meet every demand of your customer experience on every channel every single day. You’re buying a unified CCM solution for all of your applications, not just the easiest ones.