Today, an organization can benefit from survey best practice expertise and new software tools to make surveys easier, efficient, cost effective, and extremely beneficial to your bottom line for reasons of customer satisfaction, retention, increased sales, and more. Whether performing surveys in‐house or using a service provider, understanding survey best practices and what survey software can do can help you maximize the return from any survey investment. This whitepaper provides best practices and strategies to help you improve results from your survey Efforts. In addition, to illustrate how a software tool can make the survey process simpler and more productive, we include examples from Kinetic Survey.
We encourage you to contact both or either of our organizations to help you create, implement, manage and report on surveys of any type.
Goals of Survey Research
Does your company want to:
- Increase customer satisfaction and retention?
- Improve organizational performance?
- Stay ahead of the competition?
If so, a well designed customer survey program is a valuable source of actionable information for use in your quest. In order to be successful, survey programs must have a clear set of objectives. The most fundamental objectives that should be included in any customer focused survey program are:
- Verifying customer contact information and collecting demographic information.
- Understanding the requirements and expectations of various segments of your company’s customer base.
- Determining how well your organization and its competitors are meeting those requirements and expectations.
- Developing service standards based on survey findings.
- Establishing priorities and goals for improvement activities.
The most fundamental objective of all, being to continuously collect valid and reliable customer feedback that can initiate organizational strategies and lead to the retention of current customers and the maximization of their value to the organization.
In order to design a suitable program, survey sponsors must also answer these questions:
- How is the survey information used and by whom?
- Is the information gathered used to take action within the organization?
- How can the information keep existing customers and acquire new ones?
Once you have set clear objectives, answered these basic questions and gained the support of top management, you are well on your way to launching an effective survey program.
Surveys can be completed in a variety of ways:
- In person
- By telephone
- By email
- By web
- By fax
- By regular mail
Each of these methods has strengths and weaknesses. The choice of methods depends on the type of population surveyed, the characteristics of the customer Base, the type and amount of information collected, the amount of human and financial resources, and the time available to complete the project. The method chosen influences every aspect of the survey program including the design, measurement, and analysis of the data.
In‐person interviews are best suited to business‐to‐business situations where segments of the customer base are relatively well defined. Conducting personal interviews Is not only a good way to increase response rates, but it also shows customers that you are deeply interested in their opinions. This is a significant benefit to organizations with a reasonably small number of key customers.
In‐person interviews become very difficult to manage when large samples of respondents are required. This is not the only time when difficulties might occur with in‐person interviews. Problems also arise when the individual contacting the survey has a stake in the results of the survey. It is crucial to maintain independence and credibility in these types of interviews. Therefore, experts highly recommend conducting in‐person interviews by third parties outside of the organization. This recommendation then leads to higher survey costs.
Using Kinetic Survey, customer service and help desk departments can seamlessly embed surveying customers into their business processes. This allows them to be more proactive and closer to the customer. The results can lead to increased after market sales, improved marketing intelligence, better troubleshooting, and faster turnaround times on issues.
When using any of these survey completion methods, Kinetic Survey can be helpful. The application can be used for survey results data storage, reporting, an integration gateway to external
service providers, and more. For in-person or phone interviews, answers can be input directly into Kinetic Survey. Based on customer interactions, you can also generate lists and reports.
Interview (CATI) system work very well when needing samples from a relatively large end‐user products and services customer base. Telephone based interviews typically yield more representative samples of the customer base than other survey methods because randomization techniques can be Employed.
Telephone interviews offer many advantages such as their relatively low cost per completed Survey and the control that the interviewer has over the respondent. These interviews afford the opportunity for human interviewers to clarify any misunderstandings or problems that a respondent might have in answering the survey questions. Human interviewers are also able to offer a thank you and quickly terminate a call when they contact someone who is either is unable or unwilling to participate in the survey.
The main disadvantage of the telephone interview is that interviewers themselves can create unpredictable effects of their own. Only, careful training prevents interviewers stating question in a way That might have an affect on how customers respond. Because it is crucial to have well‐trained and qualified interviewers for telephone surveys, experts highly recommend conducting telephone surveys by third parties outside of the organization.
Kinetic Survey, with Voice XML plug-in capabilities, surveys customers without the bias of the human call taker or agent inputting results. Immediately after a customer interaction, your agent could transfer the call into Kinetic Survey to ask questions and handle answers.
Internet Email and Web-based Surveys
Internet‐based surveys are growing in importance as Web users become more representative of specific customer populations and offer some significant advantages. Data can be collected, transferred, and continuously updated online. Some of the challenges associated with telephone‐based surveys are overcome with these types of surveys. Often, respondents can complete the survey at their convenience and have a written version of the survey on the computer display to help keep the question and the ratings scales in mind as they complete the survey. Interviewer bias is avoided because every respondent sees exactly the same question.
Web surveys have become the preferred method of data collection where Web samples are representative of the entire customer base such as online retailers and financial services companies. Still many customer populations containing large percentages of lower income and education levels do not have direct access to the Web. Therefore, Web surveys might not be the appropriate method of collecting customer opinions. Changes in technology and infrastructure should broaden the population of Web users as time progresses. Many companies have chosen to acquire software systems to Administer and manage surveys with internal resources. Once those acquisition costs are recovered, per survey costs are relatively low.
Mail surveys remain a popular approach to customer surveys. Printed surveys can be mailed or distributed to a relatively large population of customers. Perhaps the strongest advantage of mail surveys is their relatively low cost per survey for each survey sent. However, response rates for mail surveys tend to be low. As a result, low response rates often cause mail surveys to be more expensive than telephone or Internet based surveys. Another disadvantage of these types of surveys is that the results are not representative of the customer base and can be somewhat biased.
Past research has shown that mail survey responses are more likely to be from people who have strong positive or negative feelings. The information gathered from customers with strong feelings is affected greatly by non‐response bias and might misrepresent the opinions and attitudes of the majority of the customer base.
Who to Survey
It is important to get the right person to complete your survey but it is not always that easy. There are two basic choices of whom to survey:
The decision maker OR the end user
Kinetic Survey manages the process of creating HTML or plain text email templates that notify customers of a survey. Presentation of the survey can be through the Web via a link on the email. The survey link can include an ID that ties the survey to a particular customer or customer interaction.
As with telephone and in-person methods, mail survey data can be input into Kinetic Survey for reporting capabilities. Using the Kinetic Survey database allows you to mix survey methods to meet your market needs for those without Web access, while giving respondents that do use the Internet the Web-based option. The valid answer depends on your organization’s needs and what you need to know. Who is the person that is best able to give you the feedback you need? Once that is established, you will be able to identify the individuals who should complete a particular survey.
Anonymous and Confidential Surveys
Should surveys be anonymous or confidential?
- Anonymous surveys – When is it best not to use them?
- Confidential surveys – When is it helpful to know who the respondent is?
First, when using incentives to increase response rates, you must know who the respondent is and have their contact information, if you are going to be able to award the prize. Secondly, experts consider it good practice to match existing customer data to the survey data because it allows you to use all available information that can help explain or even predict satisfaction levels.
It’s also good practice to offer the respondents an opportunity to request a response to their comments, complaints, or get resolution for possible open issues. Confidential surveys permit a wonderful opportunity for service recovery activities when things have not gone so well. If a survey collects sensitive information like financial or health related information, then an anonymous survey is most likely in order. Otherwise, surveys should be confidential rather than anonymous.
Just how many completed surveys are enough?
Truthfully, this matter can be very non‐intuitive. The answer is not necessarily based on the logic that the bigger the customer base the more completed surveys you need. In fact, the reverse is true – relatively small samples can provide a surprising level of accuracy. Required sample sizes indeed vary with the size of the customer base.
The difference in the sample size needed for customer populations of 1,000,000 and 1,000,000,000 is nil. Moreover, once the population has reached 10,000, required sample sizes increase only minimally. This is the point when cost‐benefit analysis is likely to come into play. Conversely, when very small populations are involved, more surveys need to be completed in order to provide information that is truly representative of the entire customer base. In the case of extremely small populations, samples approach near complete census levels in order to achieve acceptable levels of confidence and error rates.
Kinetic Survey capabilities include the ability to send both anonymous and confidential surveys. Rules structures permit or deny the acceptance of anonymous surveys.
Within a customer service or help desk scenario, rules can specify surveying every X number of issues or tickets. These sampling rules can apply to any list. Rules establish which events can trigger a survey such as the close of a case, or to survey in batch fashion, at scheduled intervals.
The frequency of surveys depends on the objectives of the survey program. Product research or competitive benchmarking research might only need doing once or, at most, twice a year. If the objective is to collect service quality perceptions from customers, then surveys need to be much more frequent.
Another good guideline is to only survey when your organization is committed to making the changes suggested by the results of the survey. If the organization is unable or unwilling to make changes based on survey data collected, then surveys should not be done. Surveys collected and not acted upon can send a destructive message to customers.
If the objectives of the survey program require frequent and ongoing data collection, then particular attention needs to be made to reducing the risk of overtaxing members of the sample pool. Take the effort to ensure that a particular respondent is not surveyed more than twice a year.
Kinetic Survey rules help you to not send surveys to people for X number of days, preventing customers from receiving too many surveys. Rules structures can batch and schedule surveys, such as setting a rolling yearly survey of all customers.
Inexperienced researchers typically make four common mistakes when developing a survey questionnaire.
- Not enough questions are asked.
- Surveys contain too many “open‐ended” questions.
- Too Much emphasis is placed on questions that measure attitudes.
- Not enough attention is paid to question wording.
Writing good survey questions can be the most difficult part of a survey project. The survey questionnaire must be written in such a way as to ensure the questions are valid (measure what is intended to be measured), reliable (the survey questions yield the same results at different times or to different samples), and unbiased (people are willing and able to give accurate answers).
Developing reliable, valid, and unbiased questions is not enough for creating a good survey questionnaire. Consider carefully how to organize and present questions. Effective surveys consist of a highly organized series of questions rather than a random hodge‐podge of questions. Group questions on related topics together into sections and follow a logical pattern. Well‐organized surveys work to reduce the challenges associated with surveys.
Effective surveys consist of more than just a collection of questions. Introductions and transitional statements help explain the purpose of the survey and help to enlist the respondent’s participation. For this reason, it’s imperative that the first few sentences of a survey capture the attention of the respondents and motivate them to continue completing the survey.
There are two basic question formats used in surveys: structured and unstructured. Structured questions present a list of answers from which the respondent can choose. These types of questions yield quantitative results referred to as scaled questions.
There are many different types of scales currently used in survey research. The most prevalent types are Likert, verbal frequency, and forced ranking. Likert scales are the most commonly used scales in surveys. Questions that use Likert scales are likely to ask a person’s position on a particular subject (e.g. Strongly Agree to Strongly Disagree. Scaled response choices should be unambiguous, mutually exclusive, and exhaustive).
Decisions on what scales to use in a survey have a direct impact on the type of analysis techniques that used. Therefore, it is very important to remember the objectives of the survey project and have a good understanding of the analysis techniques commonly employed. When using a Likert scale, experts recommend using a seven‐point scale. This scale offers an effective balance between potential analytical techniques, and the issues of validity and reliability. Avoid using scales of over seven points and never use a scale of less than five points.
Unstructured or open‐ended questions provide qualitative results. Every survey should include one opened question in effort to collect customer suggestions and recommendations. Also, open‐ended questions can be used as follow‐questions when respondents have rated a specific attribute with a particularly high or low rating.
Kinetic Survey makes it easy to create and modify questions and answer choices. Survey responses can be numeric ranges, customized lists of answers, or free text. Kinetic Survey scores and weights list or numeric answers.
Kinetic Survey includes many Web-standard formatting options including stylesheets, and extensive customization options to make your surveys fit within your brand identity.
The actual survey questions included in a survey are quite dependent on the objectives Of the survey and must be designed to collect data and information in support of the objective. A project having the objective of measuring customer Satisfaction with customer support would typically include questions
Measuring the following attributes:
- Agent courtesy
- Agent knowledge
- Agent authority to resolve issue
- Overall satisfaction with agent
- First contact resolution
- Overall satisfaction with organization
- Likelihood to use again
- Likelihood to recommend
Despite the best efforts of well‐meaning project teams, survey questionnaires can end up with some issues that need taking care of before launching a survey project. Common problems encountered are typos, omissions, and other types of errors. However, the problems most likely to derail or even deal a fatal blow to a research effort are poorly worded, confusing, or biased questions. Have no fear. There is a simple and effective action to take to reduce the chance of this occurring. Enter the survey pretest.
A survey pretest involves administering the survey to a small and representative sample of respondents. This gives the project team an opportunity to identify any problems that might exist within a survey. The pretest not only affords the opportunity to ferret out confusing terms or phrases, But can also reveal questions that are too difficult to answer. A pretest also gives the project team the opportunity to verify that different respondents are interpreting the questions in the same manner.
Every questionnaire can improve in some way. Even if you have a short deadline, make time for at least a quick pretest. You might save yourself from some headaches down the road. After completing the test and final tweaks, your project is ready for launch.
Kinetic Survey makes it easy to pretest your survey and do iterations within a survey. Before changing to active status, Kinetic Survey allows surveys to be in testing mode. Once active, it ensures reporting integrity, by preventing surveys from being changed. Survey cloning allows you to sunset or inactivate one survey while retaining similar questions for future surveys.
The Survey Process
Who should collect the survey data?
The crucial point to consider is whether your company has the survey expertise internally. If not, the use of external experts jumpstart your efforts and help to ensure that you set up a survey program. Outside specialists offer specific skills related to collecting and analyzing data that you might not have. They can provide the surveying, statistical analysis, interpretation skills and data warehousing capability that might be lacking within your company. This is especially important when just starting out on your journey, as they can help you design the most effective program possible and Help you to maximize the return on investment (ROI) of your investment.
Some survey methods are easier to implement using internal resources while others are more difficult. As mentioned previously, software packages are now available that allow organizations to design, implement, and manage email and Web‐based surveys with internal resources.
Improving Response Rates
Not everyone asked wants to participate in the survey therefore a 100 percent response rate is an unachievable goal. Make strenuous effort to get the highest response rate possible because the likelihood of measurement error dramatically increases as the response rate goes up.
Research shows that there is a relationship between response rates and the method of data collection. The highest response rates are achieved with personal interviews, then followed by telephone interviews. The lowest response rates are achieved with mail surveys and other self‐administered surveys such as Web‐ based and Interactive Voice Response (IVR)‐based surveys. It might seem that in‐person interviews might always be suggested. However, they’re not always feasible due to time, personnel, and cost constraints.
Many survey researchers use reminders when attempting to increase response rates. The most popular methods are:
Reminders are valuable tools for mail, email and Web survey projects. Send reminders midway through the survey‐fielding period. For example, if the fielding period for a mail survey is one month, then respondents should receive the reminder card two weeks before the cut‐off date. Typically a week after the original invitation, send reminders for email and Web surveys.
Offering incentives in an effort to increase response rates is a common practice in survey projects. Previous research has shown that the use of monetary rewards can increase survey participation by as much as 20 percent and non‐monetary rewards by five to ten percent. The key principle here is to offer an incentive in such a way that it increases survey response rates but that it does not affect the survey responses themselves.
Here are some useful rules of thumb for using incentives:
It is usually less effective to offer something other than cash or its equivalent. If you are not certain that your respondents value the item offered, it is probably best to forgo the incentive. If you are going to offer a monetary incentive, the typical maximum is five to ten dollars but even one dollar works well. A critical faux pas is to make the offer of money contingent upon completion of the survey. If you do so, you risk tainting your survey by the color of green. The chances are that respondents provide answers that are more favorable to your company. Offer the money to everyone or none at all.
A reasonable alternative to offering cash is to offer respondents an opportunity to enter a drawing for travel, a digital camera or the latest craze in personal electronics. Don’t shy away from this alternative because it can be far less expensive than giving money to everyone that is contacted.
Typically, survey results are coded and captured in a database format easily imported for use with statistical software packages. While basic statistical analyses can be performed using Microsoft® Excel, advanced statistical analysis and modeling must be performed using statistical packages such as SPSS®, SAS®, or SYSTAT®.
Commonly Used Analyses
- Frequencies and Percentage
- Measures of Central Tendency (Mean, Mode, Median)
- Measures of Dispersion
- Relationships between Variables (Correlation coefficients)
- Measures of Association (Cross tabs)
- Analysis of Variance
Because the analysis and the interpretation of the results is critical to the success of any survey project, the decisions based upon the results and resulting activities, this portion of the survey project is another instance where the use of external specialists can be very beneficial. This is especially true when the organization’s analytical resources might be in short supply.
Specialists who are knowledgeable about survey research can provide very valuable perspectives on the survey’s results. If internal resources are used, it is imperative that the resources have a good working knowledge of statistical tools and are able to determine the statistical methods which are the most appropriate to use for the data that has been collected.
The simple reporting of statistics is not enough in survey research. It is important to determine whether the survey results are different then what could have been expected by random chance. Always consider statistical significance in survey research designed to measure changes in customer attitudes or organizational performance.
Statistical significance means that there is an extremely low probability of the result rendered by a statistical test occurring if the population means are the same. In other words, we assume that the two means are identical (null hypothesis) and then test to see if they are actually different. It is common practice for the significance level to be set at p<.05, which means that there is a five percent chance or less that the survey results are due to sampling error rather than real differences.
The primary objective of the survey report is to present the results of the survey in a direct and meaningful manner. The most accepted manner to report survey results is through a document that combines a narrative and a condensed tabular report. These reports typically comprise two main sections: the executive summary and the main narrative section.
Use an executive summary as an introduction to the main body of the report. This section of the report typically contains narrative discussion of the survey results and presents the major highlights of the results. It also presents the results in table and graphical formats.
The main narrative section of the report usually presents a verbal description of the results. This section also presents an interpretation of the results and indicates the meaning of the results and relationships revealed by the survey. In addition, the research analyst can present additional insights, ideas, and information based on previous research that helps shed additional light on the results and their meanings, as well as their implications to the organization.
The main goal of the survey project’s reporting phase is to help translate the nuggets of insight gained into actionable information that can be used for service improvement activities, customer retention efforts and organizational decision making.
In order to be successful, survey programs must have a clear set of objectives, defined methodology, a representative sample of respondents, and reportable data, as well as the ability to act on the results once analyzed. By following these best practices, your survey can return a host of benefits including increased after market sales, improved marketing intelligence, and better troubleshooting to improve your product or service.
Through planning, internal and external resources, and tools such as Kinetic Survey, your next survey can be a success.
Try the Kinetic Platform Today.
When you're ready to learn how Kinetic Data can help you achieve better business outcomes, we're here to answer your questions.