Survey Resources

OIR will review proposed surveys at Tufts that are intended to be administered to Tufts students, faculty, staff, or alumni and will provide recommendations for these projects; please see OIR’s Survey Review Process for more information about this process, as well as the types of surveys for which OIR typically provides assistance. Depending on the scale, scope, or content of your proposed survey, you may also need to seek approval from leadership in your school or division.

Following the initial steps below will help you prepare a plan for your survey project and administer an effective survey. Most of these steps also align with the items in the OIR Survey Proposal Form, which we ask you to fill out to help our office assist you in your survey needs as part of the Survey Review Process.

When should you conduct a survey?

Surveys are useful tools to collect data about the perspectives, experiences, and behaviors of large groups of people. Some reasons to conduct a survey include:

• Having a large sample or population size from which you want to gather information
• Wanting a relatively cost- and time-effective way for an office, unit, or department to collect data from a large group of people
• Wanting to give your sample or population with a convenient method to provide feedback (surveys are not time and date bound like a focus group)
• Wanting to have standardized quantitative data from a population or sample
• Wanting written qualitative data from a population or sample

Alternatives to new surveys

Running a survey may not always be the most appropriate method to fulfill your research objectives. When embarking on a data collection project, you should consider whether a survey is necessary and if you are using the most appropriate data source or tool to support your research questions and objectives. If you administer a survey without thinking through the best approach to collect data for your project, the survey data you gather may not ultimately support your needs. In addition, we all should be mindful of the hazards of over-surveying the university community and only conduct surveys when it is appropriate and necessary.

Below we have listed other data sources and data collection approaches to consider when planning your project.

Institutional and administrative data: In certain contexts, institutional data may provide more accurate and detailed information than a survey. For example, if you are looking for demographic characteristics of a population of students, that information can be found through using institutional data. Here are some examples of resources that provide publicly available institutional data about Tufts:

• Tufts University Fact Book
• Fall Enrollment Calculator
• Tufts Diversity Dashboard
• Common Data Set (CDS)
• Integrated Postsecondary Education Data System (IPEDS)

Your school, division, or unit might also have other administrative records that will support your data collection needs. For example, before designing a survey to collect information about how many people have participated in certain events or used particular services, consider whether this information might already be collected elsewhere.

Focus groups: For collecting detailed and novel perspectives consider using a focus group. A focus group is a good tool for diving deep into questions, learning about differing opinions, and generating new ideas. Focus groups are also useful when the population is small. Teaching@Tufts provides guidance on using focus groups to improve classes. Focus groups can be conducted virtually or in person. Additional focus group resources can be found below:

• Conducting Focus Groups
• The definitive guide to focus groups
• Connecting for Collecting Data: Qualitative Research Online (webinar)

Academic literature: Academic literature may help answer your research questions if they have already been addressed by existing research. Tisch Library provides research resources to help investigate the literature.

Data from previous or existing surveys: Check with OIR about whether your research question can be answered by surveys currently or previously administered by OIR.

• (coming soon) Review the survey instruments listed in the OIR survey search overview page to determine if data from an existing survey may answer your research question.

Develop research questions and objectives

After you have decided to conduct a survey, you will want to develop research questions and objectives.

• It is critical to take the time to clearly define your research question(s) and the objectives for your survey project at the outset. Some questions you might consider when defining your research question and objectives are:
  • What is the main purpose and/or goal of the survey?
  • What are you trying to learn from your survey?
  • What are you trying to measure? (e.g., satisfaction with services, engagement with community, effectiveness of programs)
  • What is the most important data that will be collected by the survey?
  • Who is the audience for the survey?
  • Who will use the survey results or are the primary stakeholders? What forms of data will be most helpful to the groups receiving or using the results?
  • How will the findings from the survey be used?
  • What gaps in knowledge or information are you looking to close?
• If you are not a subject matter expert in the topic of your survey, it is important to consult with subject matter experts, either internal or external to Tufts to help formulate appropriate questions.
• Conducting an informal literature review may also be useful to understand the topic you are looking to measure.

Identify any materials that can guide your survey planning

• Are there compliance or accreditation standards related to this survey?
• Are there previous surveys administered at Tufts (either by OIR or by other offices) or surveys conducted by other institutions or organizations you can emulate?

Choose a survey population

Who should be included or excluded from your survey? For example, a survey about a service on campus may not be appropriate for students currently studying abroad.

Sample versus census

Then decide if you should use a sample of the population or a census of the entire population. For example, if you were conducting a survey for all Arts and Sciences undergraduate students, a sample of the population may be sufficient, given the large number of students in the population. Two common ways to sample for a survey are:

1. Random sampling – everyone in the population has an equal chance of being selected to take the survey
2. Representative sampling – sampling that reflects the population being surveyed

However, if you wanted the ability to compare the results by majors, you may consider sending the survey to the entire population, that is, all Arts and Sciences undergraduate students. Majors with smaller enrollments would be more likely to have usable data if you sent the survey to the population. The entire population is also known as a census.

Distribution lists

• If you plan to distribute a survey by email, we recommend sending surveys to official Tufts email addresses and using recipients’ preferred first names in communications. Depending on the needs of your project, you may also consider obtaining approval to send your survey out to a relevant e-list at Tufts using a general link. [For more information, go to the “Communications strategy and distribution” section of this page and the “Qualtrics survey link types” subsection.]
• If you don’t have an email list for your survey or need assistance obtaining permission to distribute your survey, please indicate that when filling out the Survey Proposal Form. For surveys that fall outside of OIR’s scope (as defined in the Survey Review Process), we can assist in connecting you with an appropriate office at Tufts to obtain a contact list.

Decide whether you will use a third-party vendor to administer your survey and report on the results

A third-party vendor can...

• Reduce your administrative workload
• Help compare survey results to a national standard
• Provide specialized knowledge in a topic
• Act as a neutral party to handle responses of sensitive topics

If you plan to work with a third party to administer a survey, we ask that you submit a Survey Proposal Form before signing a contract, as outlined in the Survey Review Process.

Establish a timeframe

When deciding on the administration dates for your survey…

• Consider the time to develop a survey instrument, when the survey topic would be most relevant to the survey population, what deadlines you have, and how much time it will take to analyze the survey results.
• Make sure the survey administration to a given population does not overlap with the surveys on the OIR survey calendar. Overlapping surveys can reduce the response rate of both surveys. Please note that OIR recommends that you submit a Survey Proposal Form at least three months prior to your proposed administration date for surveys planned for populations of 500 or more and one month prior for populations less than 500. This is to ensure that there is sufficient time to coordinate schedules across projects. Please read OIR's Survey Review Process to determine if this guidance is relevant to your project.

Determine who will analyze and report on survey results and the type of reporting that is needed

Determine early in the process who in your department, office, or unit will be responsible for analyzing the data and reporting the results of the surveys. At the start of a survey project, you should also plan the type of reporting that is needed for your project. Ideally, these decisions should be made before the survey is launched, as it sets the project up for success. See the “Analyzing and reporting survey data” section of this page for more information.

OIR submits all the surveys we administer to the Tufts Social, Behavioral, and Educational Research IRB (SBER IRB), and we advise others to do the same. The SBER IRB “is a panel of Tufts' faculty, staff, and community members that reviews all human subject research to ensure the safety and welfare of research participants. The IRB ensures that any research study under its jurisdiction is in compliance with federal, state, and institutional regulations. Only the IRB has the authority to approve human subject research.”

If your project is not likely to meet the regulatory definition of "research" or involve "human subjects," as is the case for many surveys that are used for internal assessment and planning purposes at Tufts, please refer to the following guidance from the Tufts IRB to submit your project for review:

If you believe that your project does not meet the regulatory definition of “research” or does not involve “human subjects,” request a formal determination from the Tufts IRB using the NR/NHSR Determination Request Form on our Forms and Templates page. You will find instructions for submitting this form in eIRB in our ‘Submit a NR/NHSR Request’ tip sheet on our Tip Sheets page.

If you have further questions after reviewing the request form and tip sheet, please do not hesitate to contact the SBER IRB office at sber@tufts.edu or (617) 627-8804, or attend our remote office hours, Tuesdays, 11 a.m. – 1 p.m. You will find the Zoom link to our office hours on our Contacts Page.

For projects that likely meet the regulatory definition of "research" or involve "human subjects," please refer to the Tufts IRB website for information about submitting a full proposal.

A well-designed survey instrument is essential for collecting high-quality and reliable data. The survey should be as short as possible and focus on the topic at hand. Questions should be clear, unbiased, and relevant to the goals of the survey. This ensures that respondents will easily understand the questions, provide accurate and useful information, and stay engaged with the survey.

General resources

• American Association for Public Opinion Research (see the “Designing your questionnaire” section)
• Creating Effective Surveys: Best Practices in Survey Design
• Qualtrics: How to Create an Effective Survey in 15 Simple Tips
• Qualtrics: Survey Design Your Respondents Will Love
• Questionnaire Design Tip Sheet

Qualtrics XM Basecamp training courses (free, requires login)

• Applying Best Practices in Questionnaire Design
• Planning and Designing a Survey

Below, we outline some best practices for designing a survey.

1. Begin the survey with a statement of purpose to motivate and inform respondents of their rights. Clearly explain:
   • Why you are conducting the survey
   • That their participation is voluntary
   • That they can skip questions they would prefer not to answer
   • Whether their responses will be treated as anonymous or confidential [For more information, go to the “Data management and security” section of this page and the “Confidentiality and anonymity” subsection.]
   • How their information will be reported and used
2. Create clear and unbiased questions.
   • Use simple and straightforward language and limit jargon to ensure that your respondents understand what is being asked.
   • Keep questions and statements neutral (i.e., avoid leading questions).
   • Limit open-ended questions and/or consider having only one “additional comments” question at the end. Open-ended responses require more effort for the respondent and the researcher.
   • Avoid double-barreled questions – those that touch on more than one issue but allow for only one response.
   • Follow Tufts’ guidelines for gender-neutral, inclusive, and bias-free language.
3. Create clear answer choices.
   • Response categories should be comprehensive, including “not applicable,” “no basis for evaluation,” “don’t know,” and/or “other” where needed. This will decrease missing values and clarify the data.
   • Response scales should be balanced, containing equal numbers of positive and negative response options (e.g., “agree” and “disagree”).
   • Keep the answer scales consistent throughout the survey. This helps with predictability and reduces error. For example, if your scale goes from “Strongly disagree” on the left to “Strongly agree” on the right, make sure that all your agreement questions follow that pattern. Whether you decide to keep negative options on the left or right, keep this consistent throughout the survey.
   • Make sure that each response option is unique and that choices do not overlap.
4. Outline the survey structure and flow.
   • Keep the survey concise. Long surveys can lead to response fatigue, which can affect the quality of choices.
   • Start with less sensitive questions. This can help build rapport and make respondents more comfortable.
   • Position high-level and important questions earlier on in the survey when possible. This protects the data from potential survey abandonment.
   • Group similar questions together and use subheadings for sections if applicable. This helps the respondent understand the context and flow of the survey.
   • Keep in mind that questions asked earlier in the survey may influence answers respondents give later.
5. Test the survey
   • If possible, have a small number of people from the population you intend to survey go through the survey and collect their feedback. Make sure the instructions and questions are clear, and that the survey experience is a positive one.
   • Once the survey is programmed (e.g., in Qualtrics), run through the survey yourself multiple times to make sure that all questions appear, that branching and display logic work as expected, and that the data records correctly. Consider inviting a colleague to test the survey as well for a fresh perspective.

Not all surveys need to offer recipients an incentive to be successful. Sending a well-designed survey to an audience interested in the survey topic can garner plenty of responses. However, not all surveys may pique the interest of recipients. In those cases, incentives can provide a tangible benefit for respondents to complete the survey and serve as a way of thanking respondents for their time.

Pros and cons to using incentives

Things to keep in mind when using incentives

• When choosing an incentive(s), be mindful of your survey audience. For example, staff members might be less likely to want a Tufts sweatshirt than students.
• Make sure the incentive matches the survey tone and topic. For example, if the survey discusses serious and sensitive issues, the incentive (if any) should be appropriate and mindful of the context.
• Consider giving a low value incentive sent to all respondents (e.g., a $5 gift card sent to everyone who completes the survey) or running a raffle for a higher value prize (e.g., 20 random respondents win a $25 gift card).

If you use a raffle incentive, you may randomly select a specified number of survey respondents to win a prize(s). Be sure to contact the winners with their prize in a timely fashion. Also, be sure not to collect raffle contact information in an anonymous survey because that would break the survey's anonymity. When your survey is anonymous, using a separate raffle survey (redirected at the end of the main survey) ensures that the respondents’ main survey responses remain anonymous (i.e., unattached from the contact information they may provide in the raffle sign up). Even when your survey is confidential, having a separate raffle sign-up survey may be more reassuring to respondents. You should always make it clear to respondents that entering the raffle is voluntary and provide them with a way to opt out of the raffle.

Examples of incentives

• Monetary (e.g., cash, gift cards, JumboCash, Rhino Bucks, coupons and discounts)
• Tufts apparel raffle (e.g., sweatshirts, hats, t-shirts, and blankets)
• Non-Tufts merchandise raffle (e.g., Apple Watch, iPad Air, art prints, and books)
• Experiential incentives (e.g., dinner with a dean, front row seats at Commencement, alumni dinner)
• Food (e.g., food trucks, ice-cream cart, sundae bar, and pizza slices)
• Tabling at the student center or a dining hall encouraging students to complete a survey in real time, and all participants receive a small gift (e.g., goodie bag, snacks, sticker, t-shirt, key chain, phone card holder, lanyard, water bottle, drawstring bag)
• A dollar amount donation to a relevant nonprofit for every respondent to the survey
• Faculty members can provide time during a class or club to fill out the survey. While this is not a traditional incentive, having the time and space to complete a survey may be the push some students need.

Resources

• How to reward your survey participants
• Using survey incentives to improve response rates
• How to offer survey incentives without sacrificing good data

Purchasing and distributing survey incentives at Tufts

If you are Tufts staff or faculty and are conducting a survey out of your department, office, or unit, be sure to check if your unit has the funds to purchase and distribute the chosen incentive(s) before your survey goes out so you can deliver what is promised. Also make sure you know how best to purchase your incentive(s), For example, gift card purchases are not permitted on a P-card. See AccessTufts for a full list of items you cannot purchase on a P-card. When in doubt about how to pay for an incentive, check with your local Tufts Finance point-of-contact about how to do so.

Tufts resources

• Tufts Procurement Card (P-Card) policy
• OVPR research incentive gift cards policies
• Guide to P-Card in PeopleSoft
• TSS contact information

If you are a Tufts student and are conducting a survey for research, please consult with your principal investigator (PI) or research advisor about funding incentives for survey participants.

Confidentiality and anonymity

OIR has established guidelines about confidentiality and anonymity of survey responses. However, when you conduct your own survey within your department, office, or unit, you will need to communicate your own expectations and definitions around confidentiality and anonymity to your survey recipients. [For more information, go to the “Communications strategy and distribution” section of this page and the “Qualtrics survey link types” subsection.]

First you will need to decide if your survey will be confidential or anonymous. Confidential surveys means that those working on the survey have access to personal, identifiable information about who took the survey, including demographic information that may have been embedded, like major, gender, ID number, or email address. However, in almost all cases of these kinds of surveys, results are aggregated and presented as a group.

Anonymous surveys mean those working on the survey do not have access to personal, identifiable information of survey respondents due to the way the survey is configured in your chosen survey software. Anonymous surveys can be conducted using an anonymous link or the anonymize responses feature in Qualtrics.

Please note that you should refrain asking large numbers of demographic questions on an anonymous surveys. And do not embed any demographic information, which would break the anonymity of the responses. The more demographic information provided in an anonymous survey, the greater chance someone could be unintentionally identified by those conducting the survey.

There are situations where it is better to select a confidential survey and situations when it is better to choose an anonymous survey. Confidential surveys typically would be the default survey choice. These allow those working on the survey to know specific population and demographic information and be able to compare groups with confidence.  Anonymous surveys are favored when conducting a survey with sensitive subject matter, and you want respondents to feel comfortable answering honestly about these topics.

Regardless of what type of survey you choose, you need to inform survey recipients about expectations and definitions about confidentiality or anonymity in the survey invitation message and/or in the survey introduction text. You also will want to provide respondents with information about how the results will be reported and shared.

Resources

• Qualtrics: Anonymous vs. confidential surveys
• Qualtrics: Survey link types
• Qualtrics: Anonymous links
• Qualtrics: Anonymize responses
• TTS: Anonymous Survey Responses

Data security and records management

Tufts has established data security and records management practices and protocols. Before beginning your survey process, review the following webpages:

• TTS: Working with Sensitive Information
• TTS: Protecting University Data
• TTS: Data Privacy
• TARC: Record Retention

Tufts’ Office of Information Security should be your first resource if you have any questions about data management and security. Tufts Archival Research Center (TARC) can answer any questions you may have about archiving survey instruments, data, and other related records.

Some tips for survey data management and keeping your data secure are below:

• Collect only the demographic and identifying information you need to understand your research question.
• Electronic files should be kept on a secure server on a Tufts computer. If you have questions about Tufts standards and server options, TTS is available to advise.
• Do not send data files via email, instead protect them on a secure server such as Box or in Departmental data storage (Q: drive).
• If applicable, paper files and completed paper surveys should be kept under two levels of locks, e.g., in a locked cabinet in a locked office.
• Password-protect your electronic data files, if needed.
• In reporting, use breakout sizes of 5 or more respondents to protect respondents’ privacy. [For more information, go to the “Analyzing and reporting survey data” section of this page and the “Reporting with fewer than 5 respondents” subsection.]
• Do not include open-ended comments in public reports that could reveal a respondent’s identity or contain sensitive information. [For more information, go to the “Analyzing and reporting survey data” section of this page and the “Prohibited ways to use/report data” subsection, as well as the “Sensitive comments” subsection in the current section.]

GDPR/UK DPA

GDPR/UK DPA – the data privacy regulations in the European Union and United Kingdom -- are relevant if individuals are in certain European countries when they take a survey. It is primarily concerned with the directly or indirectly identifying data collected in surveys or attached to survey data. If you have determined that your survey is impacted by GDPR/UK DPA, you will need to make some adjustments to your survey to account for the data privacy regulations.

See AccessTufts’ EEA & UK General Data Protection Regulation (GDPR) page for more information about how these laws apply to work at Tufts. OVPR’s GDPR Research FAQs page also provides information about how these laws apply to research (as determined by the IRB).

Below are some examples of surveys that often include respondents in EEA/UK countries and would be impacted by these laws:

• Surveys administered during the summer or between semesters
• Surveys involving international students
• Surveys involving study abroad students
• Admissions surveys administered to matriculating/non-matriculating students from outside the U.S.
• Non-student populations (e.g., alumni, faculty, staff)

Some form of GDPR/UK DPA language or logic may be required on surveys that collect identifiable data from respondents who might be located in an EEA or UK country. In general, you can choose between a few different approaches to deal with GDPR/UK DPA regulations on your survey, depending on the needs for your particular project:

1. Create an anonymous survey instead
   1. Only completely anonymous surveys are not subject to GDPR/UK DPA. For a survey to be anonymous for GDPR purposes, responses cannot be connected to any identifying information, including location/IP address, and the survey cannot collect indirectly identifiable data that, when pieced together, would allow you to identify a respondent.
   2. In Qualtrics, use an anonymous link and select the “anonymize responses” option.
2. Request consent from EEA/UK respondents using language that complies with the regulations
3. Block EEA/UK respondents from taking the survey

TTS has more information about how set up GeoIP Logic in Qualtrics, which is necessary for approaches #2 and #3.

If you are unsure whether your survey is impacted by GDPR, please contact University Counsel and/or the Office of Information Security.

Other data laws

There are other laws to consider when writing and distributing your survey, including HIPAA and FERPA. When in doubt about survey questions or procedures that may fall under one of the above listed laws or another data law, contact University Counsel.

Sensitive comments

When your survey has concluded, we recommend reading all open-ended responses as soon as possible for what are considered sensitive comments. Sensitive comments include:

• References of discrimination or harassment against a person who identifies as one of the protected characteristics (gender identity, biological sex, sexual orientation, race, color, religion, national origin, ancestry, disability status, veteran status, age, genetic information, pregnancy, or criminal record)
• References to sexual misconduct
• References to active suicidal thoughts or behaviors
• References to a desire or plan to harm other people
• References to an incident of perpetrated harm

Comments referencing discrimination, harassment, or sexual misconduct

All Tufts employees are considered mandated reporters and are legally obligated to report information about sexual misconduct, discrimination, or harassment to the Office of Equal Opportunity (OEO). This includes open-ended responses on surveys. When you encounter a survey comment that references discrimination, harassment, or sexual misconduct, we advise that you reach out directly to OEO for guidance.

Other concerning comments

If you identify other concerning open-ended responses, such as those indicating a respondent may harm themselves or others, you may want to reach out to the appropriate authority at Tufts for guidance, such as Counseling and Mental Health Services (CHMS) (or the counseling and mental health resource on your campus), Tufts University Police Department (TUPD), Human Resources, or University Counsel.

Online versus paper surveys

There are various methods to distribute your survey, though at OIR we only use online surveys. Online surveys are a very common and convenient form of survey distribution and our recommended method of distribution to the Tufts community. However, depending on your needs, target audience, and resources, you might consider another distribution type, such as a paper survey.

• Online surveys: Questionnaires administered over the internet through email, websites, and/or social media platforms. Respondents complete the survey electronically, typically via a web browser or mobile app. This type of distribution is considered the most cost-effective and allows for easy tracking of responses. OIR primarily administers online surveys to the Tufts community via email through Qualtrics, which provides an easy and convenient way to distribute surveys to large groups of people.

• Paper surveys: Questionnaires distributed in physical form, often printed on paper. Respondents fill out the survey by hand and return it via mail or in person. This traditional method can be suitable for populations without internet access or for certain settings where online surveys may not be feasible.

Getting started in Qualtrics at Tufts

All faculty, staff, and students at Tufts have access to Qualtrics, a powerful and versatile web-based platform that helps users create and distribute surveys. You can log in with your Tufts credentials from the AccessTufts Qualtrics page. Qualtrics is OIR’s preferred tool for distributing surveys due to its data security and dynamic capabilities, and therefore, we support and recommend Qualtrics to the Tufts community. For information on other available tools, visit the form building FAQs page on AccessTufts. Note that Qualtrics accounts and university’s Qualtrics license are managed by TTS, not OIR. Although OIR can provide guidance about using Qualtrics, please contact TTS for any account-related questions or problems and guidance about connecting Qualtrics to other Tufts systems.

Tufts resources

• Tufts TTS: User Guide for Qualtrics: Includes instructions for logging into Qualtrics and inviting coworkers to collaborate in Qualtrics. Links from this page contain information about creating, editing, and distributing your project; viewing results; and transferring ownership of the project.
• Tufts TTS: Qualtrics Policies and Best Practices: Includes information on ensuring that your survey complies with accessibility and GDPR requirements, anonymizing responses, and preventing fraudulent responses.
• Tufts Learning Center courses: There are several live and on-demand courses offered, including Creating your First Survey, Adding Question and Block Logic, and Digital Accessibility: Qualtrics Surveys (login required).

Creating your survey

Qualtrics has an extensive support library, and you can find instructions for almost everything you need to do by searching the Qualtrics support page. Below, we provide some especially helpful links.

Qualtrics also offers free, self-paced trainings on Qualtrics XM Basecamp such as Building a Survey Project (login required).

Qualtrics - Survey basics

• Survey basic overview: The survey builder is where you will create and edit your surveys. In the Editing pane, you can create blocks to organize your content and add and customize your questions.
• Survey flow overview: The survey flow is where you can customize how your respondents will experience the blocks of your survey. If you have a simple survey, changing the survey flow is probably not necessary. However, if you have a complex survey, the survey flow is where you can randomize block order or use branch logic to show blocks of questions only to certain respondents.
• Survey look and feel overview: Your Tufts Qualtrics account will come preloaded with the Tufts theme. There may be other options you need to customize, however.
• Survey options overview: The survey options pane is where you can customize the survey name, how responses are handled, and privacy and security. You can also change the default close date. Please see Tufts’ Qualtrics Policies and Best Practices for information on changing certain settings.
• Using logic: You can use display logic to customize which questions or answer choices are seen for certain respondents. For example, you can ask follow-up questions if someone responds to a question in a certain way or hide questions if they are not relevant to certain groups. You can also use skip logic to send someone to later in the survey or to the end based on a response.

Qualtrics - Specific tasks and topics

• Create a new survey from scratch
• Create a new survey from a copy
• Qualtrics question types: Qualtrics offers a wide range of question types. We recommend utilizing straightforward and digitally accessible question types, such as multiple choice (single and multiple answer), matrix tables (make sure “mobile friendly” is checked), form fields, text entry, file upload, text/graphic, and rank order with text box. Inaccessible or overly complicated question types include graphic slider, heat map, highlight, hot spot, pick, group, and rank, and rank order with drag and drop.
• Create, copy, and move questions
• Name your question variables: We recommend changing the default to something meaningful for future analysis and data organization.
• Format your questions
• Format your answer choices
• Question validation: You can use response requirements to force or prompt responses or validate the content of a response (e.g., a number or an email address). It is best practice not to force a respondent to answer unless needed for the rest of the survey (e.g., for informed consent).
• Insert an image or other graphic
• Anonymize responses (remove identifying contact information in dataset) [For more information, go to the “Data management and security” section of this page and the “Confidentiality and anonymity” subsection.]
• Connect Qualtrics to Tufts Systems

Other resources

• Qualtrics Beginner Tutorial (2023; 6 minutes)

Survey accessibility

Tufts’ digital and web accessibility policy requires that all surveys meet accessibility standard so that all people can access them. Please read TTS’s Policies and Best Practices, which includes specific steps to take and links to instructions. You can also attend a session on ensuring accessibility in Qualtrics through the Tufts Learning Center.

For more information on accessibility, review Qualtrics’ survey accessibility page, which lists the question types that are and are not accessible, and how to use the Check Survey Accessibility Tool. Qualtrics’ ExpertReview functionality will also flag certain accessibility issues.

Qualtrics survey link types

As mentioned above, Qualtrics is Tufts’ preferred platform for administering online surveys. With Qualtrics, you can create various survey link types for the distribution method that best fits your survey project. When choosing a link type, you must also use the most appropriate link for the survey type you have chosen – confidential or anonymous. [For more information, go to the “Data management and security” section of this page and the “Confidentiality and anonymity” subsection.]

• Individual links (unique links): For cases where you have a contact list of your target population (or a sample) with email addresses. This is the default link type in Qualtrics when sending out an email invitation using the Qualtrics mailer. With individual links, you can identify which response belongs to which recipient, track responses in progress, and send out reminder messages to unfinished respondents as well as thank you messages to your participants. These links are best for confidential surveys. However, you can employ the anonymize feature in Qualtrics if you want to run an anonymous survey but be able to send reminders to non-respondents and limit respondents to a certain population.

• Personal links (unique links): For cases where you need to use a third-party mail system (i.e., not the Qualtrics mailer) to distribute the survey invitations but also need to track which response belongs to which recipient. These links are best for confidential surveys.

• General links (single reusable link, anonymous): For cases where you would like to distribute your survey to a general audience such as sharing your survey through a newsletter, e-list, or on a website or social media and do not need to track who completes the survey. This is the simplest link option to distribute a survey. The responses you collect from the survey will be anonymous. By default, general links do not restrict access to the survey or prevent someone from taking the survey more than once, but these settings can be adjusted in the survey options. These links are best for anonymous surveys.

Resources

• Qualtrics: Anonymous vs. confidential surveys
• Qualtrics: Survey link types
• Qualtrics: Anonymous links
• Qualtrics: Anonymize responses
• TTS: Anonymous Survey Responses
• Qualtrics: Personal links

Pre-launch, survey invitation, reminder, and thank you messages

There are four main touchpoints of communication with your target audience during a survey: pre-launch (or prenotification), invitation, reminder(s), and thank you messages.

If you choose to use Qualtrics for your survey, you can utilize the Qualtrics mailer to distribute your email communications. Please note that the Qualtrics mailer can not be used for pre-launch communications.

Pre-launch communications

A pre-launch (or prenotification) message is sent to potential survey participants before the actual survey is distributed. Its purpose is to inform recipients about the upcoming survey, provide details about the survey’s purpose and importance, and sometimes to encourage participation or prepare respondents for what to expect. Its contents are like that of the survey invitation but are often shorter and less formal. They serve as an initial communication to engage potential respondents, build awareness, and increase the likelihood of survey participation when it is officially launched. This is an optional communication step and may not be necessary for every survey project. It is recommended if your survey is: new, long, or complex, high stakes, includes sensitive topics, targets specific audiences that may need an extra boost of motivation or encouragement, or if your survey requires some kind of compliance regulation that informs participants about their rights and expectations.

Inviting recipients to take your survey

The invitation email serves as the initial point of contact between you and potential respondents if you choose to not send a pre-launch email. This email sets the tone for the survey experience and plays a crucial role in engaging participants by providing essential information about the survey. Most importantly, the invitation email will include the survey link.

A good email invitation will include the following:

1. Clear email subject line
2. Appropriate sender name and “reply to” email address
3. Reason why person is receiving the survey
4. Brief explanation of survey purpose
5. Identification of the confidentiality level [For more information, go to the “Data management and security” section of this page and the “Confidentiality and anonymity” subsection.]
6. Intended use of data/responses
7. Survey link
8. Expected timeline of the survey/deadline for response
9. Estimation of time required to complete the survey
10. Announcement of incentives (if applicable)
11. Contact information for questions or if respondents encounter any issues
12. Personalization wherever possible

Reminder and thank you messages

After you send the survey invitation, you can follow up with unfinished respondents with reminder messages to prompt them to complete the survey. However, you do not want to send too many reminders. For a survey that is between 4 or 5 weeks, for example, we recommend sending no more than 3 reminders so unfinished respondents do not become annoyed and then refuse to respond to the survey. After a certain point, the response rate will not drastically change. This option is only available if you distributed individual links through the Qualtrics mailer as this survey link type tracks the progress of a recipient.

You can also send thank you emails to respondents who completed the survey to thank them for their participation. This option is also only available if you distributed individual links through the Qualtrics mailer as this survey link type tracks whether a recipient completed the survey.

If you choose to use the Qualtrics mailer to send out your survey communications, there are several options to customize your distributions:

• Schedule emails to be sent
  • Tip: The default time zone for Tufts Qualtrics accounts is set to Mountain Time. If you would like to schedule emails to be sent according to Eastern Standard Time, or another time zone, navigate to “Account Settings” and select the correct time zone.
• Customize the “From” name and email address
• Customize the “Reply-to” email address
• Used piped text features to add more personalization (e.g., “Dear [First name of recipient],”)
• Customize link expiration (this is important if you intend to keep your survey open longer than 60 days)
• Preview send your email to yourself and/or your team to see how it will appear to recipients before you send an email to recipients

Other Qualtrics communications resources

• Using the opt-out link
• Avoid being marked as spam
  • If you are sending Qualtrics links to Google and Yahoo email addresses (rather than tufts.edu email addresses), please be aware that in early 2024, these email providers began enforcing stricter spam prevention measures. Including unsubscribe links in your messages to Google and Yahoo email addresses is required so your email will not be marked as spam. Please note that the only time you should be sending survey emails to non-Tufts email addresses is in specific cases, like a survey sent to alumni.

Timing your survey communications

When planning your survey communications, it is important to consider the timing to optimize response rates and engagement. Some factors to consider when scheduling your survey and survey communications include holidays and observances, weekends, day of week, and time of day.

Generally, holiday periods tend to disrupt people’s schedules and they may be less likely to respond to survey invitations during this time. Weekends largely depend on your target audience as some people may have more free time to respond on weekends while others may be occupied with personal activities.

Regarding the day of the week and/or the time of day, consider the preferences and habits of your target audience. There is no singular rule of when a good time is to reach out as this can vary depending on the demographic and lifestyles of your potential participants. If your target audience works full-time, you may consider sending communications before or after work hours. If your target audience is students, you may consider sending communications between classes, during mealtimes, or even during evening hours when they might have more downtime. Determining a time of day or day of week to reach out to potential participants is a mixed bag and you may want to gather some insight from colleagues or a sample of your target audience before sending your survey out to everyone.

Determine who will analyze and report on the data

Determine early in the survey creation process who in your department, office, or unit will be responsible for analyzing the data and reporting the results of the survey. Ideally, this decision should be made before the survey is launched, as it sets the project up for success.

Think about the kinds of products or reporting that will be needed. Do you need tables of simple descriptive statistics or crosstabs? Do you need to create visualizations, like charts and graphs? Do you need an interactive dashboard? Does an executive summary or written report need to be created?

When you are ready to report on the data, you should look at the response rates to make sure there are enough responses to generate a report. For instance, if you surveyed 200 people, and only received 8 responses, this would be a response rate of 4% and most likely would not be enough information to create useful, representative reports.

Qualtrics reporting tools

Within Qualtrics, there are two reporting tools: Results and Reports. There is extensive documentation on the Qualtrics’ website that explains these tools, how to use them, and the difference between the two.

Qualtrics resources

• Results-Reports Basic Overview
• Advanced-Reports Basic Overview
• Results vs. Reports

Export data from Qualtrics and analyze using other software

You can also export the survey data from Qualtrics and use any data software you are familiar with to clean and analyze the survey data. Below are some tools available to work with data, as well as TTS’ definitions for these tools. Most of these tools are free for Tufts community members. More advanced statistical software, like SPSS, Stata, SAS, and R will require knowledge or training before using them. See the “Need more help?” section of this page for more information about training and offices at Tufts that help you develop your data analysis skills.

• Excel – A spreadsheet software that can be used for data cleaning and analysis, including creating pivot tables, producing charts and graphs, and removing duplicate data.
• SPSS – Statistical software package used for predictive and descriptive modeling, data mining, text analytics, forecasting and simulation, experimental design, and more.
• Stata – A general-purpose statistical software package that provides facilities for data analysis, management, and visualization.
• SAS – Statistical software package used for predictive and descriptive modeling, data mining, text analytics, forecasting and simulation, experimental design, and more.
• R (public access and no Tufts credential needed) – Free software environment for statistical computing and graphics.
• NVivo – Analyze qualitative, unstructured data and share your findings.
• Tableau Desktop (cost associated) – A desktop software used to create data visualizations and combine data sets to produce reports, graphs, maps, animated visualizations and more.
• Tableau Server – Provides online access for staff and faculty to administrative reports and dashboards.

Prohibited ways to use/report data

Please refer to the “Institutional Review Board (IRB)” and “Data management and security” sections of this page for additional information about permissible uses of data. Information about prohibited ways of reporting data is below:

• Never report on row-level/respondent-level data. All groupings of data should have 5 or more respondents. See the “Reporting with fewer than 5 respondents” subsection below for more information.
• Never provide identifiable information in public facing reports. This identifiable information is usually found in open-ended responses.

OIR also suggests reaching out to Tufts’ IRB for guidance if you have questions about the appropriateness of sharing your survey results outside of Tufts.

Reporting with fewer than 5 respondents

When analyzing data and creating any reports, including breakout reports (i.e., reports summarizing responses from subgroups of respondents), you must be mindful not to report on small groups of respondents (sometimes referred to as small N or n in statistics – i.e., the population size or sample size). With a group of fewer than 5 respondents, the chance of identification, and subsequently compromising the privacy of these respondents, is high. OIR typically only creates reports or breakouts for groups of 5 or more, both in static and interactive reporting. We strongly advise that all reports, including breakout groups, have 5 or more respondents.

Sharing your results tells your survey participants that you value their time and increases accountability to act on the responses. One reason fewer people are participating in surveys is that they feel like their survey responses do not matter. By sharing your results, you encourage people to participate in future surveys.

Dos and don’ts of sharing survey results

• Do tell your participants when and where you plan to share your results in your advertising materials and survey invitation emails.
• Do set deadlines to keep yourself accountable.
• Don’t send survey responses or other confidential information by email or as an attachment.
• Don’t share verbatim open-ended responses publicly.
• Don’t share responses of individual recipients.

Tools for sharing survey results

• Email – While this method of sharing results is convenient, it is not secure like Box and Tableau Server are, which require additional log-ins. Additionally, sharing via a portal like Box means reports stay in one location and are easily accessible in the future, rather than having to search through emails.
• Box – Store, access, and share content securely at Tufts and beyond. (Similar to: Dropbox, Google Drive, and Microsoft OneDrive)
• SharePoint – Microsoft SharePoint is an online file storage, content sharing, and collaboration service available to all members of the Tufts community.
• Tableau Server (Do not use Tableau Public) – Provides online access for staff and faculty to administrative reports and dashboards.
• Web Services for posting content on Tufts websites

Below you will find additional Tufts services and offices that can assist you.

• New to Qualtrics? Zoom classes on Qualtrics are offered to Tufts employees through the Learning Center.
• Need in-person assistance analyzing or visualizing survey results? Tufts Data Lab offers immediate walk-in services to answer your questions for quantitative data analysis using Excel, NVivo, Tableau, SPSS, and more. See the lab assistant schedule for more details.
• Looking for teaching feedback? CELT offers Teaching Feedback programs. These programs help faculty gather feedback to inform improvements to courses and create documentation for career advancement. These feedback sessions are formative and confidential, and never summative. Feedback sessions are available at the request of the faculty, not a dean or department chair.
• Need help finding academic literature to answer your research questions? Tisch librarians offer 30-60 minute in-person or virtual appointments for in-depth research help. Schedule a librarian consultation.