After assessing your organization’s capacity and research goals, you may choose to conduct a survey as one of the methods of data collection for your research analysis.
In this section:
Why conduct a survey?
A survey is an excellent way to gather information on individual organizations to capture trends at a collective level.
For example, one organization’s budget size does not tell you much about a trend in women’s rights funding, but if you know the budgets of 1,000 women’s rights organizations or even 100, you can start to form a picture of the collective state of women’s rights funding.
As you develop your survey questions, keep in mind the research framing that you developed in the previous section.
Remember: Your framing helps you determine what information you are trying to procure through your survey. The data collected from this survey should allow you to accomplish your goals, answer your key questions, and create your final products.
Back to top
Identify your survey population
This is an important step – the clearer you are about which populations you want to survey, the more refined your questions will be.
Depending on your research goals, you may want to create separate surveys for women’s rights organizations, women’s funds and donors. Or you may want to focus your survey on women’s groups and collect interviews for women’s funds and donors, as a survey for each population can be resource-intensive.
The questions you ask women’s groups may be different than ones you would ask women’s funds. If you plan on surveying more than one population, we encourage you to tailor your data collection to each population.
At the same time, some key questions for each population can and should overlap in order to draw comparative analysis from the answers.
Online survey
If you can reach your survey population online, it is useful and efficient to create an online survey.
We recommend two online tools, both which offer free versions:
Survey Gizmo allows you to convert your data for SPSS, a statistical software useful for advanced data analysis
Your data analyst person(s) will be the best person(s) to determine which tool is best for your survey based on staff capacity and analysis plans.
For accessibility, consider making a PDF form version of your survey that you can attach via email. This ensures organizations that have sporadic internet connections or those that pay for it by the minute can download the survey and complete it without requiring a constant online connection.
Paper survey
You may decide that an online approach is not sufficiently accessible or inclusive enough for your popuation.
In this case, you will need to create a paper survey and methods to reach offline populations (through popular events or through post, with pre-stamped envelopes for returning).
Back to top
Create your questions
Make it easy for participants to complete your survey.
1. Short and clear questions
If the questions are confusing or require complex answers, you risk having participants leave the survey unfinished or providing answers that are unusable for your analysis.
Ensure your questions only ask for one item of information at a time.
For example:
- What is your organization’s budget this year?
Easy to answer: participant can easily locate this information for their organization, and it is only asking for one item of information.
- What percentage of your budget have you identified as likely sources for funding for your organization, but are still unconfirmed?
Confusing and difficult to answer: are you asking for a list of unconfirmed funding sources or percentage of funding that is likely but unconfirmed?
This information is difficult to obtain: the respondent will have to calculate percentages, which they may not have on hand. This increases the risk that they will not complete the survey.
2. Simple and universal language
Many words and acronyms that are familiar to you may be unknown to survey participants, such as “resource mobilization”, “WHRD”, and “M&E”, so be sure to choose more universal language to express your questions.
If you must use industry lingo – phrases and words common to your colleagues but not widely known – then providing a definition will make your survey questions easier to understand.
Be sure to spell out any acronyms you use. For example, if you use WHRD, spell it out as “Women’s Human Rights Defenders".
3. "Closed” and "Open" questions
Closed questions:
Only one response is possible (such as “yes,” “no” or a number). Survey participants cannot answer in their own words and they typically have to choose from predetermined categories that you created or enter in a specific number. Responses to closed questions are easier to measure collectively and are often quantitative.
Example of a closed question: What is your organization’s budget?
Open-ended questions:
These are qualitative questions that are often descriptive. Respondents answer these questions entirely in their own words. These are more suitable for interviews than surveys.
They are harder to analyze at a collective level as compared to closed-end questions, especially if your survey sample is large. However, by making open-ended questions very specific, you will make it easier to analyze the responses.
Whenever possible, design your survey questions so that participants must select from a list of options instead of offering open-ended questions. This will save a lot of data cleaning and analysis time.
Example of open-ended question: What specific challenges did you face in fundraising this year?
Familiarize yourself with different types of questions
There are several ways to ask closed-ended questions. Here are some examples you can review and determine what fits best for the type of data you want to collect:
- Multiple choice questions: the participant can select one or several options you pre-entered
- Rating scales: the participant gives a note on a scale you pre-determine.
For this type of questions, make sure to clearly state what the bottom and the top of your scale mean
- Ranking: the participant will choose and organize a certain number of answers you pre-determine.
View more question types
4. Logical organization
If you plan to conduct this research at regular intervals (such as every two years), we recommend developing a baseline survey that you can repeat in order to track trends over time.
Set 1: Screening questions
Screening questions will determine the participant’s eligibility for the survey.
The online survey options we provided allow you to end the survey if respondents do not meet your eligibility criteria. Instead of completing the survey, they will be directed to a page that thanks them for their interest but explains that this survey is intended for a different type of respondent.
For example, you only want women’s rights groups in a given location to take this survey. The screening questions can determine the location of the participant and prevent respondents from other locations from continuing the survey.
Set 2: Standardized, basic demographic questions
These questions would collect data specific to the respondent, such as name and location of organization. These may overlap with your screening questions.
If resources permit, you can store these answers on a database and only ask these questions the first year an organization participates in your survey.
This way when the survey is repeated in future years, it is faster for organizations to complete the entire survey, increasing chances of completion.
Set 3: Standardized and mandatory funding questions
These questions will allow you to track income and funding sustainability. Conducted every year or every other year, this allows you to capture trends across time.
Set 4: Special issues questions
These questions account for current context. They can refer to a changing political or economic climate. They can be non-mandatory funding questions, such as attitudes towards fundraising.
For example, AWID’s 2011 WITM Global Survey asked questions on the new “women & girls” investment trend from the private sector.
5. Less than 20 mins
The shorter, the better: your survey shouldn’t exceed 20 minutes to ensure completion and respect respondents’ time.
It is natural to get excited and carried away by all the types of questions that could be asked and all the information that could be obtained. However, long surveys will lead to fatigue and abandonment from participants or loss of connection between participants and your organization.
Every additional question in your survey will add to your analytical burden once the survey is complete.
6. Simple and exciting
- Let participants know the estimated time to complete the survey before they begin
- Specify what information they will need to complete it so they have it on hand (for example, if you are asking for financial data, say it at the outset so they can prepare)
- Request information that organizations can easily access and provide – for example, requesting financial information from 20 years ago may be difficult (or impossible) for organizations to provide.
- Create an incentive to convince your survey population to complete the survey, such as a prize raffle. For example, AWID held a raffle draw for a round-trip flight to the AWID Forum as a prize for completing our 2011 WITM Global Survey.
General tips
- Ask for exact budgets instead of offering a range (in our experience, specific amounts are more useful in analysis).
- Specify currency! If necessary, ask everyone to convert their answers to the same currency or ask survey takers to clearly state the currency they are using in their financial answers.
- Ensure you collect enough demographic information on each organization to contextualize results and draw out nuanced trends.
For example, if you are analyzing WITM for a particular country, it will be useful to know what region each organization is from or at what level (rural, urban, national, local) they work in order to capture important trends such as the availability of greater funding for urban groups or specific issues.
Back to top
Test and translate
1. Your advisors
Involving your partners from the start will allow you to build deeper relationships and ensure more inclusive, higher quality research.
They will provide feedback on your draft survey, pilot test the survey, and review your draft research analysis drawn from your survey results and other data collection.
These advisors will also publicize the survey to their audiences once it is ready for release. If you plan on having the survey in multiple languages, ensure you have partners who use those languages.
If you decide to do both survey and interviews for your data collection, your advisor-partners on your survey design can also double as interviewees for your interview data collection process.
2. Draft and test
After your survey draft is complete, test it with your partners before opening it up to your respondents. This will allow you to catch and adjust any technical glitches or confusing questions in the survey.
It will also give you a realistic idea of the time it takes to take the survey.
3. Translation
Once the survey is finalized and tested in your native language, it can be translated.
Be sure to test the translated versions of your survey as well. At least some of your pilot testers should be native speakers of the translated languages to ensure clarity.
Back to top
Target the right population
1. Sample size
Your survey sample size is the number of participants that complete your survey.
Your survey sample should reflect the qualities of the larger population you intend to analyze.
For example: you would like to analyze the millions of women’s rights groups in Valyria but you lack the time and resources to survey every single one.
Instead, you can survey only 500 of the Valyrian women’s rights groups – a sample size - to represent the qualities of all the women’s groups in the region.
Recommended sample size
- 100 survey participants or less tend to be unreliable
- 250 to 400 will yield results of reasonable accuracy
- over 400 are fully adequate and will also allow accurate analysis of subgroups (for example, age groups).
Although it is not necessary to determine your exact sample size before you launch your survey, having a size in mind will allow you to determine when you have reached enough participants or whether you should extend the dates that the survey is available, in case you feel that you have not reached enough people.
2. Degree of participation
Even more important than size of a sample is the degree to which all members of the target population are able to participate in a survey.
If large or important segments of the population are systematically excluded (whether due to language, accessibility, timing, database problems, internet access or another factor) it becomes impossible to accurately assess the statistical reliability of the survey data.
In our example: you need to ensure all women’s groups in Valyria had the opportunity to participate in the survey.
If a segment of women’s groups in Valyria do not use internet, and you only pull participants for your sample through online methods, then you are missing an important segment when you have your final sample, thus it is not representative of all women’s groups in Valyria.
You cannot accurately draw conclusions on your data if segments of the population are missing in your sample size; and ensuring a representative sample allows you to avoid this mistake.
3. Database and contact list
To gain an idea of what the makeup of women’s groups for your area of research (region, population, issue, etc) looks like, it may be useful to look at databases.
- Some countries may have databases of all registered nonprofits, which will allow you to know your full population.
- If databases are not available or useful, you can generate your own list of groups in your area of research. Start with networks and coalitions, ask them to refer you to additional groups not in the membership lists.
By understanding the overall makeup of women’s groups that you plan to target, you can have an idea of what you want your sample to look like - it should be like a mini-version of the larger population.
After participants have taken your survey, you can then gauge if the resulting population you reached (your sample size) matches the makeup of the larger population. If it doesn’t match, you may then decide to do outreach to segments you believe are missing or extend the window period that your survey is open.
Do not be paralyzed if you are unsure of how representative your sample size is – do your best to spread your survey as far and wide as possible.
Back to top
Previous step
2. Frame your research
Next step
4. Collect and analyze your data
