A note about this data
This page contains analysis and opinion by Dave Verwer.
Some of the questions asked in this survey did not include “None” options. Instead, the survey asked respondents to skip questions that they did not want to answer. I did this because I wanted people to have an easy time filling in the questionnaire, but it would have been better if every question had an appropriate “None” option. This issue affected about 35% of the 109 questions asked.
What’s the issue? It’s impossible to know whether a respondent didn’t answer a question because there wasn’t an answer that was relevant to them, or whether they just skipped the question.
I want to present the raw data in the fairest possible way. So, all raw data charts may have the scale of their percentage axis toggled by a drop-down control below the chart. The toggle switches the maximum value for the percentage axis between the total number of survey respondents, and the number of respondents for that question.
For questions where the number of respondents is high, this toggle does not make a significant difference to the results. For example, 99.7% of respondents answered Question 1, and so even though there should have been an option representing “None” for this question, changing the scale does not significantly affect the results. However, where the percentage of respondents for a question is not as high, using the wrong scale for the percentage axis can present the data in a misleading way. Let’s look at two examples that highlight why a toggle is necessary for this year’s data.
Chart based on survey responses
Choosing “Chart based on survey responses” from the drop-down sets the chart to use the total number of survey respondents as the maximum value for the scale of the percentage axis. Use this scale where the question should have had a “None” option included — for example, Question 3:
If this question had included an option for “I have not developed any software for non-Apple platforms”, I believe the percentage of people answering it would have been closer to 99%, rather than 61%. In this case, the maximum scale of the percentage axis being the number of survey responses gives the best representation of the data.
Chart based on question responses
Choosing “Chart based on question responses” from the drop-down sets the chart to use total number of respondents who chose to answer that question for the maximum value of the scale of the percentage axis. Use this scale where it’s clear that the question was optional — for example, Question 41:
Why does the default scale change per-question?
I have chosen the scale for each question that I feel represents the data in the least misleading way. However, the final choice is yours. That’s why there is a drop-down scale selection next to every raw data chart.
Generally, questions that include a “None” option, or otherwise include exhaustive options, or questions that start with an “If” should use the “Chart based on question responses” scale, and all other questions should use “Chart based on survey responses”.
Why can’t the scale be toggled in the analysis articles?
The charts in the opinion/analysis articles (such as How popular is Swift?) slice and segment the data, so it rarely makes sense to allow the axis scale to be changed. All raw data charts can be toggled, all analysis/opinion charts are fixed scale by the author of the analysis.
What about next year’s survey?
Questions in next year’s survey will include a “None” option, where appropriate. 😬
Was there anything that could have been improved in this article? Let me know.