Insights from the ESS Ireland Self-Completion Pilot
Introduction
Modern societies increasingly rely on survey research to provide insights into social attitudes, political affiliation, acceptance of policy initiatives, and an array of other socio-economic and policy perspectives. However, survey research is undergoing a profound transformation as traditional face-to-face (F2F) interviewing becomes increasingly difficult to sustain. Rising costs, declining participation rates, and practical challenges related to fieldwork have prompted many large-scale surveys to experiment with alternative designs, most notably self-completion modes (Cernat & Revilla, 2021; DeLeeuw, 2018; Wolf et al., 2021). At the forefront of this change is the European Social Survey (ESS), long conducted exclusively face-to-face but now in transition: Round 12 (2025/26) will use a mixed-mode approach, while Round 13 (2027/28) will be fully implemented in self-completion mode (Fitzgerald, 2025).
As outlined in our previous paper on the Irish ESS Round 12 pilot study (Collins & Li, 2025), the move to self-completion opens opportunities for greater efficiency and flexibility, while also raising questions about data quality, representativeness, and inclusivity. The Irish pilot study provides a valuable opportunity to examine these challenges in practice. Conducted in 2024, the study implemented a mixed self-completion approach, allowing respondents to participate either via a mailed paper questionnaire or online through a secure web platform. The achieved response rate of 43% was the highest among ESS self-completion pilots to date, suggesting a strong level of engagement (Li & Collins, 2025). However, the coexistence of paper and web instruments raises important methodological and practical questions: who prefers paper, and who is willing to complete surveys online?
This paper builds on our earlier analysis of response patterns and turns to the specific issue of mode preferences. We examine the socio-demographic factors that shape respondents’ choices between paper and web, focusing on five basic characteristics: gender, age, education, domicile, and household income. In doing so, we aim to shed light on whether web completion can become the dominant mode in the future, or whether both options will need to remain available to secure representative participation.
Theoretical Background
Mode preference in self-completion surveys is shaped by a complex interplay of technological access, digital literacy, and respondent comfort (Heimel et al., 2024). Paper questionnaires have long been the default and may be perceived as more familiar, tangible, and relatively low-threshold, particularly for older respondents or those with lower levels of digital literacy. Web surveys, in contrast, offer speed, convenience, and lower costs for both researchers and participants, but they presuppose internet access, comfort with digital platforms, and often greater levels of education.
Combining paper and web introduces both opportunities and challenges. On the one hand, offering both options can maximise inclusiveness by accommodating diverse populations. On the other hand, dual modes may exacerbate mode effects, create sample composition biases, and increase logistical complexity in fieldwork and data processing.
Previous research suggests that age is one of the strongest predictors of mode choice, with older individuals tending to prefer paper (de Leeuw, 2005; Diment & Garrett-Jones, 2007). Education and income may also matter, as they correlate with digital literacy and access to technology. Domicile (whether respondents reside in urban or rural areas) could influence preferences through infrastructure and cultural factors, while gender differences are less consistently observed but still worth exploring.
Data and Methods
We use data from the ESS Round 12 Ireland pilot study (n = 631 respondents) conducted over eight weeks from June 10 to August 2, 2024. All respondents were offered the choice between completing the survey on paper (via a mailed questionnaire) or online (via a secure web link). Of those who participated, 73% returned the paper questionnaire, while 27% completed the survey online.[i]
To examine the impact of demographic characteristics on mode preference, we estimated six logistic regression models with mode choice as the dependent variable (paper = 1) and gender, age, years of full-time education, domicile and household’s income in quintiles as independent variables.
- Model 1: Gender (female=1)
- Model 2: Age (continuous variable, years)
- Model 3: Education (years of full-time education)
- Model 4: Domicile (urban=1)
- Model 5: Household income (quintiles)
- Model 6: Full model including all predictors simultaneously
Results
The regression results are presented in Table 1 (below):
Gender shows no statistically significant effect neither in the simple model nor in the full model, indicating that men and women are equally likely to choose paper versus web. Age has a strong and consistent positive effect in both models: older respondents are significantly more likely to complete the survey on paper (p < 0.001).
Education does not show significant effects in either specification, suggesting that years of formal schooling are not decisive for mode preference once age and other factors are considered. Note that models using levels of education (primary, lower secondary, etc.) showed similar results. We may not observe a significant education effect in Ireland as the Irish population, on average, is well-educated.
Domicile is significant in the simple model: urban respondents are less likely to choose paper compared to rural respondents. However, this effect dissipates when other predictors are included in the full model. Finally, household income is a strong predictor in both models: respondents in lower income quintiles are more likely to use paper, while higher-income respondents are more likely to choose the web (p < 0.001).Predicted probabilities plots help to illustrate these findings more clearly. Looking at age, Figure 1 shows that the probability of choosing paper mode rises sharply with age: while only around 25-35% of respondents in their 20s opted for paper, this increases to nearly 80% among those over 70.
Figure 1. The effect of age on the predicted probability of choosing paper mode.
Figure 2 demonstrates the household’s income effect: respondents in the lowest quintile have over 70% probability of choosing paper, compared to around 55% among those in the top quintile.
Figure 2. The effect of household’s income on the predicted probability of choosing paper mode.
Discussion and Conclusion
Our analysis highlights clear demographic divides in self-completion survey mode preferences within the Irish ESS pilot. Older respondents and those with lower household incomes remain heavily reliant on paper questionnaires, while younger and more affluent groups are more comfortable with web completion. Importantly, the persistence of these divides suggests that a fully web-based ESS would risk systematically excluding key population groups. This finding carries important implications for the ESS and for broader applications of self-completion surveys.
The results underscore the importance of maintaining both paper and web options at this stage of the transition. While web mode offers cost and efficiency advantages, paper mode continues to play a vital role in ensuring inclusiveness, particularly for older and lower-income respondents. A dual-mode approach therefore remains essential for representativeness and for securing the high-quality data on which the ESS depends.
Future research should examine whether these divides narrow over time as digital penetration and literacy increase. For now, the Irish pilot study demonstrates both the opportunities and the limits of relying solely on web-based self-completion.
References
Cernat, A., & Revilla, M. (2021). Moving from Face-to-Face to a Web Panel: Impacts on Measurement Quality. Journal of Survey Statistics and Methodology, 9(4), 745–763. https://doi.org/10.1093/jssam/smaa007
Collins, M., & Li, O. (2025). Moving to Self-Completion: Challenges and Opportunities for Survey Data Collection in Ireland. Available at: https://publicpolicy.ie/papers/moving-to-self-completion-challenges-and-opportunities-for-survey-data-collection-in-ireland/
de Leeuw, E. D. (2005). To Mix or Not to Mix Data Collection Modes in Surveys. Journal of Official Statistics, 21(2), 233.
de Leeuw, E. D. (2018). Mixed-Mode: Past, Present, and Future. Survey Research Methods, Vol 12, 75-89 Pages. https://doi.org/10.18148/SRM/2018.V12I2.7402
Diment, K., & Garrett-Jones, S. (2007). How Demographic Characteristics Affect Mode Preference in a Postal/Web Mixed-Mode Survey of Australian Researchers. Social Science Computer Review, 25(3), 410-417. https://doi.org/10.1177/0894439306295393
Fitzgerald, R. (2025, April 4). The move to self-completion interviewing on Europe’s cross-national general social survey – balancing innovation and continuity. International Methodological Conference “The Future of Survey Research,” Warsaw, Poland.
Heimel, S. K., Medway, R. L., & Horwitz, R. T. (2024). Should We Offer Web, Paper, or Both? A Comparison of Single- and Mixed-Response Mode Treatments in a Mail Survey. Journal of Survey Statistics and Methodology, 12(3), 593–623. https://doi.org/10.1093/jssam/smae021
Li, O., & Collins, M. (2025, April 8). Implementing the ESS Self-Completion Mode in Ireland: Results from the Pilot Study. 2025 International Workshop on Comparative Survey Design and Implementation, Skopje, North Macedonia.
Wolf, C., Christmann, P., Gummer, T., Schnaudt, C., & Verhoeven, S. (2021). Conducting General Social Surveys as Self-Administered Mixed-Mode Surveys. Public Opinion Quarterly, 85(2), 623–648. https://doi.org/10.1093/poq/nfab039
[i] Collins and Li (2025) provide a more detailed account of the sampling and fieldwork approach.