Insights from the ESS Poland Self-Completion Experience
Introduction
Across the world, large-scale social surveys provide the basis for understanding the context and implications of many socio-economic public policies. However, these surveys are undergoing a profound transformation in how data are collected. Traditionally, interviewer-administered surveys, whether face-to-face or via telephone, were considered the “gold standard” in ensuring data quality and representativeness. However, the combination of rising costs, falling response rates, lengthy fieldwork periods, a declining interviewer labour force, and changing communication habits has pushed survey organisations towards greater reliance on self-completion modes, either online or on paper (Cernat & Revilla, 2021; DeLeeuw, 2018; Eggleston, 2024; Heimel et al., 2024; Wolf et al., 2021).
The European Social Survey (ESS) has played a particularly important role in this transition. Since its inception in 2002, the ESS has been committed to the highest methodological standards, carefully balancing cross-national comparability with innovations in data collection. Traditionally carried out solely through face-to-face interviews, the ESS is undergoing a major shift. Round 12 (2025/26) will adopt a mixed-mode design, and Round 13 (2027/28) will mark the first full implementation of self-completion (Fitzgerald, 2025). In an earlier study of the Irish Round 12 self-completion pilot, Collins & Li (2025) highlighted that this transition offers prospects for increased efficiency and flexibility but also brings challenges regarding data quality, representativeness, and inclusiveness.
Figure 1: European Social Survey Self-Completion Response Rates, 2020-2024
The urgency of this transition did not arise in isolation. The COVID-19 pandemic accelerated the need for change: in some countries, interviewer-administered surveys became logistically impossible under strict quarantine conditions, forcing several ESS countries to move toward self-completion sooner than expected. Poland represents a particularly striking case. Faced with the impossibility of conducting face-to-face interviews, between May 2021 and June 2022 the Polish ESS team successfully implemented a full self-completion design, combining web and paper questionnaires, without prior pilot testing (Kotnarowski, 2025). As shown on Figure 1, the Polish study achieved one of the highest response rates among all self-completion ESS studies implemented. Despite the challenging context, Poland achieved robust data collection, offering important lessons both methodologically and substantively.
This paper documents this Polish experience and explores its implications. We begin by describing the methodological aspects of the transition, before turning to substantive findings with a particular focus on sensitive and cultural questions. The analysis illustrates how self-completion can shape responses on topics where social desirability bias and cultural norms play a significant role.
The Polish ESS Experience with Self-Completion
When COVID-19 restrictions made face-to-face fieldwork impossible, the Polish ESS team had to pivot rapidly to self-completion. The decision to adopt a self-completion design was taken under considerable time pressure in May 2021, in order to ensure Poland’s continued participation in the ESS.
The first pilot, launched in October 2021 with a sample of 100, provided respondents with an invitation letter, a link to the web questionnaire and an unconditional incentive of PLN 25 (~EUR 5.45)[i]. As the Polish team applied a sequential approach, subsequently a paper version with a return envelope was included with the first reminder sent seven days after the invitation. Two additional reminders were sent later, and the achieved response rate was 17%.
After the first attempt, the Polish team changed their approach to the study and conducted a second pilot in November 2021 (sample size = 300). Here a concurrent approach was adapted, and respondents received an invitation letter with the link to the web questionnaire, together with a paper questionnaire and return envelope. The incentive structure was adjusted: an unconditional incentive of PLN 10 (~EUR 2.18) was combined with a conditional incentive of PLN 80 (~EUR 17.43) for those who completed within three weeks after the invitation and PLN 60 (~EUR 13.07) for those completing in the following three weeks. After three reminders, the response rate achieved was 33%.
Building on these experiences, full fieldwork was launched in January 2022 and concluded in March 2022, using the second pilot’s design. The sample size was 5,278 with the achieved response rate of 32%.
To boost participation further, an additional fieldwork phase took place from April to June 2022. Fieldworkers visited non-responding addresses, not to conduct interviews, but to deliver another invitation letter, a paper questionnaire, the link to the online version, and a conditional incentive of PLN 60 (~EUR 13.07). This supplementary effort increased the overall response rate to 37%.
Figure 2 presents the distribution of response rates by mode (web, paper, and fieldworker-assisted). This mixed-mode approach ultimately proved important in broadening participation and enhancing data quality under highly constrained circumstances.
Figure 2: European Social Survey Poland Self-Completion Response Rates by Mode
Responses to Sensitive and Cultural Questions
Beyond methodological innovations, the Polish self-completion experience offers important insights into how respondents engage with questions considered sensitive or culturally loaded. We use “sensitive” here not only in terms of issues prone to social desirability bias, such as attitudes toward minorities, sexuality, or corruption, but also in relation to questions embedded in specific cultural or political contexts, where respondents may feel discomfort in interviewer-administered settings.
As Li and Collins (2025) show in their study of survey mode and sensitive questions based on the ESS Round 12 self-completion pilot study in Ireland, the mode of data collection may influence respondents’ answers to sensitive questions. Self-completion provides respondents with a greater sense of anonymity, potentially reducing the pressure to provide socially acceptable answers. Examples of sensitive and cultural questions from the ESS Poland Round 10 include items on sexual minorities, church attendance, and climate change attitudes.
Figure 3 presents the distribution of responses (weighted to be nationally representative) to the question on sexual minorities in ESS Poland across all available rounds. It asked whether respondents agree thar gays and lesbians are free to live life as they wish. Overall, the data indicate a long-term trend of growing acceptance, with rising proportions of respondents agreeing or strongly agreeing that gays and lesbians should be free to live life as they wish. The share of “agree strongly” peaks in the self-completion round (Round 10) before declining somewhat in Round 11, when data collection returned to face-to-face mode. The “agree” category follows the opposite trajectory, dipping in Round 10 and increasing again in Round 11. This pattern suggests that in the self-completion mode respondents may feel more comfortable expressing positive attitudes toward sexual minorities. In addition, the proportion selecting “don’t know” has declined over time, reaching its lowest level in Round 11, further supporting the argument of the growing acceptance trend.
Figure 3: Distribution of “Gays and lesbians free to live life as they wish” variable in the ESS Poland, by rounds.
Note: data are weighted to be nationally representative
Figure 4 (below) illustrates the trend in religiosity in Poland across all ESS rounds. It presents mean values on a scale from 0 (not at all religious) to 10 (very religious) for a question asking: “how religious would you say you are?” The results show a slow decline between Rounds 1 – 9, reaching its lowest point in the self-completion round (Round 10), where it dropped from 6.1 in Round 9 to 5.2. When the ESS survey returned to face-to-face mode, the mean religiosity figure increased again to 6.0 in Round 11. This pattern suggests that in the presence of an interviewer, respondents in Poland may provide more socially desirable answers, reflecting the country’s strong religious context.
Figure 4: Mean response to “how religious would you say you are” in the ESS Poland, by rounds.
Note: data are weighted to be nationally representative
Figure 5 depicts Polish respondents’ views on the causes of climate change in Rounds 8, 10, and 11. They were asked whether climate change is mainly the result of natural processes, human activity, or both. The share of respondents attributing climate change “about equally” to natural processes and human activity fell from above 50% in Round 8 to around 40% in the self-completion round (Round 10), before rising again to 50% in Round 11. Conversely, the proportion who saw climate change as caused “mainly by human activity” increased from under 30% in Round 8 to 40% in Round 10 and then dropped back to 30% in Round 11. These shifts suggest that respondents who might normally tell an interviewer that climate change is caused equally by both factors were more likely, in self-completion mode, to select “mainly by human activity”. This pattern suggests that self-completion may elicit more truthful and less socially desirable responses.
Figure 5: Distribution of “views on climate change variable in the ESS Poland, by rounds.
Note: data are weighted to be nationally representative
Discussion and Conclusion
The evidence from ESS Poland suggests that survey mode can shape how respondents approach questions with social, cultural, or political sensitivities. In each of the domains examined (sexual minorities, church attendance, and climate change), self-completion mode appears to reduce interviewer effects and social desirability bias. Respondents demonstrated greater willingness to express supportive views of sexual minorities, to acknowledge lower levels of religious practice, and to attribute climate change mainly to human activity. The Polish data suggest that by reducing interviewer effects and providing greater privacy, self-completion fosters more honest reporting on socially delicate issues. These findings align with earlier evidence from the ESS pilot in Ireland (Li & Collins, 2025), reinforcing the idea that the mode of data collection matters not only for methodological outcomes such as response rates, but also for the substantive interpretation of public opinion trends.
The Polish case also demonstrates that even under crisis conditions, a rapid shift to self-completion can yield high-quality survey data. With appropriate fieldwork support and a dual-mode design, Poland successfully preserved both response rates and data integrity.
These lessons extend beyond Poland. As the ESS moves toward full self-completion in Round 13 (2027/28), the Polish experience highlights the importance of flexibility, resilience, and methodological creativity. In situations of uncertainty, whether due to pandemics, budget constraints, or other disruptions, well-designed self-completion approaches can safeguard the continuity of high-quality cross-national data collection. The insights gained from this experience underline both the obstacles and potential of self-completion. While it may be a necessary adaptation, it offers an opportunity for innovation but a challenge in interpreting time series trends in social research.
References
Cernat, A., & Revilla, M. (2021). Moving from Face-to-Face to a Web Panel: Impacts on Measurement Quality. Journal of Survey Statistics and Methodology, 9(4), 745–763. https://doi.org/10.1093/jssam/smaa007
Collins, M., & Li, O. (2025). Moving to Self-Completion: Challenges and Opportunities for Survey Data Collection in Ireland. Available at: https://publicpolicy.ie/papers/moving-to-self-completion-challenges-and-opportunities-for-survey-data-collection-in-ireland/
de Leeuw, E. D. (2018). Mixed-Mode: Past, Present, and Future. Survey Research Methods, Vol 12, 75-89 Pages. https://doi.org/10.18148/SRM/2018.V12I2.7402
Eggleston, J. (2024). Frequent Survey Requests and Declining Response Rates: Evidence from the 2020 Census and Household Surveys. Journal of Survey Statistics and Methodology, 12(5), 1138–1156. https://doi.org/10.1093/jssam/smae022
Fitzgerald, R. (2025, April 4). The move to self-completion interviewing on Europe’s cross-national general social survey – balancing innovation and continuity. International Methodological Conference “The Future of Survey Research,” Warsaw, Poland.
Heimel, S. K., Medway, R. L., & Horwitz, R. T. (2024). Should We Offer Web, Paper, or Both? A Comparison of Single- and Mixed-Response Mode Treatments in a Mail Survey. Journal of Survey Statistics and Methodology, 12(3), 593–623. https://doi.org/10.1093/jssam/smae021
Li, O. & Collins, M. (2025). Moving to Survey Self-Completion: Mode Choice and Sensitive Questions. Available at: https://publicpolicy.ie/papers/moving-to-survey-self-completion-mode-choice-and-sensitive-questions/
Kotnarowski, M. (2025, April 8). Is the self-completion a remedy to the problems with implementing face to face interviews? Experiences of the ESS in Poland in transition to self-completion mode. 2025 International Workshop on Comparative Survey Design and Implementation, Skopje, North Macedonia.
Wolf, C., Christmann, P., Gummer, T., Schnaudt, C., & Verhoeven, S. (2021). Conducting General Social Surveys as Self-Administered Mixed-Mode Surveys. Public Opinion Quarterly, 85(2), 623–648. https://doi.org/10.1093/poq/nfab039
[i] Exchange rates for October 2021.
[ii] Michał Kotnarowski is an assistant professor at the Institute of Philosophy and Sociology of the Polish Academy of Sciences and a national coordinator of ESS Poland. This paper was completed as part of the UCD Geary Institute for Public Policy 2025/26 – Building International Collaborations Supports (BICS) visiting researcher programme.