Moving to Self-Completion: Challenges and Opportunities for Survey Data Collection in Ireland

Introduction: The Push Toward Self-Completion in Survey Research

Data gathered by large-scale nationally representative surveys continue to be the basis of much of the evidence used to understand our societies. However, the way this data are collected is changing with surveys worldwide transitioning away from traditional face-to-face data collection due to rising costs, a shrinking pool of experienced interviewers, fewer fieldwork agencies, and increasingly lengthy fieldwork periods (Cernat & Revilla, 2021; DeLeeuw, 2018; Wolf et al., 2021). Compounding these operational challenges is a steady decline in response rates across many international survey programmes (Jabkowski & Cichocki, 2025; Koen Beullens, 2018; Luiten et al., 2020).

Figure 1: European Social Survey Response Rates, 2002-2024. Note: ESS Round 1 (R1) took place over 2002/03 with each subsequent round occurring bi-annually to R9 (2018/19). R10 ran over the COVID-19 period 2020-22 and R11 ran over 2023/24.

The European Social Survey (ESS) exemplifies this trend. While historically conducted through face-to-face interviews, ESS has begun exploring self-completion modes. As Figure 1 illustrates, response rates across ESS countries have declined from 60.6% in Round 1 (2002/03) to 39.4% in Round 11 (2023/24); with a particular decline from Round 7 (2014/15) onwards. The scale and persistence of this decline highlights the urgency for change. Unlike most ESS participating countries, Ireland has not followed this steadily declining trend. Its response rates have remained relatively stable across rounds, with only isolated drops in Rounds 4 (2008/09) and 7 (2014/15), and a more pronounced decline in Round 10 to 36.3%, which was largely attributable to the impact of the COVID-19 pandemic. While this pattern makes Ireland’s trajectory quite unique in the ESS context, it still reflects a concerning direction of travel.

The ESS is not merely adapting but taking the lead in moving from face-to-face to self-completion (Fitzgerald, 2025; Xena & Lynn, 2025). In recent studies, mixed-mode designs, especially those integrating paper-based and web-based survey completion options, have shown promise in improving participation while reducing costs, though optimal design strategies remain under discussion (Edwards et al., 2023; Heimel et al., 2024).

Several ESS countries were required to switch to self-completion during the COVID-19 pandemic (Round 10). Drawing on promising results from those countries, the ESS is now gradually transitioning toward a full self-completion approach. While Round 12 (2025/26) will be conducted using a mixed-mode approach (half face-to-face and half self-completion), Round 13 (2027/28) will be implemented fully in self-completion mode.

To test the feasibility of this transition, the ESS undertook a self-completion pilot study for Round 12 in two countries, Bulgaria and Ireland, during 2024. That study marks the first large-scale self-completion implementation in Ireland. In this paper, we discuss some findings from the Irish pilot that offer key insights into the challenges and opportunities of switching to a self-completion mode.

ESS Ireland and the Round 12 Self-Completion Pilot

Ireland has participated in every ESS round. For Rounds 9 to 12, ESS Ireland has been hosted by the UCD Geary Institute for Public Policy with support from the Irish Research Council/Research Ireland. The Round 12 self-completion pilot study was run by the ESS Headquarters (City University, London) with fieldwork completed by IPSOS/B&A. The fieldwork period was just eight weeks, from June 10 to August 2, 2024, which is brief compared to the fieldwork period for ESS Round 11 (June 27, 2023 to January 3, 2024).

For the pilot, a sample of addresses was drawn across Ireland from the GeoDirectory. Out of 1,500 addresses, 1,040 (69%) were contacted with a postal-first approach. At the same time, for 460 non-unique addresses (31%), where An Post could not deliver letters reliably due to issues with Eircodes, a fieldworker-first approach was employed. In these cases, fieldworkers were sent to visit the selected addresses, attempt contact and leave survey materials without conducting an interview.

For the postal-first component, the initial mailout included a one-page invitation letter with a web link, a 3-page leaflet explaining the ESS, a 36-page paper questionnaire, a return envelope, and a €5 unconditional incentive. For the fieldworker-first component, fieldworkers left the same materials but without any unconditional incentive. A €10 voucher was offered for everyone who completed the survey.

Figure 2:  European Social Survey Recent Self-Completion Response Rates

The ESS 12 pilot study achieved an overall response rate of 43.2% (n = 638), with 73% of these respondents completing and returning the paper questionnaire, while 27% completed the survey online. The ESS 12 Ireland pilot achieved the highest response rate among all ESS self-completion implementations to date (see Figure 2). The Bulgarian ESS 12 pilot achieved a 33.4% response rate, while most other attempts achieved response rates of between 30-40%.

While promising, the ESS 12 pilot experience also highlighted several challenges: difficulties with initiating contact, variability in response, and uncertainties regarding costs, missing data, and subjective question quality (Li & Collins, 2025). At the same time, key opportunities emerged, including a substantially shorter fieldwork period and improved sample composition. We examine these findings in the next section.

Sample Composition by Demographics

The sample composition from the Irish self-completion pilot provides valuable insights into who responds to different modes of data collection and how representative the self-completion sample is when compared to earlier ESS rounds and the Irish population. Table 1 (below) presents the distribution of respondents across key demographic variables: gender, age, education, and domicile, and compares figures from the ESS 12 pilot with Census 2022 and ESS Round 11 data. The ESS 12 pilot data are also decomposed by mode.

In terms of gender, women are overrepresented in both the pilot study and in Round 11, continuing a long-standing trend in survey research. However, the pilot showed a slightly higher response rate for women compared to the previous round. Despite efforts to balance representation, men remained less likely to respond, particularly via paper mode.

Age composition reveals more pronounced differences. Younger cohorts, especially those aged 15–29, were underrepresented in both the face-to-face and self-completion rounds. However, web mode attracted more young people than paper. At the other end of the spectrum, those aged 65 and over were overrepresented and showed a strong preference for paper.

The educational distribution of the pilot sample skews toward higher education levels. Respondents with a Master’s degree or above are overrepresented in the pilot study, particularly in the web mode. Among respondents who completed the survey online, nearly one-third held postgraduate qualifications. In contrast, individuals with less than lower secondary education were notably underrepresented in the web mode (1.7% of respondents).

The distribution by domicile in the pilot study shows a closer match to the population than that achieved in Round 11. While paper mode attracted more rural respondents, web participation was higher in urban areas. This balance demonstrates that the mixed-mode approach, offering both web and paper, helps mitigate coverage issues that arise when relying solely on one mode.

Table 1: Comparison of ESS 12 Pilot Sample Composition Notes: Census 2022 occurred on April 3, 2022; ESS Round 11 fieldwork took place between June 2023 and January 2024; the ESS 12 pilot fieldwork took place between June 10 to August 2, 2024. Both ESS 11 and ESS 12p results are based on the unweighted sample data. For ESS 11 n=2,017, for ESS 12p n=638.

Overall, the experience from the ESS 12 pilot study suggests that self-completion can produce a reasonably representative sample with clear strengths in some areas (better representation for education and domicile using paper and web) and challenges in others (older and lower-educated underrepresentation in web).

Who Chooses What? Mode Preferences by Demographics

Understanding who prefers web versus paper is key to designing effective self-completion strategies (Heimel et al., 2024; Olson et al., 2012). The ESS Ireland pilot study offers insights into mode preferences across demographic lines, and we use four visual breakdowns by gender, age, education, and domicile to explore this further.

Figure 3: ESS 12 Pilot Mode Preferences, by demographic characteristics. Note: Overall, 73% of respondents completed the pilot survey by paper and 27% by web.

Gender did not significantly influence mode preference. Both men and women overwhelmingly chose the paper version, with over 70% of respondents in each group opting for it. This suggests that the preference for paper is not linked to gender, but likely to other factors.

Age, however, played a critical role in mode selection. The youngest age group (15–29) had the highest relative preference for web surveys, with 44.6% completing the questionnaire online. Conversely, among those aged 65 and over, only 8.4% used the web, while 91.6% completed the paper version. This pattern highlights the importance of maintaining paper options to engage older populations effectively.

The data suggest that education level is strongly associated with mode choice. Only 5.1% of respondents with less than lower secondary education used the web mode, compared to 41% of those with a Master’s degree or higher. This steep gradient demonstrates a digital divide in survey participation. The web mode clearly appeals more to those with higher educational attainment, whereas less-educated respondents rely more heavily on paper.

Finally, domicile also influenced mode selection. Urban residents were slightly more inclined to complete the survey online compared to those from rural areas. This likely reflects differences in internet access and familiarity with digital tools across geographic contexts.

These findings reinforce the importance of a mixed-mode approach. While the web mode brings clear benefits, particularly in cost and turnaround, it is insufficient on its own to ensure a balanced, inclusive and representative sample. Retaining a paper option remains essential for reaching certain demographic groups and avoiding systematic biases.

Discussion and Conclusion: Navigating the Transition

The Irish pilot for ESS Round 12 demonstrates that self-completion can yield encouraging response rates, although these are about one-fifth lower than in recent face-to-face rounds. Although challenges persist, including contact difficulties and item nonresponse, the opportunities are substantial.

A key takeaway is that a paper completion option remains essential for engaging older, rural, and less-educated respondents. Conversely, the web mode appeals more to younger, urban, and highly educated individuals. Together, a mixed-mode self-completion approach enhances sample representativeness and provides greater flexibility.

Self-completion also brings some operational advantages: shorter fieldwork periods, more efficient logistics, and some data quality improvements. That said, uncertainties remain around cost-effectiveness, subjective item validity, and the long-term sustainability of this new approach.

Our findings suggest that future ESS rounds will need to strike a balance: supporting diverse access while gradually encouraging digital completion. The Irish case shows that while the transition is complex, it is both necessary and promising.

References

Cernat, A., & Revilla, M. (2021). Moving from Face-to-Face to a Web Panel: Impacts on Measurement Quality. Journal of Survey Statistics and Methodology, 9(4), 745–763. https://doi.org/10.1093/jssam/smaa007

DeLeeuw, E. D. (2018). Mixed-Mode: Past, Present, and Future. Survey Research Methods, Vol 12, 75-89 Pages. https://doi.org/10.18148/SRM/2018.V12I2.7402

Edwards, P. J., Roberts, I., Clarke, M. J., DiGuiseppi, C., Woolf, B., & Perkins, C. (2023). Methods to increase response to postal and electronic questionnaires. Cochrane Database of Systematic Reviews, 2023(11). https://doi.org/10.1002/14651858.MR000008.pub5

Fitzgerald, R. (2025, April 4). The move to self-completion interviewing on Europe’s cross-national general social survey – balancing innovation and continuity. International Methodological Conference “The Future of Survey Research,” Warsaw, Poland.

Heimel, S. K., Medway, R. L., & Horwitz, R. T. (2024). Should We Offer Web, Paper, or Both? A Comparison of Single- and Mixed-Response Mode Treatments in a Mail Survey. Journal of Survey Statistics and Methodology, 12(3), 593–623. https://doi.org/10.1093/jssam/smae021

Jabkowski, P., & Cichocki, P. (2025). Survey response rates in European comparative surveys: A 20-year decline irrespective of sampling frames or survey modes. Quality & Quantity, 59(S1), 635–655. https://doi.org/10.1007/s11135-024-01993-9

Koen Beullens, K. L. (2018). Response Rates in the European Social Survey: Increasing, Decreasing, or a Matter of Fieldwork Efforts? https://doi.org/10.13094/SMIF-2018-00003

Li, O., & Collins, M. (2025, April 8). Implementing the ESS Self-Completion Mode in Ireland: Results from the Pilot Study. 2025 International Workshop on Comparative Survey Design and Implementation, Skopje, North Macedonia.

Luiten, A., Hox, J., & De Leeuw, E. (2020). Survey Nonresponse Trends and Fieldwork Effort in the 21st Century: Results of an International Study across Countries and Surveys. Journal of Official Statistics, 36(3), 469–487. https://doi.org/10.2478/jos-2020-0025

Olson, K., Smyth, J. D., & Wood, H. M. (2012). Does Giving People Their Preferred Survey Mode Actually Increase Survey Participation Rates? An Experimental Examination. Public Opinion Quarterly, 76(4), 611–635. https://doi.org/10.1093/poq/nfs024

Wolf, C., Christmann, P., Gummer, T., Schnaudt, C., & Verhoeven, S. (2021). Conducting General Social Surveys as Self-Administered Mixed-Mode Surveys. Public Opinion Quarterly, 85(2), 623–648. https://doi.org/10.1093/poq/nfab039

Xena, C., & Lynn, P. (2025). A Comparison of Non-Response Patterns over Time in Countries that Switched to Self-Completion at Round 10. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.5118891