Publication


Tom Emery, Eugenio Paglino
Evaluating interviewer manipulation in the new round of the Generations and Gender Survey
Demographic Research, 2020
URL, JabRef BibTex, Abstract
Background: Past research has criticized the quality of the Generations and Gender Survey retrospective fertility and partnership histories. For example, fatigue and learning effects were deemed responsible for distortions in the Generations and Gender Survey in Germany. Objective: We assess the quality of the Generations and Gender Survey for Belarus (GGS-BL) in 2017 to assess whether the new centralized fieldwork system and monitoring procedures are effective in preventing distortions in life history data. Methods: We conduct a range of analyses to find evidence of fatigue and learning effects on the part of both interviewers and respondents. Multilevel models, comparison of crucial indicators with other sources, and descriptive analysis of item-nonresponse are used. Results: In a preliminary analysis, we find no evidence of severe distortions. An in-depth analysis into interviewer and respondent effects reveals some small signs of possible manipulation. However, when assessing the impact of anomalous interviewers on the indicators more likely to be affected, we find no evidence of harm to data quality. Conclusions: The new data collection procedure adopted by the Generations and Gender Survey seems to be effective in preventing detectable manipulation and fabrication. Furthermore, we dismiss the hypothesis that fatigue and learning effects are a source of bias in the collection of life history data. Contribution: This paper delivers three key messages: (1) the Generations and Gender Survey for Belarus is a reliable source for retrospective histories, (2) in-field checks are an effective tool to prevent fabrication, and (3) extensive use of inexperienced interviewers does not seem to harm data quality when adequate monitoring and monitoring is in place.

Reference


@article{Emery2020a,
  author = {Tom Emery, Eugenio Paglino},
  title = {Evaluating interviewer manipulation in the new round of the Generations and Gender Survey},
  year = {2020},
  journal = {Demographic Research},
  volume = {43},
  number = {50},
  pages = {1461-1494},
  month = {Dec},
  url = {https://www.demographic-research.org/volumes/vol43/50/},
  timestamp = {16.12.2020},
  abstract = {

Background: Past research has criticized the quality of the Generations and Gender Survey retrospective fertility and partnership histories. For example, fatigue and learning effects were deemed responsible for distortions in the Generations and Gender Survey in Germany.

Objective: We assess the quality of the Generations and Gender Survey for Belarus (GGS-BL) in 2017 to assess whether the new centralized fieldwork system and monitoring procedures are effective in preventing distortions in life history data.

Methods: We conduct a range of analyses to find evidence of fatigue and learning effects on the part of both interviewers and respondents. Multilevel models, comparison of crucial indicators with other sources, and descriptive analysis of item-nonresponse are used.

Results: In a preliminary analysis, we find no evidence of severe distortions. An in-depth analysis into interviewer and respondent effects reveals some small signs of possible manipulation. However, when assessing the impact of anomalous interviewers on the indicators more likely to be affected, we find no evidence of harm to data quality.

Conclusions: The new data collection procedure adopted by the Generations and Gender Survey seems to be effective in preventing detectable manipulation and fabrication. Furthermore, we dismiss the hypothesis that fatigue and learning effects are a source of bias in the collection of life history data.

Contribution: This paper delivers three key messages: (1) the Generations and Gender Survey for Belarus is a reliable source for retrospective histories, (2) in-field checks are an effective tool to prevent fabrication, and (3) extensive use of inexperienced interviewers does not seem to harm data quality when adequate monitoring and monitoring is in place.}
}
Start your research with GGP Data today

Subscribe to our Mailing List!

Fill the form below with your contact information to receive our monthly GGP at a glance newsletter.