Field rules and bias in random surveys with quota samples. An assessment of CIS surveys

José Manuel Pavía Miralles; Cristina Aybar Arias

Ayuda

Field rules and bias in random surveys with quota samples. An assessment of CIS surveys

Jose M. Pavía ^[1] ; Cristina Aybar ^[1]
1. [1] Universitat de València
  
  Universitat de València
  
  Valencia, España
Localización: Sort: Statistics and Operations Research Transactions, ISSN 1696-2281, Vol. 42, Nº. 2, 2018, págs. 183-206
Idioma: inglés
Enlaces
- Texto completo (pdf)

Dialnet Métricas: 2 Citas

Resumen
- Surveys applying quota sampling in their final step are widely used in opinion and market re- search all over the world. This is also the case in Spain, where the surveys carried out by CIS (a public institution for sociological research supported by the government) have become a point of reference. The rules used by CIS to select individuals within quotas, however, could be improved as they lead to biases in age distributions. Analysing more than 545,000 responses collected in the 220 monthly barometers conducted between 1997 and 2016 by CIS, we compare the empiri- cal distributions of the barometers with the expec ted distributions from the sample design and/or target populations. Among other results, we find, as a consequence of the rules used, significant overrepresentations in the observed propor tions of respondents with ages equal to the minimum and maximum of each quota (age and gender group). Furthermore, in line with previous litera- ture, we also note a significant overrepresentation of ages ending in zero. After offering simple solutions to avoid all these biases, we discuss some of their consequences for modelling and inference and about limitations and potentialities of CIS data.
Referencias bibliográficas
- Bachi, R. (1951). The tendency to round off age returns: measurement and correction. Bulletin of the International Statistical Institute,...
- Badal-Valero, E., Alvarez-JarenÌo, J. A. and PavÄ±Ìa, J. M. (2018). Combining Benford’s law and machine learning to detect money laundering....
- Brian, A., Baten, J. and Crayen, D. (2009). Quantifying quantitative literacy: Age heaping and the history of human capital. The Journal of...
- Burnap, P., Gibson, R., Sloan, L., Southern, R. and Williams, M. (2016). 140 characters to victory?: Using twitter to predict the UK 2015...
- Carrier, N. (1959). A note on the measurement of digital preference in age recordings. Journal of the Institute of Actuaries, 85, 71–85.
- Cea D’Ancona, M. A. (2004). MeÌtodos de encuesta: teorÄ±Ìa y praÌctica, errores y mejora. SÄ±Ìntesis, Madrid.
- Cleveland, W. (1993). Visualizing Data. New Jersey: Hobart Press.
- Del Vicario, M., Scala, A., Caldarelli, G., Stanley, H. and Quattrociocchi, W. (2017). Modeling confirmation bias and polarization. Scientific...
- DÄ±Ìaz de Rada, V. (2005). Manual de trabajo de campo en la encuesta. Cuadernos MetodoloÌgicos del CIS, 36. Centro de Investigaciones SocioloÌgicas,...
- DÄ±Ìaz de Rada, V. (2008). La seleccioÌn de los entrevistados uÌltimos en encuestas presenciales: un anaÌlisis de la utilizacioÌn conjunta...
- DÄ±Ìaz de Rada, V. (2014). Analysis of incidents in face-to-face surveys: Improvements in fieldwork. Revista EspanÌola de Investigaciones...
- DÄ±Ìaz de Rada, V. (2015). Manual de Trabajo de Campo en la Encuesta. Cuadernos MetodoloÌgicos del CIS, 36. Centro de Investigaciones SocioloÌgicas,...
- Ebrahimi, M., Yazdavar, H. and Sheth, A. (2017). Challenges of sentiment analysis for dynamic events. IEEE Intelligent Systems, 32, 70–75....
- Groves, R., Fowler, F., Couper, M., Lepkowski, J., Singer, E. and Tourangeau, R. (2009). Survey Methodology. John Wiley & Sons, New Jersey.
- Hope, A. (1968). A simplified Monte Carlo significance test procedure. Journal of the Royal Statistical Society, Series B, 30, 582–598.
- Jungherr, A., Schoen, H. Posegga, O. and JuÌrgens, P. (2017). Digital trace data in the study of public opinion: An indicator of attention...
- Kalampokis, E., Karamanou, A., Tambouris, E. and Tarabanis, K. (2017). On predicting election results using twitter and linked open data:...
- Kennedy, C., Mercer, A., Keeter, S., Hatley, N., McGeeney, K. and Gimenez, A. (2016). Evaluating Online Nonprobability Surveys. Pew Research...
- Kish, L. (1995). Survey Sampling. John Wiley and Sons, Inc, New York.
- Lind, K., Link, M. and Oldendick, R. (2000). A comparison of the accuracy of the last birthday versus the next birthday methods for random...
- LledoÌ, J., PavÄ±Ìa, J. M. and Morillas, F. G. (2017). Assessing implicit hypothesis in life table construction. Scandinavian Acturial Journal,...
- Lyons-Amos, M. and Stones, T. (2017). Trends in demographic and health survey data quality: An analysis of age heaping over time in 34 countries...
- Marsh, C. and Scarbrough, E. (1990). Testing nine hypotheses about quota sampling. Journal of the Market Research Society, 32, 485–506.
- Mellon, J. and Prosser, C. (2017). Twitter and facebook are not representative of the general population: Political attitudes and demographics...
- Meng, X.-L. (2016). Statistical paradises and paradoxes in big data. In Royal Statistical Society Annual Conference 2016. https://www.youtube.com/watch?v=8YLdIDOMEZs.
- Mosteller, F., Hyman, H., McCarthy, P., Marks, E. and Truman, D. (1949). The Pre-Election Polls of 1948. Social Science Research Council,...
- Myers, R. (1940). Errors and bias in the reporting of ages in census data. Transactions of the Actuarial Society of America, 41-2, 395–415.
- O’Rourke, D. and Blair, J. (1983). Improving random respondent selection in telephone surveys. Journal of Marketing Research, 20, 428–432....
- PavÄ±Ìa, J. M., Badal, E. and GarcÄ±Ìa-CaÌrceles, B. (2016). Spanish exit polls: Sampling error or nonresponse bias? Revista Internacional...
- PavÄ±Ìa, J. M. and GarcÄ±Ìa-CaÌrceles, B. (2012). Una aproximacioÌn empÄ±Ìrica al error de disenÌo muestral en las encuestas electorales...
- PavÄ±Ìa, J. M. and Larraz, B. (2012). Nonresponse bias and superpopulation models in electoral polls. Revista EspanÌola de Investigaciones...
- R Core Team (2016). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. htpps://www.R-project.org/.
- RodrÄ±Ìguez Osuna, J. (1991). MeÌtodos de Muestreo. Cuadernos MetodoloÌgicos del CIS, 1. Centro de Investigaciones SocioloÌgicas, Madrid.
- RodrÄ±Ìguez Osuna, J. (2005). MeÌtodos de Muestreo. Casos PraÌcticos. Cuadernos MetodoloÌgicos del CIS, 1. Centro de Investigaciones SocioloÌgicas,...
- Smith, T. (1983). On the validity of inference from non-random samples. Journal of the Royal Statistical Society, Series A, 146, 394–403....
- Stephenson, C. B. (1978). A comparison of full-probability and probability-with-quotas sampling techniques in the general social survey. GSS...
- Sudman, S. (1976). Applied Sampling. Academic Press, New York.
- Trochim, W. and Donnelly, J. (2006). The Research Methods Knowledge Base. Boston, Massachusetts: Cengage Learning, Boston, Massachusetts.
- Vavreck, L. and Rivers, D. (2008). The 2006 cooperative congressional election study. Journal of Elections, Public Opinion and Parties, 18,...
- Vehovar, V. (1999). Field substitution and unit nonresponse. Journal of Official Statistics, 15, 335–350.
- Yang, K. and Banamah, A. (2014). Quota sampling as an alternative to probability sampling? an experimental study. Sociological Research Online,...