THIS DOI METADATA WAS LAST UPDATED: 2024-01-10 22:48
Full Title
SOCIOLOGIA E POLITICHE SOCIALI
Publisher
FrancoAngeli
ISSN
1591-2027 (Printed Journal)
1972-5116 (Online Journal)
Journal Issue Number
2
Journal Issue Designation
2
Journal Issue Date (YYYY/MM)
2015/09
Full Title
La valutazione dei pari nelle scienze sociali e politiche. la lezione della VQR 2004-2010
By (author)
First Page
44
Last Page
72
Language of text
Italian
Publication Date
2015/09
Copyright
2015 FrancoAngeli srl
Introduction or preface
A scientific evaluation must follow rigorous, codified and reproducible procedures. The methodological adequacy of the peer review procedure in the latest Italian research evaluation exercise (VQR 2004- 2010) is the key focus of this work. First and foremost the analysis regards the referee selection procedure. Afterward the matching between the referees and the research products to be evaluated is considered. Also the evaluation form available to the referees is examined, together with the synthesis procedure of the scores. The last focus concerns the peer judgments stability and homogeneity. Also concrete proposals for the improvement of the evaluation procedures are presented. The study is limited to the Political and Social Sciences Area, because of its epistemic peculiarities.
Unstructured Citation
Agodi, M.C., 1999. Lazarsfeld e la «natura» della classificazione nelle scienze sociali. Sociologia e ricerca sociale, 20(58/59), pp. 117-149.
Unstructured Citation
Anvur, 2011. Valutazione della qualità della ricerca (VQR 2004-2010). Bando di partecipazione.
Unstructured Citation
Anvur, 2013a. Valutazione della Qualità della Ricerca 2004-2010 – Rapporto Finale. Parte Prima: Statistiche e risultati di compendio. --http://www.Anvur.org/ rapporto/main.php?page=intro 30.07.13. Anvur, 2013b. Valutazione della Qualità della Ricerca 2004-2010 – Rapporti di area. --http://www.Anvur.org/ rapporto/main.php?page=intro 30.07.13.
Unstructured Citation
Anvur, 2014a. Criteri di assegnazione delle classi di merito nel caso di valutazioni peer review con valutazioni non coincidenti da parte dei referee. --http://www.Anvur.org/attachments/article/244/Criteri%20di%20assegnazione%20della%20classe%20di%20merito%20.pdf 09.06.14. Anvur, 2014b. Elenco alfabetico dei revisori Vqr. --http://www.anvur.org/attachments/article/244/Referee%20VQR.pdf 09.06.14.
Unstructured Citation
Bailar, J.C., 1991. Reliability, Fairness, Objectivity and Other Inappropriate Goals in Peer Review. Behavioral and Brain
Sciences, 14(01), pp. 137-138.
https://doi.org/10.1017/S0140525X00065705
Unstructured Citation
Baxt, W.G., Waeckerle, J.F., Berlin, J.A., Callaham, M.L., 1998. Who Reviews the Reviewers? Feasibility of Using a Fictitious
Manuscript to Evaluate Peer Reviewer Performance. Annals of Emergency Medicine, 32(3), pp. 310-317.
https://doi.org/10.1016/S0196-0644(98)70006-X
Unstructured Citation
Benos, D.J., Bashari, E., Chaves, J.M., Gaggar, A., Kapoor, N., LaFrance, M. et al., 2007. The Ups and Downs of Peer Review.
Advances in Physiology Education, 31(2), pp. 145-152.
https://doi.org/10.1152/advan.00104.2006
Unstructured Citation
Bentley, R., Blackburn, R., 1990. Changes in Academic Research Performance Over Time: A Study of Institutional Accumulative
Advantage. Research in Higher Education, 31(4), pp. 327-353.
https://doi.org/10.1007/BF00992271
Unstructured Citation
Bornmann, L., 2008. Scientific Peer Review: An Analysis of the Peer review Process from the Perspective of Sociology of Science Theories. Human Architecture: Journal of the Sociology of Self-Knowledge, 6(2), pp. 23-37.
Unstructured Citation
Bornmann, L., 2011. Scientific Peer Review.Annual Review of Information Science and Technology,45(1), pp. 197-245.
https://doi.org/10.1002/aris.2011.1440450112
Unstructured Citation
Bornmann, L., Mutz, R., Daniel, H.D., 2007. Gender Differences in Grant Peer Review: A Meta-analysis. Journal of Informetrics,
1(3), pp. 226-238.
https://doi.org/10.1016/j.joi.2007.03.001
Unstructured Citation
Cho, M.K., Justice, A.C., Winker, M.A., Berlin, J. A., Waeckerle, J.F., Callaham, M.L., Rennie, D., 1998. Masking Author Identity
in Peer Review. JAMA: The Journal of the American Medical Association,280(3), pp. 243-245.
https://doi.org/10.1001/jama.280.3.243
Unstructured Citation
CIVR, 2006. VTR 2001-2003. Relazione finale. http://vtr2006.cineca.it Crane, D., 1965. Scientists at Major and Minor Universities: A Study of Productivity and Recognition. American Sociological Review, 30(5), pp. 699-714.
Unstructured Citation
Crane, D., 1967. The Gatekeepers of Science. Some Factors Affecting the Selection of Articles in Scientific Journals. American Sociologist, 2(4), pp. 195-201.
Unstructured Citation
Eckberg, D. L., 1991. When Non Reliability of Reviews Indicates Solid Science. Behavioral and Brain Sciences 14(1), pp. 145-146.
https://doi.org/10.1017/S0140525X00065791
Unstructured Citation
Fasanella, A., Di Benedetto, A., 2014. Luci ed ombre nella VQR 2004- 2010: un focus sulla scheda di valutazione peer nell'Area 14, Sociologia e ricerca sociale, 35(104), pp. 59-84.
Unstructured Citation
Gläser, J., Laudel, G., 2006. Advantages and Dangers of ‘Remote’ Peer Evaluation. Research Evaluation 14(3), pp. 186-198. Godlee, F., Gale, C.R., Martyn, C.N., 1998. Effect on the Quality of Peer Review of Blinding Reviewers and Asking them to Sign their Reports.JAMA: Journal of the American Medical Association,280(3), pp. 237-240.
Unstructured Citation
Grant, J., Burden, S., Breen, G., 1997. No Evidence of Sexism in Peer Review. Nature, 390(6659), pp. 438-438.
https://doi.org/10.1038/37213
Unstructured Citation
Hojat, M., Gonnella, J.S., Caelleigh, A.S., 2003. Impartial Judgment by the “Gatekeepers” of Science: Fallibility and Accountability in the Peer Review Process.Advances in Health Sciences Education, 8(1), pp. 75-96. --http://www.Anvur.org/attachments/article/122/bando_vqr_def_07_11.pdf 30.07.13.
Unstructured Citation
Jayasinghe, U.W., Marsh, H.W., Bond, N., 2003. A Multilevel Crossclassified Modelling Approach to Peer Review of Grant Proposals:
the Effects of Assessor and Researcher Attributes on Assessor Ratings.Journal of the Royal Statistical Society: Series A,
166(3), pp. 279-300.
https://doi.org/10.1111/1467-985X.00278
Unstructured Citation
Justice, A.C., Cho, M.K., Winker, M.A., Berlin, J.A., Rennie, D., 1998. Does Masking Author Identity Improve Peer Review Quality?.JAMA:
Journal of the American Medical Association,280(3), pp. 240-242.
https://doi.org/10.1001/jama.280.3.240
Unstructured Citation
Krippendorff, K., 1980. Content Analysis. An Introduction to its Methodology. London, Sage. Tr. it. 1983, Analisi del contenuto. Introduzione metodologica. Torino: Eri.
Unstructured Citation
Kuhn, T.S., 1962. The Structure of Scientific Revolutions, Chicago: University of Chicago Press.
Unstructured Citation
Lane, D. 2008. Double-blind Review: Easy to Guess in Specialist Fields. Nature, 452(7183), p. 28.
https://doi.org/10.1038/452028c
Unstructured Citation
Lazarsfeld, P.F., 1959. Problems in Methodology. In Merton et al. (eds.), 1959, pp. 39-78.
Unstructured Citation
Lee, C.J., Sugimoto, C.R., Zhang, G., Cronin, B., 2013. Bias in Peer Review. Journal of the American Society for Information
Science and Technology, 64(1), pp. 2-17.
https://doi.org/10.1002/asi.22784
Unstructured Citation
Lutynski, J., 1988. Un centro di ricerca sulle tecniche di raccolta dei dati. In Marradi (a c. di), 1988, pp. 117-132.
Unstructured Citation
Mahoney, M.J., 1977. Publication Prejudices: An Experimental Study of Confirmatory Bias in the Peer Review System. Cognitive
therapy and research, 1(2), pp. 161-175.
https://doi.org/10.1007/BF01173636
Unstructured Citation
Marradi A. (a cura di) 1988. Costruire il dato. Milano: FrancoAngeli.
Unstructured Citation
Marradi, A., 1980. Concetti e metodi per la ricerca sociale. Firenze: La Giustina.
Unstructured Citation
Marradi, A., 2007. Metodologia della ricerca sociale.Bologna: Il mulino.
Unstructured Citation
Mauceri, S., 2003. Per la qualità del dato nella ricerca sociale. Strategie di progettazione e conduzione dell’intervista con questionario. Milano: FrancoAngeli.
Unstructured Citation
Merton, R.K., 1968a. The Matthew Effect in Science. Science, 159(3810), pp. 56-63.
https://doi.org/10.1126/science.159.3810.56
Unstructured Citation
Merton, R.K., 1968b. Social Theory and Social Structure. Glencoe, Ill; tr. it. Teoria e struttura sociale Bologna: Il Mulino, 2000.
Unstructured Citation
Merton, R.K., 1973. The sociology of Science: Theoretical and Empirical Investigations. Chicago: University of Chicago Press Merton, R.K., Broom, L., Cottrell, L.S. (eds.), 1959. Sociology Today: Problems and Prospects. New York: Basic books.
Unstructured Citation
Moody, J., 2004. The Structure of a Social Science Collaboration Network: Disciplinary Cohesion from 1963 to 1999.American
Sociological Review, 69(2), pp. 213-238.
https://doi.org/10.1177/000312240406900204
Unstructured Citation
Nobile, S., 1997. La credibilità dell’analisi del contenuto. Milano: FrancoAngeli.
Unstructured Citation
Nobile, S., 2008. La chiusura del cerchio. La costruzione degli indici nella ricerca sociale. Acireale-Roma: Bonanno.
Unstructured Citation
Palumbo, M., 2001. Il processo di valutazione. Decidere, programmare, valutare, Milano: FrancoAngeli.
Unstructured Citation
Pitrone, M.C., 2009. Sondaggi e interviste.Lo studio dell'opinione pubblica nella ricerca sociale. Milano: FrancoAngeli.
Unstructured Citation
Preston, C.C., Colman, A.M. (2000). Optimal Number of Response Categories in Rating Scales: Reliability, Validity, Discriminating
Power, and Respondent Preferences. Acta Psychologica, 104(1), pp. 1-15.
https://doi.org/10.1016/S0001-6918(99)00050-5
Unstructured Citation
Reale, E., Barbara, A., Costantini, A., 2007. Peer Review for the Evaluation of Academic Research: lessons from the Italian
experience. Research Evaluation, 16(3), pp. 216-228.
https://doi.org/10.3152/095820207X227501
Unstructured Citation
Sandström, U., Hällsten, M., 2008. Persistent Nepotism in Peer Review. Scientometrics, 74(2), pp. 175-189.
Unstructured Citation
Schroter, S., Black, N., Evans, S., Carpenter, J., Godlee, F., Smith, R., 2004. Effects of Training on Quality of Peer Review:
Randomised Controlled Trial. British Medical Journal, 328(7441), 673.
https://doi.org/10.1136/bmj.38023.700775.AE
Unstructured Citation
Shatz, D., 2004. Peer Review: a Critical Inquiry. Lanham, Rowman & Littlefield.
Unstructured Citation
van Rooyen, S., Godlee, F., Evans, S., Black, N., Smith, R., 1999. Effect of Open Peer Review on Quality of Reviews and on
Reviewers' Recommendations: a Randomised Trial.BMJ: British Medical Journal,318(7175), pp. 23-27.
https://doi.org/10.1136/bmj.318.7175.23
Unstructured Citation
van Rooyen, S., Godlee, F., Evans, S., Smith, R., Black, N. 1998. Effect of Blinding and Unmasking on the Quality of Peer
Review: a Randomized Trial. JAMA: Journal of the American Medical Association,280(3), pp. 234-237.
https://doi.org/10.1001/jama.280.3.234
Unstructured Citation
Wennerâs, C., Wold, A., 1997. Nepotism and Sexism in Peer Review. Nature, 387(6631), pp. 341-343.
Unstructured Citation
Zuckerman, H., Merton, R.K., 1973. Patterns of Evaluation of Science: Institutionalization, Structure and Functions of the referee system. In: Merton, 1973, pp. 569-609.