Use and Misuse of Bibliometric Measures for Assessment of Academic Performance, Tenure and Publication Support
•
3 likes•460 views
Bibliometric methods such as journal impact factor and article influence score based on the number of citations were developed to measure and compare the quality of journals listed in citation indexes. Yet, they are increasingly being used nowadays for research assessment, hiring, tenure and academic promotion, research funding and publication support even though such metrics have not been developed to measure the quality of individual researchers or scientific articles. In this paper, we review the use of journal impact factor, cited half-life, article influence score and h index for academic performance assessment, academic promotion and publication support by Turkish universities and the Turkish Scientific and Technological Research Center (TUBITAK). Examples are provided regarding the consequences of using bibliometric measures beyond what they were originally designed for, and some recommendations are offered.
Recommended
Recommended
More Related Content
What's hot
What's hot (19)
Similar to Use and Misuse of Bibliometric Measures for Assessment of Academic Performance, Tenure and Publication Support
Similar to Use and Misuse of Bibliometric Measures for Assessment of Academic Performance, Tenure and Publication Support (20)
Recently uploaded
Recently uploaded (20)
Use and Misuse of Bibliometric Measures for Assessment of Academic Performance, Tenure and Publication Support
- 1. Use and Misuse of BibliometricMeasures for Assessment of Academic Performance, Tenure and Publication Support Yaşar Tonta Hacettepe University Department of Information Management 06800 Beytepe, Ankara, Turkey yunus.hacettepe.edu.tr/~tonta/tonta.html yasartonta@gmail.com @yasartonta ASIST2014 SIG-MET Workshop on Informetric and Scientometric Research, Nov. 5, 2014, Seattle, WA
- 2. "Not everything that counts can be counted, and not everything that can be counted counts." --Albert Einstein
- 3. Outline •Citation indexes and research assessment •Journal impact factors and publication support •H index and research assessment •Conclusions
- 4. Turkey •Higher Education Council (HEC) •184 universities •121,995 faculty (55,232 professors) •5,5M HE students •HEC’s minimum tenure requirements http://www.yok.gov.tr/web/guest/ogretim-elemanlari-dagilimi
- 6. Citation indexes •Journal impact factor (JIF) •Skewed distributions •JIFs vary by domain and can be manipulated •Transparent data •Changing policies of publishers
- 7. Number of journals in JCR (2000-2012) http://yunus.hacettepe.edu.tr/~umutal/publications/war.pdf
- 8. JIFs and publication support •Turkish Scientific and Technological Research Council (TUBITAK) •TUBITAK’s use of JIFs –Till 2013: Journals categorized by their JIFs as A, B, C, D –2013: TUBITAK’s own JIF: 5-year IF * cited half-life, max/min support for ±2 SDsof average, support for the ones in between transformed using a linear formula –2014: Article influence score
- 9. TUBITAK’s support to Archaeology journals (2012-2014)
- 10. TUBITAK’s support to Archaeology journals (2012-2014) 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 AmericanAntiquity CambridgeArchaeol. J. J. ofArchaeol. Science J. of FieldArchaeology Antiquity Adalya Oxford J. ofArchaeology American J. ofArchaeology WorldArchaeology J. of NearEasternStudies IranicaAntique Olba Belleten 2012 support (in TL) 2013 support (in TL) 2014 support (in TL)
- 11. Issues with indicators •Use of cited half-life. . . –Skewed –Has nothing much to do with the paper quality –Obsolescence •Correlation between 2013 and 2014 lists r= .58 •56% of 286 Geology journals ranked lower •49% misranked •Correlation between JIFs and AISs r=.90 –(N=5,900 journals, JCR 2007 edition) •Higher JIFs => Higher AISs •TUBITAK support program should be reviewed
- 12. H index and research assessment
- 13. Issues with h index •Co-authors are not taken into account (Hirsch, 2007) •Citations below/above one’s h index do not count •Vary by disciplines •Not suitable for short-term research assessment •Vary by publishers’ policies
- 14. h = 10 http://www.theguardian.com/science/2013/dec/06/peter-higgs-boson-academic-system
- 15. Conclusions •JIFs: "poor man’s citation analysis" (Marx & Bornmann, 2013) •"fatal attraction" (Van Raan, 2005) •IEEE: "...bibliometricperformance indicators should be applied only as a collective group (and not individually), and in conjunction with peer review following a clearly stated code of conduct"(original emphasis) (IEEE, 2013)
- 16. Goodhart’s Law (1975) "When a measure becomes a target, it ceases to be a good measure." http://en.wikipedia.org/wiki/Goodhart%27s_law
- 18. Use and Misuse of BibliometricMeasures for Assessment of Academic Performance, Tenure and Publication Support Yaşar Tonta Hacettepe University Department of Information Management 06800 Beytepe, Ankara, Turkey yunus.hacettepe.edu.tr/~tonta/tonta.html yasartonta@gmail.com @yasartonta ASIST2014 SIG-MET Workshop on Informetric and Scientometric Research, Nov. 5, 2014, Seattle, WA Full text of the paper: http://bit.ly/1ur3cGN