OHIO and Kent State researchers propose a set of new evaluation indices for scientific publication quality

Professor Jixin Chen at Ohio University and Professor Hanbin Mao at Kent State University have analyzed the publication and citation record of 179,597 scientists listed in the online database of World Top 2% Scholar. They have proposed a set of indices to evaluate the average quality of the publications of each scholar.

The h-index has been one of the most popular parameters to reflect the productivity and quality of the publications of a scholar. However, it is difficult to distinguish the publication quality of scholars at different stages and with different amounts of resources and reputations, etc. For individual publications, their quality is usually estimated based on the number of citations each has received at the current date. This is a value that will grow over time, and it will be difficult to judge its future values. Thus, there is a need to come up with some alternative parameters for publication-quality measurement.

The idea is to use the readily available data in the h-index, number of publications, and number of total citations to construct a parameter for each scholar that has a normal distribution property. The value is then divided by the average value of scholars with the same number of publications such that the average value becomes the standard of comparison among peers in the subgroup. This method placed the scholar in a similar stage of their career together and compared the average quality of their publications roughly correlated with the h-index and average number of citations of their publications. As such, the extensive quantity such as the h-index and the number of citations is converted to an intensive quantity that is adjusted to the average value of the top 2% of scientists, i.e., now we can roughly estimate the current reputation of a scholar has with respect to his/her peers and predict the future values. An example difference between extensive quantity and intensive quantity is the weight and density of materials. For example, 100 grams of water is heavier than 10 grams of gold, but gold has a much higher density than water. The extensive quantity such as weight will become larger as adding more materials but the intensive quantity such as density does not change.

"It is very exciting to observe that the average quality of the publications of a scholar is likely to be a conserved value over his/her career. Thus, it is highly indicative of the scholarship impact at an early stage which is what is needed for many purposes of such an index, for example, to predict future success during the hiring process. It is also interesting to find that the highest average quality belongs to those scholars with about 100 publications among the top 2% list, i.e. there is a trade between quantity and quality in the larger than ~100 number of publications. The theory is that extra effort is need to produce high quality paper from each contributors of the paper and there is an time budget limit for each author, especially the corresponding author (Figure 1)."

Figure 1. A scheme of publication stage of each individual scholar.

This paper has been accepted for publication in the Journal of Scientometric Research. A preprint can be found at ChemRxiv.

Chen is an Associate Professor in the Department of Chemistry and Biochemistry in the College of Arts and Science, and a member of Nanoscale and Quantum Phenomena Institute (NQPI) at Ohio University.

Mao is a professor in the Department of Chemistry and Biochemistry and a member of the Advanced Materials and Liquid Crystal Institute at Kent State University.