Journal Ranking By Google?

By Ian

A new method for ranking science journals is being proposed. What's it based on? Google, of course.

The most popular index of a journal's status is the ISI Impact Factor (IF), produced by Thomson Scientific. It counts the total number of citations a journal's papers receive, and divides it by the number of papers the journal publishes. But the rise of online journals, coupled with sophisticated search engines that permit rankings of web resources, is triggering a wave of other measures. Last year, for example, physicist Jorge Hirsch of the University of California, San Diego, proposed a metric called the h-index for assessing the quality of researchers' publications (see Nature 436, 900; 2005).

Now Johan Bollen and his colleagues at the Research Library of Los Alamos National Laboratory in New Mexico are focusing on Google's PageRank (PR) algorithm. The algorithm provides a kind of peer assessment of the value of a web page, by counting not just the number of pages linking to it, but also the number of pages pointing to those links, and so on. So a link from a popular page is given a higher weighting than one from an unpopular page.

Only hearing second-hand information about the way tenure review goes for academics, I was under the impression that the relative "importance" of the journals in which one might publish also has a big impact on tenure decisions. Ranking systems then seem to be deeply involved in the way research occurs if it impacts who gets financial support either throught grants or university support.

My question is whether the "PageRank" metric might be highly vulnerable to information cascades and manipulation in much the same way as Google's process for ranking sites.

-- February 28, 2006 9:48 AM |

Comments

Mike Giberson wrote:

It is much easier, faster, and cheaper to produce a webpage than to get a journal article published, so manipulating your scientific journal's "PageRank" metric would take longer. In addition, the relative weight of the existing universe of documents is probably much much higher in the science journal universe. It is one thing to manipulate Google rankings for "energy policy blogs" for which the "literature" is only a few years deep, and another thing to try to beef up the scholarly ranking of a new academic journal in the face of a century of American Economic Reviews.

Nonetheless, if there is value in the rankings, editors and publishers can be expected to try to improve their quality in ways that are measured in the rankings. Some will also try to improve their ranking in ways that have nothing to do with quality -- i.e. manipulate -- and certainly anyone proposing a ranking system should put some thought into the issue. In any case, anyone proposing to rely upon a ranking system to allocate rewards (tenure! bonuses! esteem!), should consider the reliability of the system.

-- February 28, 2006 11:44 AM ∞

Mike Giberson wrote:

Did you notice that Nature scored 6 in the ISI Impact Factor list, but topped both the "PageRank" and "Y Value Factor" list?

Did you also notice that the linked article was produced by Nature.

Not that there is anything wrong with that....

-- February 28, 2006 11:49 AM ∞

Ian wrote:

I saw that too. Which is actually what made me wonder about informational issues. If, under IF, Nature doesn't fare as well, I wondered what the other methods are using to move up the score? That is, who is it that cites Nature that itself is well cited that isn't captured under the old IF system?

And, I hope I didn't imply that web pages are akin to journal articles in terms of effort to create. I was thinking more that a lot of pages now ranked include PDFs (of articles, often) as well as things like web-friendly powerpoint, pictures, etc. Cites like Del.icio.ous make a cascade more possible since one person watching the RSS feeds would be interested in something a few others are linking to, then include that URL in their Del bookmarks...and so on. In my feedreader I get a few Del tags, and everyday there are long strings of the same site being bookmarked.

With a more Google-like system, would it be more likely that people will cite pieces in journals ranked highly based on a notion of quality since "everyone else seems to be citing them as well"?

-- February 28, 2006 1:41 PM ∞

Pakistan Journal of Statistics wrote:

Can we check the impact factor and ranking of our Journal i.e. Pakistan Journal of Statistics?

-- March 17, 2006 10:24 AM ∞

Pakistan Journal of Statistics wrote:

Please send us the impact factor and/or ranking of the "Pakistan Journal of Statistics".
Thanks

-- April 7, 2006 5:13 AM ∞

M. Ahmad wrote:

The self impact facor of the Pakistan Journal of Statistics has been computed as 1.2

-- April 16, 2006 4:39 AM ∞

Dr. Munir Ahmad wrote:

Can we check the impact factor of "Pakistan Journal of Statistics"

-- April 17, 2006 11:57 AM ∞

DR. MUIR COHEN wrote:

Can some help me to find the Impact Factor of Pakistan Journal of Statistics?

-- September 30, 2009 10:11 AM ∞

Post a comment