Post n°272. In order to assess the quality of its research output, the Banque de France has constructed its own ranking of academic journals in economics and finance. The objective is to encourage the production of research on issues relevant to the Banque de France. This post explains the ten principles underlying the construction of this ranking.
The emphasis on research by central banks is growing steadily. According to the IDEAS-REPEC website, the number of researchers in euro area central banks increased threefold between 2012 and 2022 (Chart 1). Furthermore, more and more academic research papers are referred to in central bankers' speeches: this holds true both for the number of speeches and for the average number of references per speech (Chart 2).
Central banks rely on research for decision-making, for informing the economic public debate, and for contributing to academic production. These three objectives shape the metric for assessing the quality of research constructed by the Banque de France. Its aim is to provide a tool for assessing the quality of publications while preserving the academic freedom of researchers.
This journal ranking is designed to encourage the production of high-quality research while preserving the diversity and equality between different types of research (theory, empirics, history, experimental) and different research topics. While high-quality research contributes to high-quality decision-making, the need for diversity aims to respond to the situations described by Jean-Claude Trichet in 2010: "in the face of the crisis, we felt abandoned by conventional tools. ...[W]e were helped by one of the areas of economic literature: historical analysis".
In order to meet these objectives, the Banque de France has defined ten principles for constructing its ranking of academic journals. These principles were defined by a working group made up of representatives of the Bank's research-oriented directorates, whose members are the authors of this post, in consultation with researchers in order to take into account their reactions and suggestions.
Combining research quality and diversity
A ranking embodies a scientific policy that must take a long-term perspective. Principle 1 establishes the need to rank journals in accordance with the recommendations made in the “External evaluation of research at the Banque de France” report produced by Patrick Bolton, Stephen Cecchetti and Lucrezia Reichlin in 2016.
Research should aim for excellence, and should be checked and validated by anonymous external reviewers who guarantee the quality of the scientific work. Principle 2 therefore requires that the Banque de France ranks only peer-reviewed journals referenced on the REPEC website.
Each journal is characterised by a governance system monitoring the work of both the editors and the anonymous reviewers (referees). This governance and the quality of the referees’ work, as well as the reputational effects linked to the age of the journal, often account for the quality of the best journals. Principle 3 therefore concerns the determination of the academic journals with the highest ranking in each field with the restriction that, by definition of what is a top field journal, this group includes at most two best journal(s) per research field.
Chart 3 shows the diversity of the research fields by the Banque de France.
Principle 4 defines the types of academic journals ranked, with the aim of representing the diversity of expertise needs. In addition to the "general interest" journals, eleven fields typical of the usual fields of central banking research are listed. In view of the critical need for interdisciplinary research, a "social sciences" field is included, while the "climate and environment" field reflects the integration of these issues by central banks.
Encouraging researchers to produce high-quality research on a regular basis
In order for each researcher to take it into account and in order to use it for assessing research activity, the ranking system integrates signals that are useful to the institution. Principles 5 to 8 aim at ensuring that 1) short papers published in quality journals with a rapid validation process are taken into account, 2) the ranking and therefore individual evaluations are predictable; 3) co-authorship in research is encouraged and 4) researchers' careers between different institutions are fluid.
Principle 5, in particular, concerns papers published by some academic journals in a short form, i.e. as “Papers & Proceedings” or guest papers, which are downgraded one notch compared to the journal concerned.
Research is a collaborative activity par excellence that requires a wide range of skills. Principle 6 aims at encouraging collective work by weighting co-authorships favourably, while avoiding the juxtaposition of too many authors (less than seven).
Principle 7 defines the effective allocation of a number of points to researchers for a publication in each academic journal within a given field. In order to take into account the diversity of research needs, the fields are not ranked, regardless of the number of references or the popularity of the field in the academic community. The journals in each field are ranked according to the five layers of quality. Each publication in a journal grants to its authors a number of points ranging from 25 for the lowest-ranked journals to 400 for the best.
Researchers' careers rarely take place in a single institution. Research requires exchanges, and mobility should not be penalised. Principle 8 therefore ensures harmonisation with other journal rankings made by other central banks by stating that no journal may be ranked by the Banque de France below the ranks attributed to the same journal by similar institutions.
These different principles make it possible to define for each researcher a minimum expected publication activity, defined over a cycle of several years, which can thus be used in the same way as the number of citations to determine some bonuses or for career promotion.
A ranking of academic journals should be regularly updated. Principles 9 and 10 provide that the ranking can be completed during the year to add a journal not yet ranked. A group of researchers from the Banque de France is conveyed for this purpose.
To conclude
Research is a two-pronged activity, as it is based on both emulation and cooperation. By clarifying the principles for ranking academic journals, our aim was to establish a framework governing the evaluation of each researcher's contributions in order to allow for argued and informed discussion. Since the demand for one type of research or another is partly unpredictable, it was necessary to preserve the incentives to publish in all fields that are potentially useful for the Banque de France to fulfill its missions. As shown in Chart 4, this led to equalising the weighting of each field compared to the REPEC ranking.
Source: Authors' calculations based on the REPEC ranking "Recursive Discounted Impact Factors (Last 10 Years)" and the Banque de France journal ranking.
Key: a researcher publishing two papers in the first two categories of econometrics journals obtains 14% of the value of these same two papers with the REPEC ranking if they were published in the eight best economics and finance journals, against 60% in the Banque de France ranking.
Note: each bar gives, for each of the fields defined by the Banque de France, the score obtained by a researcher with two research articles published in the first two categories of journals in his or her field. This metric is calculated relative to the value assigned by the REPEC and Banque de France rankings to the top eight economics and finance journals (i.e. the top 5 in economics and the top 3 in finance).