CROSS-CUTTING

Scientometric analysis

Quantitative study of science as a system: production, collaboration, citations, impact, field dynamics. Differs from bibliometrics by broader scope (policies, national indicators). Methods: network analysis, text mining, temporal analysis.

Extended definition

Scientometrics (from Russian naukometriya) is the quantitative study of science as a social and cognitive system, covering scientific production, collaborations, citation flows, field dynamics, economic and social impact. Operational distinction: bibliometrics is the application of statistical methods to publications and citations; scientometrics has broader scope, including analysis of science policy, funding, researcher training, national indicators, and international comparisons. Nalimov and Mulchenko (1971, Naukometriya) formally introduced the term; Garfield (creator of SCI) and Price (Little Science, Big Science, 1963) built the empirical foundation of the field. Contemporaneously: Zeng et al. (2017, Physics Reports) synthesized “science of science” as physics of complex systems — models of production growth, collaboration networks, citation patterns. Central methods: network analysis (coauthorship, cocitation, bibliographic coupling), text mining of abstracts/titles, temporal analysis of emerging fields, composite indicators (impact indices, scientific maps). Standard databases: Web of Science, Scopus, Dimensions, Lens.org, OpenAlex (more recent, open-access). Applications in science policy: institutional evaluation (CAPES Qualis, UK REF), international benchmarking (university rankings), identification of research frontiers, monitoring strategic areas (AI, climate change, COVID-19).

When it applies

Scientometric analysis applies in macro studies of scientific fields (mapping emerging areas, identifying communities), in institutional evaluation (department production, compared impact), in formulating science policies (investment prioritization, graduate programs), in historical studies of science (paradigm evolution, classical vs. ephemeral citation). It applies in sociological research of science: national/international collaboration networks, researcher mobility, demographic diversity in fields. It applies in corporate strategic analysis: emerging technologies via patent-publication cross-analysis. It applies in consulting for funding agencies (CNPq, FAPESP, NSF, ERC) for portfolio mapping.

When it does not apply

It does not apply as individual researcher evaluation in isolation: aggregate indicators (h-index, FWCI, JCR) have documented limitations; the DORA Declaration warns against reductionist use. It does not apply as a single measure of quality: classical literature on Bradford, Lotka, and Price’s Laws shows highly skewed distributions that invalidate naive comparisons. It does not apply in fields with very specific editorial culture without adjustment (humanities with books as dominant vehicle, areas with single-authorship tradition). It does not apply in very short temporal windows: citations mature 3-5 years post-publication; 2024 analysis with 2023 data distorts. It does not apply as definitive diagnosis: a complement, not a substitute, for qualitative peer review.

Applications by field

Science policy: CAPES Qualis evaluation, UK REF, ARC Australia; university rankings; international benchmarking. — Sociology of science: collaboration networks, mobility, demographic diversity, gender in STEM. — Innovation studies: patent-publication cross-analysis; university-industry technology transfer. — History of science: paradigm evolution; classical cocitation networks; identification of “sleeping papers”.

Common pitfalls

The first pitfall is confusing descriptive bibliometrics with explanatory scientometrics: counting articles is describing; explaining dynamics requires models. The second is using incomplete databases without documenting: Web of Science and Scopus have linguistic biases (under-representation of non-English), disciplinary biases (under-representation of humanities), and geographic biases (under-representation of Global South); OpenAlex improved coverage. The third is comparing areas with distinct norms without normalization: mathematics has ~5 average citations/article in 5 years; biomedicine has ~20; naive comparison confounds. The fourth is neglecting temporal bias: older articles have accumulated citation advantage; comparable windows (3-5 years) are necessary. The fifth is applying composite indicators without methodological auditing: h-index, FWCI, SJR have specific assumptions; applying without review produces fragile conclusions.

Last updated —