Research Visibility versus Research Impact

The digital transformation of scholarly communication has fundamentally altered how research is produced, disseminated, discovered and evaluated. Research outputs now circulate through interconnected systems comprising scholarly journals, institutional repositories, indexing services, researcher identifiers, citation databases, preprint platforms, datasets, software repositories, social networks and institutional research portals.

This transformation has increased the importance of research visibility. A study cannot be readily used by researchers who cannot discover it. A dataset cannot contribute to reproducible science if potential users cannot locate it. A researcher's contribution can be incorrectly attributed when scholarly identity systems fail to connect publications to the correct author.

However, visibility should not be confused with impact. Research evaluation has historically relied heavily on bibliometric indicators, particularly citation based measures. Such indicators can provide useful evidence concerning scholarly communication and influence, but the Leiden Manifesto emphasises that quantitative indicators should support rather than replace expert assessment, and that evaluation must account for disciplinary, institutional and contextual differences (Hicks et al., 2015).

The distinction becomes increasingly important as research assessment expands beyond citations toward downloads, views, social media attention, policy references, practitioner use and, more recently, quality signals generated by large language models (Thelwall, 2026). The central question of this Bulletin is therefore:

When does research visibility constitute evidence of research impact, and when does it merely represent an opportunity for impact?

To address this question, the Bulletin draws together evidence from bibliometrics, altmetrics, open science policy and research evaluation scholarship, and integrates this evidence into a single conceptual architecture that can inform how platforms such as ResearchTrics measure and report research contribution.

The first ResearchTrics Research Bulletin defined research visibility as:

The degree to which a research output, researcher, research group or institution can be discovered, accessed, recognised, connected, engaged with and represented within relevant scholarly and knowledge ecosystems (Oladunmoye, 2026a).

That definition provides the conceptual starting point for this Bulletin. Research visibility encompasses multiple processes, summarised in Table 1.

Table 1. Dimensions of research visibility identified in Bulletin No. 001

Source: adapted from Oladunmoye (2026a).

These dimensions describe the conditions under which research can enter and circulate through knowledge ecosystems. Research impact addresses a different question: what changed because the research existed, was used, or influenced subsequent action? This distinction is foundational to the ResearchTrics research programme and is developed further in the sections that follow.

The visibility and impact constructs discussed in this Bulletin did not emerge in isolation. They sit within a longer scholarly conversation about how research value should be measured, and a brief review of that conversation clarifies why the distinction matters in practice.

3.1 Responsible metrics and the Leiden Manifesto

The Leiden Manifesto for research metrics, published in Nature by Hicks, Wouters, Waltman, de Rijcke and Rafols (2015), remains the most widely cited statement of responsible bibliometric practice. Its ten principles caution against the substitution of quantitative indicators for expert judgement, call for evaluation to be sensitive to field and mission differences, and warn against false precision in the use of numerical scores. These principles apply directly to the visibility and impact distinction: a visibility indicator can be measured with reasonable precision, yet precision in measuring visibility does not transfer into precision about impact.

3.2 The altmetrics movement

Priem, Taraborelli, Groth and Neylon's Altmetrics Manifesto (2010) argued that traditional peer review and citation counting were too slow and too narrow to capture the full range of scholarly and public engagement with research, and called for new, web based indicators of attention. Altmetrics have since been widely adopted by publishers and research information systems as evidence of reach. However, subsequent commentary has observed that the manifesto does not define impact precisely, and that altmetric attention is frequently treated as a proxy for impact without adequate justification. This is one of the clearest illustrations of the conceptual slippage this Bulletin seeks to correct.

3.3 Societal impact assessment

Bornmann's (2013) literature survey of societal impact assessment catalogued the wide variety of frameworks, indicators and case study approaches that have been used to demonstrate the effects of research on policy, practice, the economy and society. Bornmann's review makes clear that societal impact is typically assessed through qualitative narrative and case evidence rather than through bibliometric proxies, reinforcing the argument that visibility indicators and impact indicators belong to different evidentiary traditions.

3.4 Large language models and the next generation of evaluation

Thelwall (2026) extends the Leiden Manifesto to consider the emerging role of large language models in research quality evaluation, noting that model based quality predictions can correlate more strongly with expert judgement than citation counts in some fields, while raising new concerns about transparency, since the internal reasoning of such models cannot be fully inspected. This development is significant for the ResearchTrics programme because it signals that future visibility and influence indicators may increasingly be generated by automated systems rather than by direct human citation behaviour, which strengthens rather than weakens the case for keeping visibility and impact as separate, clearly defined evidentiary layers.

3.5 Open access and the visibility literature

A growing body of systematic review evidence examines whether open access publishing increases visibility, most commonly proxied through citation counts. Langham-Putrow, Bakker and Riegelman (2021) reviewed 134 studies and found that only 47.8 per cent confirmed an open access citation advantage outright, 27.6 per cent found no advantage, 23.9 per cent found an advantage only within certain subsets of their sample, and the remainder were inconclusive. A more recent review by Dote Pardo (2026), covering 45 studies published between 2006 and 2024, similarly found that open access enhances academic visibility in a heterogeneous rather than a uniform manner, with green open access and preprints showing more consistent advantages than gold open access, and with effects varying by discipline and geography. Figure 1 summarises the Langham-Putrow findings.

These findings are important for the ResearchTrics framework because they demonstrate that even a well studied visibility intervention, making an output openly accessible, does not translate consistently into citation gain, let alone into demonstrable research impact. Accessibility should therefore be treated as one contributing dimension of visibility rather than as a guaranteed pathway to influence.

Research visibility concerns the observable presence and accessibility of research within relevant knowledge environments. A visible research output may:

• appear in scholarly databases;
• be discoverable through search engines;
• have complete bibliographic metadata;
• be linked to an identifiable researcher through a persistent identifier;
• be accessible through an open repository;
• attract readers;
• be downloaded;
• be discussed on scholarly or public platforms;
• be cited;
• be incorporated into research networks; or
• reach communities beyond the original publication venue.

Visibility therefore concerns the opportunity for research to be encountered and engaged with. It does not, by itself, establish that the research produced a meaningful change.

Research impact is a broader and more consequential construct. In practical research evaluation, impact can encompass changes attributable to research in areas such as scientific knowledge, theory, research methods, clinical practice, education, policy, organisational practice, technology, professional practice, economic activity, public understanding and societal wellbeing (Bornmann, 2013).

Importantly, impact should not be inferred automatically from visibility. A highly cited paper may have influenced subsequent scholarship substantially, but citation count alone does not establish whether it changed policy, professional practice or society. Similarly, a research project may have substantial local policy or community influence while generating relatively few citations.

This is one reason responsible research assessment frameworks emphasise contextual interpretation rather than reliance on isolated numerical indicators. The Leiden Manifesto explicitly calls for indicators to be aligned with the mission of the researcher, group or institution, and for differences in publication and citation practices across fields to be taken into account (Hicks et al., 2015).

The distinction between the related constructs of productivity, quality, visibility, influence and impact can be summarised as shown in Table 2.

Table 2. Five related but distinct constructs in research evaluation

Source: author's own analysis, extending Oladunmoye (2026a).

The constructs interact, but none should be treated as synonymous with another. Treating visibility indicators as if they were impact indicators, for example equating a high download count with demonstrable policy change, represents a category error that responsible evaluation systems should avoid.

A useful conceptual proposition is that research visibility creates opportunities for engagement, engagement creates opportunities for influence, and influence may contribute to impact. The resulting conceptual pathway is illustrated in Figure 2.

This is a conceptual model rather than a deterministic causal sequence. Research may move through these stages in different ways, illustrated by the four pathways in Table 3.

Table 3. Illustrative pathways from research production to impact

Source: author's own analysis.

Pathway D is particularly important because conventional bibliometric systems may capture it poorly. Research that produces genuine local benefit can appear, from a citation based dashboard alone, to be low value simply because the audience it reached does not publish in indexed journals.

Citation is one of the most important signals within scholarly communication. When one publication cites another, there is evidence that the cited work has entered the citing author's scholarly discourse. However, a citation demonstrates a relationship within scholarly communication; it does not automatically demonstrate substantive impact.

Citations can occur for different reasons. A paper may be cited because it provides theoretical background, describes a methodology, supplies a measurement instrument, reports contradictory findings, provides historical context, is used as an illustrative example, is being criticised, is highly authoritative, or has genuinely influenced subsequent research.

Consequently, citation counts should be interpreted as one form of evidence rather than as a complete measure of research impact. This position is consistent with responsible metrics principles, which caution against false precision and recommend contextual interpretation of indicators (Hicks et al., 2015).

The issue is particularly relevant to ResearchTrics because the platform's long term objective includes research visibility measurement. A Research Visibility Metric should therefore avoid simply renaming citation counts as impact.

The emergence of digital scholarly communication has produced a growing number of attention indicators, including page views, downloads, social media mentions, bookmarks, repository activity, news coverage, policy document references, practitioner discussion and other forms of online engagement (Priem, Taraborelli, Groth & Neylon, 2010).

Such indicators can provide useful evidence of attention and reach. However, the following distinctions should be maintained:

• attention is not the same as engagement;
• engagement is not the same as influence; and
• influence is not the same as impact.

For example, a controversial article may receive substantial online attention without contributing positively to scientific knowledge. Conversely, a technically important methodological paper may attract modest social media attention but become heavily used by researchers over many years. Therefore, the interpretation of digital indicators requires conceptual discipline, and platforms that surface such indicators should label them accurately as measures of attention rather than as measures of impact.

Visibility and impact also differ temporally. Visibility can sometimes emerge immediately after publication through indexing, social dissemination, repository deposits, media attention, conference presentation or researcher network sharing.

Impact may take substantially longer to emerge. A methodological article, for example, might be published in one year, begin accumulating citations some years later, become widely adopted by researchers after that, and only influence research practice substantially over a much longer horizon. Consequently, early visibility should not be interpreted as established impact. Similarly, low early citation counts should not necessarily be interpreted as evidence of low research value. Research evaluation systems should therefore incorporate appropriate temporal windows, an approach consistent with the Leiden Manifesto's caution against short term, indicator driven judgements (Hicks et al., 2015).

Research visibility can occur throughout the research lifecycle rather than only after publication, as summarised in Table 4.

Table 4. Visibility across the research lifecycle

Source: author's own analysis, extending Oladunmoye (2026a).

This suggests that research visibility is better conceptualised as a longitudinal process than as a single post publication event, and that visibility measurement systems should track outputs across the full lifecycle rather than only from the point of formal publication.

Accessibility is an important component of research visibility. As introduced in Section 3.5, Dote Pardo's (2026) systematic review of 45 studies published between 2006 and 2024 found that open access enhances academic visibility, but in a heterogeneous rather than uniform manner. Green open access and preprints were associated with more consistent increases in citations and readership, whereas gold open access produced mixed outcomes, being positive in medicine and biology but neutral or negative in economics, library science and translation studies.

This finding is important for the ResearchTrics framework because it demonstrates why accessibility should be treated as one dimension of visibility rather than assumed to be equivalent to citation impact. Open science policy also increasingly emphasises making scientific knowledge discoverable, accessible and reusable. UNESCO's Recommendation on Open Science calls on member states to promote open access to scientific publications, data, metadata and other research outputs as a means of improving the transparency, accessibility and reach of research (UNESCO, 2021).

Thus, accessibility can increase the opportunity for research to be engaged with, but accessibility alone does not establish impact.

The distinction between visibility and impact has direct implications for research assessment. A responsible evaluation system should not ask only how many citations a researcher has accumulated. Instead, it should ask the questions summarised in Table 5.

Table 5. Questions for a responsible research evaluation system

Source: author's own analysis, informed by Hicks et al. (2015).

These questions closely correspond with the Leiden Manifesto's recommendations concerning qualitative judgement, context, transparency, field differences and the avoidance of false precision (Hicks et al., 2015).

Contemporary discussion is also extending responsible research evaluation into the era of artificial intelligence. Thelwall (2026) argues that large language models introduce additional considerations for responsible bibliometrics and research evaluation, since model generated quality scores can correlate more strongly with expert judgement than citation based indicators in some fields, while remaining difficult to audit because the underlying reasoning of the model cannot be fully inspected. This reinforces the need for research information systems to distinguish measurement of observable signals from interpretation of scholarly value.

Building on Bulletin No. 001, this Bulletin proposes the conceptual architecture set out below, structured as five sequential stages.

Stage 1: Production

A researcher generates a publication, dataset, instrument, piece of software, report, protocol, book, research project or other scholarly output.

Stage 2: Visibility

The output becomes discoverable, accessible, identifiable, indexed and connected.

Stage 3: Engagement

Relevant audiences read, download, discuss, share, examine and reuse the output.

Stage 4: Influence

The research informs subsequent research, influences methodology, shapes scholarly debate, informs professional decisions, informs policy discussion, or supports innovation.

Stage 5: Impact

The research contributes to observable changes in knowledge, policy, practice, organisations, technology, education, health, the economy or society.

The important conceptual distinction that runs through all five stages is that visibility describes the opportunity for research to be encountered, whereas impact describes consequential change associated with research.

The relationship between visibility and impact becomes clearer when the two constructs are treated as independent dimensions rather than as points on a single scale. Table 6 and Figure 3 present this two axis model.

Table 6. Visibility and impact as independent dimensions

Source: author's own analysis.

The bottom right quadrant of Figure 3 represents the ideal outcome, but the top left quadrant deserves particular analytical attention. High impact, low visibility research may occur when research is locally disseminated, influences practice without generating citations, is published in poorly indexed outlets, is inaccessible internationally, exists in reports rather than journals, is available in non dominant languages, or addresses community specific problems.

A sophisticated research intelligence system should therefore be able to detect contributions that conventional citation based systems may underestimate, rather than treating the absence of bibliometric visibility as evidence of the absence of value.

The distinction developed in this Bulletin suggests that ResearchTrics should maintain separate conceptual layers for research visibility, research engagement, research influence and research impact, summarised in Table 7.

Table 7. Proposed evidence layers for the ResearchTrics platform

Source: author's own analysis, extending Oladunmoye (2026a).

This architecture prevents a common methodological error, namely collapsing different constructs into a single composite score without empirical justification. If ResearchTrics eventually develops a Research Visibility Metric, the metric should primarily measure visibility rather than silently incorporating impact variables. Impact can instead be represented as a complementary evidence layer, reported alongside visibility rather than merged into it.

This distinction creates a logical foundation for subsequent ResearchTrics publications. Bulletin No. 001 established the concept of research visibility. Bulletin No. 002 establishes the distinction between visibility and impact. Future bulletins can therefore investigate the topics listed in Table 8.

Table 8. Indicative topics for future ResearchTrics bulletins

Source: author's own analysis.

This produces a cumulative conceptual literature rather than a series of disconnected publications.

The conceptual framework developed in this Bulletin generates the following propositions for future empirical testing.

1. Research visibility and research impact are conceptually distinct constructs.
2. Research visibility is an enabling condition for, but not a sufficient determinant of, research impact.
3. Citation activity represents one component of scholarly visibility and influence but should not be treated as a universal measure of research impact.
4. Accessibility and discoverability contribute to opportunities for scholarly engagement.
5. The relationship between research visibility and impact is moderated by disciplinary, institutional, geographical and temporal contexts.
6. Research evaluation systems that combine quantitative visibility indicators with contextual qualitative evidence provide a more defensible representation of research contribution than systems relying on a single metric.
7. Research impact may occur without high conventional bibliometric visibility, particularly where research influences local policy, practice, communities or organisations.

These propositions provide a foundation for later empirical ResearchTrics studies, including validation work using citation databases, altmetric providers and institutional repository analytics.

This Bulletin is conceptual rather than empirical, and its propositions have not yet been tested against ResearchTrics platform data. Three limitations are worth noting.

19.1 Reliance on existing indicator infrastructure

Much of the evidence cited in this Bulletin, including the open access citation advantage reviews, was generated using existing bibliometric and altmetric infrastructure that was not designed with the visibility and impact distinction in mind. Future empirical work should test whether visibility indicators, once separated conceptually from influence and impact indicators, behave as distinct constructs statistically as well as theoretically.

19.2 Disciplinary and linguistic coverage

The literature underpinning this Bulletin is drawn predominantly from English language, Global North contexts. As Table 3's Pathway D suggests, research with strong local or community impact in other linguistic and regional contexts may be systematically underrepresented in the evidence base used here, and this should be treated as a caution rather than a settled conclusion.

19.3 The evolving role of artificial intelligence

As Thelwall (2026) observes, large language models are beginning to generate quality and relevance signals that sit alongside, and sometimes outperform, traditional citation based indicators. The implications of this shift for the visibility and impact distinction proposed here have not yet been tested empirically, and represent a priority area for the next Bulletin in this series.

Research visibility and research impact are closely related but fundamentally different constructs. Visibility concerns whether research can be discovered, accessed, recognised, connected and engaged with. Impact concerns the consequences or changes associated with the research.

The relationship between them is therefore best understood as one of potential and progression rather than equivalence. A research output must often become visible before it can be widely engaged with, but visibility does not guarantee meaningful influence or impact. Likewise, meaningful research impact can occur without strong representation in conventional bibliometric systems.

This distinction has significant implications for contemporary research evaluation. Research institutions, researchers and research information systems should avoid interpreting citation counts, downloads, views or other visibility signals as complete representations of research value. Instead, research evaluation should distinguish at least four dimensions, moving from productivity, to quality, to visibility, to impact, with each dimension answering a different question.

ResearchTrics therefore proposes that future research evaluation systems should represent visibility and impact as complementary but non equivalent evidence domains. This conceptual distinction provides the basis for subsequent ResearchTrics research on the measurement, validation and application of research visibility.

Recommended citation

Oladunmoye, E. O. (2026). Research visibility versus research impact: Distinguishing related constructs in contemporary research evaluation. ResearchTrics Research Bulletin, 2, 1 to 34.

In text citation: (Oladunmoye, 2026b)

Recommended citation sentence for subsequent ResearchTrics papers

Research visibility should be distinguished from research impact because visibility represents the opportunity for scholarly engagement, whereas impact concerns consequential changes associated with research (Oladunmoye, 2026a, 2026b).