empirical research laboratory

Skip to main content
Skip to primary sidebar
Skip to footer
QuestionPro

Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case AskWhy Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research

Empirical Research: Definition, Methods, Types and Examples

Content Index

Empirical research: Definition

Empirical research: origin, quantitative research methods, qualitative research methods, steps for conducting empirical research, empirical research methodology cycle, advantages of empirical research, disadvantages of empirical research, why is there a need for empirical research.

Empirical research is defined as any research where conclusions of the study is strictly drawn from concretely empirical evidence, and therefore “verifiable” evidence.

This empirical evidence can be gathered using quantitative market research and qualitative market research methods.

For example: A research is being conducted to find out if listening to happy music in the workplace while working may promote creativity? An experiment is conducted by using a music website survey on a set of audience who are exposed to happy music and another set who are not listening to music at all, and the subjects are then observed. The results derived from such a research will give empirical evidence if it does promote creativity or not.

LEARN ABOUT: Behavioral Research

You must have heard the quote” I will not believe it unless I see it”. This came from the ancient empiricists, a fundamental understanding that powered the emergence of medieval science during the renaissance period and laid the foundation of modern science, as we know it today. The word itself has its roots in greek. It is derived from the greek word empeirikos which means “experienced”.

In today’s world, the word empirical refers to collection of data using evidence that is collected through observation or experience or by using calibrated scientific instruments. All of the above origins have one thing in common which is dependence of observation and experiments to collect data and test them to come up with conclusions.

LEARN ABOUT: Causal Research

Types and methodologies of empirical research

Empirical research can be conducted and analysed using qualitative or quantitative methods.

Quantitative research : Quantitative research methods are used to gather information through numerical data. It is used to quantify opinions, behaviors or other defined variables . These are predetermined and are in a more structured format. Some of the commonly used methods are survey, longitudinal studies, polls, etc
Qualitative research: Qualitative research methods are used to gather non numerical data. It is used to find meanings, opinions, or the underlying reasons from its subjects. These methods are unstructured or semi structured. The sample size for such a research is usually small and it is a conversational type of method to provide more insight or in-depth information about the problem Some of the most popular forms of methods are focus groups, experiments, interviews, etc.

Data collected from these will need to be analysed. Empirical evidence can also be analysed either quantitatively and qualitatively. Using this, the researcher can answer empirical questions which have to be clearly defined and answerable with the findings he has got. The type of research design used will vary depending on the field in which it is going to be used. Many of them might choose to do a collective research involving quantitative and qualitative method to better answer questions which cannot be studied in a laboratory setting.

LEARN ABOUT: Qualitative Research Questions and Questionnaires

Quantitative research methods aid in analyzing the empirical evidence gathered. By using these a researcher can find out if his hypothesis is supported or not.

Survey research: Survey research generally involves a large audience to collect a large amount of data. This is a quantitative method having a predetermined set of closed questions which are pretty easy to answer. Because of the simplicity of such a method, high responses are achieved. It is one of the most commonly used methods for all kinds of research in today’s world.

Previously, surveys were taken face to face only with maybe a recorder. However, with advancement in technology and for ease, new mediums such as emails , or social media have emerged.

For example: Depletion of energy resources is a growing concern and hence there is a need for awareness about renewable energy. According to recent studies, fossil fuels still account for around 80% of energy consumption in the United States. Even though there is a rise in the use of green energy every year, there are certain parameters because of which the general population is still not opting for green energy. In order to understand why, a survey can be conducted to gather opinions of the general population about green energy and the factors that influence their choice of switching to renewable energy. Such a survey can help institutions or governing bodies to promote appropriate awareness and incentive schemes to push the use of greener energy.

Learn more: Renewable Energy Survey Template Descriptive Research vs Correlational Research

Experimental research: In experimental research , an experiment is set up and a hypothesis is tested by creating a situation in which one of the variable is manipulated. This is also used to check cause and effect. It is tested to see what happens to the independent variable if the other one is removed or altered. The process for such a method is usually proposing a hypothesis, experimenting on it, analyzing the findings and reporting the findings to understand if it supports the theory or not.

For example: A particular product company is trying to find what is the reason for them to not be able to capture the market. So the organisation makes changes in each one of the processes like manufacturing, marketing, sales and operations. Through the experiment they understand that sales training directly impacts the market coverage for their product. If the person is trained well, then the product will have better coverage.

Correlational research: Correlational research is used to find relation between two set of variables . Regression analysis is generally used to predict outcomes of such a method. It can be positive, negative or neutral correlation.

LEARN ABOUT: Level of Analysis

For example: Higher educated individuals will get higher paying jobs. This means higher education enables the individual to high paying job and less education will lead to lower paying jobs.

Longitudinal study: Longitudinal study is used to understand the traits or behavior of a subject under observation after repeatedly testing the subject over a period of time. Data collected from such a method can be qualitative or quantitative in nature.

For example: A research to find out benefits of exercise. The target is asked to exercise everyday for a particular period of time and the results show higher endurance, stamina, and muscle growth. This supports the fact that exercise benefits an individual body.

Cross sectional: Cross sectional study is an observational type of method, in which a set of audience is observed at a given point in time. In this type, the set of people are chosen in a fashion which depicts similarity in all the variables except the one which is being researched. This type does not enable the researcher to establish a cause and effect relationship as it is not observed for a continuous time period. It is majorly used by healthcare sector or the retail industry.

For example: A medical study to find the prevalence of under-nutrition disorders in kids of a given population. This will involve looking at a wide range of parameters like age, ethnicity, location, incomes and social backgrounds. If a significant number of kids coming from poor families show under-nutrition disorders, the researcher can further investigate into it. Usually a cross sectional study is followed by a longitudinal study to find out the exact reason.

Causal-Comparative research : This method is based on comparison. It is mainly used to find out cause-effect relationship between two variables or even multiple variables.

For example: A researcher measured the productivity of employees in a company which gave breaks to the employees during work and compared that to the employees of the company which did not give breaks at all.

LEARN ABOUT: Action Research

Some research questions need to be analysed qualitatively, as quantitative methods are not applicable there. In many cases, in-depth information is needed or a researcher may need to observe a target audience behavior, hence the results needed are in a descriptive analysis form. Qualitative research results will be descriptive rather than predictive. It enables the researcher to build or support theories for future potential quantitative research. In such a situation qualitative research methods are used to derive a conclusion to support the theory or hypothesis being studied.

LEARN ABOUT: Qualitative Interview

Case study: Case study method is used to find more information through carefully analyzing existing cases. It is very often used for business research or to gather empirical evidence for investigation purpose. It is a method to investigate a problem within its real life context through existing cases. The researcher has to carefully analyse making sure the parameter and variables in the existing case are the same as to the case that is being investigated. Using the findings from the case study, conclusions can be drawn regarding the topic that is being studied.

For example: A report mentioning the solution provided by a company to its client. The challenges they faced during initiation and deployment, the findings of the case and solutions they offered for the problems. Such case studies are used by most companies as it forms an empirical evidence for the company to promote in order to get more business.

Observational method: Observational method is a process to observe and gather data from its target. Since it is a qualitative method it is time consuming and very personal. It can be said that observational research method is a part of ethnographic research which is also used to gather empirical evidence. This is usually a qualitative form of research, however in some cases it can be quantitative as well depending on what is being studied.

For example: setting up a research to observe a particular animal in the rain-forests of amazon. Such a research usually take a lot of time as observation has to be done for a set amount of time to study patterns or behavior of the subject. Another example used widely nowadays is to observe people shopping in a mall to figure out buying behavior of consumers.

One-on-one interview: Such a method is purely qualitative and one of the most widely used. The reason being it enables a researcher get precise meaningful data if the right questions are asked. It is a conversational method where in-depth data can be gathered depending on where the conversation leads.

For example: A one-on-one interview with the finance minister to gather data on financial policies of the country and its implications on the public.

Focus groups: Focus groups are used when a researcher wants to find answers to why, what and how questions. A small group is generally chosen for such a method and it is not necessary to interact with the group in person. A moderator is generally needed in case the group is being addressed in person. This is widely used by product companies to collect data about their brands and the product.

For example: A mobile phone manufacturer wanting to have a feedback on the dimensions of one of their models which is yet to be launched. Such studies help the company meet the demand of the customer and position their model appropriately in the market.

Text analysis: Text analysis method is a little new compared to the other types. Such a method is used to analyse social life by going through images or words used by the individual. In today’s world, with social media playing a major part of everyone’s life, such a method enables the research to follow the pattern that relates to his study.

For example: A lot of companies ask for feedback from the customer in detail mentioning how satisfied are they with their customer support team. Such data enables the researcher to take appropriate decisions to make their support team better.

Sometimes a combination of the methods is also needed for some questions that cannot be answered using only one type of method especially when a researcher needs to gain a complete understanding of complex subject matter.

We recently published a blog that talks about examples of qualitative data in education ; why don’t you check it out for more ideas?

Learn More: Data Collection Methods: Types & Examples

Since empirical research is based on observation and capturing experiences, it is important to plan the steps to conduct the experiment and how to analyse it. This will enable the researcher to resolve problems or obstacles which can occur during the experiment.

Step #1: Define the purpose of the research

This is the step where the researcher has to answer questions like what exactly do I want to find out? What is the problem statement? Are there any issues in terms of the availability of knowledge, data, time or resources. Will this research be more beneficial than what it will cost.

Before going ahead, a researcher has to clearly define his purpose for the research and set up a plan to carry out further tasks.

Step #2 : Supporting theories and relevant literature

The researcher needs to find out if there are theories which can be linked to his research problem . He has to figure out if any theory can help him support his findings. All kind of relevant literature will help the researcher to find if there are others who have researched this before, or what are the problems faced during this research. The researcher will also have to set up assumptions and also find out if there is any history regarding his research problem

Step #3: Creation of Hypothesis and measurement

Before beginning the actual research he needs to provide himself a working hypothesis or guess what will be the probable result. Researcher has to set up variables, decide the environment for the research and find out how can he relate between the variables.

Researcher will also need to define the units of measurements, tolerable degree for errors, and find out if the measurement chosen will be acceptable by others.

Step #4: Methodology, research design and data collection

In this step, the researcher has to define a strategy for conducting his research. He has to set up experiments to collect data which will enable him to propose the hypothesis. The researcher will decide whether he will need experimental or non experimental method for conducting the research. The type of research design will vary depending on the field in which the research is being conducted. Last but not the least, the researcher will have to find out parameters that will affect the validity of the research design. Data collection will need to be done by choosing appropriate samples depending on the research question. To carry out the research, he can use one of the many sampling techniques. Once data collection is complete, researcher will have empirical data which needs to be analysed.

LEARN ABOUT: Best Data Collection Tools

Step #5: Data Analysis and result

Data analysis can be done in two ways, qualitatively and quantitatively. Researcher will need to find out what qualitative method or quantitative method will be needed or will he need a combination of both. Depending on the unit of analysis of his data, he will know if his hypothesis is supported or rejected. Analyzing this data is the most important part to support his hypothesis.

Step #6: Conclusion

A report will need to be made with the findings of the research. The researcher can give the theories and literature that support his research. He can make suggestions or recommendations for further research on his topic.

A.D. de Groot, a famous dutch psychologist and a chess expert conducted some of the most notable experiments using chess in the 1940’s. During his study, he came up with a cycle which is consistent and now widely used to conduct empirical research. It consists of 5 phases with each phase being as important as the next one. The empirical cycle captures the process of coming up with hypothesis about how certain subjects work or behave and then testing these hypothesis against empirical data in a systematic and rigorous approach. It can be said that it characterizes the deductive approach to science. Following is the empirical cycle.

Observation: At this phase an idea is sparked for proposing a hypothesis. During this phase empirical data is gathered using observation. For example: a particular species of flower bloom in a different color only during a specific season.
Induction: Inductive reasoning is then carried out to form a general conclusion from the data gathered through observation. For example: As stated above it is observed that the species of flower blooms in a different color during a specific season. A researcher may ask a question “does the temperature in the season cause the color change in the flower?” He can assume that is the case, however it is a mere conjecture and hence an experiment needs to be set up to support this hypothesis. So he tags a few set of flowers kept at a different temperature and observes if they still change the color?
Deduction: This phase helps the researcher to deduce a conclusion out of his experiment. This has to be based on logic and rationality to come up with specific unbiased results.For example: In the experiment, if the tagged flowers in a different temperature environment do not change the color then it can be concluded that temperature plays a role in changing the color of the bloom.
Testing: This phase involves the researcher to return to empirical methods to put his hypothesis to the test. The researcher now needs to make sense of his data and hence needs to use statistical analysis plans to determine the temperature and bloom color relationship. If the researcher finds out that most flowers bloom a different color when exposed to the certain temperature and the others do not when the temperature is different, he has found support to his hypothesis. Please note this not proof but just a support to his hypothesis.
Evaluation: This phase is generally forgotten by most but is an important one to keep gaining knowledge. During this phase the researcher puts forth the data he has collected, the support argument and his conclusion. The researcher also states the limitations for the experiment and his hypothesis and suggests tips for others to pick it up and continue a more in-depth research for others in the future. LEARN MORE: Population vs Sample

LEARN MORE: Population vs Sample

There is a reason why empirical research is one of the most widely used method. There are a few advantages associated with it. Following are a few of them.

It is used to authenticate traditional research through various experiments and observations.
This research methodology makes the research being conducted more competent and authentic.
It enables a researcher understand the dynamic changes that can happen and change his strategy accordingly.
The level of control in such a research is high so the researcher can control multiple variables.
It plays a vital role in increasing internal validity .

Even though empirical research makes the research more competent and authentic, it does have a few disadvantages. Following are a few of them.

Such a research needs patience as it can be very time consuming. The researcher has to collect data from multiple sources and the parameters involved are quite a few, which will lead to a time consuming research.
Most of the time, a researcher will need to conduct research at different locations or in different environments, this can lead to an expensive affair.
There are a few rules in which experiments can be performed and hence permissions are needed. Many a times, it is very difficult to get certain permissions to carry out different methods of this research.
Collection of data can be a problem sometimes, as it has to be collected from a variety of sources through different methods.

LEARN ABOUT: Social Communication Questionnaire

Empirical research is important in today’s world because most people believe in something only that they can see, hear or experience. It is used to validate multiple hypothesis and increase human knowledge and continue doing it to keep advancing in various fields.

For example: Pharmaceutical companies use empirical research to try out a specific drug on controlled groups or random groups to study the effect and cause. This way, they prove certain theories they had proposed for the specific drug. Such research is very important as sometimes it can lead to finding a cure for a disease that has existed for many years. It is useful in science and many other fields like history, social sciences, business, etc.

LEARN ABOUT: 12 Best Tools for Researchers

With the advancement in today’s world, empirical research has become critical and a norm in many fields to support their hypothesis and gain more knowledge. The methods mentioned above are very useful for carrying out such research. However, a number of new methods will keep coming up as the nature of new investigative questions keeps getting unique or changing.

Create a single source of real data with a built-for-insights platform. Store past data, add nuggets of insights, and import research data from various sources into a CRM for insights. Build on ever-growing research with a real-time dashboard in a unified research management platform to turn insights into knowledge.

LEARN MORE FREE TRIAL

MORE LIKE THIS

QuestionPro: Leading the Charge in Customer Journey Management and Voice of the Customer Platforms

Sep 17, 2024

What is Driver Analysis? Importance and Best Practices

Was The Experience Memorable? (Part II) — Tuesday CX Thoughts

Data Discovery: What it is, Importance, Process + Use Cases

Sep 16, 2024

Empirical Research

Reference work entry
First Online: 01 January 2020
Cite this reference work entry

Emeka Thaddues Njoku 3

252 Accesses

1 Citations

The term “empirical” entails gathered data based on experience, observations, or experimentation. In empirical research, knowledge is developed from factual experience as opposed to theoretical assumption and usually involved the use of data sources like datasets or fieldwork, but can also be based on observations within a laboratory setting. Testing hypothesis or answering definite questions is a primary feature of empirical research. Empirical research, in other words, involves the process of employing working hypothesis that are tested through experimentation or observation. Hence, empirical research is a method of uncovering empirical evidence.

Through the process of gathering valid empirical data, scientists from a variety of fields, ranging from the social to the natural sciences, have to carefully design their methods. This helps to ensure quality and accuracy of data collection and treatment. However, any error in empirical data collection process could inevitably render such...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save.

Get 10 units per month
Download Article/Chapter or eBook
1 Unit = 1 Article or 1 Chapter
Cancel anytime
Available as PDF
Read on any device
Instant download
Own it forever
Available as EPUB and PDF
Durable hardcover edition
Dispatched in 3 to 5 business days
Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Bibliography

Bhattacherjee, A. (2012). Social science research: Principles, methods, and practices. Textbooks Collection . Book 3.

Google Scholar

Comte, A., & Bridges, J. H. (Tr.) (1865). A general view of positivism . Trubner and Co. (reissued by Cambridge University Press, 2009).

Dilworth, C. B. (1982). Empirical research in the literature class. English Journal, 71 (3), 95–97.

Article Google Scholar

Heisenberg, W. (1971). Positivism, metaphysics and religion. In R. N. Nanshen (Ed.), Werner Heisenberg – Physics and beyond – Encounters and conversations , World Perspectives. 42. Translator: Arnold J. Pomerans. New York: Harper and Row.

Hossain, F. M. A. (2014). A critical analysis of empiricism. Open Journal of Philosophy, 2014 (4), 225–230.

Kant, I. (1783). Prolegomena to any future metaphysic (trans: Bennett, J.). Early Modern Texts. www.earlymoderntexts.com

Koch, S. (1992). Psychology’s Bridgman vs. Bridgman’s Bridgman: An essay in reconstruction. Theory and Psychology, 2 (3), 261–290.

Matin, A. (1968). An outline of philosophy . Dhaka: Mullick Brothers.

Mcleod, S. (2008). Psychology as science. http://www.simplypsychology.org/science-psychology.html

Popper, K. (1963). Conjectures and refutations: The growth of scientific knowledge . London: Routledge.

Simmel, G. (1908). The problem areas of sociology in Kurt H. Wolf: The sociology of Georg Simmel . London: The Free Press.

Weber, M. (1991). The nature of social action. In W. G. Runciman (Ed.), Weber: Selections in translation . Cambridge: Cambridge University Press.

Download references

Author information

Authors and affiliations.

Department of Political Science, University of Ibadan, Ibadan, Oyo, Nigeria

Emeka Thaddues Njoku

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Emeka Thaddues Njoku .

Editor information

Editors and affiliations.

University of Connecticut, Storrs, CT, USA

David A. Leeming

Blanton-Peale Institute, New York, NY, USA

Rights and permissions

Reprints and permissions

Copyright information

About this entry

Cite this entry.

Njoku, E.T. (2020). Empirical Research. In: Leeming, D.A. (eds) Encyclopedia of Psychology and Religion. Springer, Cham. https://doi.org/10.1007/978-3-030-24348-7_200051

Download citation

DOI : https://doi.org/10.1007/978-3-030-24348-7_200051

Published : 12 June 2020

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-24347-0

Online ISBN : 978-3-030-24348-7

eBook Packages : Behavioral Science and Psychology Reference Module Humanities and Social Sciences Reference Module Business, Economics and Social Sciences

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Publish with us

Policies and ethics

Find a journal
Track your research

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

View all journals
Explore content
About the journal
Publish with us
Sign up for alerts
Review Article
Published: 01 June 2023

Data, measurement and empirical methods in the science of science

Lu Liu 1 , 2 , 3 , 4 ,
Benjamin F. Jones ORCID: orcid.org/0000-0001-9697-9388 1 , 2 , 3 , 5 , 6 ,
Brian Uzzi ORCID: orcid.org/0000-0001-6855-2854 1 , 2 , 3 &
Dashun Wang ORCID: orcid.org/0000-0002-7054-2206 1 , 2 , 3 , 7

Nature Human Behaviour volume 7 , pages 1046–1058 ( 2023 ) Cite this article

20k Accesses

19 Citations

117 Altmetric

Metrics details

Scientific community

The advent of large-scale datasets that trace the workings of science has encouraged researchers from many different disciplinary backgrounds to turn scientific methods into science itself, cultivating a rapidly expanding ‘science of science’. This Review considers this growing, multidisciplinary literature through the lens of data, measurement and empirical methods. We discuss the purposes, strengths and limitations of major empirical approaches, seeking to increase understanding of the field’s diverse methodologies and expand researchers’ toolkits. Overall, new empirical developments provide enormous capacity to test traditional beliefs and conceptual frameworks about science, discover factors associated with scientific productivity, predict scientific outcomes and design policies that facilitate scientific progress.

SciSciNet: A large-scale open data lake for the science of science research

A dataset for measuring the impact of research data and their curation

Envisioning a “science diplomacy 2.0”: on data, global challenges, and multi-layered networks

Scientific advances are a key input to rising standards of living, health and the capacity of society to confront grand challenges, from climate change to the COVID-19 pandemic 1 , 2 , 3 . A deeper understanding of how science works and where innovation occurs can help us to more effectively design science policy and science institutions, better inform scientists’ own research choices, and create and capture enormous value for science and humanity. Building on these key premises, recent years have witnessed substantial development in the ‘science of science’ 4 , 5 , 6 , 7 , 8 , 9 , which uses large-scale datasets and diverse computational toolkits to unearth fundamental patterns behind scientific production and use.

The idea of turning scientific methods into science itself is long-standing. Since the mid-20th century, researchers from different disciplines have asked central questions about the nature of scientific progress and the practice, organization and impact of scientific research. Building on these rich historical roots, the field of the science of science draws upon many disciplines, ranging from information science to the social, physical and biological sciences to computer science, engineering and design. The science of science closely relates to several strands and communities of research, including metascience, scientometrics, the economics of science, research on research, science and technology studies, the sociology of science, metaknowledge and quantitative science studies 5 . There are noticeable differences between some of these communities, mostly around their historical origins and the initial disciplinary composition of researchers forming these communities. For example, metascience has its origins in the clinical sciences and psychology, and focuses on rigour, transparency, reproducibility and other open science-related practices and topics. The scientometrics community, born in library and information sciences, places a particular emphasis on developing robust and responsible measures and indicators for science. Science and technology studies engage the history of science and technology, the philosophy of science, and the interplay between science, technology and society. The science of science, which has its origins in physics, computer science and sociology, takes a data-driven approach and emphasizes questions on how science works. Each of these communities has made fundamental contributions to understanding science. While they differ in their origins, these differences pale in comparison to the overarching, common interest in understanding the practice of science and its societal impact.

Three major developments have encouraged rapid advances in the science of science. The first is in data 9 : modern databases include millions of research articles, grant proposals, patents and more. This windfall of data traces scientific activity in remarkable detail and at scale. The second development is in measurement: scholars have used data to develop many new measures of scientific activities and examine theories that have long been viewed as important but difficult to quantify. The third development is in empirical methods: thanks to parallel advances in data science, network science, artificial intelligence and econometrics, researchers can study relationships, make predictions and assess science policy in powerful new ways. Together, new data, measurements and methods have revealed fundamental new insights about the inner workings of science and scientific progress itself.

With multiple approaches, however, comes a key challenge. As researchers adhere to norms respected within their disciplines, their methods vary, with results often published in venues with non-overlapping readership, fragmenting research along disciplinary boundaries. This fragmentation challenges researchers’ ability to appreciate and understand the value of work outside of their own discipline, much less to build directly on it for further investigations.

Recognizing these challenges and the rapidly developing nature of the field, this paper reviews the empirical approaches that are prevalent in this literature. We aim to provide readers with an up-to-date understanding of the available datasets, measurement constructs and empirical methodologies, as well as the value and limitations of each. Owing to space constraints, this Review does not cover the full technical details of each method, referring readers to related guides to learn more. Instead, we will emphasize why a researcher might favour one method over another, depending on the research question.

Beyond a positive understanding of science, a key goal of the science of science is to inform science policy. While this Review mainly focuses on empirical approaches, with its core audience being researchers in the field, the studies reviewed are also germane to key policy questions. For example, what is the appropriate scale of scientific investment, in what directions and through what institutions 10 , 11 ? Are public investments in science aligned with public interests 12 ? What conditions produce novel or high-impact science 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 ? How do the reward systems of science influence the rate and direction of progress 13 , 21 , 22 , 23 , 24 , and what governs scientific reproducibility 25 , 26 , 27 ? How do contributions evolve over a scientific career 28 , 29 , 30 , 31 , 32 , and how may diversity among scientists advance scientific progress 33 , 34 , 35 , among other questions relevant to science policy 36 , 37 .

Overall, this review aims to facilitate entry to science of science research, expand researcher toolkits and illustrate how diverse research approaches contribute to our collective understanding of science. Section 2 reviews datasets and data linkages. Section 3 reviews major measurement constructs in the science of science. Section 4 considers a range of empirical methods, focusing on one study to illustrate each method and briefly summarizing related examples and applications. Section 5 concludes with an outlook for the science of science.

Historically, data on scientific activities were difficult to collect and were available in limited quantities. Gathering data could involve manually tallying statistics from publications 38 , 39 , interviewing scientists 16 , 40 , or assembling historical anecdotes and biographies 13 , 41 . Analyses were typically limited to a specific domain or group of scientists. Today, massive datasets on scientific production and use are at researchers’ fingertips 42 , 43 , 44 . Armed with big data and advanced algorithms, researchers can now probe questions previously not amenable to quantification and with enormous increases in scope and scale, as detailed below.

Publication datasets cover papers from nearly all scientific disciplines, enabling analyses of both general and domain-specific patterns. Commonly used datasets include the Web of Science (WoS), PubMed, CrossRef, ORCID, OpenCitations, Dimensions and OpenAlex. Datasets incorporating papers’ text (CORE) 45 , 46 , 47 , data entities (DataCite) 48 , 49 and peer review reports (Publons) 33 , 50 , 51 have also become available. These datasets further enable novel measurement, for example, representations of a paper’s content 52 , 53 , novelty 15 , 54 and interdisciplinarity 55 .

Notably, databases today capture more diverse aspects of science beyond publications, offering a richer and more encompassing view of research contexts and of researchers themselves (Fig. 1 ). For example, some datasets trace research funding to the specific publications these investments support 56 , 57 , allowing high-scale studies of the impact of funding on productivity and the return on public investment. Datasets incorporating job placements 58 , 59 , curriculum vitae 21 , 59 and scientific prizes 23 offer rich quantitative evidence on the social structure of science. Combining publication profiles with mentorship genealogies 60 , 61 , dissertations 34 and course syllabi 62 , 63 provides insights on mentoring and cultivating talent.

This figure presents commonly used data types in science of science research, information contained in each data type and examples of data sources. Datasets in the science of science research have not only grown in scale but have also expanded beyond publications to integrate upstream funding investments and downstream applications that extend beyond science itself.

Finally, today’s scope of data extends beyond science to broader aspects of society. Altmetrics 64 captures news media and social media mentions of scientific articles. Other databases incorporate marketplace uses of science, including through patents 10 , pharmaceutical clinical trials and drug approvals 65 , 66 . Policy documents 67 , 68 help us to understand the role of science in the halls of government 69 and policy making 12 , 68 .

While datasets of the modern scientific enterprise have grown exponentially, they are not without limitations. As is often the case for data-driven research, drawing conclusions from specific data sources requires scrutiny and care. Datasets are typically based on published work, which may favour easy-to-publish topics over important ones (the streetlight effect) 70 , 71 . The publication of negative results is also rare (the file drawer problem) 72 , 73 . Meanwhile, English language publications account for over 90% of articles in major data sources, with limited coverage of non-English journals 74 . Publication datasets may also reflect biases in data collection across research institutions or demographic groups. Despite the open science movement, many datasets require paid subscriptions, which can create inequality in data access. Creating more open datasets for the science of science, such as OpenAlex, may not only improve the robustness and replicability of empirical claims but also increase entry to the field.

As today’s datasets become larger in scale and continue to integrate new dimensions, they offer opportunities to unveil the inner workings and external impacts of science in new ways. They can enable researchers to reach beyond previous limitations while conducting original studies of new and long-standing questions about the sciences.

Measurement

Here we discuss prominent measurement approaches in the science of science, including their purposes and limitations.

Modern publication databases typically include data on which articles and authors cite other papers and scientists. These citation linkages have been used to engage core conceptual ideas in scientific research. Here we consider two common measures based on citation information: citation counts and knowledge flows.

First, citation counts are commonly used indicators of impact. The term ‘indicator’ implies that it only approximates the concept of interest. A citation count is defined as how many times a document is cited by subsequent documents and can proxy for the importance of research papers 75 , 76 as well as patented inventions 77 , 78 , 79 . Rather than treating each citation equally, measures may further weight the importance of each citation, for example by using the citation network structure to produce centrality 80 , PageRank 81 , 82 or Eigenfactor indicators 83 , 84 .

Citation-based indicators have also faced criticism 84 , 85 . Citation indicators necessarily oversimplify the construct of impact, often ignoring heterogeneity in the meaning and use of a particular reference, the variations in citation practices across fields and institutional contexts, and the potential for reputation and power structures in science to influence citation behaviour 86 , 87 . Researchers have started to understand more nuanced citation behaviours ranging from negative citations 86 to citation context 47 , 88 , 89 . Understanding what a citation actually measures matters in interpreting and applying many research findings in the science of science. Evaluations relying on citation-based indicators rather than expert judgements raise questions regarding misuse 90 , 91 , 92 . Given the importance of developing indicators that can reliably quantify and evaluate science, the scientometrics community has been working to provide guidance for responsible citation practices and assessment 85 .

Second, scientists use citations to trace knowledge flows. Each citation in a paper is a link to specific previous work from which we can proxy how new discoveries draw upon existing ideas 76 , 93 and how knowledge flows between fields of science 94 , 95 , research institutions 96 , regions and nations 97 , 98 , 99 , and individuals 81 . Combinations of citation linkages can also approximate novelty 15 , disruptiveness 17 , 100 and interdisciplinarity 55 , 95 , 101 , 102 . A rapidly expanding body of work further examines citations to scientific articles from other domains (for example, patents, clinical drug trials and policy documents) to understand the applied value of science 10 , 12 , 65 , 66 , 103 , 104 , 105 .

Individuals

Analysing individual careers allows researchers to answer questions such as: How do we quantify individual scientific productivity? What is a typical career lifecycle? How are resources and credits allocated across individuals and careers? A scholar’s career can be examined through the papers they publish 30 , 31 , 106 , 107 , 108 , with attention to career progression and mobility, publication counts and citation impact, as well as grant funding 24 , 109 , 110 and prizes 111 , 112 , 113 ,

Studies of individual impact focus on output, typically approximated by the number of papers a researcher publishes and citation indicators. A popular measure for individual impact is the h -index 114 , which takes both volume and per-paper impact into consideration. Specifically, a scientist is assigned the largest value h such that they have h papers that were each cited at least h times. Later studies build on the idea of the h -index and propose variants to address limitations 115 , these variants ranging from emphasizing highly cited papers in a career 116 , to field differences 117 and normalizations 118 , to the relative contribution of an individual in collaborative works 119 .

To study dynamics in output over the lifecycle, individuals can be studied according to age, career age or the sequence of publications. A long-standing literature has investigated the relationship between age and the likelihood of outstanding achievement 28 , 106 , 111 , 120 , 121 . Recent studies further decouple the relationship between age, publication volume and per-paper citation, and measure the likelihood of producing highly cited papers in the sequence of works one produces 30 , 31 .

As simple as it sounds, representing careers using publication records is difficult. Collecting the full publication list of a researcher is the foundation to study individuals yet remains a key challenge, requiring name disambiguation techniques to match specific works to specific researchers. Although algorithms are increasingly capable at identifying millions of career profiles 122 , they vary in accuracy and robustness. ORCID can help to alleviate the problem by offering researchers the opportunity to create, maintain and update individual profiles themselves, and it goes beyond publications to collect broader outputs and activities 123 . A second challenge is survivorship bias. Empirical studies tend to focus on careers that are long enough to afford statistical analyses, which limits the applicability of the findings to scientific careers as a whole. A third challenge is the breadth of scientists’ activities, where focusing on publications ignores other important contributions such as mentorship and teaching, service (for example, refereeing papers, reviewing grant proposals and editing journals) or leadership within their organizations. Although researchers have begun exploring these dimensions by linking individual publication profiles with genealogical databases 61 , 124 , dissertations 34 , grants 109 , curriculum vitae 21 and acknowledgements 125 , scientific careers beyond publication records remain under-studied 126 , 127 . Lastly, citation-based indicators only serve as an approximation of individual performance with similar limitations as discussed above. The scientific community has called for more appropriate practices 85 , 128 , ranging from incorporating expert assessment of research contributions to broadening the measures of impact beyond publications.

Over many decades, science has exhibited a substantial and steady shift away from solo authorship towards coauthorship, especially among highly cited works 18 , 129 , 130 . In light of this shift, a research field, the science of team science 131 , 132 , has emerged to study the mechanisms that facilitate or hinder the effectiveness of teams. Team size can be proxied by the number of coauthors on a paper, which has been shown to predict distinctive types of advance: whereas larger teams tend to develop ideas, smaller teams tend to disrupt current ways of thinking 17 . Team characteristics can be inferred from coauthors’ backgrounds 133 , 134 , 135 , allowing quantification of a team’s diversity in terms of field, age, gender or ethnicity. Collaboration networks based on coauthorship 130 , 136 , 137 , 138 , 139 offer nuanced network-based indicators to understand individual and institutional collaborations.

However, there are limitations to using coauthorship alone to study teams 132 . First, coauthorship can obscure individual roles 140 , 141 , 142 , which has prompted institutional responses to help to allocate credit, including authorship order and individual contribution statements 56 , 143 . Second, coauthorship does not reflect the complex dynamics and interactions between team members that are often instrumental for team success 53 , 144 . Third, collaborative contributions can extend beyond coauthorship in publications to include members of a research laboratory 145 or co-principal investigators (co-PIs) on a grant 146 . Initiatives such as CRediT may help to address some of these issues by recording detailed roles for each contributor 147 .

Institutions

Research institutions, such as departments, universities, national laboratories and firms, encompass wider groups of researchers and their corresponding outputs. Institutional membership can be inferred from affiliations listed on publications or patents 148 , 149 , and the output of an institution can be aggregated over all its affiliated researchers 150 . Institutional research information systems (CRIS) contain more comprehensive research outputs and activities from employees.

Some research questions consider the institution as a whole, investigating the returns to research and development investment 104 , inequality of resource allocation 22 and the flow of scientists 21 , 148 , 149 . Other questions focus on institutional structures as sources of research productivity by looking into the role of peer effects 125 , 151 , 152 , 153 , how institutional policies impact research outcomes 154 , 155 and whether interdisciplinary efforts foster innovation 55 . Institution-oriented measurement faces similar limitations as with analyses of individuals and teams, including name disambiguation for a given institution and the limited capacity of formal publication records to characterize the full range of relevant institutional outcomes. It is also unclear how to allocate credit among multiple institutions associated with a paper. Moreover, relevant institutional employees extend beyond publishing researchers: interns, technicians and administrators all contribute to research endeavours 130 .

In sum, measurements allow researchers to quantify scientific production and use across numerous dimensions, but they also raise questions of construct validity: Does the proposed metric really reflect what we want to measure? Testing the construct’s validity is important, as is understanding a construct’s limits. Where possible, using alternative measurement approaches, or qualitative methods such as interviews and surveys, can improve measurement accuracy and the robustness of findings.

Empirical methods

In this section, we review two broad categories of empirical approaches (Table 1 ), each with distinctive goals: (1) to discover, estimate and predict empirical regularities; and (2) to identify causal mechanisms. For each method, we give a concrete example to help to explain how the method works, summarize related work for interested readers, and discuss contributions and limitations.

Descriptive and predictive approaches

Empirical regularities and generalizable facts.

The discovery of empirical regularities in science has had a key role in driving conceptual developments and the directions of future research. By observing empirical patterns at scale, researchers unveil central facts that shape science and present core features that theories of scientific progress and practice must explain. For example, consider citation distributions. de Solla Price first proposed that citation distributions are fat-tailed 39 , indicating that a few papers have extremely high citations while most papers have relatively few or even no citations at all. de Solla Price proposed that citation distribution was a power law, while researchers have since refined this view to show that the distribution appears log-normal, a nearly universal regularity across time and fields 156 , 157 . The fat-tailed nature of citation distributions and its universality across the sciences has in turn sparked substantial theoretical work that seeks to explain this key empirical regularity 20 , 156 , 158 , 159 .

Empirical regularities are often surprising and can contest previous beliefs of how science works. For example, it has been shown that the age distribution of great achievements peaks in middle age across a wide range of fields 107 , 121 , 160 , rejecting the common belief that young scientists typically drive breakthroughs in science. A closer look at the individual careers also indicates that productivity patterns vary widely across individuals 29 . Further, a scholar’s highest-impact papers come at a remarkably constant rate across the sequence of their work 30 , 31 .

The discovery of empirical regularities has had important roles in shaping beliefs about the nature of science 10 , 45 , 161 , 162 , sources of breakthrough ideas 15 , 163 , 164 , 165 , scientific careers 21 , 29 , 126 , 127 , the network structure of ideas and scientists 23 , 98 , 136 , 137 , 138 , 139 , 166 , gender inequality 57 , 108 , 126 , 135 , 143 , 167 , 168 , and many other areas of interest to scientists and science institutions 22 , 47 , 86 , 97 , 102 , 105 , 134 , 169 , 170 , 171 . At the same time, care must be taken to ensure that findings are not merely artefacts due to data selection or inherent bias. To differentiate meaningful patterns from spurious ones, it is important to stress test the findings through different selection criteria or across non-overlapping data sources.

Regression analysis

When investigating correlations among variables, a classic method is regression, which estimates how one set of variables explains variation in an outcome of interest. Regression can be used to test explicit hypotheses or predict outcomes. For example, researchers have investigated whether a paper’s novelty predicts its citation impact 172 . Adding additional control variables to the regression, one can further examine the robustness of the focal relationship.

Although regression analysis is useful for hypothesis testing, it bears substantial limitations. If the question one wishes to ask concerns a ‘causal’ rather than a correlational relationship, regression is poorly suited to the task as it is impossible to control for all the confounding factors. Failing to account for such ‘omitted variables’ can bias the regression coefficient estimates and lead to spurious interpretations. Further, regression models often have low goodness of fit (small R 2 ), indicating that the variables considered explain little of the outcome variation. As regressions typically focus on a specific relationship in simple functional forms, regressions tend to emphasize interpretability rather than overall predictability. The advent of predictive approaches powered by large-scale datasets and novel computational techniques offers new opportunities for modelling complex relationships with stronger predictive power.

Mechanistic models

Mechanistic modelling is an important approach to explaining empirical regularities, drawing from methods primarily used in physics. Such models predict macro-level regularities of a system by modelling micro-level interactions among basic elements with interpretable and modifiable formulars. While theoretical by nature, mechanistic models in the science of science are often empirically grounded, and this approach has developed together with the advent of large-scale, high-resolution data.

Simplicity is the core value of a mechanistic model. Consider for example, why citations follow a fat-tailed distribution. de Solla Price modelled the citing behaviour as a cumulative advantage process on a growing citation network 159 and found that if the probability a paper is cited grows linearly with its existing citations, the resulting distribution would follow a power law, broadly aligned with empirical observations. The model is intentionally simplified, ignoring myriad factors. Yet the simple cumulative advantage process is by itself sufficient in explaining a power law distribution of citations. In this way, mechanistic models can help to reveal key mechanisms that can explain observed patterns.

Moreover, mechanistic models can be refined as empirical evidence evolves. For example, later investigations showed that citation distributions are better characterized as log-normal 156 , 173 , prompting researchers to introduce a fitness parameter to encapsulate the inherent differences in papers’ ability to attract citations 174 , 175 . Further, older papers are less likely to be cited than expected 176 , 177 , 178 , motivating more recent models 20 to introduce an additional aging effect 179 . By combining the cumulative advantage, fitness and aging effects, one can already achieve substantial predictive power not just for the overall properties of the system but also the citation dynamics of individual papers 20 .

In addition to citations, mechanistic models have been developed to understand the formation of collaborations 136 , 180 , 181 , 182 , 183 , knowledge discovery and diffusion 184 , 185 , topic selection 186 , 187 , career dynamics 30 , 31 , 188 , 189 , the growth of scientific fields 190 and the dynamics of failure in science and other domains 178 .

At the same time, some observers have argued that mechanistic models are too simplistic to capture the essence of complex real-world problems 191 . While it has been a cornerstone for the natural sciences, representing social phenomena in a limited set of mathematical equations may miss complexities and heterogeneities that make social phenomena interesting in the first place. Such concerns are not unique to the science of science, as they represent a broader theme in computational social sciences 192 , 193 , ranging from social networks 194 , 195 to human mobility 196 , 197 to epidemics 198 , 199 . Other observers have questioned the practical utility of mechanistic models and whether they can be used to guide decisions and devise actionable policies. Nevertheless, despite these limitations, several complex phenomena in the science of science are well captured by simple mechanistic models, showing a high degree of regularity beneath complex interacting systems and providing powerful insights about the nature of science. Mixing such modelling with other methods could be particularly fruitful in future investigations.

Machine learning

The science of science seeks in part to forecast promising directions for scientific research 7 , 44 . In recent years, machine learning methods have substantially advanced predictive capabilities 200 , 201 and are playing increasingly important parts in the science of science. In contrast to the previous methods, machine learning does not emphasize hypotheses or theories. Rather, it leverages complex relationships in data and optimizes goodness of fit to make predictions and categorizations.

Traditional machine learning models include supervised, semi-supervised and unsupervised learning. The model choice depends on data availability and the research question, ranging from supervised models for citation prediction 202 , 203 to unsupervised models for community detection 204 . Take for example mappings of scientific knowledge 94 , 205 , 206 . The unsupervised method applies network clustering algorithms to map the structures of science. Related visualization tools make sense of clusters from the underlying network, allowing observers to see the organization, interactions and evolution of scientific knowledge. More recently, supervised learning, and deep neural networks in particular, have witnessed especially rapid developments 207 . Neural networks can generate high-dimensional representations of unstructured data such as images and texts, which encode complex properties difficult for human experts to perceive.

Take text analysis as an example. A recent study 52 utilizes 3.3 million paper abstracts in materials science to predict the thermoelectric properties of materials. The intuition is that the words currently used to describe a material may predict its hitherto undiscovered properties (Fig. 2 ). Compared with a random material, the materials predicted by the model are eight times more likely to be reported as thermoelectric in the next 5 years, suggesting that machine learning has the potential to substantially speed up knowledge discovery, especially as data continue to grow in scale and scope. Indeed, predicting the direction of new discoveries represents one of the most promising avenues for machine learning models, with neural networks being applied widely to biology 208 , physics 209 , 210 , mathematics 211 , chemistry 212 , medicine 213 and clinical applications 214 . Neural networks also offer a quantitative framework to probe the characteristics of creative products ranging from scientific papers 53 , journals 215 , organizations 148 , to paintings and movies 32 . Neural networks can also help to predict the reproducibility of papers from a variety of disciplines at scale 53 , 216 .

This figure illustrates the word2vec skip-gram methods 52 , where the goal is to predict useful properties of materials using previous scientific literature. a , The architecture and training process of the word2vec skip-gram model, where the 3-layer, fully connected neural network learns the 200-dimensional representation (hidden layer) from the sparse vector for each word and its context in the literature (input layer). b , The top two principal components of the word embedding. Materials with similar features are close in the 2D space, allowing prediction of a material’s properties. Different targeted words are shown in different colours. Reproduced with permission from ref. 52 , Springer Nature Ltd.

While machine learning can offer high predictive accuracy, successful applications to the science of science face challenges, particularly regarding interpretability. Researchers may value transparent and interpretable findings for how a given feature influences an outcome, rather than a black-box model. The lack of interpretability also raises concerns about bias and fairness. In predicting reproducible patterns from data, machine learning models inevitably include and reproduce biases embedded in these data, often in non-transparent ways. The fairness of machine learning 217 is heavily debated in applications ranging from the criminal justice system to hiring processes. Effective and responsible use of machine learning in the science of science therefore requires thoughtful partnership between humans and machines 53 to build a reliable system accessible to scrutiny and modification.

Causal approaches

The preceding methods can reveal core facts about the workings of science and develop predictive capacity. Yet, they fail to capture causal relationships, which are particularly useful in assessing policy interventions. For example, how can we test whether a science policy boosts or hinders the performance of individuals, teams or institutions? The overarching idea of causal approaches is to construct some counterfactual world where two groups are identical to each other except that one group experiences a treatment that the other group does not.

Towards causation

Before engaging in causal approaches, it is useful to first consider the interpretative challenges of observational data. As observational data emerge from mechanisms that are not fully known or measured, an observed correlation may be driven by underlying forces that were not accounted for in the analysis. This challenge makes causal inference fundamentally difficult in observational data. An awareness of this issue is the first step in confronting it. It further motivates intermediate empirical approaches, including the use of matching strategies and fixed effects, that can help to confront (although not fully eliminate) the inference challenge. We first consider these approaches before turning to more fully causal methods.

Matching. Matching utilizes rich information to construct a control group that is similar to the treatment group on as many observable characteristics as possible before the treatment group is exposed to the treatment. Inferences can then be made by comparing the treatment and the matched control groups. Exact matching applies to categorical values, such as country, gender, discipline or affiliation 35 , 218 . Coarsened exact matching considers percentile bins of continuous variables and matches observations in the same bin 133 . Propensity score matching estimates the probability of receiving the ‘treatment’ on the basis of the controlled variables and uses the estimates to match treatment and control groups, which reduces the matching task from comparing the values of multiple covariates to comparing a single value 24 , 219 . Dynamic matching is useful for longitudinally matching variables that change over time 220 , 221 .

Fixed effects. Fixed effects are a powerful and now standard tool in controlling for confounders. A key requirement for using fixed effects is that there are multiple observations on the same subject or entity (person, field, institution and so on) 222 , 223 , 224 . The fixed effect works as a dummy variable that accounts for the role of any fixed characteristic of that entity. Consider the finding where gender-diverse teams produce higher-impact papers than same-gender teams do 225 . A confounder may be that individuals who tend to write high-impact papers may also be more likely to work in gender-diverse teams. By including individual fixed effects, one accounts for any fixed characteristics of individuals (such as IQ, cultural background or previous education) that might drive the relationship of interest.

In sum, matching and fixed effects methods reduce potential sources of bias in interpreting relationships between variables. Yet, confounders may persist in these studies. For instance, fixed effects do not control for unobserved factors that change with time within the given entity (for example, access to funding or new skills). Identifying casual effects convincingly will then typically require distinct research methods that we turn to next.

Quasi-experiments

Researchers in economics and other fields have developed a range of quasi-experimental methods to construct treatment and control groups. The key idea here is exploiting randomness from external events that differentially expose subjects to a particular treatment. Here we review three quasi-experimental methods: difference-in-differences, instrumental variables and regression discontinuity (Fig. 3 ).

a – c , This figure presents illustrations of ( a ) differences-in-differences, ( b ) instrumental variables and ( c ) regression discontinuity methods. The solid line in b represents causal links and the dashed line represents the relationships that are not allowed, if the IV method is to produce causal inference.

Difference-in-differences. Difference-in-difference regression (DiD) investigates the effect of an unexpected event, comparing the affected group (the treated group) with an unaffected group (the control group). The control group is intended to provide the counterfactual path—what would have happened were it not for the unexpected event. Ideally, the treated and control groups are on virtually identical paths before the treatment event, but DiD can also work if the groups are on parallel paths (Fig. 3a ). For example, one study 226 examines how the premature death of superstar scientists affects the productivity of their previous collaborators. The control group are collaborators of superstars who did not die in the time frame. The two groups do not show significant differences in publications before a death event, yet upon the death of a star scientist, the treated collaborators on average experience a 5–8% decline in their quality-adjusted publication rates compared with the control group. DiD has wide applicability in the science of science, having been used to analyse the causal effects of grant design 24 , access costs to previous research 155 , 227 , university technology transfer policies 154 , intellectual property 228 , citation practices 229 , evolution of fields 221 and the impacts of paper retractions 230 , 231 , 232 . The DiD literature has grown especially rapidly in the field of economics, with substantial recent refinements 233 , 234 .

Instrumental variables. Another quasi-experimental approach utilizes ‘instrumental variables’ (IV). The goal is to determine the causal influence of some feature X on some outcome Y by using a third, instrumental variable. This instrumental variable is a quasi-random event that induces variation in X and, except for its impact through X , has no other effect on the outcome Y (Fig. 3b ). For example, consider a study of astronomy that seeks to understand how telescope time affects career advancement 235 . Here, one cannot simply look at the correlation between telescope time and career outcomes because many confounds (such as talent or grit) may influence both telescope time and career opportunities. Now consider the weather as an instrumental variable. Cloudy weather will, at random, reduce an astronomer’s observational time. Yet, the weather on particular nights is unlikely to correlate with a scientist’s innate qualities. The weather can then provide an instrumental variable to reveal a causal relationship between telescope time and career outcomes. Instrumental variables have been used to study local peer effects in research 151 , the impact of gender composition in scientific committees 236 , patents on future innovation 237 and taxes on inventor mobility 238 .

Regression discontinuity. In regression discontinuity, policies with an arbitrary threshold for receiving some benefit can be used to construct treatment and control groups (Fig. 3c ). Take the funding paylines for grant proposals as an example. Proposals with scores increasingly close to the payline are increasingly similar in their both observable and unobservable characteristics, yet only those projects with scores above the payline receive the funding. For example, a study 110 examines the effect of winning an early-career grant on the probability of winning a later, mid-career grant. The probability has a discontinuous jump across the initial grant’s payline, providing the treatment and control groups needed to estimate the causal effect of receiving a grant. This example utilizes the ‘sharp’ regression discontinuity that assumes treatment status to be fully determined by the cut-off. If we assume treatment status is only partly determined by the cut-off, we can use ‘fuzzy’ regression discontinuity designs. Here the probability of receiving a grant is used to estimate the future outcome 11 , 110 , 239 , 240 , 241 .

Although quasi-experiments are powerful tools, they face their own limitations. First, these approaches identify causal effects within a specific context and often engage small numbers of observations. How representative the samples are for broader populations or contexts is typically left as an open question. Second, the validity of the causal design is typically not ironclad. Researchers usually conduct different robustness checks to verify whether observable confounders have significant differences between the treated and control groups, before treatment. However, unobservable features may still differ between treatment and control groups. The quality of instrumental variables and the specific claim that they have no effect on the outcome except through the variable of interest, is also difficult to assess. Ultimately, researchers must rely partly on judgement to tell whether appropriate conditions are met for causal inference.

This section emphasized popular econometric approaches to causal inference. Other empirical approaches, such as graphical causal modelling 242 , 243 , also represent an important stream of work on assessing causal relationships. Such approaches usually represent causation as a directed acyclic graph, with nodes as variables and arrows between them as suspected causal relationships. In the science of science, the directed acyclic graph approach has been applied to quantify the causal effect of journal impact factor 244 and gender or racial bias 245 on citations. Graphical causal modelling has also triggered discussions on strengths and weaknesses compared to the econometrics methods 246 , 247 .

Experiments

In contrast to quasi-experimental approaches, laboratory and field experiments conduct direct randomization in assigning treatment and control groups. These methods engage explicitly in the data generation process, manipulating interventions to observe counterfactuals. These experiments are crafted to study mechanisms of specific interest and, by designing the experiment and formally randomizing, can produce especially rigorous causal inference.

Laboratory experiments. Laboratory experiments build counterfactual worlds in well-controlled laboratory environments. Researchers randomly assign participants to the treatment or control group and then manipulate the laboratory conditions to observe different outcomes in the two groups. For example, consider laboratory experiments on team performance and gender composition 144 , 248 . The researchers randomly assign participants into groups to perform tasks such as solving puzzles or brainstorming. Teams with a higher proportion of women are found to perform better on average, offering evidence that gender diversity is causally linked to team performance. Laboratory experiments can allow researchers to test forces that are otherwise hard to observe, such as how competition influences creativity 249 . Laboratory experiments have also been used to evaluate how journal impact factors shape scientists’ perceptions of rewards 250 and gender bias in hiring 251 .

Laboratory experiments allow for precise control of settings and procedures to isolate causal effects of interest. However, participants may behave differently in synthetic environments than in real-world settings, raising questions about the generalizability and replicability of the results 252 , 253 , 254 . To assess causal effects in real-world settings, researcher use randomized controlled trials.

Randomized controlled trials. A randomized controlled trial (RCT), or field experiment, is a staple for causal inference across a wide range of disciplines. RCTs randomly assign participants into the treatment and control conditions 255 and can be used not only to assess mechanisms but also to test real-world interventions such as policy change. The science of science has witnessed growing use of RCTs. For instance, a field experiment 146 investigated whether lower search costs for collaborators increased collaboration in grant applications. The authors randomly allocated principal investigators to face-to-face sessions in a medical school, and then measured participants’ chance of writing a grant proposal together. RCTs have also offered rich causal insights on peer review 256 , 257 , 258 , 259 , 260 and gender bias in science 261 , 262 , 263 .

While powerful, RCTs are difficult to conduct in the science of science, mainly for two reasons. The first concerns potential risks in a policy intervention. For instance, while randomizing funding across individuals could generate crucial causal insights for funders, it may also inadvertently harm participants’ careers 264 . Second, key questions in the science of science often require a long-time horizon to trace outcomes, which makes RCTs costly. It also raises the difficulty of replicating findings. A relative advantage of the quasi-experimental methods discussed earlier is that one can identify causal effects over potentially long periods of time in the historical record. On the other hand, quasi-experiments must be found as opposed to designed, and they often are not available for many questions of interest. While the best approaches are context dependent, a growing community of researchers is building platforms to facilitate RCTs for the science of science, aiming to lower their costs and increase their scale. Performing RCTs in partnership with science institutions can also contribute to timely, policy-relevant research that may substantially improve science decision-making and investments.

Research in the science of science has been empowered by the growth of high-scale data, new measurement approaches and an expanding range of empirical methods. These tools provide enormous capacity to test conceptual frameworks about science, discover factors impacting scientific productivity, predict key scientific outcomes and design policies that better facilitate future scientific progress. A careful appreciation of empirical techniques can help researchers to choose effective tools for questions of interest and propel the field. A better and broader understanding of these methodologies may also build bridges across diverse research communities, facilitating communication and collaboration, and better leveraging the value of diverse perspectives. The science of science is about turning scientific methods on the nature of science itself. The fruits of this work, with time, can guide researchers and research institutions to greater progress in discovery and understanding across the landscape of scientific inquiry.

Bush, V . S cience–the Endless Frontier: A Report to the President on a Program for Postwar Scientific Research (National Science Foundation, 1990).

Mokyr, J. The Gifts of Athena (Princeton Univ. Press, 2011).

Jones, B. F. in Rebuilding the Post-Pandemic Economy (eds Kearney, M. S. & Ganz, A.) 272–310 (Aspen Institute Press, 2021).

Wang, D. & Barabási, A.-L. The Science of Science (Cambridge Univ. Press, 2021).

Fortunato, S. et al. Science of science. Science 359 , eaao0185 (2018).

Article PubMed PubMed Central Google Scholar

Azoulay, P. et al. Toward a more scientific science. Science 361 , 1194–1197 (2018).

Article PubMed Google Scholar

Clauset, A., Larremore, D. B. & Sinatra, R. Data-driven predictions in the science of science. Science 355 , 477–480 (2017).

Article CAS PubMed Google Scholar

Zeng, A. et al. The science of science: from the perspective of complex systems. Phys. Rep. 714 , 1–73 (2017).

Article Google Scholar

Lin, Z., Yin. Y., Liu, L. & Wang, D. SciSciNet: a large-scale open data lake for the science of science research. Sci. Data, https://doi.org/10.1038/s41597-023-02198-9 (2023).

Ahmadpoor, M. & Jones, B. F. The dual frontier: patented inventions and prior scientific advance. Science 357 , 583–587 (2017).

Azoulay, P., Graff Zivin, J. S., Li, D. & Sampat, B. N. Public R&D investments and private-sector patenting: evidence from NIH funding rules. Rev. Econ. Stud. 86 , 117–152 (2019).

Yin, Y., Dong, Y., Wang, K., Wang, D. & Jones, B. F. Public use and public funding of science. Nat. Hum. Behav. 6 , 1344–1350 (2022).

Merton, R. K. The Sociology of Science: Theoretical and Empirical Investigations (Univ. Chicago Press, 1973).

Kuhn, T. The Structure of Scientific Revolutions (Princeton Univ. Press, 2021).

Uzzi, B., Mukherjee, S., Stringer, M. & Jones, B. Atypical combinations and scientific impact. Science 342 , 468–472 (2013).

Zuckerman, H. Scientific Elite: Nobel Laureates in the United States (Transaction Publishers, 1977).

Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566 , 378–382 (2019).

Wuchty, S., Jones, B. F. & Uzzi, B. The increasing dominance of teams in production of knowledge. Science 316 , 1036–1039 (2007).

Foster, J. G., Rzhetsky, A. & Evans, J. A. Tradition and innovation in scientists’ research strategies. Am. Sociol. Rev. 80 , 875–908 (2015).

Wang, D., Song, C. & Barabási, A.-L. Quantifying long-term scientific impact. Science 342 , 127–132 (2013).

Clauset, A., Arbesman, S. & Larremore, D. B. Systematic inequality and hierarchy in faculty hiring networks. Sci. Adv. 1 , e1400005 (2015).

Ma, A., Mondragón, R. J. & Latora, V. Anatomy of funded research in science. Proc. Natl Acad. Sci. USA 112 , 14760–14765 (2015).

Article CAS PubMed PubMed Central Google Scholar

Ma, Y. & Uzzi, B. Scientific prize network predicts who pushes the boundaries of science. Proc. Natl Acad. Sci. USA 115 , 12608–12615 (2018).

Azoulay, P., Graff Zivin, J. S. & Manso, G. Incentives and creativity: evidence from the academic life sciences. RAND J. Econ. 42 , 527–554 (2011).

Schor, S. & Karten, I. Statistical evaluation of medical journal manuscripts. JAMA 195 , 1123–1128 (1966).

Platt, J. R. Strong inference: certain systematic methods of scientific thinking may produce much more rapid progress than others. Science 146 , 347–353 (1964).

Ioannidis, J. P. Why most published research findings are false. PLoS Med. 2 , e124 (2005).

Simonton, D. K. Career landmarks in science: individual differences and interdisciplinary contrasts. Dev. Psychol. 27 , 119 (1991).

Way, S. F., Morgan, A. C., Clauset, A. & Larremore, D. B. The misleading narrative of the canonical faculty productivity trajectory. Proc. Natl Acad. Sci. USA 114 , E9216–E9223 (2017).

Sinatra, R., Wang, D., Deville, P., Song, C. & Barabási, A.-L. Quantifying the evolution of individual scientific impact. Science 354 , aaf5239 (2016).

Liu, L. et al. Hot streaks in artistic, cultural, and scientific careers. Nature 559 , 396–399 (2018).

Liu, L., Dehmamy, N., Chown, J., Giles, C. L. & Wang, D. Understanding the onset of hot streaks across artistic, cultural, and scientific careers. Nat. Commun. 12 , 5392 (2021).

Squazzoni, F. et al. Peer review and gender bias: a study on 145 scholarly journals. Sci. Adv. 7 , eabd0299 (2021).

Hofstra, B. et al. The diversity–innovation paradox in science. Proc. Natl Acad. Sci. USA 117 , 9284–9291 (2020).

Huang, J., Gates, A. J., Sinatra, R. & Barabási, A.-L. Historical comparison of gender inequality in scientific careers across countries and disciplines. Proc. Natl Acad. Sci. USA 117 , 4609–4616 (2020).

Gläser, J. & Laudel, G. Governing science: how science policy shapes research content. Eur. J. Sociol. 57 , 117–168 (2016).

Stephan, P. E. How Economics Shapes Science (Harvard Univ. Press, 2012).

Garfield, E. & Sher, I. H. New factors in the evaluation of scientific literature through citation indexing. Am. Doc. 14 , 195–201 (1963).

Article CAS Google Scholar

de Solla Price, D. J. Networks of scientific papers. Science 149 , 510–515 (1965).

Etzkowitz, H., Kemelgor, C. & Uzzi, B. Athena Unbound: The Advancement of Women in Science and Technology (Cambridge Univ. Press, 2000).

Simonton, D. K. Scientific Genius: A Psychology of Science (Cambridge Univ. Press, 1988).

Khabsa, M. & Giles, C. L. The number of scholarly documents on the public web. PLoS ONE 9 , e93949 (2014).

Xia, F., Wang, W., Bekele, T. M. & Liu, H. Big scholarly data: a survey. IEEE Trans. Big Data 3 , 18–35 (2017).

Evans, J. A. & Foster, J. G. Metaknowledge. Science 331 , 721–725 (2011).

Milojević, S. Quantifying the cognitive extent of science. J. Informetr. 9 , 962–973 (2015).

Rzhetsky, A., Foster, J. G., Foster, I. T. & Evans, J. A. Choosing experiments to accelerate collective discovery. Proc. Natl Acad. Sci. USA 112 , 14569–14574 (2015).

Poncela-Casasnovas, J., Gerlach, M., Aguirre, N. & Amaral, L. A. Large-scale analysis of micro-level citation patterns reveals nuanced selection criteria. Nat. Hum. Behav. 3 , 568–575 (2019).

Hardwicke, T. E. et al. Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition. R. Soc. Open Sci. 5 , 180448 (2018).

Nagaraj, A., Shears, E. & de Vaan, M. Improving data access democratizes and diversifies science. Proc. Natl Acad. Sci. USA 117 , 23490–23498 (2020).

Bravo, G., Grimaldo, F., López-Iñesta, E., Mehmani, B. & Squazzoni, F. The effect of publishing peer review reports on referee behavior in five scholarly journals. Nat. Commun. 10 , 322 (2019).

Tran, D. et al. An open review of open review: a critical analysis of the machine learning conference review process. Preprint at https://doi.org/10.48550/arXiv.2010.05137 (2020).

Tshitoyan, V. et al. Unsupervised word embeddings capture latent knowledge from materials science literature. Nature 571 , 95–98 (2019).

Yang, Y., Wu, Y. & Uzzi, B. Estimating the deep replicability of scientific findings using human and artificial intelligence. Proc. Natl Acad. Sci. USA 117 , 10762–10768 (2020).

Mukherjee, S., Uzzi, B., Jones, B. & Stringer, M. A new method for identifying recombinations of existing knowledge associated with high‐impact innovation. J. Prod. Innov. Manage. 33 , 224–236 (2016).

Leahey, E., Beckman, C. M. & Stanko, T. L. Prominent but less productive: the impact of interdisciplinarity on scientists’ research. Adm. Sci. Q. 62 , 105–139 (2017).

Sauermann, H. & Haeussler, C. Authorship and contribution disclosures. Sci. Adv. 3 , e1700404 (2017).

Oliveira, D. F. M., Ma, Y., Woodruff, T. K. & Uzzi, B. Comparison of National Institutes of Health grant amounts to first-time male and female principal investigators. JAMA 321 , 898–900 (2019).

Yang, Y., Chawla, N. V. & Uzzi, B. A network’s gender composition and communication pattern predict women’s leadership success. Proc. Natl Acad. Sci. USA 116 , 2033–2038 (2019).

Way, S. F., Larremore, D. B. & Clauset, A. Gender, productivity, and prestige in computer science faculty hiring networks. In Proc. 25th International Conference on World Wide Web 1169–1179. (ACM 2016)

Malmgren, R. D., Ottino, J. M. & Amaral, L. A. N. The role of mentorship in protege performance. Nature 465 , 622–626 (2010).

Ma, Y., Mukherjee, S. & Uzzi, B. Mentorship and protégé success in STEM fields. Proc. Natl Acad. Sci. USA 117 , 14077–14083 (2020).

Börner, K. et al. Skill discrepancies between research, education, and jobs reveal the critical need to supply soft skills for the data economy. Proc. Natl Acad. Sci. USA 115 , 12630–12637 (2018).

Biasi, B. & Ma, S. The Education-Innovation Gap (National Bureau of Economic Research Working papers, 2020).

Bornmann, L. Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics. J. Informetr. 8 , 895–903 (2014).

Cleary, E. G., Beierlein, J. M., Khanuja, N. S., McNamee, L. M. & Ledley, F. D. Contribution of NIH funding to new drug approvals 2010–2016. Proc. Natl Acad. Sci. USA 115 , 2329–2334 (2018).

Spector, J. M., Harrison, R. S. & Fishman, M. C. Fundamental science behind today’s important medicines. Sci. Transl. Med. 10 , eaaq1787 (2018).

Haunschild, R. & Bornmann, L. How many scientific papers are mentioned in policy-related documents? An empirical investigation using Web of Science and Altmetric data. Scientometrics 110 , 1209–1216 (2017).

Yin, Y., Gao, J., Jones, B. F. & Wang, D. Coevolution of policy and science during the pandemic. Science 371 , 128–130 (2021).

Sugimoto, C. R., Work, S., Larivière, V. & Haustein, S. Scholarly use of social media and altmetrics: a review of the literature. J. Assoc. Inf. Sci. Technol. 68 , 2037–2062 (2017).

Dunham, I. Human genes: time to follow the roads less traveled? PLoS Biol. 16 , e3000034 (2018).

Kustatscher, G. et al. Understudied proteins: opportunities and challenges for functional proteomics. Nat. Methods 19 , 774–779 (2022).

Rosenthal, R. The file drawer problem and tolerance for null results. Psychol. Bull. 86 , 638 (1979).

Franco, A., Malhotra, N. & Simonovits, G. Publication bias in the social sciences: unlocking the file drawer. Science 345 , 1502–1505 (2014).

Vera-Baceta, M.-A., Thelwall, M. & Kousha, K. Web of Science and Scopus language coverage. Scientometrics 121 , 1803–1813 (2019).

Waltman, L. A review of the literature on citation impact indicators. J. Informetr. 10 , 365–391 (2016).

Garfield, E. & Merton, R. K. Citation Indexing: Its Theory and Application in Science, Technology, and Humanities (Wiley, 1979).

Kelly, B., Papanikolaou, D., Seru, A. & Taddy, M. Measuring Technological Innovation Over the Long Run Report No. 0898-2937 (National Bureau of Economic Research, 2018).

Kogan, L., Papanikolaou, D., Seru, A. & Stoffman, N. Technological innovation, resource allocation, and growth. Q. J. Econ. 132 , 665–712 (2017).

Hall, B. H., Jaffe, A. & Trajtenberg, M. Market value and patent citations. RAND J. Econ. 36 , 16–38 (2005).

Google Scholar

Yan, E. & Ding, Y. Applying centrality measures to impact analysis: a coauthorship network analysis. J. Am. Soc. Inf. Sci. Technol. 60 , 2107–2118 (2009).

Radicchi, F., Fortunato, S., Markines, B. & Vespignani, A. Diffusion of scientific credits and the ranking of scientists. Phys. Rev. E 80 , 056103 (2009).

Bollen, J., Rodriquez, M. A. & Van de Sompel, H. Journal status. Scientometrics 69 , 669–687 (2006).

Bergstrom, C. T., West, J. D. & Wiseman, M. A. The eigenfactor™ metrics. J. Neurosci. 28 , 11433–11434 (2008).

Cronin, B. & Sugimoto, C. R. Beyond Bibliometrics: Harnessing Multidimensional Indicators of Scholarly Impact (MIT Press, 2014).

Hicks, D., Wouters, P., Waltman, L., De Rijcke, S. & Rafols, I. Bibliometrics: the Leiden Manifesto for research metrics. Nature 520 , 429–431 (2015).

Catalini, C., Lacetera, N. & Oettl, A. The incidence and role of negative citations in science. Proc. Natl Acad. Sci. USA 112 , 13823–13826 (2015).

Alcacer, J. & Gittelman, M. Patent citations as a measure of knowledge flows: the influence of examiner citations. Rev. Econ. Stat. 88 , 774–779 (2006).

Ding, Y. et al. Content‐based citation analysis: the next generation of citation analysis. J. Assoc. Inf. Sci. Technol. 65 , 1820–1833 (2014).

Teufel, S., Siddharthan, A. & Tidhar, D. Automatic classification of citation function. In Proc. 2006 Conference on Empirical Methods in Natural Language Processing, 103–110 (Association for Computational Linguistics 2006)

Seeber, M., Cattaneo, M., Meoli, M. & Malighetti, P. Self-citations as strategic response to the use of metrics for career decisions. Res. Policy 48 , 478–491 (2019).

Pendlebury, D. A. The use and misuse of journal metrics and other citation indicators. Arch. Immunol. Ther. Exp. 57 , 1–11 (2009).

Biagioli, M. Watch out for cheats in citation game. Nature 535 , 201 (2016).

Jo, W. S., Liu, L. & Wang, D. See further upon the giants: quantifying intellectual lineage in science. Quant. Sci. Stud. 3 , 319–330 (2022).

Boyack, K. W., Klavans, R. & Börner, K. Mapping the backbone of science. Scientometrics 64 , 351–374 (2005).

Gates, A. J., Ke, Q., Varol, O. & Barabási, A.-L. Nature’s reach: narrow work has broad impact. Nature 575 , 32–34 (2019).

Börner, K., Penumarthy, S., Meiss, M. & Ke, W. Mapping the diffusion of scholarly knowledge among major US research institutions. Scientometrics 68 , 415–426 (2006).

King, D. A. The scientific impact of nations. Nature 430 , 311–316 (2004).

Pan, R. K., Kaski, K. & Fortunato, S. World citation and collaboration networks: uncovering the role of geography in science. Sci. Rep. 2 , 902 (2012).

Jaffe, A. B., Trajtenberg, M. & Henderson, R. Geographic localization of knowledge spillovers as evidenced by patent citations. Q. J. Econ. 108 , 577–598 (1993).

Funk, R. J. & Owen-Smith, J. A dynamic network measure of technological change. Manage. Sci. 63 , 791–817 (2017).

Yegros-Yegros, A., Rafols, I. & D’este, P. Does interdisciplinary research lead to higher citation impact? The different effect of proximal and distal interdisciplinarity. PLoS ONE 10 , e0135095 (2015).

Larivière, V., Haustein, S. & Börner, K. Long-distance interdisciplinarity leads to higher scientific impact. PLoS ONE 10 , e0122565 (2015).

Fleming, L., Greene, H., Li, G., Marx, M. & Yao, D. Government-funded research increasingly fuels innovation. Science 364 , 1139–1141 (2019).

Bowen, A. & Casadevall, A. Increasing disparities between resource inputs and outcomes, as measured by certain health deliverables, in biomedical research. Proc. Natl Acad. Sci. USA 112 , 11335–11340 (2015).

Li, D., Azoulay, P. & Sampat, B. N. The applied value of public investments in biomedical research. Science 356 , 78–81 (2017).

Lehman, H. C. Age and Achievement (Princeton Univ. Press, 2017).

Simonton, D. K. Creative productivity: a predictive and explanatory model of career trajectories and landmarks. Psychol. Rev. 104 , 66 (1997).

Duch, J. et al. The possible role of resource requirements and academic career-choice risk on gender differences in publication rate and impact. PLoS ONE 7 , e51332 (2012).

Wang, Y., Jones, B. F. & Wang, D. Early-career setback and future career impact. Nat. Commun. 10 , 4331 (2019).

Bol, T., de Vaan, M. & van de Rijt, A. The Matthew effect in science funding. Proc. Natl Acad. Sci. USA 115 , 4887–4890 (2018).

Jones, B. F. Age and great invention. Rev. Econ. Stat. 92 , 1–14 (2010).

Newman, M. Networks (Oxford Univ. Press, 2018).

Mazloumian, A., Eom, Y.-H., Helbing, D., Lozano, S. & Fortunato, S. How citation boosts promote scientific paradigm shifts and nobel prizes. PLoS ONE 6 , e18975 (2011).

Hirsch, J. E. An index to quantify an individual’s scientific research output. Proc. Natl Acad. Sci. USA 102 , 16569–16572 (2005).

Alonso, S., Cabrerizo, F. J., Herrera-Viedma, E. & Herrera, F. h-index: a review focused in its variants, computation and standardization for different scientific fields. J. Informetr. 3 , 273–289 (2009).

Egghe, L. An improvement of the h-index: the g-index. ISSI Newsl. 2 , 8–9 (2006).

Kaur, J., Radicchi, F. & Menczer, F. Universality of scholarly impact metrics. J. Informetr. 7 , 924–932 (2013).

Majeti, D. et al. Scholar plot: design and evaluation of an information interface for faculty research performance. Front. Res. Metr. Anal. 4 , 6 (2020).

Sidiropoulos, A., Katsaros, D. & Manolopoulos, Y. Generalized Hirsch h-index for disclosing latent facts in citation networks. Scientometrics 72 , 253–280 (2007).

Jones, B. F. & Weinberg, B. A. Age dynamics in scientific creativity. Proc. Natl Acad. Sci. USA 108 , 18910–18914 (2011).

Dennis, W. Age and productivity among scientists. Science 123 , 724–725 (1956).

Sanyal, D. K., Bhowmick, P. K. & Das, P. P. A review of author name disambiguation techniques for the PubMed bibliographic database. J. Inf. Sci. 47 , 227–254 (2021).

Haak, L. L., Fenner, M., Paglione, L., Pentz, E. & Ratner, H. ORCID: a system to uniquely identify researchers. Learn. Publ. 25 , 259–264 (2012).

Malmgren, R. D., Ottino, J. M. & Amaral, L. A. N. The role of mentorship in protégé performance. Nature 465 , 662–667 (2010).

Oettl, A. Reconceptualizing stars: scientist helpfulness and peer performance. Manage. Sci. 58 , 1122–1140 (2012).

Morgan, A. C. et al. The unequal impact of parenthood in academia. Sci. Adv. 7 , eabd1996 (2021).

Morgan, A. C. et al. Socioeconomic roots of academic faculty. Nat. Hum. Behav. 6 , 1625–1633 (2022).

San Francisco Declaration on Research Assessment (DORA) (American Society for Cell Biology, 2012).

Falk‐Krzesinski, H. J. et al. Advancing the science of team science. Clin. Transl. Sci. 3 , 263–266 (2010).

Cooke, N. J. et al. Enhancing the Effectiveness of Team Science (National Academies Press, 2015).

Börner, K. et al. A multi-level systems perspective for the science of team science. Sci. Transl. Med. 2 , 49cm24 (2010).

Leahey, E. From sole investigator to team scientist: trends in the practice and study of research collaboration. Annu. Rev. Sociol. 42 , 81–100 (2016).

AlShebli, B. K., Rahwan, T. & Woon, W. L. The preeminence of ethnic diversity in scientific collaboration. Nat. Commun. 9 , 5163 (2018).

Hsiehchen, D., Espinoza, M. & Hsieh, A. Multinational teams and diseconomies of scale in collaborative research. Sci. Adv. 1 , e1500211 (2015).

Koning, R., Samila, S. & Ferguson, J.-P. Who do we invent for? Patents by women focus more on women’s health, but few women get to invent. Science 372 , 1345–1348 (2021).

Barabâsi, A.-L. et al. Evolution of the social network of scientific collaborations. Physica A 311 , 590–614 (2002).

Newman, M. E. Scientific collaboration networks. I. Network construction and fundamental results. Phys. Rev. E 64 , 016131 (2001).

Newman, M. E. Scientific collaboration networks. II. Shortest paths, weighted networks, and centrality. Phys. Rev. E 64 , 016132 (2001).

Palla, G., Barabási, A.-L. & Vicsek, T. Quantifying social group evolution. Nature 446 , 664–667 (2007).

Ross, M. B. et al. Women are credited less in science than men. Nature 608 , 135–145 (2022).

Shen, H.-W. & Barabási, A.-L. Collective credit allocation in science. Proc. Natl Acad. Sci. USA 111 , 12325–12330 (2014).

Merton, R. K. Matthew effect in science. Science 159 , 56–63 (1968).

Ni, C., Smith, E., Yuan, H., Larivière, V. & Sugimoto, C. R. The gendered nature of authorship. Sci. Adv. 7 , eabe4639 (2021).

Woolley, A. W., Chabris, C. F., Pentland, A., Hashmi, N. & Malone, T. W. Evidence for a collective intelligence factor in the performance of human groups. Science 330 , 686–688 (2010).

Feldon, D. F. et al. Postdocs’ lab engagement predicts trajectories of PhD students’ skill development. Proc. Natl Acad. Sci. USA 116 , 20910–20916 (2019).

Boudreau, K. J. et al. A field experiment on search costs and the formation of scientific collaborations. Rev. Econ. Stat. 99 , 565–576 (2017).

Holcombe, A. O. Contributorship, not authorship: use CRediT to indicate who did what. Publications 7 , 48 (2019).

Murray, D. et al. Unsupervised embedding of trajectories captures the latent structure of mobility. Preprint at https://doi.org/10.48550/arXiv.2012.02785 (2020).

Deville, P. et al. Career on the move: geography, stratification, and scientific impact. Sci. Rep. 4 , 4770 (2014).

Edmunds, L. D. et al. Why do women choose or reject careers in academic medicine? A narrative review of empirical evidence. Lancet 388 , 2948–2958 (2016).

Waldinger, F. Peer effects in science: evidence from the dismissal of scientists in Nazi Germany. Rev. Econ. Stud. 79 , 838–861 (2012).

Agrawal, A., McHale, J. & Oettl, A. How stars matter: recruiting and peer effects in evolutionary biology. Res. Policy 46 , 853–867 (2017).

Fiore, S. M. Interdisciplinarity as teamwork: how the science of teams can inform team science. Small Group Res. 39 , 251–277 (2008).

Hvide, H. K. & Jones, B. F. University innovation and the professor’s privilege. Am. Econ. Rev. 108 , 1860–1898 (2018).

Murray, F., Aghion, P., Dewatripont, M., Kolev, J. & Stern, S. Of mice and academics: examining the effect of openness on innovation. Am. Econ. J. Econ. Policy 8 , 212–252 (2016).

Radicchi, F., Fortunato, S. & Castellano, C. Universality of citation distributions: toward an objective measure of scientific impact. Proc. Natl Acad. Sci. USA 105 , 17268–17272 (2008).

Waltman, L., van Eck, N. J. & van Raan, A. F. Universality of citation distributions revisited. J. Am. Soc. Inf. Sci. Technol. 63 , 72–77 (2012).

Barabási, A.-L. & Albert, R. Emergence of scaling in random networks. Science 286 , 509–512 (1999).

de Solla Price, D. A general theory of bibliometric and other cumulative advantage processes. J. Am. Soc. Inf. Sci. 27 , 292–306 (1976).

Cole, S. Age and scientific performance. Am. J. Sociol. 84 , 958–977 (1979).

Ke, Q., Ferrara, E., Radicchi, F. & Flammini, A. Defining and identifying sleeping beauties in science. Proc. Natl Acad. Sci. USA 112 , 7426–7431 (2015).

Bornmann, L., de Moya Anegón, F. & Leydesdorff, L. Do scientific advancements lean on the shoulders of giants? A bibliometric investigation of the Ortega hypothesis. PLoS ONE 5 , e13327 (2010).

Mukherjee, S., Romero, D. M., Jones, B. & Uzzi, B. The nearly universal link between the age of past knowledge and tomorrow’s breakthroughs in science and technology: the hotspot. Sci. Adv. 3 , e1601315 (2017).

Packalen, M. & Bhattacharya, J. NIH funding and the pursuit of edge science. Proc. Natl Acad. Sci. USA 117 , 12011–12016 (2020).

Zeng, A., Fan, Y., Di, Z., Wang, Y. & Havlin, S. Fresh teams are associated with original and multidisciplinary research. Nat. Hum. Behav. 5 , 1314–1322 (2021).

Newman, M. E. The structure of scientific collaboration networks. Proc. Natl Acad. Sci. USA 98 , 404–409 (2001).

Larivière, V., Ni, C., Gingras, Y., Cronin, B. & Sugimoto, C. R. Bibliometrics: global gender disparities in science. Nature 504 , 211–213 (2013).

West, J. D., Jacquet, J., King, M. M., Correll, S. J. & Bergstrom, C. T. The role of gender in scholarly authorship. PLoS ONE 8 , e66212 (2013).

Gao, J., Yin, Y., Myers, K. R., Lakhani, K. R. & Wang, D. Potentially long-lasting effects of the pandemic on scientists. Nat. Commun. 12 , 6188 (2021).

Jones, B. F., Wuchty, S. & Uzzi, B. Multi-university research teams: shifting impact, geography, and stratification in science. Science 322 , 1259–1262 (2008).

Chu, J. S. & Evans, J. A. Slowed canonical progress in large fields of science. Proc. Natl Acad. Sci. USA 118 , e2021636118 (2021).

Wang, J., Veugelers, R. & Stephan, P. Bias against novelty in science: a cautionary tale for users of bibliometric indicators. Res. Policy 46 , 1416–1436 (2017).

Stringer, M. J., Sales-Pardo, M. & Amaral, L. A. Statistical validation of a global model for the distribution of the ultimate number of citations accrued by papers published in a scientific journal. J. Assoc. Inf. Sci. Technol. 61 , 1377–1385 (2010).

Bianconi, G. & Barabási, A.-L. Bose-Einstein condensation in complex networks. Phys. Rev. Lett. 86 , 5632 (2001).

Bianconi, G. & Barabási, A.-L. Competition and multiscaling in evolving networks. Europhys. Lett. 54 , 436 (2001).

Yin, Y. & Wang, D. The time dimension of science: connecting the past to the future. J. Informetr. 11 , 608–621 (2017).

Pan, R. K., Petersen, A. M., Pammolli, F. & Fortunato, S. The memory of science: Inflation, myopia, and the knowledge network. J. Informetr. 12 , 656–678 (2018).

Yin, Y., Wang, Y., Evans, J. A. & Wang, D. Quantifying the dynamics of failure across science, startups and security. Nature 575 , 190–194 (2019).

Candia, C. & Uzzi, B. Quantifying the selective forgetting and integration of ideas in science and technology. Am. Psychol. 76 , 1067 (2021).

Milojević, S. Principles of scientific research team formation and evolution. Proc. Natl Acad. Sci. USA 111 , 3984–3989 (2014).

Guimera, R., Uzzi, B., Spiro, J. & Amaral, L. A. N. Team assembly mechanisms determine collaboration network structure and team performance. Science 308 , 697–702 (2005).

Newman, M. E. Coauthorship networks and patterns of scientific collaboration. Proc. Natl Acad. Sci. USA 101 , 5200–5205 (2004).

Newman, M. E. Clustering and preferential attachment in growing networks. Phys. Rev. E 64 , 025102 (2001).

Iacopini, I., Milojević, S. & Latora, V. Network dynamics of innovation processes. Phys. Rev. Lett. 120 , 048301 (2018).

Kuhn, T., Perc, M. & Helbing, D. Inheritance patterns in citation networks reveal scientific memes. Phys. Rev. 4 , 041036 (2014).

Jia, T., Wang, D. & Szymanski, B. K. Quantifying patterns of research-interest evolution. Nat. Hum. Behav. 1 , 0078 (2017).

Zeng, A. et al. Increasing trend of scientists to switch between topics. Nat. Commun. https://doi.org/10.1038/s41467-019-11401-8 (2019).

Siudem, G., Żogała-Siudem, B., Cena, A. & Gagolewski, M. Three dimensions of scientific impact. Proc. Natl Acad. Sci. USA 117 , 13896–13900 (2020).

Petersen, A. M. et al. Reputation and impact in academic careers. Proc. Natl Acad. Sci. USA 111 , 15316–15321 (2014).

Jin, C., Song, C., Bjelland, J., Canright, G. & Wang, D. Emergence of scaling in complex substitutive systems. Nat. Hum. Behav. 3 , 837–846 (2019).

Hofman, J. M. et al. Integrating explanation and prediction in computational social science. Nature 595 , 181–188 (2021).

Lazer, D. et al. Computational social science. Science 323 , 721–723 (2009).

Lazer, D. M. et al. Computational social science: obstacles and opportunities. Science 369 , 1060–1062 (2020).

Albert, R. & Barabási, A.-L. Statistical mechanics of complex networks. Rev. Mod. Phys. 74 , 47 (2002).

Newman, M. E. The structure and function of complex networks. SIAM Rev. 45 , 167–256 (2003).

Song, C., Qu, Z., Blumm, N. & Barabási, A.-L. Limits of predictability in human mobility. Science 327 , 1018–1021 (2010).

Alessandretti, L., Aslak, U. & Lehmann, S. The scales of human mobility. Nature 587 , 402–407 (2020).

Pastor-Satorras, R. & Vespignani, A. Epidemic spreading in scale-free networks. Phys. Rev. Lett. 86 , 3200 (2001).

Pastor-Satorras, R., Castellano, C., Van Mieghem, P. & Vespignani, A. Epidemic processes in complex networks. Rev. Mod. Phys. 87 , 925 (2015).

Goodfellow, I., Bengio, Y. & Courville, A. Deep Learning (MIT Press, 2016).

Bishop, C. M. Pattern Recognition and Machine Learning (Springer, 2006).

Dong, Y., Johnson, R. A. & Chawla, N. V. Will this paper increase your h-index? Scientific impact prediction. In Proc. 8th ACM International Conference on Web Search and Data Mining, 149–158 (ACM 2015)

Xiao, S. et al. On modeling and predicting individual paper citation count over time. In IJCAI, 2676–2682 (IJCAI, 2016)

Fortunato, S. Community detection in graphs. Phys. Rep. 486 , 75–174 (2010).

Chen, C. Science mapping: a systematic review of the literature. J. Data Inf. Sci. 2 , 1–40 (2017).

CAS Google Scholar

Van Eck, N. J. & Waltman, L. Citation-based clustering of publications using CitNetExplorer and VOSviewer. Scientometrics 111 , 1053–1070 (2017).

LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521 , 436–444 (2015).

Senior, A. W. et al. Improved protein structure prediction using potentials from deep learning. Nature 577 , 706–710 (2020).

Krenn, M. & Zeilinger, A. Predicting research trends with semantic and neural networks with an application in quantum physics. Proc. Natl Acad. Sci. USA 117 , 1910–1916 (2020).

Iten, R., Metger, T., Wilming, H., Del Rio, L. & Renner, R. Discovering physical concepts with neural networks. Phys. Rev. Lett. 124 , 010508 (2020).

Guimerà, R. et al. A Bayesian machine scientist to aid in the solution of challenging scientific problems. Sci. Adv. 6 , eaav6971 (2020).

Segler, M. H., Preuss, M. & Waller, M. P. Planning chemical syntheses with deep neural networks and symbolic AI. Nature 555 , 604–610 (2018).

Ryu, J. Y., Kim, H. U. & Lee, S. Y. Deep learning improves prediction of drug–drug and drug–food interactions. Proc. Natl Acad. Sci. USA 115 , E4304–E4311 (2018).

Kermany, D. S. et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172 , 1122–1131.e9 (2018).

Peng, H., Ke, Q., Budak, C., Romero, D. M. & Ahn, Y.-Y. Neural embeddings of scholarly periodicals reveal complex disciplinary organizations. Sci. Adv. 7 , eabb9004 (2021).

Youyou, W., Yang, Y. & Uzzi, B. A discipline-wide investigation of the replicability of psychology papers over the past two decades. Proc. Natl Acad. Sci. USA 120 , e2208863120 (2023).

Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K. & Galstyan, A. A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR) 54 , 1–35 (2021).

Way, S. F., Morgan, A. C., Larremore, D. B. & Clauset, A. Productivity, prominence, and the effects of academic environment. Proc. Natl Acad. Sci. USA 116 , 10729–10733 (2019).

Li, W., Aste, T., Caccioli, F. & Livan, G. Early coauthorship with top scientists predicts success in academic careers. Nat. Commun. 10 , 5170 (2019).

Hendry, D. F., Pagan, A. R. & Sargan, J. D. Dynamic specification. Handb. Econ. 2 , 1023–1100 (1984).

Jin, C., Ma, Y. & Uzzi, B. Scientific prizes and the extraordinary growth of scientific topics. Nat. Commun. 12 , 5619 (2021).

Azoulay, P., Ganguli, I. & Zivin, J. G. The mobility of elite life scientists: professional and personal determinants. Res. Policy 46 , 573–590 (2017).

Slavova, K., Fosfuri, A. & De Castro, J. O. Learning by hiring: the effects of scientists’ inbound mobility on research performance in academia. Organ. Sci. 27 , 72–89 (2016).

Sarsons, H. Recognition for group work: gender differences in academia. Am. Econ. Rev. 107 , 141–145 (2017).

Campbell, L. G., Mehtani, S., Dozier, M. E. & Rinehart, J. Gender-heterogeneous working groups produce higher quality science. PLoS ONE 8 , e79147 (2013).

Azoulay, P., Graff Zivin, J. S. & Wang, J. Superstar extinction. Q. J. Econ. 125 , 549–589 (2010).

Furman, J. L. & Stern, S. Climbing atop the shoulders of giants: the impact of institutions on cumulative research. Am. Econ. Rev. 101 , 1933–1963 (2011).

Williams, H. L. Intellectual property rights and innovation: evidence from the human genome. J. Polit. Econ. 121 , 1–27 (2013).

Rubin, A. & Rubin, E. Systematic Bias in the Progress of Research. J. Polit. Econ. 129 , 2666–2719 (2021).

Lu, S. F., Jin, G. Z., Uzzi, B. & Jones, B. The retraction penalty: evidence from the Web of Science. Sci. Rep. 3 , 3146 (2013).

Jin, G. Z., Jones, B., Lu, S. F. & Uzzi, B. The reverse Matthew effect: consequences of retraction in scientific teams. Rev. Econ. Stat. 101 , 492–506 (2019).

Azoulay, P., Bonatti, A. & Krieger, J. L. The career effects of scandal: evidence from scientific retractions. Res. Policy 46 , 1552–1569 (2017).

Goodman-Bacon, A. Difference-in-differences with variation in treatment timing. J. Econ. 225 , 254–277 (2021).

Callaway, B. & Sant’Anna, P. H. Difference-in-differences with multiple time periods. J. Econ. 225 , 200–230 (2021).

Hill, R. Searching for Superstars: Research Risk and Talent Discovery in Astronomy Working Paper (Massachusetts Institute of Technology, 2019).

Bagues, M., Sylos-Labini, M. & Zinovyeva, N. Does the gender composition of scientific committees matter? Am. Econ. Rev. 107 , 1207–1238 (2017).

Sampat, B. & Williams, H. L. How do patents affect follow-on innovation? Evidence from the human genome. Am. Econ. Rev. 109 , 203–236 (2019).

Moretti, E. & Wilson, D. J. The effect of state taxes on the geographical location of top earners: evidence from star scientists. Am. Econ. Rev. 107 , 1858–1903 (2017).

Jacob, B. A. & Lefgren, L. The impact of research grant funding on scientific productivity. J. Public Econ. 95 , 1168–1177 (2011).

Li, D. Expertise versus bias in evaluation: evidence from the NIH. Am. Econ. J. Appl. Econ. 9 , 60–92 (2017).

Pearl, J. Causal diagrams for empirical research. Biometrika 82 , 669–688 (1995).

Pearl, J. & Mackenzie, D. The Book of Why: The New Science of Cause and Effect (Basic Books, 2018).

Traag, V. A. Inferring the causal effect of journals on citations. Quant. Sci. Stud. 2 , 496–504 (2021).

Traag, V. & Waltman, L. Causal foundations of bias, disparity and fairness. Preprint at https://doi.org/10.48550/arXiv.2207.13665 (2022).

Imbens, G. W. Potential outcome and directed acyclic graph approaches to causality: relevance for empirical practice in economics. J. Econ. Lit. 58 , 1129–1179 (2020).

Heckman, J. J. & Pinto, R. Causality and Econometrics (National Bureau of Economic Research, 2022).

Aggarwal, I., Woolley, A. W., Chabris, C. F. & Malone, T. W. The impact of cognitive style diversity on implicit learning in teams. Front. Psychol. 10 , 112 (2019).

Balietti, S., Goldstone, R. L. & Helbing, D. Peer review and competition in the Art Exhibition Game. Proc. Natl Acad. Sci. USA 113 , 8414–8419 (2016).

Paulus, F. M., Rademacher, L., Schäfer, T. A. J., Müller-Pinzler, L. & Krach, S. Journal impact factor shapes scientists’ reward signal in the prospect of publication. PLoS ONE 10 , e0142537 (2015).

Williams, W. M. & Ceci, S. J. National hiring experiments reveal 2:1 faculty preference for women on STEM tenure track. Proc. Natl Acad. Sci. USA 112 , 5360–5365 (2015).

Collaboration, O. S. Estimating the reproducibility of psychological science. Science 349 , aac4716 (2015).

Camerer, C. F. et al. Evaluating replicability of laboratory experiments in economics. Science 351 , 1433–1436 (2016).

Camerer, C. F. et al. Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nat. Hum. Behav. 2 , 637–644 (2018).

Duflo, E. & Banerjee, A. Handbook of Field Experiments (Elsevier, 2017).

Tomkins, A., Zhang, M. & Heavlin, W. D. Reviewer bias in single versus double-blind peer review. Proc. Natl Acad. Sci. USA 114 , 12708–12713 (2017).

Blank, R. M. The effects of double-blind versus single-blind reviewing: experimental evidence from the American Economic Review. Am. Econ. Rev. 81 , 1041–1067 (1991).

Boudreau, K. J., Guinan, E. C., Lakhani, K. R. & Riedl, C. Looking across and looking beyond the knowledge frontier: intellectual distance, novelty, and resource allocation in science. Manage. Sci. 62 , 2765–2783 (2016).

Lane, J. et al. When Do Experts Listen to Other Experts? The Role of Negative Information in Expert Evaluations for Novel Projects Working Paper #21-007 (Harvard Business School, 2020).

Teplitskiy, M. et al. Do Experts Listen to Other Experts? Field Experimental Evidence from Scientific Peer Review (Harvard Business School, 2019).

Moss-Racusin, C. A., Dovidio, J. F., Brescoll, V. L., Graham, M. J. & Handelsman, J. Science faculty’s subtle gender biases favor male students. Proc. Natl Acad. Sci. USA 109 , 16474–16479 (2012).

Forscher, P. S., Cox, W. T., Brauer, M. & Devine, P. G. Little race or gender bias in an experiment of initial review of NIH R01 grant proposals. Nat. Hum. Behav. 3 , 257–264 (2019).

Dennehy, T. C. & Dasgupta, N. Female peer mentors early in college increase women’s positive academic experiences and retention in engineering. Proc. Natl Acad. Sci. USA 114 , 5964–5969 (2017).

Azoulay, P. Turn the scientific method on ourselves. Nature 484 , 31–32 (2012).

Download references

Acknowledgements

The authors thank all members of the Center for Science of Science and Innovation (CSSI) for invaluable comments. This work was supported by the Air Force Office of Scientific Research under award number FA9550-19-1-0354, National Science Foundation grant SBE 1829344, and the Alfred P. Sloan Foundation G-2019-12485.

Author information

Authors and affiliations.

Center for Science of Science and Innovation, Northwestern University, Evanston, IL, USA

Lu Liu, Benjamin F. Jones, Brian Uzzi & Dashun Wang

Northwestern Institute on Complex Systems, Northwestern University, Evanston, IL, USA

Kellogg School of Management, Northwestern University, Evanston, IL, USA

College of Information Sciences and Technology, Pennsylvania State University, University Park, PA, USA

National Bureau of Economic Research, Cambridge, MA, USA

Benjamin F. Jones

Brookings Institution, Washington, DC, USA

McCormick School of Engineering, Northwestern University, Evanston, IL, USA

Dashun Wang

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dashun Wang .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Nature Human Behaviour thanks Ludo Waltman, Erin Leahey and Sarah Bratt for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cite this article.

Liu, L., Jones, B.F., Uzzi, B. et al. Data, measurement and empirical methods in the science of science. Nat Hum Behav 7 , 1046–1058 (2023). https://doi.org/10.1038/s41562-023-01562-4

Download citation

Received : 30 June 2022

Accepted : 17 February 2023

Published : 01 June 2023

Issue Date : July 2023

DOI : https://doi.org/10.1038/s41562-023-01562-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Publication, funding, and experimental data in support of human reference atlas construction and usage.

Yongxin Kong
Katy Börner

Scientific Data (2024)

Quantifying attrition in science: a cohort-based, longitudinal study of scientists in 38 OECD countries

Marek Kwiek
Lukasz Szymula

Higher Education (2024)

Rescaling the disruption index reveals the universality of disruption distributions in science

Alex J. Yang
Hongcun Gong
Sanhong Deng

Scientometrics (2024)

Signs of Dysconscious Racism and Xenophobiaism in Knowledge Production and the Formation of Academic Researchers: A National Study

Dina Zoe Belluigi

Journal of Academic Ethics (2024)

Scientific Data (2023)

Quick links

Explore articles by subject
Guide to authors
Editorial policies

University of Memphis Libraries
Research Guides

Empirical Research: Defining, Identifying, & Finding

Defining empirical research, what is empirical research, quantitative or qualitative.

Introduction
Database Tools
Search Terms
Image Descriptions

Calfee & Chambliss (2005) (UofM login required) describe empirical research as a "systematic approach for answering certain types of questions." Those questions are answered "[t]hrough the collection of evidence under carefully defined and replicable conditions" (p. 43).

The evidence collected during empirical research is often referred to as "data."

Characteristics of Empirical Research

Emerald Publishing's guide to conducting empirical research identifies a number of common elements to empirical research:

A research question , which will determine research objectives.
A particular and planned design for the research, which will depend on the question and which will find ways of answering it with appropriate use of resources.
The gathering of primary data , which is then analysed.
A particular methodology for collecting and analysing the data, such as an experiment or survey.
The limitation of the data to a particular group, area or time scale, known as a sample [emphasis added]: for example, a specific number of employees of a particular company type, or all users of a library over a given time scale. The sample should be somehow representative of a wider population.
The ability to recreate the study and test the results. This is known as reliability .
The ability to generalize from the findings to a larger sample and to other situations.

If you see these elements in a research article, you can feel confident that you have found empirical research. Emerald's guide goes into more detail on each element.

Empirical research methodologies can be described as quantitative, qualitative, or a mix of both (usually called mixed-methods).

Ruane (2016) (UofM login required) gets at the basic differences in approach between quantitative and qualitative research:

Quantitative research -- an approach to documenting reality that relies heavily on numbers both for the measurement of variables and for data analysis (p. 33).
Qualitative research -- an approach to documenting reality that relies on words and images as the primary data source (p. 33).

Both quantitative and qualitative methods are empirical . If you can recognize that a research study is quantitative or qualitative study, then you have also recognized that it is empirical study.

Below are information on the characteristics of quantitative and qualitative research. This video from Scribbr also offers a good overall introduction to the two approaches to research methodology:

Characteristics of Quantitative Research

Researchers test hypotheses, or theories, based in assumptions about causality, i.e. we expect variable X to cause variable Y. Variables have to be controlled as much as possible to ensure validity. The results explain the relationship between the variables. Measures are based in pre-defined instruments.

Examples: experimental or quasi-experimental design, pretest & post-test, survey or questionnaire with closed-ended questions. Studies that identify factors that influence an outcomes, the utility of an intervention, or understanding predictors of outcomes.

Characteristics of Qualitative Research

Researchers explore “meaning individuals or groups ascribe to social or human problems (Creswell & Creswell, 2018, p3).” Questions and procedures emerge rather than being prescribed. Complexity, nuance, and individual meaning are valued. Research is both inductive and deductive. Data sources are multiple and varied, i.e. interviews, observations, documents, photographs, etc. The researcher is a key instrument and must be reflective of their background, culture, and experiences as influential of the research.

Examples: open question interviews and surveys, focus groups, case studies, grounded theory, ethnography, discourse analysis, narrative, phenomenology, participatory action research.

Calfee, R. C. & Chambliss, M. (2005). The design of empirical research. In J. Flood, D. Lapp, J. R. Squire, & J. Jensen (Eds.), Methods of research on teaching the English language arts: The methodology chapters from the handbook of research on teaching the English language arts (pp. 43-78). Routledge. http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=nlebk&AN=125955&site=eds-live&scope=site .

Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Thousand Oaks: Sage.

How to... conduct empirical research . (n.d.). Emerald Publishing. https://www.emeraldgrouppublishing.com/how-to/research-methods/conduct-empirical-research .

Scribbr. (2019). Quantitative vs. qualitative: The differences explained [video]. YouTube. https://www.youtube.com/watch?v=a-XtVF7Bofg .

Ruane, J. M. (2016). Introducing social research methods : Essentials for getting the edge . Wiley-Blackwell. http://ezproxy.memphis.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=nlebk&AN=1107215&site=eds-live&scope=site .

<< Previous: Home
Next: Identifying Empirical Research >>
Last Updated: Apr 2, 2024 11:25 AM
URL: https://libguides.memphis.edu/empirical-research

Ask a Librarian

Research: Overview & Approaches

Getting Started with Undergraduate Research
Planning & Getting Started
Building Your Knowledge Base
Locating Sources
Reading Scholarly Articles
Creating a Literature Review
Productivity & Organizing Research
Scholarly and Professional Relationships

Introduction to Empirical Research

Databases for finding empirical research, guided search, google scholar, examples of empirical research, sources and further reading.

Interpretive Research
Action-Based Research
Creative & Experimental Approaches

Your Librarian

Introductory Video This video covers what empirical research is, what kinds of questions and methods empirical researchers use, and some tips for finding empirical research articles in your discipline.

Guided Search: Finding Empirical Research Articles This is a hands-on tutorial that will allow you to use your own search terms to find resources.

Study on radiation transfer in human skin for cosmetics
Long-Term Mobile Phone Use and the Risk of Vestibular Schwannoma: A Danish Nationwide Cohort Study
Emissions Impacts and Benefits of Plug-In Hybrid Electric Vehicles and Vehicle-to-Grid Services
Review of design considerations and technological challenges for successful development and deployment of plug-in hybrid electric vehicles
Endocrine disrupters and human health: could oestrogenic chemicals in body care cosmetics adversely affect breast cancer incidence in women?

<< Previous: Scholarly and Professional Relationships
Next: Interpretive Research >>
Last Updated: Aug 13, 2024 12:18 PM
URL: https://guides.lib.purdue.edu/research_approaches

Penn State University Libraries

Empirical research in the social sciences and education.

What is Empirical Research and How to Read It
Finding Empirical Research in Library Databases
Designing Empirical Research
Ethics, Cultural Responsiveness, and Anti-Racism in Research
Citing, Writing, and Presenting Your Work

Contact the Librarian at your campus for more help!

Introduction: What is Empirical Research?

Empirical research is based on observed and measured phenomena and derives knowledge from actual experience rather than from theory or belief.

How do you know if a study is empirical? Read the subheadings within the article, book, or report and look for a description of the research "methodology." Ask yourself: Could I recreate this study and test these results?

Key characteristics to look for:

Specific research questions to be answered
Definition of the population, behavior, or phenomena being studied
Description of the process used to study this population or phenomena, including selection criteria, controls, and testing instruments (such as surveys)

Another hint: some scholarly journals use a specific layout, called the "IMRaD" format, to communicate empirical research findings. Such articles typically have 4 components:

Introduction: sometimes called "literature review" -- what is currently known about the topic -- usually includes a theoretical framework and/or discussion of previous studies
Methodology: sometimes called "research design" -- how to recreate the study -- usually describes the population, research process, and analytical tools used in the present study
Results: sometimes called "findings" -- what was learned through the study -- usually appears as statistical data or as substantial quotations from research participants
Discussion: sometimes called "conclusion" or "implications" -- why the study is important -- usually describes how the research results influence professional practices or future studies

Reading and Evaluating Scholarly Materials

Reading research can be a challenge. However, the tutorials and videos below can help. They explain what scholarly articles look like, how to read them, and how to evaluate them:

CRAAP Checklist A frequently-used checklist that helps you examine the currency, relevance, authority, accuracy, and purpose of an information source.
IF I APPLY A newer model of evaluating sources which encourages you to think about your own biases as a reader, as well as concerns about the item you are reading.
Credo Video: How to Read Scholarly Materials (4 min.)
Credo Tutorial: How to Read Scholarly Materials
Credo Tutorial: Evaluating Information
Credo Video: Evaluating Statistics (4 min.)
Credo Tutorial: Evaluating for Diverse Points of View
Next: Finding Empirical Research in Library Databases >>
Last Updated: Aug 13, 2024 3:16 PM
URL: https://guides.libraries.psu.edu/emp

What is Empirical Research? Definition, Methods, Examples

Appinio Research · 09.02.2024 · 36min read

Ever wondered how we gather the facts, unveil hidden truths, and make informed decisions in a world filled with questions? Empirical research holds the key.

In this guide, we'll delve deep into the art and science of empirical research, unraveling its methods, mysteries, and manifold applications. From defining the core principles to mastering data analysis and reporting findings, we're here to equip you with the knowledge and tools to navigate the empirical landscape.

What is Empirical Research?

Empirical research is the cornerstone of scientific inquiry, providing a systematic and structured approach to investigating the world around us. It is the process of gathering and analyzing empirical or observable data to test hypotheses, answer research questions, or gain insights into various phenomena. This form of research relies on evidence derived from direct observation or experimentation, allowing researchers to draw conclusions based on real-world data rather than purely theoretical or speculative reasoning.

Characteristics of Empirical Research

Empirical research is characterized by several key features:

Observation and Measurement : It involves the systematic observation or measurement of variables, events, or behaviors.
Data Collection : Researchers collect data through various methods, such as surveys, experiments, observations, or interviews.
Testable Hypotheses : Empirical research often starts with testable hypotheses that are evaluated using collected data.
Quantitative or Qualitative Data : Data can be quantitative (numerical) or qualitative (non-numerical), depending on the research design.
Statistical Analysis : Quantitative data often undergo statistical analysis to determine patterns , relationships, or significance.
Objectivity and Replicability : Empirical research strives for objectivity, minimizing researcher bias . It should be replicable, allowing other researchers to conduct the same study to verify results.
Conclusions and Generalizations : Empirical research generates findings based on data and aims to make generalizations about larger populations or phenomena.

Importance of Empirical Research

Empirical research plays a pivotal role in advancing knowledge across various disciplines. Its importance extends to academia, industry, and society as a whole. Here are several reasons why empirical research is essential:

Evidence-Based Knowledge : Empirical research provides a solid foundation of evidence-based knowledge. It enables us to test hypotheses, confirm or refute theories, and build a robust understanding of the world.
Scientific Progress : In the scientific community, empirical research fuels progress by expanding the boundaries of existing knowledge. It contributes to the development of theories and the formulation of new research questions.
Problem Solving : Empirical research is instrumental in addressing real-world problems and challenges. It offers insights and data-driven solutions to complex issues in fields like healthcare, economics, and environmental science.
Informed Decision-Making : In policymaking, business, and healthcare, empirical research informs decision-makers by providing data-driven insights. It guides strategies, investments, and policies for optimal outcomes.
Quality Assurance : Empirical research is essential for quality assurance and validation in various industries, including pharmaceuticals, manufacturing, and technology. It ensures that products and processes meet established standards.
Continuous Improvement : Businesses and organizations use empirical research to evaluate performance, customer satisfaction , and product effectiveness. This data-driven approach fosters continuous improvement and innovation.
Human Advancement : Empirical research in fields like medicine and psychology contributes to the betterment of human health and well-being. It leads to medical breakthroughs, improved therapies, and enhanced psychological interventions.
Critical Thinking and Problem Solving : Engaging in empirical research fosters critical thinking skills, problem-solving abilities, and a deep appreciation for evidence-based decision-making.

Empirical research empowers us to explore, understand, and improve the world around us. It forms the bedrock of scientific inquiry and drives progress in countless domains, shaping our understanding of both the natural and social sciences.

How to Conduct Empirical Research?

So, you've decided to dive into the world of empirical research. Let's begin by exploring the crucial steps involved in getting started with your research project.

1. Select a Research Topic

Selecting the right research topic is the cornerstone of a successful empirical study. It's essential to choose a topic that not only piques your interest but also aligns with your research goals and objectives. Here's how to go about it:

Identify Your Interests : Start by reflecting on your passions and interests. What topics fascinate you the most? Your enthusiasm will be your driving force throughout the research process.
Brainstorm Ideas : Engage in brainstorming sessions to generate potential research topics. Consider the questions you've always wanted to answer or the issues that intrigue you.
Relevance and Significance : Assess the relevance and significance of your chosen topic. Does it contribute to existing knowledge? Is it a pressing issue in your field of study or the broader community?
Feasibility : Evaluate the feasibility of your research topic. Do you have access to the necessary resources, data, and participants (if applicable)?

2. Formulate Research Questions

Once you've narrowed down your research topic, the next step is to formulate clear and precise research questions . These questions will guide your entire research process and shape your study's direction. To create effective research questions:

Specificity : Ensure that your research questions are specific and focused. Vague or overly broad questions can lead to inconclusive results.
Relevance : Your research questions should directly relate to your chosen topic. They should address gaps in knowledge or contribute to solving a particular problem.
Testability : Ensure that your questions are testable through empirical methods. You should be able to gather data and analyze it to answer these questions.
Avoid Bias : Craft your questions in a way that avoids leading or biased language. Maintain neutrality to uphold the integrity of your research.

3. Review Existing Literature

Before you embark on your empirical research journey, it's essential to immerse yourself in the existing body of literature related to your chosen topic. This step, often referred to as a literature review, serves several purposes:

Contextualization : Understand the historical context and current state of research in your field. What have previous studies found, and what questions remain unanswered?
Identifying Gaps : Identify gaps or areas where existing research falls short. These gaps will help you formulate meaningful research questions and hypotheses.
Theory Development : If your study is theoretical, consider how existing theories apply to your topic. If it's empirical, understand how previous studies have approached data collection and analysis.
Methodological Insights : Learn from the methodologies employed in previous research. What methods were successful, and what challenges did researchers face?

4. Define Variables

Variables are fundamental components of empirical research. They are the factors or characteristics that can change or be manipulated during your study. Properly defining and categorizing variables is crucial for the clarity and validity of your research. Here's what you need to know:

Independent Variables : These are the variables that you, as the researcher, manipulate or control. They are the "cause" in cause-and-effect relationships.
Dependent Variables : Dependent variables are the outcomes or responses that you measure or observe. They are the "effect" influenced by changes in independent variables.
Operational Definitions : To ensure consistency and clarity, provide operational definitions for your variables. Specify how you will measure or manipulate each variable.
Control Variables : In some studies, controlling for other variables that may influence your dependent variable is essential. These are known as control variables.

Understanding these foundational aspects of empirical research will set a solid foundation for the rest of your journey. Now that you've grasped the essentials of getting started, let's delve deeper into the intricacies of research design.

Empirical Research Design

Now that you've selected your research topic, formulated research questions, and defined your variables, it's time to delve into the heart of your empirical research journey – research design . This pivotal step determines how you will collect data and what methods you'll employ to answer your research questions. Let's explore the various facets of research design in detail.

Types of Empirical Research

Empirical research can take on several forms, each with its own unique approach and methodologies. Understanding the different types of empirical research will help you choose the most suitable design for your study. Here are some common types:

Experimental Research : In this type, researchers manipulate one or more independent variables to observe their impact on dependent variables. It's highly controlled and often conducted in a laboratory setting.
Observational Research : Observational research involves the systematic observation of subjects or phenomena without intervention. Researchers are passive observers, documenting behaviors, events, or patterns.
Survey Research : Surveys are used to collect data through structured questionnaires or interviews. This method is efficient for gathering information from a large number of participants.
Case Study Research : Case studies focus on in-depth exploration of one or a few cases. Researchers gather detailed information through various sources such as interviews, documents, and observations.
Qualitative Research : Qualitative research aims to understand behaviors, experiences, and opinions in depth. It often involves open-ended questions, interviews, and thematic analysis.
Quantitative Research : Quantitative research collects numerical data and relies on statistical analysis to draw conclusions. It involves structured questionnaires, experiments, and surveys.

Your choice of research type should align with your research questions and objectives. Experimental research, for example, is ideal for testing cause-and-effect relationships, while qualitative research is more suitable for exploring complex phenomena.

Experimental Design

Experimental research is a systematic approach to studying causal relationships. It's characterized by the manipulation of one or more independent variables while controlling for other factors. Here are some key aspects of experimental design:

Control and Experimental Groups : Participants are randomly assigned to either a control group or an experimental group. The independent variable is manipulated for the experimental group but not for the control group.
Randomization : Randomization is crucial to eliminate bias in group assignment. It ensures that each participant has an equal chance of being in either group.
Hypothesis Testing : Experimental research often involves hypothesis testing. Researchers formulate hypotheses about the expected effects of the independent variable and use statistical analysis to test these hypotheses.

Observational Design

Observational research entails careful and systematic observation of subjects or phenomena. It's advantageous when you want to understand natural behaviors or events. Key aspects of observational design include:

Participant Observation : Researchers immerse themselves in the environment they are studying. They become part of the group being observed, allowing for a deep understanding of behaviors.
Non-Participant Observation : In non-participant observation, researchers remain separate from the subjects. They observe and document behaviors without direct involvement.
Data Collection Methods : Observational research can involve various data collection methods, such as field notes, video recordings, photographs, or coding of observed behaviors.

Survey Design

Surveys are a popular choice for collecting data from a large number of participants. Effective survey design is essential to ensure the validity and reliability of your data. Consider the following:

Questionnaire Design : Create clear and concise questions that are easy for participants to understand. Avoid leading or biased questions.
Sampling Methods : Decide on the appropriate sampling method for your study, whether it's random, stratified, or convenience sampling.
Data Collection Tools : Choose the right tools for data collection, whether it's paper surveys, online questionnaires, or face-to-face interviews.

Case Study Design

Case studies are an in-depth exploration of one or a few cases to gain a deep understanding of a particular phenomenon. Key aspects of case study design include:

Single Case vs. Multiple Case Studies : Decide whether you'll focus on a single case or multiple cases. Single case studies are intensive and allow for detailed examination, while multiple case studies provide comparative insights.
Data Collection Methods : Gather data through interviews, observations, document analysis, or a combination of these methods.

Qualitative vs. Quantitative Research

In empirical research, you'll often encounter the distinction between qualitative and quantitative research . Here's a closer look at these two approaches:

Qualitative Research : Qualitative research seeks an in-depth understanding of human behavior, experiences, and perspectives. It involves open-ended questions, interviews, and the analysis of textual or narrative data. Qualitative research is exploratory and often used when the research question is complex and requires a nuanced understanding.
Quantitative Research : Quantitative research collects numerical data and employs statistical analysis to draw conclusions. It involves structured questionnaires, experiments, and surveys. Quantitative research is ideal for testing hypotheses and establishing cause-and-effect relationships.

Understanding the various research design options is crucial in determining the most appropriate approach for your study. Your choice should align with your research questions, objectives, and the nature of the phenomenon you're investigating.

Data Collection for Empirical Research

Now that you've established your research design, it's time to roll up your sleeves and collect the data that will fuel your empirical research. Effective data collection is essential for obtaining accurate and reliable results.

Sampling Methods

Sampling methods are critical in empirical research, as they determine the subset of individuals or elements from your target population that you will study. Here are some standard sampling methods:

Random Sampling : Random sampling ensures that every member of the population has an equal chance of being selected. It minimizes bias and is often used in quantitative research.
Stratified Sampling : Stratified sampling involves dividing the population into subgroups or strata based on specific characteristics (e.g., age, gender, location). Samples are then randomly selected from each stratum, ensuring representation of all subgroups.
Convenience Sampling : Convenience sampling involves selecting participants who are readily available or easily accessible. While it's convenient, it may introduce bias and limit the generalizability of results.
Snowball Sampling : Snowball sampling is instrumental when studying hard-to-reach or hidden populations. One participant leads you to another, creating a "snowball" effect. This method is common in qualitative research.
Purposive Sampling : In purposive sampling, researchers deliberately select participants who meet specific criteria relevant to their research questions. It's often used in qualitative studies to gather in-depth information.

The choice of sampling method depends on the nature of your research, available resources, and the degree of precision required. It's crucial to carefully consider your sampling strategy to ensure that your sample accurately represents your target population.

Data Collection Instruments

Data collection instruments are the tools you use to gather information from your participants or sources. These instruments should be designed to capture the data you need accurately. Here are some popular data collection instruments:

Questionnaires : Questionnaires consist of structured questions with predefined response options. When designing questionnaires, consider the clarity of questions, the order of questions, and the response format (e.g., Likert scale , multiple-choice).
Interviews : Interviews involve direct communication between the researcher and participants. They can be structured (with predetermined questions) or unstructured (open-ended). Effective interviews require active listening and probing for deeper insights.
Observations : Observations entail systematically and objectively recording behaviors, events, or phenomena. Researchers must establish clear criteria for what to observe, how to record observations, and when to observe.
Surveys : Surveys are a common data collection instrument for quantitative research. They can be administered through various means, including online surveys, paper surveys, and telephone surveys.
Documents and Archives : In some cases, data may be collected from existing documents, records, or archives. Ensure that the sources are reliable, relevant, and properly documented.

To streamline your process and gather insights with precision and efficiency, consider leveraging innovative tools like Appinio . With Appinio's intuitive platform, you can harness the power of real-time consumer data to inform your research decisions effectively. Whether you're conducting surveys, interviews, or observations, Appinio empowers you to define your target audience, collect data from diverse demographics, and analyze results seamlessly.

By incorporating Appinio into your data collection toolkit, you can unlock a world of possibilities and elevate the impact of your empirical research. Ready to revolutionize your approach to data collection?

Book a Demo

Data Collection Procedures

Data collection procedures outline the step-by-step process for gathering data. These procedures should be meticulously planned and executed to maintain the integrity of your research.

Training : If you have a research team, ensure that they are trained in data collection methods and protocols. Consistency in data collection is crucial.
Pilot Testing : Before launching your data collection, conduct a pilot test with a small group to identify any potential problems with your instruments or procedures. Make necessary adjustments based on feedback.
Data Recording : Establish a systematic method for recording data. This may include timestamps, codes, or identifiers for each data point.
Data Security : Safeguard the confidentiality and security of collected data. Ensure that only authorized individuals have access to the data.
Data Storage : Properly organize and store your data in a secure location, whether in physical or digital form. Back up data to prevent loss.

Ethical Considerations

Ethical considerations are paramount in empirical research, as they ensure the well-being and rights of participants are protected.

Informed Consent : Obtain informed consent from participants, providing clear information about the research purpose, procedures, risks, and their right to withdraw at any time.
Privacy and Confidentiality : Protect the privacy and confidentiality of participants. Ensure that data is anonymized and sensitive information is kept confidential.
Beneficence : Ensure that your research benefits participants and society while minimizing harm. Consider the potential risks and benefits of your study.
Honesty and Integrity : Conduct research with honesty and integrity. Report findings accurately and transparently, even if they are not what you expected.
Respect for Participants : Treat participants with respect, dignity, and sensitivity to cultural differences. Avoid any form of coercion or manipulation.
Institutional Review Board (IRB) : If required, seek approval from an IRB or ethics committee before conducting your research, particularly when working with human participants.

Adhering to ethical guidelines is not only essential for the ethical conduct of research but also crucial for the credibility and validity of your study. Ethical research practices build trust between researchers and participants and contribute to the advancement of knowledge with integrity.

With a solid understanding of data collection, including sampling methods, instruments, procedures, and ethical considerations, you are now well-equipped to gather the data needed to answer your research questions.

Empirical Research Data Analysis

Now comes the exciting phase of data analysis, where the raw data you've diligently collected starts to yield insights and answers to your research questions. We will explore the various aspects of data analysis, from preparing your data to drawing meaningful conclusions through statistics and visualization.

Data Preparation

Data preparation is the crucial first step in data analysis. It involves cleaning, organizing, and transforming your raw data into a format that is ready for analysis. Effective data preparation ensures the accuracy and reliability of your results.

Data Cleaning : Identify and rectify errors, missing values, and inconsistencies in your dataset. This may involve correcting typos, removing outliers, and imputing missing data.
Data Coding : Assign numerical values or codes to categorical variables to make them suitable for statistical analysis. For example, converting "Yes" and "No" to 1 and 0.
Data Transformation : Transform variables as needed to meet the assumptions of the statistical tests you plan to use. Common transformations include logarithmic or square root transformations.
Data Integration : If your data comes from multiple sources, integrate it into a unified dataset, ensuring that variables match and align.
Data Documentation : Maintain clear documentation of all data preparation steps, as well as the rationale behind each decision. This transparency is essential for replicability.

Effective data preparation lays the foundation for accurate and meaningful analysis. It allows you to trust the results that will follow in the subsequent stages.

Descriptive Statistics

Descriptive statistics help you summarize and make sense of your data by providing a clear overview of its key characteristics. These statistics are essential for understanding the central tendencies, variability, and distribution of your variables. Descriptive statistics include:

Measures of Central Tendency : These include the mean (average), median (middle value), and mode (most frequent value). They help you understand the typical or central value of your data.
Measures of Dispersion : Measures like the range, variance, and standard deviation provide insights into the spread or variability of your data points.
Frequency Distributions : Creating frequency distributions or histograms allows you to visualize the distribution of your data across different values or categories.

Descriptive statistics provide the initial insights needed to understand your data's basic characteristics, which can inform further analysis.

Inferential Statistics

Inferential statistics take your analysis to the next level by allowing you to make inferences or predictions about a larger population based on your sample data. These methods help you test hypotheses and draw meaningful conclusions. Key concepts in inferential statistics include:

Hypothesis Testing : Hypothesis tests (e.g., t-tests , chi-squared tests ) help you determine whether observed differences or associations in your data are statistically significant or occurred by chance.
Confidence Intervals : Confidence intervals provide a range within which population parameters (e.g., population mean) are likely to fall based on your sample data.
Regression Analysis : Regression models (linear, logistic, etc.) help you explore relationships between variables and make predictions.
Analysis of Variance (ANOVA) : ANOVA tests are used to compare means between multiple groups, allowing you to assess whether differences are statistically significant.

Chi-Square Calculator :

t-Test Calculator :

One-way ANOVA Calculator :

Inferential statistics are powerful tools for drawing conclusions from your data and assessing the generalizability of your findings to the broader population.

Qualitative Data Analysis

Qualitative data analysis is employed when working with non-numerical data, such as text, interviews, or open-ended survey responses. It focuses on understanding the underlying themes, patterns, and meanings within qualitative data. Qualitative analysis techniques include:

Thematic Analysis : Identifying and analyzing recurring themes or patterns within textual data.
Content Analysis : Categorizing and coding qualitative data to extract meaningful insights.
Grounded Theory : Developing theories or frameworks based on emergent themes from the data.
Narrative Analysis : Examining the structure and content of narratives to uncover meaning.

Qualitative data analysis provides a rich and nuanced understanding of complex phenomena and human experiences.

Data Visualization

Data visualization is the art of representing data graphically to make complex information more understandable and accessible. Effective data visualization can reveal patterns, trends, and outliers in your data. Common types of data visualization include:

Bar Charts and Histograms : Used to display the distribution of categorical data or discrete data .
Line Charts : Ideal for showing trends and changes in data over time.
Scatter Plots : Visualize relationships and correlations between two variables.
Pie Charts : Display the composition of a whole in terms of its parts.
Heatmaps : Depict patterns and relationships in multidimensional data through color-coding.
Box Plots : Provide a summary of the data distribution, including outliers.
Interactive Dashboards : Create dynamic visualizations that allow users to explore data interactively.

Data visualization not only enhances your understanding of the data but also serves as a powerful communication tool to convey your findings to others.

As you embark on the data analysis phase of your empirical research, remember that the specific methods and techniques you choose will depend on your research questions, data type, and objectives. Effective data analysis transforms raw data into valuable insights, bringing you closer to the answers you seek.

How to Report Empirical Research Results?

At this stage, you get to share your empirical research findings with the world. Effective reporting and presentation of your results are crucial for communicating your research's impact and insights.

1. Write the Research Paper

Writing a research paper is the culmination of your empirical research journey. It's where you synthesize your findings, provide context, and contribute to the body of knowledge in your field.

Title and Abstract : Craft a clear and concise title that reflects your research's essence. The abstract should provide a brief summary of your research objectives, methods, findings, and implications.
Introduction : In the introduction, introduce your research topic, state your research questions or hypotheses, and explain the significance of your study. Provide context by discussing relevant literature.
Methods : Describe your research design, data collection methods, and sampling procedures. Be precise and transparent, allowing readers to understand how you conducted your study.
Results : Present your findings in a clear and organized manner. Use tables, graphs, and statistical analyses to support your results. Avoid interpreting your findings in this section; focus on the presentation of raw data.
Discussion : Interpret your findings and discuss their implications. Relate your results to your research questions and the existing literature. Address any limitations of your study and suggest avenues for future research.
Conclusion : Summarize the key points of your research and its significance. Restate your main findings and their implications.
References : Cite all sources used in your research following a specific citation style (e.g., APA, MLA, Chicago). Ensure accuracy and consistency in your citations.
Appendices : Include any supplementary material, such as questionnaires, data coding sheets, or additional analyses, in the appendices.

Writing a research paper is a skill that improves with practice. Ensure clarity, coherence, and conciseness in your writing to make your research accessible to a broader audience.

2. Create Visuals and Tables

Visuals and tables are powerful tools for presenting complex data in an accessible and understandable manner.

Clarity : Ensure that your visuals and tables are clear and easy to interpret. Use descriptive titles and labels.
Consistency : Maintain consistency in formatting, such as font size and style, across all visuals and tables.
Appropriateness : Choose the most suitable visual representation for your data. Bar charts, line graphs, and scatter plots work well for different types of data.
Simplicity : Avoid clutter and unnecessary details. Focus on conveying the main points.
Accessibility : Make sure your visuals and tables are accessible to a broad audience, including those with visual impairments.
Captions : Include informative captions that explain the significance of each visual or table.

Compelling visuals and tables enhance the reader's understanding of your research and can be the key to conveying complex information efficiently.

3. Interpret Findings

Interpreting your findings is where you bridge the gap between data and meaning. It's your opportunity to provide context, discuss implications, and offer insights. When interpreting your findings:

Relate to Research Questions : Discuss how your findings directly address your research questions or hypotheses.
Compare with Literature : Analyze how your results align with or deviate from previous research in your field. What insights can you draw from these comparisons?
Discuss Limitations : Be transparent about the limitations of your study. Address any constraints, biases, or potential sources of error.
Practical Implications : Explore the real-world implications of your findings. How can they be applied or inform decision-making?
Future Research Directions : Suggest areas for future research based on the gaps or unanswered questions that emerged from your study.

Interpreting findings goes beyond simply presenting data; it's about weaving a narrative that helps readers grasp the significance of your research in the broader context.

With your research paper written, structured, and enriched with visuals, and your findings expertly interpreted, you are now prepared to communicate your research effectively. Sharing your insights and contributing to the body of knowledge in your field is a significant accomplishment in empirical research.

Examples of Empirical Research

To solidify your understanding of empirical research, let's delve into some real-world examples across different fields. These examples will illustrate how empirical research is applied to gather data, analyze findings, and draw conclusions.

Social Sciences

In the realm of social sciences, consider a sociological study exploring the impact of socioeconomic status on educational attainment. Researchers gather data from a diverse group of individuals, including their family backgrounds, income levels, and academic achievements.

Through statistical analysis, they can identify correlations and trends, revealing whether individuals from lower socioeconomic backgrounds are less likely to attain higher levels of education. This empirical research helps shed light on societal inequalities and informs policymakers on potential interventions to address disparities in educational access.

Environmental Science

Environmental scientists often employ empirical research to assess the effects of environmental changes. For instance, researchers studying the impact of climate change on wildlife might collect data on animal populations, weather patterns, and habitat conditions over an extended period.

By analyzing this empirical data, they can identify correlations between climate fluctuations and changes in wildlife behavior, migration patterns, or population sizes. This empirical research is crucial for understanding the ecological consequences of climate change and informing conservation efforts.

Business and Economics

In the business world, empirical research is essential for making data-driven decisions. Consider a market research study conducted by a business seeking to launch a new product. They collect data through surveys , focus groups , and consumer behavior analysis.

By examining this empirical data, the company can gauge consumer preferences, demand, and potential market size. Empirical research in business helps guide product development, pricing strategies, and marketing campaigns, increasing the likelihood of a successful product launch.

Psychological studies frequently rely on empirical research to understand human behavior and cognition. For instance, a psychologist interested in examining the impact of stress on memory might design an experiment. Participants are exposed to stress-inducing situations, and their memory performance is assessed through various tasks.

By analyzing the data collected, the psychologist can determine whether stress has a significant effect on memory recall. This empirical research contributes to our understanding of the complex interplay between psychological factors and cognitive processes.

These examples highlight the versatility and applicability of empirical research across diverse fields. Whether in medicine, social sciences, environmental science, business, or psychology, empirical research serves as a fundamental tool for gaining insights, testing hypotheses, and driving advancements in knowledge and practice.

Conclusion for Empirical Research

Empirical research is a powerful tool for gaining insights, testing hypotheses, and making informed decisions. By following the steps outlined in this guide, you've learned how to select research topics, collect data, analyze findings, and effectively communicate your research to the world. Remember, empirical research is a journey of discovery, and each step you take brings you closer to a deeper understanding of the world around you. Whether you're a scientist, a student, or someone curious about the process, the principles of empirical research empower you to explore, learn, and contribute to the ever-expanding realm of knowledge.

How to Collect Data for Empirical Research?

Introducing Appinio , the real-time market research platform revolutionizing how companies gather consumer insights for their empirical research endeavors. With Appinio, you can conduct your own market research in minutes, gaining valuable data to fuel your data-driven decisions.

Appinio is more than just a market research platform; it's a catalyst for transforming the way you approach empirical research, making it exciting, intuitive, and seamlessly integrated into your decision-making process.

Here's why Appinio is the go-to solution for empirical research:

From Questions to Insights in Minutes : With Appinio's streamlined process, you can go from formulating your research questions to obtaining actionable insights in a matter of minutes, saving you time and effort.
Intuitive Platform for Everyone : No need for a PhD in research; Appinio's platform is designed to be intuitive and user-friendly, ensuring that anyone can navigate and utilize it effectively.
Rapid Response Times : With an average field time of under 23 minutes for 1,000 respondents, Appinio delivers rapid results, allowing you to gather data swiftly and efficiently.
Global Reach with Targeted Precision : With access to over 90 countries and the ability to define target groups based on 1200+ characteristics, Appinio empowers you to reach your desired audience with precision and ease.

Get free access to the platform!

Join the loop 💌

Be the first to hear about new updates, product news, and data insights. We'll send it all straight to your inbox.

Get the latest market research news straight to your inbox! 💌

Wait, there's more

Creative Checkup – Optimize Advertising Slogans & Creatives for maximum ROI

16.09.2024 | 9min read

Creative Checkup – Optimize Advertising Slogans & Creatives for ROI

03.09.2024 | 5min read

Get your brand Holiday Ready: 4 Essential Steps to Smash your Q4

Beyond Demographics: Psychographic Power in target group identification

03.09.2024 | 8min read

Beyond Demographics: Psychographics power in target group identification

Empirical Research

What is Empirical Research?
Finding Empirical Research in Library Databases

Choosing Your Research Design

Library resources on research design.

Case Sudies

Image from: Vogt, W. P., Gardner, D. C., & Haeffele, L. M. (2012). When to use what research design. New York, NY: The Guilford Press. pg. 14

<< Previous: Finding Empirical Research in Library Databases
Next: Case Sudies >>
Last Updated: Sep 9, 2024 2:14 PM
URL: https://enmu.libguides.com/EmpiricalResearch

Canvas | University | Ask a Librarian

Library Homepage
Arrendale Library

Empirical & Non-Empirical Research

Empirical Research

Introduction: What is Empirical Research?

Quantitative methods, qualitative methods.

Quantitative vs. Qualitative
Reference Works for Social Sciences Research
What is Non-Empirical Research?
Contact Us!

Call us at 706-776-0111

Chat with a Librarian

Send Us Email

Library Hours

Empirical research is based on phenomena that can be observed and measured. Empirical research derives knowledge from actual experience rather than from theory or belief.

Key characteristics of empirical research include:

Specific research questions to be answered;
Definitions of the population, behavior, or phenomena being studied;
Description of the methodology or research design used to study this population or phenomena, including selection criteria, controls, and testing instruments (such as surveys);
Two basic research processes or methods in empirical research: quantitative methods and qualitative methods (see the rest of the guide for more about these methods).

(based on the original from the Connelly LIbrary of LaSalle University)

Empirical Research: Qualitative vs. Quantitative

Learn about common types of journal articles that use APA Style, including empirical studies; meta-analyses; literature reviews; and replication, theoretical, and methodological articles.

Academic Writer

More about Academic Writer ...

Quantitative Research

A quantitative research project is characterized by having a population about which the researcher wants to draw conclusions, but it is not possible to collect data on the entire population.

For an observational study, it is necessary to select a proper, statistical random sample and to use methods of statistical inference to draw conclusions about the population.
For an experimental study, it is necessary to have a random assignment of subjects to experimental and control groups in order to use methods of statistical inference.

Statistical methods are used in all three stages of a quantitative research project.

For observational studies, the data are collected using statistical sampling theory. Then, the sample data are analyzed using descriptive statistical analysis. Finally, generalizations are made from the sample data to the entire population using statistical inference.

For experimental studies, the subjects are allocated to experimental and control group using randomizing methods. Then, the experimental data are analyzed using descriptive statistical analysis. Finally, just as for observational data, generalizations are made to a larger population.

Iversen, G. (2004). Quantitative research . In M. Lewis-Beck, A. Bryman, & T. Liao (Eds.), Encyclopedia of social science research methods . (pp. 897-898). Thousand Oaks, CA: SAGE Publications, Inc.

Qualitative Research

What makes a work deserving of the label qualitative research is the demonstrable effort to produce richly and relevantly detailed descriptions and particularized interpretations of people and the social, linguistic, material, and other practices and events that shape and are shaped by them.

Qualitative research typically includes, but is not limited to, discerning the perspectives of these people, or what is often referred to as the actor’s point of view. Although both philosophically and methodologically a highly diverse entity, qualitative research is marked by certain defining imperatives that include its case (as opposed to its variable) orientation, sensitivity to cultural and historical context, and reflexivity.

In its many guises, qualitative research is a form of empirical inquiry that typically entails some form of purposive sampling for information-rich cases; in-depth interviews and open-ended interviews, lengthy participant/field observations, and/or document or artifact study; and techniques for analysis and interpretation of data that move beyond the data generated and their surface appearances.

Sandelowski, M. (2004). Qualitative research . In M. Lewis-Beck, A. Bryman, & T. Liao (Eds.), Encyclopedia of social science research methods . (pp. 893-894). Thousand Oaks, CA: SAGE Publications, Inc.

Next: Quantitative vs. Qualitative >>
Last Updated: Jul 24, 2024 12:04 PM
URL: https://library.piedmont.edu/empirical-research
Ebooks & Online Video
New Materials
Renew Checkouts
Faculty Resources
Library Friends
Library Services
Our Mission
Library History
Ask a Librarian!
Making Citations
Working Online

Arrendale Library Piedmont University 706-776-0111

Empirical Research: What is Empirical Research?

What is Empirical Research?
Finding Empirical Research in Library Databases
Designing Empirical Research

Introduction

Empirical research is based on observed and measured phenomena and derives knowledge from actual experience rather than from theory or belief.

Key characteristics to look for:

Specific research questions to be answered
Definition of the population, behavior, or phenomena being studied
Description of the process used to study this population or phenomena, including selection criteria, controls, and testing instruments (such as surveys)

Another hint: some scholarly journals use a specific layout, called the "IMRaD" format (Introduction – Method – Results – and – Discussion), to communicate empirical research findings. Such articles typically have 4 components:

Introduction : sometimes called "literature review" -- what is currently known about the topic -- usually includes a theoretical framework and/or discussion of previous studies
Methodology : sometimes called "research design" -- how to recreate the study -- usually describes the population, research process, and analytical tools
Results : sometimes called "findings" -- what was learned through the study -- usually appears as statistical data or as substantial quotations from research participants
Discussion : sometimes called "conclusion" or "implications" -- why the study is important -- usually describes how the research results influence professional practices or future studies

Empirical research is published in books and in scholarly, peer-reviewed journals .

Make sure to select the peer-review box within each database!

Next: Finding Empirical Research in Library Databases >>
Last Updated: Nov 21, 2022 8:55 AM
URL: https://libguides.lahc.edu/empirical

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

Knowledge Base

Methodology

Types of Research Designs Compared | Guide & Examples

Types of Research Designs Compared | Guide & Examples

Published on June 20, 2019 by Shona McCombes . Revised on June 22, 2023.

When you start planning a research project, developing research questions and creating a research design , you will have to make various decisions about the type of research you want to do.

There are many ways to categorize different types of research. The words you use to describe your research depend on your discipline and field. In general, though, the form your research design takes will be shaped by:

The type of knowledge you aim to produce
The type of data you will collect and analyze
The sampling methods , timescale and location of the research

This article takes a look at some common distinctions made between different types of research and outlines the key differences between them.

Types of research aims, types of research data, types of sampling, timescale, and location, other interesting articles.

The first thing to consider is what kind of knowledge your research aims to contribute.

Type of research	What’s the difference?	What to consider
Basic vs. applied	Basic research aims to , while applied research aims to .	Do you want to expand scientific understanding or solve a practical problem?
vs.	Exploratory research aims to , while explanatory research aims to .	How much is already known about your research problem? Are you conducting initial research on a newly-identified issue, or seeking precise conclusions about an established issue?
	aims to , while aims to .	Is there already some theory on your research problem that you can use to develop , or do you want to propose new theories based on your findings?

Prevent plagiarism. Run a free check.

The next thing to consider is what type of data you will collect. Each kind of data is associated with a range of specific research methods and procedures.

Type of research	What’s the difference?	What to consider
Primary research vs secondary research	Primary data is (e.g., through or ), while secondary data (e.g., in government or scientific publications).	How much data is already available on your topic? Do you want to collect original data or analyze existing data (e.g., through a )?
	, while .	Is your research more concerned with measuring something or interpreting something? You can also create a research design that has elements of both.
vs	Descriptive research gathers data , while experimental research .	Do you want to identify characteristics, patterns and or test causal relationships between ?

Finally, you have to consider three closely related questions: how will you select the subjects or participants of the research? When and how often will you collect data from your subjects? And where will the research take place?

Keep in mind that the methods that you choose bring with them different risk factors and types of research bias . Biases aren’t completely avoidable, but can heavily impact the validity and reliability of your findings if left unchecked.

Type of research	What’s the difference?	What to consider
	allows you to , while allows you to draw conclusions .	Do you want to produce knowledge that applies to many contexts or detailed knowledge about a specific context (e.g. in a )?
vs	Cross-sectional studies , while longitudinal studies .	Is your research question focused on understanding the current situation or tracking changes over time?
Field research vs laboratory research	Field research takes place in , while laboratory research takes place in .	Do you want to find out how something occurs in the real world or draw firm conclusions about cause and effect? Laboratory experiments have higher but lower .
Fixed design vs flexible design	In a fixed research design the subjects, timescale and location are begins, while in a flexible design these aspects may .	Do you want to test hypotheses and establish generalizable facts, or explore concepts and develop understanding? For measuring, testing and making generalizations, a fixed research design has higher .

Choosing between all these different research types is part of the process of creating your research design , which determines exactly how your research will be conducted. But the type of research is only the first step: next, you have to make more concrete decisions about your research methods and the details of the study.

Read more about creating a research design

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

Normal distribution
Degrees of freedom
Null hypothesis
Discourse analysis
Control groups
Mixed methods research
Non-probability sampling
Quantitative research
Ecological validity

Research bias

Rosenthal effect
Implicit bias
Cognitive bias
Selection bias
Negativity bias
Status quo bias

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, June 22). Types of Research Designs Compared | Guide & Examples. Scribbr. Retrieved September 16, 2024, from https://www.scribbr.com/methodology/types-of-research/

Is this article helpful?

Shona McCombes

Other students also liked, what is a research design | types, guide & examples, qualitative vs. quantitative research | differences, examples & methods, what is a research methodology | steps & tips, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Empirical evidence: A definition

Empirical evidence is information that is acquired by observation or experimentation.

The scientific method

Types of empirical research, identifying empirical evidence, empirical law vs. scientific law, empirical, anecdotal and logical evidence, additional resources and reading, bibliography.

Empirical evidence is information acquired by observation or experimentation. Scientists record and analyze this data. The process is a central part of the scientific method , leading to the proving or disproving of a hypothesis and our better understanding of the world as a result.

Empirical evidence might be obtained through experiments that seek to provide a measurable or observable reaction, trials that repeat an experiment to test its efficacy (such as a drug trial, for instance) or other forms of data gathering against which a hypothesis can be tested and reliably measured.

"If a statement is about something that is itself observable, then the empirical testing can be direct. We just have a look to see if it is true. For example, the statement, 'The litmus paper is pink', is subject to direct empirical testing," wrote Peter Kosso in " A Summary of Scientific Method " (Springer, 2011).

"Science is most interesting and most useful to us when it is describing the unobservable things like atoms , germs , black holes , gravity , the process of evolution as it happened in the past, and so on," wrote Kosso. Scientific theories , meaning theories about nature that are unobservable, cannot be proven by direct empirical testing, but they can be tested indirectly, according to Kosso. "The nature of this indirect evidence, and the logical relation between evidence and theory, are the crux of scientific method," wrote Kosso.

The scientific method begins with scientists forming questions, or hypotheses , and then acquiring the knowledge through observations and experiments to either support or disprove a specific theory. "Empirical" means "based on observation or experience," according to the Merriam-Webster Dictionary . Empirical research is the process of finding empirical evidence. Empirical data is the information that comes from the research.

Before any pieces of empirical data are collected, scientists carefully design their research methods to ensure the accuracy, quality and integrity of the data. If there are flaws in the way that empirical data is collected, the research will not be considered valid.

The scientific method often involves lab experiments that are repeated over and over, and these experiments result in quantitative data in the form of numbers and statistics. However, that is not the only process used for gathering information to support or refute a theory.

This methodology mostly applies to the natural sciences. "The role of empirical experimentation and observation is negligible in mathematics compared to natural sciences such as psychology, biology or physics," wrote Mark Chang, an adjunct professor at Boston University, in " Principles of Scientific Methods " (Chapman and Hall, 2017).

"Empirical evidence includes measurements or data collected through direct observation or experimentation," said Jaime Tanner, a professor of biology at Marlboro College in Vermont. There are two research methods used to gather empirical measurements and data: qualitative and quantitative.

Qualitative research, often used in the social sciences, examines the reasons behind human behavior, according to the National Center for Biotechnology Information (NCBI) . It involves data that can be found using the human senses . This type of research is often done in the beginning of an experiment. "When combined with quantitative measures, qualitative study can give a better understanding of health related issues," wrote Dr. Sanjay Kalra for NCBI.

Quantitative research involves methods that are used to collect numerical data and analyze it using statistical methods, ."Quantitative research methods emphasize objective measurements and the statistical, mathematical, or numerical analysis of data collected through polls, questionnaires, and surveys, or by manipulating pre-existing statistical data using computational techniques," according to the LeTourneau University . This type of research is often used at the end of an experiment to refine and test the previous research.

Identifying empirical evidence in another researcher's experiments can sometimes be difficult. According to the Pennsylvania State University Libraries , there are some things one can look for when determining if evidence is empirical:

Can the experiment be recreated and tested?
Does the experiment have a statement about the methodology, tools and controls used?
Is there a definition of the group or phenomena being studied?

The objective of science is that all empirical data that has been gathered through observation, experience and experimentation is without bias. The strength of any scientific research depends on the ability to gather and analyze empirical data in the most unbiased and controlled fashion possible.

However, in the 1960s, scientific historian and philosopher Thomas Kuhn promoted the idea that scientists can be influenced by prior beliefs and experiences, according to the Center for the Study of Language and Information .

— Amazing Black scientists

— Marie Curie: Facts and biography

— What is multiverse theory?

"Missing observations or incomplete data can also cause bias in data analysis, especially when the missing mechanism is not random," wrote Chang.

Because scientists are human and prone to error, empirical data is often gathered by multiple scientists who independently replicate experiments. This also guards against scientists who unconsciously, or in rare cases consciously, veer from the prescribed research parameters, which could skew the results.

The recording of empirical data is also crucial to the scientific method, as science can only be advanced if data is shared and analyzed. Peer review of empirical data is essential to protect against bad science, according to the University of California .

Empirical laws and scientific laws are often the same thing. "Laws are descriptions — often mathematical descriptions — of natural phenomenon," Peter Coppinger, associate professor of biology and biomedical engineering at the Rose-Hulman Institute of Technology, told Live Science.

Empirical laws are scientific laws that can be proven or disproved using observations or experiments, according to the Merriam-Webster Dictionary . So, as long as a scientific law can be tested using experiments or observations, it is considered an empirical law.

Empirical, anecdotal and logical evidence should not be confused. They are separate types of evidence that can be used to try to prove or disprove and idea or claim.

Logical evidence is used proven or disprove an idea using logic. Deductive reasoning may be used to come to a conclusion to provide logical evidence. For example, "All men are mortal. Harold is a man. Therefore, Harold is mortal."

Anecdotal evidence consists of stories that have been experienced by a person that are told to prove or disprove a point. For example, many people have told stories about their alien abductions to prove that aliens exist. Often, a person's anecdotal evidence cannot be proven or disproven.

There are some things in nature that science is still working to build evidence for, such as the hunt to explain consciousness .

Meanwhile, in other scientific fields, efforts are still being made to improve research methods, such as the plan by some psychologists to fix the science of psychology .

" A Summary of Scientific Method " by Peter Kosso (Springer, 2011)

"Empirical" Merriam-Webster Dictionary

" Principles of Scientific Methods " by Mark Chang (Chapman and Hall, 2017)

"Qualitative research" by Dr. Sanjay Kalra National Center for Biotechnology Information (NCBI)

"Quantitative Research and Analysis: Quantitative Methods Overview" LeTourneau University

"Empirical Research in the Social Sciences and Education" Pennsylvania State University Libraries

"Thomas Kuhn" Center for the Study of Language and Information

"Misconceptions about science" University of California

Sign up for the Live Science daily newsletter now

Get the world’s most fascinating discoveries delivered straight to your inbox.

Science news this week: 'Thorin' the last Neanderthal and a 'smiley face' on Mars

Live Science x HowTheLightGetsIn — Get discounted tickets to the world’s largest ideas and music festival

Pregnancy shrinks parts of the brain, leaving 'permanent etchings' postpartum

Reference Guide: Searching for Empirical Articles

Open Access Journals
Placing a hold on a book
Requesting Items from OneSearch
Submitting an ILL request manually
Checking on your Requests/Loans
Google Scholar
Faculty Resources
Primary & Secondary Sources
Looking up if it is Peer-Reviewed
Grey Literature
Videos & Tutorials
Searching for Empirical Articles
Impact Factors
Annotated Bibliography vs. Literature Review

What is Empirical Research?

Empirical research is conducted based on observed and measured phenomena and derives knowledge from actual experience, rather than from theory or belief. Empirical research articles are examples of primary research.

How do you know if a study is empirical?

Read the subheadings within the article, book, or report and look for a description of the research methodology. Ask yourself: Could I recreate this study and test these results?

Key characteristics to look for:

Specific research questions to be answered
Definition of the population, behavior, or phenomena being studied
Description of the process used to study this population or phenomena, including selection criteria, controls, and testing instruments (such as surveys)
The article abstract mentions a study, observation, analysis, # of participants/subjects .
The article includes charts , graphs , or statistical analysis .
The article is substantial in size, likely to be more than 5 pages long.
The article contains the following sections (the exact terms may vary): abstract, introduction, methodology , results , discussion, references.
Empirical research is often (but not always) published in peer-reviewed academic journals.

Finding Empirical Research in the Databases

Most databases will not have a simple way to only look at empirical research. In the window below are some suggestions for specific databases, but here are some good rules of thumb to follow:

Search subject-specific databases - Multipurpose databases can definitely contain empirical research, but it's almost always easier to use the databases devoted to your topic, which should have more topical results and will respond better to your keywords.

Select "Peer-reviewed Journals" - Not all empirical research is published in academic journals. Grey literature is a great place to search, particularly in the health sciences. However, grey literature can be difficult to identify, so it is recommended to search the databases until you are more comfortable identifying empirical literature.

Check the abstract / methods - Most articles will not have the phrase "empirical research" in their title, or even in the whole article. A better place to get an idea of what the article contains is by looking at the abstract and the methods section. In the abstract, there will usually be a description of what was done in the article. If there isn't, look in the methods. Ideally, you can get an idea of whether original research is being conducted or if it's reviewing it from other sources.

Consider your keywords - Think about what types of methods are used in empirical research and incorporate those into your keywords. or example, searching for "sleep loss" will certainly bring back many articles about that subject, but "sleep loss and study" might yield some results describing studies being conducted on sleep loss.

The box to the right features some typical methods of conducting empirical research that you might consider including in your search terms.

Empirical research search terms

observation
questionnaire
participants

Specific database examples

CINAHL Plus
APA PsychINFO
Science Direct
Linguistics and Language Behavior Abstracts
CINAHL Complete This link opens in a new window CINAHL, the Cumulative Index to Nursing & Allied Health Literature, is a comprehensive research tool for nursing, allied health, public health, biomedicine, and related fields. It provides indexing for articles from 5,400 journals in the fields of nursing and allied health. This database provides full text access to more than 1,300 journals dating back to 1937.
Use the "Advanced Search"
Type your keywords into the search boxes
Below the search windows, check off "Evidence-Based Practice" in the "Special Interests" menu
Choose other limits, such as published date, if needed
Click on the "Search" button
Empirical Research
Experimental Studies
Nonexperimental Studies
Qualitative Studies
Quantitative Studies
PubMed This link opens in a new window A comprehensive index to biomedical and life sciences journals with citations to over 18 million articles back to 1948. Note: To limit to full-text articles, search PUBMED CENTRAL.

There are 2 ways to find empirical articles in PubMed:

One technique is to limit your search results after you perform a search:

Type in your keywords and click on the "Search" button
To the left of your results, under "Article Types," click on "Customize"
Choose the types of studies that interest you, and click on the "Show" button

Another alternative is to construct a more sophisticated search:

From PubMed's main screen, click on "Advanced" link underneath the search box
On the Advance Search Builder screen type your keywords into the search boxes
Change one of the empty boxes from "All Fields" to "Publication Type"
To the right of Publication Type, click on "Show Index List" and choose a methodology that interests you. You can choose more than one by holding down the "Ctrl" or "⌘" on your keyboard as you click on each methodology
APA PsycINFO This link opens in a new window Available via EBSCO. The American Psychological Associations (APA) notable database for locating abstracts of scholarly journal articles, book chapters, books, and dissertations. This resource is the largest of its kind dedicated to peer-reviewed literature in behavioral science and mental health, and it also includes information about the psychological aspects of related fields such as medicine, psychiatry, nursing, sociology, education, pharmacology, technology, linguistics, anthropology, business, and law. Material is drawn from over 2,000 periodicals in more than 20 languages.

To find empirical articles in PsycINFO:

Scroll down the page to "Methodology," and choose "Empirical Study." There are more specific methodologies below.
Choose other limits, such as publication date, if needed

To find empirical articles in ScienceDirect:

Click on "Advanced Search" to the right of the search windows
On next page, click on "Show all fields"
Under "Article Types," select "Research Articles," or any other type of article which might be helpful.
Slick Search
Case Studies
Qualitative Analysis
Quantitative Analysis
Statistical Analysis
ERIC This link opens in a new window Abstracts (and in some cases, full-text) articles, reports, book reviews and government documents covering all aspects of education from 1966 to the present
Action Research
Ethnography
Evaluation Methods
Evaluation Research
Experiments
Focus Groups
Field Studies
Mail Surveys
Mixed Methods Research
Naturalistic Observation
Online Surveys
Participant Observation
Participatory Research
Qualitative Research
Questionnaires
Statistical Studies
Telephone Surveys

Empirical Articles - Sample Research Tips -- CAS & PSYC 101 / PSYC 341 IN-PERSON & ONLINE -- ACCESSIBLE VERSION

This guide helps to identify the major parts of an empirical article and covers sample strategies for locating them through databases such as APA PsycInfo and ERIC . There are also general tips applicable to other databases.

<< Previous: Videos & Tutorials
Next: Impact Factors >>
Last Updated: Sep 9, 2024 6:56 PM
URL: https://libraryguides.fullerton.edu/reference

This site is maintained by Pollak Library .

Web Accessibility

CSUF is committed to ensuring equal accessibility to our users. Let us know about any accessibility problems you encounter using this website. We'll do our best to improve things and get you the information you need.

Adobe Reader
Microsoft Viewers
Report An ATI Issue
Accessible @ CSUF

How to find Psychology Articles
Using APA Thesaurus

Empirical Articles

How to Limit to Empirical Articles
What are they?
How to Read them?
Main Sections

Empirical articles are those in which authors report on their own study. The authors will have collected data to answer a research question. Empirical research contains observed and measured examples that inform or answer the research question. The data can be collected in a variety of ways such as interviews, surveys, questionnaires, observations, and various other quantitative and qualitative research methods.

Empirical research is based on observed and measured phenomena and derives knowledge from actual experience rather than from theory or belief.

Key characteristics to look for:

Specific research questions to be answered
Definition of the population, behavior, or phenomena being studied
Description of the process used to study this population or phenomena, including selection criteria, controls, and testing instruments (such as surveys)

Another hint: some scholarly journals use a specific layout, called the "IMRaD" format, to communicate empirical research findings. Such articles typically have 4 components:

Introduction : sometimes called "literature review" -- what is currently known about the topic -- usually includes a theoretical framework and/or discussion of previous studies
Methodology: sometimes called "research design" -- how to recreate the study -- usually describes the population, research process, and analytical tools
Results : sometimes called "findings" -- what was learned through the study -- usually appears as statistical data or as substantial quotations from research participants
Discussion : sometimes called "conclusion" or "implications" -- why the study is important -- usually describes how the research results influence professional practices or future studies

General Advice

Plan to read the article more than once
Don't read it all the way through in one sitting, read strategically first.
Identify relevant conclusions and limitations of study

Abstract: Get a sense of the article’s purpose and findings. Use it to assess if the article is useful for your research.

Skim: Review headings to understand the structure and label parts if needed.

Introduction/Literature Review: Identify the main argument, problem, previous work, proposed next steps, and hypothesis.

Methodology: Understand data collection methods, data sources, and variables.

Findings/Results: Examine tables and figures to see if they support the hypothesis without relying on captions.

Discussion/Conclusion: Determine if the findings support the argument/hypothesis and if the authors acknowledge any limitations.

Anatomy of a Research Paper by Richard D. Branson published in Respir Care. 2004 October; 49(10): 1222–1228.

How to Read a Scholarly Chemistry Artricle - Rider tutorial.

How to read and understand a scientific paper - a guide for non-scientists - Violent Metaphors (blog post).

Compare your article to this table to help determine you have located an empirical study/research report.

Look for the following words in the title/abstract: empirical, experiment, research, or study.


Abstract	A short synopsis of the article’s content
Introduction	Need and rational of this particular research project with research question, statement, and hypothesis.
Literature Review (sometimes included in the Introduction)	Supporting their ideas with other scholarly research
Methods	Describes the methodology including a description of the participants, and a description of the research method, measure, research design, or approach to data analysis.
Results or Findings	Uses narrative, charts, tables, graphs, or other graphics to describe the findings of the paper
Discussion/Conclusion/Implications	Provides a discussion, summary, or conclusion, bringing together the research question, statement,
References	References all the articlesdiscussed and cited in the paper- mostly in the literature or results sections

<< Previous: Using APA Thesaurus
Next: How to Limit to Empirical Articles >>
Last Updated: Sep 16, 2024 2:30 PM
URL: https://guides.rider.edu/psy

Our systems are now restored following recent technical disruption, and we’re working hard to catch up on publishing. We apologise for the inconvenience caused. Find out more: https://www.cambridge.org/universitypress/about-us/news-and-blogs/cambridge-university-press-publishing-update-following-technical-disruption

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings .

Login Alert

> Journals
> Nationalities Papers
> Volume 51 Issue 6
> Chechnya’s Paradiplomacy 2000–2020: The Emergence and...

Article contents

Introduction, paradiplomacy and the chechen case, paradiplomacy, governance, territorial acquisition, and the creation of chechnya’s institutions, chechnya’s paradiplomacy 2000–2020, conclusions, disclosures, chechnya’s paradiplomacy 2000–2020: the emergence and evolution of external relations of a reincorporated territory.

Published online by Cambridge University Press: 25 April 2022

From the year 2000 on, Chechen official international relations – called “paradiplomacy” – have centered around legitimacy-building, security cooperation and investment attraction, priorities set by the republic’s first official, pro-Russian president, Akhmat Kadyrov (in office 2000–2004). Kadyrov’s successors, Alu Alkhanov (2004–2007) and Ramzan Kadyrov (2007–to date) developed Grozny’s international engagements further, introducing new partners – such as China – and new dimensions to the external action – such as militarization. At each step, Grozny operated between full autonomy and collaboration with Moscow, involving itself in high-level diplomacy and furthering Moscow’s agenda abroad, primarily in the Middle East. In this article, I argue that Chechen paradiplomacy is an instrument for the inclusion of Chechnya into the governance structures of Russia’s federal order. The argument rests on two premises: Chechnya’s paradiplomacy is framed by the Kremlin’s proactive support and coordination, and Chechnya’s paradiplomacy is closely connected to the Kremlin’s security priorities. Since reincorporation, Chechen paradiplomacy has not been an addition to Russian federal relations but an intrinsic part of the post-2000 political arrangement between Grozny and Moscow. To empirically ground this argument, I trace the evolution of Chechen paradiplomacy across the three post-incorporation presidencies, ending in 2020.

From 2015 on, the international relations of the Chechen Republic, a federal subject of the Russian Federation, have been brought to attention by their scope and influence in the Middle East. Today, Grozny is known to have deployed special forces in Syria, hold close contact with the leadership of the Gulf monarchies, and have a highly autonomous perspective on international affairs that occasionally clashes with Moscow. Indeed, Chechnya stands out as a region that has remained highly autonomous in spite of the centralizing trends in the Russian Federation. Because of this, Chechnya’s sub-state diplomacy (also called “paradiplomacy”; for a discussion of the concept, see Aguirre Reference Aguirre 1999 ) has been scrutinized by literature in the past (e.g., Cornago Reference Cornago 1999 ). Yet, a comprehensive analysis of the transformations of Chechen paradiplomacy is missing from the literature, and, as I argue below, the Chechen case is potentially revealing of certain central dynamics of paradiplomacy in general.

How to understand the agency that Chechnya has had at the international stage following its de facto reincorporation into Russia? What has driven and enabled Grozny to have the international role it currently has? The existing paradiplomacy literature has suggested a few different ways to understand the underlying political dynamics behind Grozny’s external action. First, Chechen paradiplomacy may be seen as a way of conflict management. In this account, Moscow gave Chechnya paradiplomacy competencies to deactivate the local conflict, making Chechnya’s case fall into the pattern of paradiplomacy as problem-solving (Joenniemi and Sergunin Reference Joenniemi and Sergunin 2014 ; Tavares Reference Tavares 2016 , 44). Second, Chechen paradiplomacy may be seen as part of the political bargain between the Kadyrov family and Putin to ensure Chechnya’s loyalty to the Russian Federation. According to this version, Moscow gave the Kadyrov clan the opportunity to entrench their autocratic rule over Chechnya in exchange for their services as intermediaries with the Muslim world (Luzin Reference Luzin 2018 ). Third, Chechen paradiplomacy can be seen as an outcome of center-region relations, bargaining, and the competition for federal funding. In this version, Grozny exploits Chechnya’s unique culture and connections to the Middle East to demonstrate the republic’s value to Moscow and, in turn, is rewarded with stability in federal funding (Makarychev and Yatsyk Reference Makarychev and Yatsyk 2018 , 917).

All these accounts overlap in their underlying assumption of paradiplomacy as the outcome of a win-win arrangement between Moscow and the pro-Russian Chechen leadership. Building on these interpretations, I trace the evolution of Chechen paradiplomacy and the ways it operationalizes Russian foreign policy. I proceed on the argument that Grozny’s international engagements were enabled and encouraged by Moscow since the territorial reincorporation of Chechnya in 2000. In an attempt to answer the questions regarding the facilitating conditions for Chechen paradiplomacy, this article pursues two objectives, an empirical one and a conceptual one. First, I trace the evolution of Chechen paradiplomacy to evince the interaction between opportunity structures and Grozny’s international agency in the 2000–2020 period. Second, on the basis of the preceding, I argue for the relevance of imposition in the creation of paradiplomacy institutions in newly incorporated regions. On the basis of established understandings of the role of leadership in paradiplomacy, I propose a periodization focused on the changing heads of the Chechen republic, namely Akhmat Kadyrov, Alu Alkhanov, and Ramzan Kadyrov. Because of my lack of access to Chechen officials, I rely on press reports and the existing literature to empirically ground my argument.

The article has two contributions to the literature. The reconstruction of Chechnya’s paradiplomacy 2000–2020 adds value to our understanding of its transformations by drawing attention to the long-term trends in Chechnya’s international relations. Then, by suggesting that Chechen paradiplomacy is a component of Russia’s sovereign reassertion over Chechnya, I draw attention to paradiplomacy institutions and their role in keeping newly incorporated regions governable. This interpretation adds to our broader understanding of paradiplomacy as it further elaborates on the political dynamics of the normalization of paradiplomacy in general (Cornago Reference Cornago 2010 ) and on the dynamics of territorial change in particular. In addition, as a secondary aim, this article adds to the challenge to the established notion that paradiplomacy is structurally bound to service provision (“low” politics) by exploring a case of militarized paradiplomacy.

Paradiplomacy is the label given to the broad phenomenon of the official relations between sub-state territorial governments (“regions” for short) and international partners, such as foreign companies, governments, non-governmental organizations, and other institutions based abroad. Whilst in the past it was considered to be an atypical phenomenon limited to Western, federal democracies, today it is considered to be a “normal” phenomenon (Cornago Reference Cornago 2010 ). This “normality” implies both expectations and prevalence in the world. Today, constituents and regional governments expect to have a right to carry out some form of international engagement, at a minimum for investment attraction. At most, sub-state regions can carry out a complex, rich, ambitious external action that includes cultural events, scientific exchange programs, trade and investment promotion, policy coordination, international events, among other activities, with foreign partners (Tavares Reference Tavares 2016 , 117–150). Regarding prevalence, there has been growing attention to the phenomenon of paradiplomacy beyond Europe and North America, where this scholarship began. Indeed, from 2000 on, there has been a proliferation of studies bringing in evidence of paradiplomacy in Africa, Asia, South America, the Middle East, among other parts of the world (Kuznetsov Reference Kuznetsov 2015 , 41–42).

This literature has proceeded on the basis of case studies as analyzed through a well-established conceptual grid (Lecours Reference Lecours 2002 , 93). The scholarship offers a wealth of typologies and a growing number of explanatory frameworks (Kuznetsov Reference Kuznetsov 2015 ). A thorough analysis of this literature is beyond the scope of this paper, yet a few contributions are important to highlight, as they offer the conceptual toolkit of the present study. Two pathbreaking works laid the conceptual basis for most of the literature on paradiplomacy. First, Ivo Duchacek’s work of paradiplomacy sheds light on the ways to analyze the international activities of sub-state governments as they respond to internal and external pressures and opportunities (Duchacek Reference Duchacek, Michelmann and Soldatos 1991 ). New approaches would specify that paradiplomacy can be a form of problem-solving and capacity-building (Joenniemi and Sergunin Reference Joenniemi and Sergunin 2014 ) or pursue primarily political, economic, cultural, or border-oriented agendas (Kuznetsov Reference Kuznetsov 2015 , 116). Second, Panayotis Soldatos’s ( Reference Soldatos 1991 ) work on center-region relations offers a typology to conceptualize harmonious and disharmonious relations between center and region regarding international matters. In addition, Lecours ( Reference Lecours 2002 ) adds a historical institutional outlook assessing the evolution of paradiplomacy; this perspective places the focus of analysis on the evolution of concrete institutions that govern, regulate, and articulate a region’s international relations ( Reference Lecours 2002 , 97; see also Bursens and Deforche Reference Bursens and Deforche 2010 ). Moreover, the institutional framework compels the incorporation of a broader understanding of the context in which paradiplomacy is embedded, namely to include the transformations of the country’s international relations and of the region’s own institutions. Finally, I focus here on the leadership changes as a key driver in change in paradiplomacy. Paradiplomacy is important for governors and regional elites as they frequently use it to promote their own leadership and invest in an international image. Then, many sub-state governments are set up in a way that foreign relations are concentrated in the executive – meaning that paradiplomacy tends to be structurally “executive driven” (Thurer and MacLaren Reference Thurer and Maclaren 2009 ; Tavares Reference Tavares 2016 , 41–42b; see Stremoukhov Reference Stremoukhov 2021 ). A focus on the executive power, pressures, and center-region relations are the main features of the conceptual grid used here to address the case of Chechnya.

The case of official (the one led by the pro-Russian leadership, as opposed to rebel leadership) Chechen paradiplomacy is revealing and worthwhile as a case study. First, the Chechen case challenges the established understandings of the relationship between paradiplomacy and what are considered the exclusive tools of statecraft. The idea of the exclusive realm of the state vis-à-vis paradiplomacy and regional governments was further elaborated upon by Bartmann ( Reference Bartmann 2006 , 544). According to Bartmann, the state, as the legitimate participant of the international system, decides whether its constituent regional governments may act abroad and to what extent. Typically, this conditions paradiplomacy to act only at the level of “low politics,” as these international interactions primarily focus on service provision and do not interfere in the exclusive purview of the state (see Tavares Reference Tavares 2016 , 6–7). Yet a growing literature has demonstrated the existence of instances where paradiplomacy impacts the “high politics” of diplomacy, security, and defense (Cornago Reference Cornago 1999 ; Morin and Poliquin Reference Morin and Poliquin 2016 ). For instance, Paquin presents ( Reference Paquin 2004 , 147–163) several ways in which paradiplomacy contributes to the security policy of states, listing regional bloc cohesion, participation in peace missions, and conflict management.

Paradiplomacy has been consistently present in the Russian Federation since 1991. During the 1990s, the decentralization of national politics and the relative power of regions over the federal center meant that regional governors had a large scope to act abroad autonomously. In turn, these regions were able to contribute to Russia’s globalization (Makarychev Reference Makarychev 2000 ). The tide turned in 2000 with the beginning of Vladimir Putin’s first term as president, and Russian regional politics began to gradually trend towards centralization and a diminution in the space for paradiplomacy. Some regions would remain especially active abroad, adapting to the changes in the distribution of power in center-region relations, yet the overall trend in federal politics has been towards a concentration of power in Moscow (Renaldi Reference Renaldi 2019 ; Stremoukhov Reference Stremoukhov 2021 ; Arteev and Kentros Klyszcz Reference Arteev and Klyszcz 2021 ).

Chechnya’s international relations have been described and scrutinized in the past from a variety of perspectives. To start, after the fall of Grozny in 2000, the non-official, rebel diplomacy has received attention from the literature (Akhmadov and Daniloff Reference Akhmadov and Daniloff 2013 ). As Chechnya was reincorporated into Russia following the capture of Grozny in 2000, the topic of Chechnya’s international relations became less pressing as a constitutive issue for regional security and Russia’s state integrity. Another strand of scholarship on Chechnya’s international relations focuses on the paradiplomacy of Grozny under Ramzan Kadyrov’s leadership (2007–to date). This official, “pro-Russian” paradiplomacy has been analyzed primarily through its special role in Russia’s overall international relations – namely as a bridge to the Middle East, with whom Chechnya has developed close relations (Hallbach Reference Halbach 2018 ; Pietrasiak and Słowikowski Reference Pietrasiak and Słowikowski 2018 ; Kosach Reference Kosach and Laruelle 2019 ). Then, some studies analyze the international dimension of the Chechen conflict, focusing on the relevant, overarching geopolitical factors (Avioutskii Reference Avioutskii 2005 ). Finally, Chechnya’s connections to other countries have been considered within studies of North Caucasus broader international relations, such as with China (Babayan Reference Babayan 2016 ).

This diverse scholarship approaches Chechnya’s international relations from several perspectives, involving various conceptual frameworks and sources of empirical evidence. Yet Grozny’s official paradiplomacy has been understudied in general, with change and continuity remaining unexplained. The presidencies of Akhmat Kadyrov and Alu Alkhanov are typically not contemplated at all or addressed only insofar as they connect to Ramzan Kadyrov’s own activities abroad. A gap is thus left in our understanding of the transformations of Chechnya’s official paradiplomacy and its post-2000 trajectory, which is the empirical contribution of this article.

In this article, I maintain the scope of “paradiplomacy” widely, incorporating the persistent, institutional elements of a region’s international activities ( pace Duran Reference Duran 2015 , 21). Functionally, many external relations initiatives of sub-state entities can amount to Track Two diplomacy, such as when the region helps to maintain channels of communication between the parent state and another government (Jones Reference Jones 2015 , 24). Much of Chechen paradiplomacy, especially during Ramzan Kadyrov’s tenure, revolved around such functions. Yet, this overlap with Track Two diplomacy leaves out the fact that paradiplomacy takes place at the “antechamber” of sovereignty (Bartmann Reference Bartmann 2006 ). Namely, essential to the concept of paradiplomacy is the affirmation of political subjectivity of an entity that has many of the core features of a state – in the Weberian sense, territory, population, and law. Consequently, whilst both paradiplomacy and Track Two diplomacy contemplate ad hoc and permanent initiatives, paradiplomacy is based on the capacities that the region’s governing institutions have. For instance, regions frequently have an executive-level institution with legal competencies for carrying out their external engagements. In the Chechen case, the creation of the (pro-Russian) Chechen Republic in 2000 involved not only Russia gaining control of the region’s territory but also building governing institutions that would facilitate the governance of the region as a Russian federal subject. This has been a critical challenge for Moscow, one approached through the creation of governing institutions for Chechnya that differed from those of other federal subjects (Avioutskii Reference Avioutskii 2005 , 134; Mac-Glandières Reference Mac-Glandières 2017 , 206). That some of these institutions have competencies for external engagements is significant and places my analytical focus on both Chechen paradiplomatic initiatives and the Chechen institutions involved.

This conceptual arrangement could be applied to any case where the creation of paradiplomacy institutions happened in a top-down process involving territorial conquest or reunification. To deploy this concept on Chechen paradiplomacy, I start from the fact that Chechnya’s institutions were created in the context of the re-incorporation of Chechnya into the Russian state. In particular, Cornago’s Foucaldian explanation for the normalization of paradiplomacy sheds light on the transformations of Chechen paradiplomacy. According to Cornago, top-down processes of globalization and regionalization – as well as bottom-up processes of center-region negotiation and local government mobilization – have led to paradiplomacy becoming prevalent throughout the planet (Cornago Reference Cornago 2010 ). Chechnya’s case is revealing of the underlying power dynamics involved in the process of normalization of paradiplomacy – namely, as a process meant to facilitate governance.

Chechnya’s case is revealing as it is a case where political institutions were imposed by force on a region, including the institutions for paradiplomacy (Cornago Reference Cornago 2010 , 30). The authority for Grozny to carry out its own external affairs was given from the Russian state as part of Moscow’s conflict management strategy in Chechnya. The close connection between Grozny’s international activities, Russia’s reassertion of sovereignty over Chechen territory, and the synergy between Grozny and Moscow on international affairs suggest that Chechen paradiplomacy is inherently connected to the emerging power structures in post-Communist Russia. This widened scope of Chechen official authority would persist beyond the immediate years after 2000 (see Mac-Glandières Reference Mac-Glandières 2017 , 209).

Regarding paradiplomacy specifically, its inclusion in the government structures imposed by Russia in Chechnya is revealing of the micro-dynamics of the normalization of paradiplomacy. Especially relevant to Cornago’s framework of normalization is Moscow’s encouragement and involvement of the pro-Russian Chechen leadership in international relations. Then, the international agenda of Chechnya was, from the start in 2000, connected to central elements of Russia’s sovereign power and national security. As the description below points to, Chechen paradiplomacy was meant to tackle perceived threats to the Russian state – from within (radicalization caused by lack of investment) and from without (government-in-exile and the perceived threat of foreign intervention). Later on, as Chechen paradiplomacy expanded into high-level diplomacy with Middle Eastern governments, the focus went from direct threats to Russia to capacity building, with the notable development of a Chechen expeditionary force. Yet, this military paradiplomacy has been regarded by Moscow as an integral part of its strategic reassertion over the Middle East. This perception has kept Grozny’s international agenda in close connection to Moscow’s power politics. In sum, it is possible to make the hypothesis that, from 2000 on, Moscow set out broad parameters for Grozny’s self-led diplomatic action, with said parameters designed to enhance the governance of Chechnya.

To ground this argument empirically, in the following, I describe the emergence of Chechen paradiplomacy and its evolution from 2000 on. I proceed by reconstructing the paradiplomacy of Akhmat Kadyrov and Alu Alkhanov in detail, to then offer a broader portrait of Ramzan Kadyrov’s expansive external activities. This way, the article tackles its conceptual aim of arguing for the relevance of institutions for paradiplomacy and its empirical aim of offering an account of change and continuity in Chechen paradiplomacy. To maintain a focused analysis, I center my attention on analyzing official visits abroad, leaving other forms of external engagement to a secondary place. As the analysis shows further below, the emergence and evolution of Chechen paradiplomacy have been closely connected to the evolution of Moscow’s statecraft in the Vladimir Putin era.

Emergence and Early Trends: The Akhmat Kadyrov Period 2000–2004

The emergence of the official, pro-Russian Chechen paradiplomacy is embedded in the security juncture that surrounded the aftermath of the second Chechen war. In 1999, responding to a Chechen incursion into neighboring Dagestan, Moscow launched a military campaign known as the second Chechen war that ended with the effective control of Chechnya by the year 2000 (Seely Reference Seely 2005 ; Galeotti Reference Galeotti 2014 ). Upon recapturing Chechnya, the Russian government faced several challenges: the remaining insurgent forces, the Chechen “government in exile” that claimed to be the legitimate government of Chechnya (Akhmadov and Daniloff Reference Akhmadov and Daniloff 2013 ), and international condemnation from the war (Russell Reference Russell 2007 , 162). To a great extent, these challenges were tackled within Chechnya. First, part of the secessionist leadership was co-opted; notably, Akhmat-Haji Kadyrov became the head of the pro-Russia Chechen administration in 2000. Akhmat Kadyrov was a significant figure among Chechnya’s rebel leadership. In 1995, he was appointed by the Chechen rebel leadership Chief Mufti of Grozny and developed a large following within Chechnya. Russia’s move to co-opt Akhmat Kadyrov is an instance of what Malejacq calls outsourcing security to a local warlord by Moscow ( Reference Malejacq 2020 , 22). The Kremlin presented Akhmat Kadyrov as a “reformed separatist,” a message that bolstered the claim that the war in Chechnya was not a war on Chechens but on terrorism (Souleimanov Reference Souleimanov 2015 , 102–104). Second, the enduring insurgency was countered by a heavy-handed security apparatus that drew its strength from effective incorporation of indigenous forces from co-opted Chechen political factions. These, the literature argues, are among the essential pillars for the endurance of the Kadyrov regime and the relative stability of Chechnya itself (Taylor Reference Taylor 2007 ; Souleimanov and Aliyev Reference Souleimanov and Aliyev 2016 ).

The international dimension was just as complex, necessitating proactive diplomacy to remedy the fallout from the war. Crucially, the Kremlin saw the presence of Arab fighters and international channels of funds and arms for the rebels as a threat (Polyakov Reference Polyakov 2001 , 62–94; Avioutskii Reference Avioutskii 2005 , 252; 261). Even worse, the Kremlin saw Russia’s diminished reputation in the Muslim world as connected to emerging terrorist threats as many international terrorist organizations claimed the Chechen cause (Esposito Reference Esposito 2003 , 22). The threat was becoming tangible already next to Chechnya, as the risk that Dagestan would follow its neighbor and become another trouble spot was present at the time (Ware and Kisriev Reference Ware and Kisriev 1998 ). Russian diplomacy was deployed extensively to meet these challenges. First, the Kremlin rhetorically aligned with Washington on the “war on terror” by painting the Chechen rebels as part of the same international “jihadi” terrorist network to which Al Qaeda and the Taliban belong. This move was meant to place Russia’s use of force within the accepted international norms on the use of force, and it had enough success to gain the implicit acquiescence of the West regarding its war on Chechnya (Russell Reference Russell 2007 ; Kentros Klyszcz Reference Kentros Klyszcz 2019 ). This diplomatic move, whilst relatively successful, did not address the lack of legitimacy of the imposed authorities in Grozny in international eyes, especially among the Chechen diaspora and in the Middle East.

Akhmat Kadyrov’s role in Moscow’s diplomacy would become clear soon after the Russian capture of Grozny and his appointment as acting head of the administration of Chechnya in June 2000. Between Akhmat Kadyrov’s appointment and his assassination in May 2004, Akhmat Kadyrov went on official visits abroad a number of times, among others to Egypt, Germany, Iraq, Jordan, Saudi Arabia, Syria, Switzerland, and the United States. In these visits, he met with government officials, international organizations, NGOs, religious authorities, and representatives of diaspora organizations. In the following, I recount these visits in light of their significance to Moscow’s security and diplomacy as described above. Akhmat Kadyrov’s visits abroad generally pursued two objectives: (1) advocating for the legitimacy of his administration against the claims of the rebel government-in-exile and (2) attracting funding for Chechnya’s post-war reconstruction. Regarding the former, we can find that purpose in all his visits abroad: in September 2001, Kadyrov visited Egypt, Jordan, Iraq, and Syria to advocate against the Chechen rebel government-in-exile. In particular, he asked the governments of these countries to close down the existing channels of material support for the Chechen rebels present in their countries. In the process, he aimed at building good relations with some of the Middle East states that were previously critical of Moscow’s Chechnya military campaign (Kommersant 2001 ). Kadyrov would go on to advocate for his government and against the rebel claims to legitimacy in the Parliamentary Assembly of the Council of Europe (PACE) in November 2001 (Izvestia 2001 ), in Berlin in February 2002, at the United Nations in September 2003 (Kommersant 2003 ), at the UN Human Rights Committee in Geneva in October 2003 (UNHR 2003 ), at the 10th meeting of the Organization of Islamic Cooperation (OIC) in Malaysia in October 2003, and in Saudi Arabia in January 2004. The OIC meeting stands out as an example of the coordination between Kadyrov and the Kremlin; it was in that meeting that Putin announced Russia’s intention to join the OIC. The move in general, and the inclusion of Kadyrov in Putin’s delegation in particular, was meant to dissuade the accusations of Islamophobia in Russia and as a driver of the second Chechen campaign (Kosach Reference Kosach and Laruelle 2019 , 5–7). The visit to the US in 2003 was also planned with a degree of cooperation with the Kremlin (RIA 2003a ). Some of these visits had a public diplomacy component, as Kadyrov met with NGOs in Berlin in his 2002 visit (DW 2002 ) and visited the Ground Zero site in New York accompanied by Russian and US journalists ( Nezavisimaya Gazeta 2003 ); further, part of the 2004 Saudi Arabia visit involved film screenings to convince the Saudi leadership and public of the illegitimacy of the Chechen rebel forces (Kommersant 2004 ). The topics of humanitarian aid and reconstruction were part of the discussions and agendas pursued by Kadyrov in most of these visits as well. Even though Russia was experiencing rapid economic growth in the 2000s, its ability and willingness to fully finance Chechnya’s reconstruction was limited. Thus, states, NGOs, diaspora communities, and international organizations were all courted by the Kremlin and Kadyrov for aid and reconstruction funds. For instance, part of the purpose of his September 2001 Middle East tour was to gain support among the Chechen diaspora abroad and to mobilize diaspora and governments to send humanitarian help to Chechnya. Illustrative of the challenges Grozny found in these outreach efforts at the time was that Kadyrov failed to meet with the Jordanian Chechen diaspora on that tour (Kommersant 2001 ). A subsequent tour to the Middle East in 2003 would again aim at improving the image of the North Caucasus in a number of countries of the region. In Jordan, the objective was, in Kadyrov’s words, “showing the Jordanians the truth about the situation in Chechnya” (RIA 2003b ), as it was in Libya. In that country, Kadyrov even met with President Muammar Gaddafi (Kommersant 2003a ). The 2001 visit to Strasbourg and the 2003 visit to the UN also included the topic of humanitarian aid, as Kadyrov advocated for international partners to commit to more aid for Chechnya’s reconstruction (Izvestia 2001 ).

Throughout many of these visits, especially those concerning Muslim-majority states and the OIC, the external engagement discourse adopted by Kadyrov acquired a religious dimension. As mentioned above, the Kremlin saw the discourse of “Russia’s war on Islam” as potentially threatening, so the religious credentials of Kadyrov were at the forefront of his paradiplomacy. The January 2004 visit to Saudi Arabia – Kadyrov’s first trip abroad after his election as president of Chechnya – is illustrative of how Chechen paradiplomacy at the time combined religious discourse, the pursuit of legitimacy, and the attraction of reconstruction funds. In Kadyrov’s words, the purpose of that visit was to tell the “Muslim” public “where is politics and where is banditism” (Kommersant 2004 ). In a press conference, Kadyrov emphasized that his journey to Saudi Arabia demonstrated that the claims by the maskhadovtsy Footnote 1 about Kadyrov’s supposed lack of faith were false (Kommersant 2004a ); to emphasize his devotion, Kadyrov carried out the “small hajj” pilgrimage during his visit. The visit also addressed investment as Kadyrov met with the Saudi business community and the president of the Islamic Development Bank (Kommersant 2004b ). This visit stands out as Kadyrov was then the head of the delegation – rather than merely a member as he was in the 2003 US visit. Due to a still adverse security situation, Saudi investments would not be forthcoming in the short term.

Finally, the government of Akhmat Kadyrov also saw the institutionalization of Chechen paradiplomacy. In the spring of 2000, a press and communications committee was created in the provisional (pro-Russian) government, to be upgraded into a ministry in August of that year. It was reorganized in 2001, and in 2003 it was merged with the Ministry of the Chechen Republic for Ethnic Affairs, Regional Policy and Foreign Relations. Footnote 2 The Ministry acknowledges that the context of its creation was the war and counter-insurgency campaign, namely, with the mission to advocate for the legitimacy of the (pro-Russian) Chechen government (Ministerstvo n.d .).

Chechen Paradiplomacy under Alu Alkhanov 2004–2007

Between the assassination of Akhmat Kadyrov by the Chechen Islamist rebels in May 2004 and the rise of his son Ramzan Kadyrov as Head of Chechnya in February 2007, Chechnya was governed by acting president Alu Alkhanov, Chechnya’s minister of interior. Like Akhmat Kadyrov, Alkhanov’s rise to the presidency was possible thanks to Kremlin support, both in the aftermath of Kadyrov’s assassination and in the rigged August 2004 election. Alkhanov’s presidency was marred by the still intense conflict with the Chechen rebels and the slow pace of reconstruction. Then, the September 2004 Beslan school siege brought renewed federal attention to the North Caucasus and increased the Kremlin’s drive to centralize power in the federation. Internal power competition also undermined Alkhanov’s position from the very start. Alkhanov’s deputy prime minister and Kadyrov’s son, Ramzan, would compete with him for the control of Chechnya’s government. In fact, some describe Alkhanov’s presidency as powerless, given the extent to which Chechnya’s institutions were de facto under Kadyrov’s control, especially the Chechen security forces (Vatchagaev Reference Vatchagaev 2006 ). In February 2007, Alkhanov stepped down in a move widely attributed to Kadyrov’s successful maneuvering in intra-Chechen politics and in his relationship with the Kremlin. Particularly important was Putin’s personal rapport with Kadyrov father and later with Kadyrov son. Chechnya’s paradiplomacy under Alkhanov showed the endurance of his predecessor’s agenda but also new innovations in Chechen paradiplomacy.

The changing fortunes of the Chechen insurgency and changes in the international environment would also impact Chechen paradiplomacy during Alkhanov’s tenure. In spite of the persistent low-intensity fight and the Beslan attack, the Chechen insurgency entered a period of decline during Alkhanov’s tenure. In 2004, the conflict “stagnated” with the parties locked into a confrontational logic (Baev Reference Baev 2004 ). By 2007, the insurgency reconstituted around Doku Umarov’s “Caucasus Emirate” organization, abolishing the separatist project of the past decade and a half. Then, the assassination of rebel leader Aslan Maskhadov in March 2005 and his commander Shamil Basayev in October 2005 marked the beginning of the end for the low-intensity war of the second Chechen war (Galeotti Reference Galeotti 2014 , 79). From then on, the consolidation of pro-Russian rule in Chechnya was seen as inevitable, which was the message that Alkhanov’s paradiplomacy was meant to convey abroad (Jamestown 2004 ). Nevertheless, instability in Chechnya would endure throughout the rest of the 2000 decade, hindering large-scale Russian private investment and Foreign Direct Investment (FDI). Yet the international dimension of the Chechen conflict would continue to improve in the eyes of the Kremlin. Crucially, foreign state support for the Chechen insurgency was becoming a less acute threat in Russia’s eyes. As the OIC granted Russia observer status in 2005 and Western states remained inconsequential in their criticisms about human rights violations in Chechnya, the need for proactive diplomacy diminished. The only exception was the break with Washington that happened in 2004 when the US adopted a perspective on Chechnya that distinguished radical and moderate Chechen rebels. This perspective, coupled with the 2003 Iraq invasion, renewed Moscow’s sense of insecurity in the Caucasus. This time, however, the perceived threat was no longer about secessionism but about US influence in the post-Soviet space (Williams Reference Williams 2004 , 206–207).

These changing domestic and international circumstances had an impact in the scope and character of Chechen paradiplomacy. Similar to his predecessor, Alkhanov’s paradiplomacy was harmonious with Moscow’s foreign policy, with many visits abroad being planned in coordination with and with support from Moscow. Also, like Akhmat Kadyrov, Alkhanov’s few visits abroad focused on funding Chechnya’s reconstruction, consolidating relations with certain Middle Eastern states, and advocating for the legitimacy of his government in front of international audiences. The only trend under Akhmat Kadyrov that Alkhanov paused was the official, public contact with Gaddafi. Then, some ongoing trends would change; for example, Western engagements were becoming less relevant compared with the Middle East. Three tours abroad are particularly revealing of these trends: Alkhanov’s Middle East tour in 2004, his visit to PACE in 2004, and his visit to China in 2006. Shortly after the assassination of Akhmat Kadyrov, a follow-up Middle East tour in Jordan, Syria, and Saudi Arabia by Alkhanov in September 2004 reaffirmed Chechnya’s Middle East paradiplomacy. The main message, as before, was aimed against the rebels’ claims to legitimacy. The tour also aimed at improving the still negative image of the Chechen government in front of the diaspora. Regarding the Jordanian Chechen diaspora, there is no evidence that Alkhanov was substantially more successful than his predecessor in establishing himself as legitimate in their eyes (Jamestown 2004 ). In October 2004, Alkhanov visited PACE as a member of the Russian delegation. His message there centered on the renewal of the Chechen economy, reconstruction and the return of refugees. This message was meant to convey to the members of the Assembly the determination of Chechen and federal authorities to improve Chechens’ lives. Also, he dismissed the possibility of dialogue with the remaining rebels as advocated by some members of the Assembly; as mentioned above, Grozny and Moscow pushed the line that the remaining insurgents numbered in the hundreds. In spite of these attempts at improving the image of Chechnya’s official authorities, PACE voted in favor of condemning Russia’s human rights violations in Chechnya ( Rossiyskaya Gazeta 2004 ). Finally, in October 2006, Alkhanov headed a delegation to Zhejiang province, China, primarily with an economic agenda. During his stay, he extolled the growing stability of the republic, promoted business contacts, invited investment, and signed a bilateral cooperation agreement with the governor of Zhejiang ( Rossiyskaya Gazeta 2006 ). The China Development Bank would finance some of the promised investments into Chechnya, making China the first foreign country to invest in Chechnya’s economy (Moscow Times 2006 ). From that moment on, China would rise in prominence in Chechnya’s trade and investment (Babayan Reference Babayan 2016 , 6). Finally, the institutionalization of paradiplomacy in Chechnya would plateau under Alkhanov’s brief tenure. During his administration, the Ministry of the Chechen Republic for National Policy, Information and External Relations was reorganized once in 2005. The Ministry placed a greater emphasis on internal matters, with the external connections of Chechnya featuring less prominently in its mission (Stolitsa Plyus 2005 ; Stolitsa Plyus 2005a ). Yet, it would be under Alkhanov that the new concept of nationalities’ policy – which contemplates external relations – would be created, to be approved by his successor later on (see below).

An important trend outside of Alkhanov’s paradiplomacy was the diversification of means for Chechen external relations. Two in particular stand out in hindsight: the militarization of Chechen external relations and the rise in prominence of the Kadyrov Foundation as an arm of Chechen external relations policy. Regarding the former, during Alkhanov’s period, Chechen forces would become officially involved in Middle Eastern affairs. In October 2006, Russia’s Minister of Defense Sergei Ivanov mentioned that Chechen forces would join the Russian peacekeeping mission to Lebanon to protect a military engineer battalion operating there (Lenta 2006 ). The Chechen forces were considered by Moscow to be particularly suitable for deployment in the Middle East, given their experience and Muslim background. In addition, the improving security situation in Chechnya encouraged the Russian military to divert some resources from the Caucasus to other missions ( Rossiyskaya Gazeta 2006a ). As the security forces were under Ramzan Kadyrov’s control, it is possible that this decision involved him to at least some extent and that it was meant as a show of loyalty to Moscow (McGregor Reference McGregor 2006 ). The creation of the Akhmat Kadyrov Foundation in June 2004 would also have repercussions for Chechnya’s international relations, especially during Ramzan Kadyrov’s presidency. A lavishly funded and opaque NGO, the Kadyrov Foundation would promote Chechnya abroad through charitable actions, frequently through actions with religious symbolism. Oil wealth and a parallel tax system are credited as the sources of funding for this organization (RFERL 2015 ; Halbach Reference Halbach 2018 ).

Alkhanov, as weakened as his position was because of Ramzan Kadyrov’s de facto power and growing influence in Chechnya, was still the legitimate international face of Chechnya’s government. Consequently, he represented Chechnya abroad in his few official visits outside of Russia, albeit with an agenda broadly set by his predecessor. Further, this agenda remained harmonious with Moscow’s own foreign policy. In this sense, Alkhanov frequently connected Chechnya’s paradiplomacy to Russia’s foreign policy: “We would like to find common ground on many issues, especially in the field of business, economy, education, culture, religion, and this, in our opinion, could be facilitated by the simplification of the visa regime between Jordan and Russia,” Alkhanov said of his visit to Jordan (Belgorodskaia pravda 2004 ). As the economic fortune of Russia improved and Chechen reconstruction slowly went underway, the emphasis of Chechnya’s paradiplomacy shifted from aid to investment attraction, with early investments into Chechnya coming from China. In spite of his influence in the republic, Ramzan Kadyrov made no similar international visits as deputy prime minister. Yet, by October 2006, it was clear that he would succeed Alkhanov and become president of Chechnya ( Courrier International 2006 ).

Chechen Paradiplomacy under Ramzan Kadyrov 2007–2020: From the “War on Terror” to Confrontation with the West

Under Kadyrov, Chechnya’s international relations have grown in scope and importance, facilitated by a good disposition of the federal center and the further incorporation of Chechen paradiplomacy in the country’s foreign policy. In spite of the centralizing trends in the Russian Federation, Grozny was able to enhance its autonomy partly thanks to the Russian army withdrawal in 2009 (Souleimanov Reference Souleimanov 2015 , 102–104) and the fact that Putin considered Kadyrov to be an essential asset, both in the North Caucasus and abroad. The esteem of the Kremlin persisted even during Medvedev’s presidency (Black Reference Black 2015 , 53; Falkowski Reference Falkowski 2015 , 26–33). Kadyrov depends on Putin too, namely for federal transfers and regime security (Souleimanov and Jasutis Reference Souleimanov and Jasutis 2016 ). Then, since 2007, Russia’s confrontational stance towards the West made Grozny’s contacts with China and the Middle East more valuable, inducing Moscow to support their development. Simultaneously, the 2008 war with Georgia, the 2014 Sochi Olympics, the conflict in Ukraine, and the 2015 Syria campaign all made Russia’s southern flank even more sensitive for Moscow, bringing more attention to Chechnya’s regional role. Finally, from (at least) 2014 on, Moscow would increase the militarization of its foreign policy, in turn relying more on private military companies for plausible deniability in its power projection. Kadyrov – insisting on his loyalty to Moscow and his ability to quell the Chechen insurgency – also militarized Chechnya’s external relations, with Moscow’s support. In the following, I offer an overview based around the main features of Chechnya’s paradiplomacy under Ramzan Kadyrov, focusing on change and continuity from his predecessors’ times.

By 2007, paradiplomacy in the Russian Federation had changed extensively from the 1990s. Notably, the space for governors to act in disharmonious ways with the federal center had narrowed significantly (Renaldi Reference Renaldi 2019 ). From then on, paradiplomacy in Russia would diminish in intensity overall and progressively be driven by federal policy to focus on economic matters, trade, and investment attraction (Stremoukhov Reference Stremoukhov 2021 ). And yet, the importance of external engagements was clear from the start of the Ramzan Kadyrov presidency. The pace of his international activities was intense, meeting investors and making his first official visit abroad in the weeks following his investiture (RIA 2007 ; Vesti Respubliki 2007b ). Then, the institutionalization of external relations was clear from early on. The first decree issued by Ramzan Kadyrov as President on April 9, 2007, was “On the Concept of the State National Policy of the Chechen Republic,” Footnote 3 which tasked the Ministry of the Chechen Republic for National Policy, Press and Information with a new policy on nationalities, including on international connections. Regarding external engagements, the ministry was tasked with handling the external relations of the Chechen Republic on matters of trade, science, culture, sports, and partnerships with governments, investors, and the diaspora ( Vesti Respubliki 2007 ).

The geographic scope of official Chechen paradiplomacy has remained consistent in the Ramzan Kadyrov era, carrying its broad contours from his predecessors. Like Alkhanov, Kadyrov focused his official international activities on the Middle East, with Europe being mostly relegated to the target of covert operations. Russia’s own confrontational stance towards the West would set the stage for Kadyrov’s position towards Europe, yet this was not the case at the very start of his presidency. As evidence of a modicum of good will, the European Union allocated 20 million euros for the North Caucasus region, primarily targeting Chechnya and Ingushetia, under the circumstances of consolidating stability in Chechnya in 2007 (Black Reference Black 2015 , 50). Nevertheless, Kadyrov’s brutal rule would quickly erode Western interest in investment in Chechnya. In 2014, Kadyrov was sanctioned by the EU over his support of the Russian annexation of Crimea (Reuters 2014 ). Relations would sink further in 2017 following reports of widespread persecution against LGBT+ minorities in Chechnya that prompted international outcry as well as condemnation from the UN ( Novaya Gazeta 2017 ). In fact, Kadyrov would become the target of further sanctions from the US to a great extent as a response to his LGBT+ persecution campaign (NBC 2017 ). Since then, Kadyrov has disparaged the West, calling it a threat against Russia (RT 2017 ). France has also stood out as a target, particularly in the wake of the Charlie Hebdo attacks (RT 2015a ). The lull in diplomatic engagements has not prevented some official Chechen external activities from continuing in Europe. Evidence of this is the announcement of a diaspora outreach agency that contemplates Europe in its scope. This has been referred to as an instrument to launder the reputation of Kadyrov’s covert operations in Europe (Jamestown 2020 ). While relations with Europe declined, relations with China improved, primarily in trade and investment, continuing the trend initiated under Alkhanov. Until 2010, China played a small role in Chechnya’s commerce, to then become one of the most important countries for Chechnya’s trade. Then, the Alkhanov-era economic partnership with Zhejiang province was renewed and enhanced further in 2013 (Babayan Reference Babayan 2016 , 5–6). Finally, in 2017, Chechnya opened its strategic oil extraction sector to foreign investment, specifically from China (Regnum 2017 ).

The importance of the Middle East for Chechnya’s external engagement under Kadyrov has been unparalleled, with engagements growing in intensity at a brisk pace from 2007 on. Illustrative of this turn to the Middle East was the “Eastern Alternative” project. Days after Kadyrov became president, the Chechen government began to plan to systematically court investment from the Middle East as an alternative to Western investors ( Vesti Respubliki 2007a ). Thus, unlike Alkhanov, Ramzan Kadyrov has sustained a highly engaged partnership with the Middle East throughout his tenure. Shortly after becoming president in August 2007, Kadyrov visited Saudi Arabia and met with King Abdullah (RIA 2007 ); since then, Kadyrov would have a high-level visit or tour to the Middle East at least once every year, if not more (Luzin Reference Luzin 2018 ). There has also been a greater degree of institutionalization, with Kadyrov creating a special representative to the Middle East soon after taking office. He would also have special envoys active in sensitive parts of the region, as was done in Libya (Kommersant 2019 ; Hauer Reference Hauer 2020 ). These engagements follow the trends set by Akhmat Kadyrov, such as meeting with the same heads of state in the Gulf monarchies and reviving public connections with Muammar Gaddafi (Lenta 2008 ; Grozny Inform 2010 ). These diplomatic relations have assisted Moscow in keeping open communication channels with the heads of state of the region (Luzin Reference Luzin 2018 ). At the same time, Kadyrov benefits from these high-level encounters to raise his personal profile inside and outside Russia. Namely, his legitimacy is confirmed from outside and his influence grows among Russia’s and Central Asia’s Muslims, especially among the Vainakh diaspora (Laruelle Reference Laruelle 2017 ; Markedonov Reference Markedonov 2017 ). In addition, these contacts reportedly help Chechnya gain foreign investment and establish business connections (BBC 2018 ; Luzin Reference Luzin 2018 ). While all Gulf monarchies have expressed interest and invested in Chechnya, the United Arab Emirate (UAE) stands out as an engaged investor. Namely, the UAE sees investment in Chechnya as a way to build rapport with Chechnya, given the latter’s involvement in those conflicts seen as strategic by the UAE (Karasik Reference Karasik 2017 ; National 2018 ). The attraction of foreign investment has also required Moscow’s initiative; by the mid-2010s Moscow was actively pursuing its Middle Eastern partners to invest in the North Caucasus (Blank Reference Blank 2016 ). These diplomatic maneuvers have been successful – although not without setbacks, such as the brief 2016 rift in diplomatic relations with Riyadh over Kadyrov’s condemnation of Wahhabism (Jamestown 2016 ).

Kadyrov’s Middle East paradiplomacy, like his father’s, relies to an extent on a discourse of religious kinship. Kadyrov has promoted a national discourse that portrays Chechnya as “naturally” Islamic and himself as a pious believer (Avedissian Reference Avedissian 2016 ). Externally, this policy has been operationalized in several ways. Chechnya’s place in world Islam has been articulated by hosting major international conferences of Islamic scholars, sponsoring the building of mosques abroad, engaging religious authorities in foreign states, and cultivating the image of a defender of Islam against the West and against the Islamic State organization (ISIS). The religious discourse may also be shaping Grozny’s choices in developing partnerships abroad; the relationship with the Palestinian Authority stands out as it substantiates and operationalizes Chechnya’s religious paradiplomacy discourse. Indeed, trade and diaspora connections are not substantive with the Palestinian territories, yet relations between Mahmoud Abbas and Kadyrov have been close since 2008. Emblematic of their rapport was Kadyrov being awarded the Star of Jerusalem Order to the service of Palestine in 2019 (Caucasian Knot 2008 ; RIA 2019 ). Kadyrov often embraces the cause of Palestine in public declarations, most recently speaking against the 2020 US Middle East peace plan for the region (Laruelle Reference Laruelle 2017 , 23; Gazeta 2020 ). Finally, religious discourse sometimes directly facilitates other external action goals. For instance, in 2008, the Mufti of Chechnya assisted a major Islamic conference in Libya, where he conveyed the official narrative of the state of Chechnya at the time against the competing rebel narratives (Grozny Inform 2008 ). Then, the Kadyrov Foundation contributes to Chechnya’s image abroad frequently with initiatives that feature a religious tone, such as building and restoring mosques abroad. The foundation has also sent aid to Myanmar and Somalia (RFERL 2016 ; Hauer Reference Hauer 2018 ; RT 2018 ).

Beyond the discourse on Islam, it is the convergence in security interests between Chechnya, Russia, and certain Middle Eastern countries – the Gulf monarchies above all – that has driven Grozny’s diplomacy in that region. Chief among these interests is the fight against international terrorist networks, starting with those connected to the Caucasus Emirate and, later, those to ISIS. Grozny’s interest in developing capabilities and partnerships for military paradiplomacy has been partly driven by the enduring threat of Russian-speaking fighters operating outside of or returning to the North Caucasus. To meet this challenge, Chechnya’s external action would gradually militarize, gaining diplomatic clout by participating in expeditionary campaigns in Lebanon – during Alkhanov’s period – and in Syria. In terms of cooperation, arms sales and international training are the main features of this military paradiplomacy (Hauer Reference Hauer 2019 ). Thanks to lavish federal transfers and the oil wealth of Chechnya, Kadyrov has been able to fund a Chechen military force and a “spetsnaz academy” for Chechen and international teams. This academy would become capable of training foreign specialists from 2019 on, although those ambitions were announced as early as 2016 (RT 2016 ; Hauer Reference Hauer 2018a ). Due to the covert nature of many elements of Chechen external action, it is hard to establish a clear timeline of the militarization of Chechnya’s paradiplomacy under Kadyrov beyond the Syrian war. The watershed moment was in 2014 when Chechen troops reportedly joined Russian operations in Ukraine, namely to fight the pro-Ukrainian Chechen battalions (Luzin Reference Luzin 2018 ; Galeotti Reference Galeotti 2019 , 55). While Chechen involvement became evident later on, much remains to be known about the scope and timing of the Chechen operation in Syria. In 2014, Kadyrov announced the creation of a Chechen special security unit meant to fight terrorism both in the North Caucasus and beyond. The potential threat of fighters coming to Russia was given as the rationale for the creation of this unit (MEMO 2014 ). In October 2015, shortly after Russia’s air campaign in Syria began, Kadyrov asked Putin to deploy Chechen fighters in Syria to fight ISIS (RT 2015 ). In October 2015, Kadyrov said that no Chechen forces were operating in Syria and that he would be ready to send special forces were Putin to request it (Reuters 2016 ). By 2016, it became known that Chechen forces were officially on the ground in Syria. The number of Chechen forces deployed officially as “military police” was reportedly no more than 500, with an additional 300–400 Ingush forces present (Hauer Reference Hauer 2017 ).

Chechnya’s expansive external relations have been primarily harmonious with the Kremlin’s foreign policy. Kadyrov’s cultivated public image has always been that of Putin’s loyal “soldier,” yet occasionally disagreements arise. The one incident that stands out is the 2017 spat over the Myanmar Rohingya crisis. During a large rally that took place on September 4, 2017, Kadyrov openly criticized the Kremlin for inaction over the Rohingya plight (France24 2017 ). For Kadyrov, the move was meant to position him as a world figure in Islam ( Wall Street Journal 2017 ). Whilst it did not sour center-regions relations for good, the incident illustrated the wide scope of freedom afforded to Grozny in internal and external matters (Markedonov Reference Markedonov 2017 ).

The Chechen case problematizes the distinction between internal and external politics. It also highlights the importance of top-down influence on a region in shaping the emergence and evolution of its paradiplomacy. The creation of Chechnya’s paradiplomacy institutions reflects Russia’s imperatives in the Chechen conflict, especially since the de facto incorporation of the breakaway territory in 2000. The account presented above shows continuity and change in Chechnya’s international relations as they developed for two decades. Unlike the often unharmonious paradiplomacy that featured among many constituents of the Russian Federation in the 1990s, Grozny’s paradiplomacy since 2000 was mostly in harmony with the Kremlin. Chechen paradiplomacy also featured a rich discursive and thematic scope. Further, unlike the broader trends in the federation post-2006, Chechen paradiplomacy remained intensive and maintained a broad scope of action. Adverse security and diplomatic junctures prompted Moscow to give Grozny extensive external autonomy unlike any other Russian region, in spite of the risk of disharmony in external matters. While frequently proceeding with Moscow’s encouragement and direction, Chechnya’s external relations have operated in a self-led manner, tightly directed by the head of the republic.

Chechnya’s case is revealing because of its extreme conditions (e.g., war, extensive autonomy in a federal framework), yet it is also indicative of a more general implicit condition underlying the normalization of paradiplomacy. Namely, Chechnya shows how sovereign state interests and sub-state diplomatic action meet. Central to this argument is the fact that Chechnya, like other sub-state entities, depends on some measure of state approval for carrying out its international activities. At the same time, however, the Russian state saw in the encouragement of Chechen paradiplomacy a legitimate instrument for its own strategic interests outside and inside the country. Instead of emerging from bottom-up pressures, Chechen paradiplomacy is an institution that was grafted onto the pro-Russian government by Moscow to pursue – however autonomously – an external agenda that facilitates Russia’s governance of Chechnya. Thus, paradiplomacy neatly aligns with the state’s goals regarding external interactions and internal governance.

Regarding the external dimension, Chechnya’s paradiplomacy contributes to Russia’s relations to the Middle East in numerous and essential ways. The circumstances of the emergence of this agenda (Moscow’s assertion of sovereignty over Chechnya and the building of a pro-Moscow arrangement in Chechen politics), the persistence of this agenda beyond those circumstances (the “defeat” of the insurgency and of the “NATO intervention”), and the importance of the agenda (ensuring Russian sovereignty, supporting its Middle East strategy) suggest that Chechen paradiplomacy has had a critical role in the Russian governance of Chechnya. In other words, Chechen paradiplomacy is a persistent institution that supports Moscow’s governance of Chechnya itself, both asserting its sovereignty over its territory, bolstering its claims to legitimacy, and even supporting its economic plans. To summarize the argument: Moscow pursued the creation of Chechnya’s self-led, highly-personalistic paradiplomacy, which in turn was meant to assist Moscow’s security imperatives in the context of the Chechen conflict.

The implications of this case study for the field of paradiplomacy is that it broadens the understanding of paradiplomacy, pointing to a central paradox in this phenomenon. The Chechen case suggests that paradiplomacy can be regarded by central governments as a mechanism for affirming their control of their constituent regions by, in fact, giving up a portion of the expected monopoly of external relations, even in the military sphere. In this light, the case of Chechnya further adds evidence of the blur between paradiplomacy and the exclusive functions of the state in diplomacy and security matters. This points to the relevance of institutions in cases of top-down creation of paradiplomacy, which is shown by describing the creation of Chechnya’s official external relations, their overlap with Russia’s foreign and security policy, and their institutionalization. The implications for the understanding of the case of Chechen paradiplomacy are more straightforward: first, Chechen paradiplomacy has remained mostly consistent in spite of changes of leadership; and second, Chechen paradiplomacy is neither a supplement to Russian foreign policy nor an outgrowth of the Kadyrov regime. Chechen paradiplomacy is an integral component of the post-war institutional arrangement that renders Chechnya governable by the Russian Federation.

This article has been cited by the following publications. This list is generated based on data provided by Crossref .

Google Scholar

View all Google Scholar citations for this article.

Save article to Kindle

To save this article to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle .

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Volume 51, Issue 6
Ivan Ulises Klyszcz (a1)
DOI: https://doi.org/10.1017/nps.2022.8

Save article to Dropbox

To save this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about saving content to Dropbox .

Save article to Google Drive

To save this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about saving content to Google Drive .

Reply to: Submit a response

- No HTML tags allowed - Web page URLs will display as text only - Lines and paragraphs break automatically - Attachments, images or tables are not permitted

Your details

Your email address will be used in order to notify you when your comment has been reviewed by the moderator and in case the author(s) of the article or the moderator need to contact you directly.

You have entered the maximum number of contributors

Conflicting interests.

Please list any fees and grants from, employment by, consultancy for, shared ownership in or any close relationship with, at any time over the preceding 36 months, any organisation whose interests may be affected by the publication of the response. Please also list any non-financial associations or interests (personal, professional, political, institutional, religious or other) that a reasonable reader would want to know about in relation to the submitted work. This pertains to all the authors of the piece, their spouses or partners.

Yuri Kozyrev: Photographing 15 Years of Chechnya’s Troubled History

Worshipers leave evening prayer at the Heart of Chechnya Mosque in Grozny, April 17, 2015.Yuri kozyrev—NOOR for TIME

Yuri Kozyrev recalls the winter of 1999 as one of the most trying and tragic of his career as a photographer. It was the eve of Vladimir Putin’s ascent to the Russian presidency, and the height of the Russian bombardment of Chechnya, when entire towns in that breakaway republic were, as the Russians often put it, “made level with the earth.”

Kozyrev, a native of Moscow, documented both of Chechnya’s wars against Russia in the 1990s. The first one, fought between 1994 and 1996, had resulted in a humiliating defeat for Russia. But the carnage was far worse when the conflict resumed under Putin in 1999.

Arriving in Chechnya that fall, Kozyrev’s plan was to find and photograph two men amid the chaos of the Russian invasion. The first was Major General Alexander Ivanovich Otrakovsky, who was then commanding the Russian marines from his encampment near the town of Tsentaroy, a key stronghold of the Chechen separatists. The second was the general’s son, Captain Ivan Otrakovsky, who was serving on the front lines not far from the base, in one of the most hotly contested patches of territory.

The aim, says Kozyrev, was to document the two generations of Russian servicemen involved in the conflict – the elder brought up at the height of Soviet power during the Cold War, the younger in the dying years of Moscow’s empire. After weeks of negotiations, he finally managed to embed with the marines and to track down their general, a stocky man with a sly smile and a distinctive mole on the right side of his nose.

At the time, his command center was in an abandoned storage facility for crude oil, Chechnya’s most plentiful and lucrative commodity – and one of the main reasons why Russia refused to allow the region to secede. “It was incredible,” Kozyrev says of his first encounter with the general. “Here were these commanders living inside of a giant oil bunker.”

He recalls Otrakovsky as a kindly intellectual, nothing like the Russian cutthroats who would later be accused of committing atrocities in Chechnya. The general, whose troops referred to him affectionately as Dyed, or Grandpa, was willing to help Kozyrev. But he explained that reaching his son on the front lines would be extremely dangerous, as it would require passing through enemy territory around Tsentaroy.

That town was well known in Chechnya as the home of the Kadyrov clan, an extended family of rebel fighters whose patriarch, the mufti Akhmad Kadyrov, had served as the religious leader of the rebellion. During the first war for independence in the 1990s, he had even declared a state of jihad against Russia, instructing all Chechens that it was their duty to “kill as many Russians as they could.”

At the start of the second war, however, Kadyrov switched sides and agreed to help the Russians, causing a fateful split within the rebel ranks. While the more recalcitrant insurgents had turned to the tactics of terrorism and the ideology of radical Islam, Akhmad Kadyrov abandoned his previous calls for jihad and agreed to serve as Putin’s proxy leader in Chechnya in the fall of 1999.

That did not stop the fighting around his home village, as various insurgent groups continued attacking Russian and loyalist forces positioned around Tsentaroy. So none of the Russian marines were especially keen to move around the area unless they had good reason, and it took Kozyrev days to convince the Russian commander to allow him to reach the front lines. Eventually Gen. Otrakovsky consented, providing the photographer with an escort of about ten marines and two armored personnel carriers.

They set out on what Kozyrev recalls as an especially cold day, rumbling through fog or mist that made it difficult to see the surrounding terrain. As the general had feared, the group was ambushed. From multiple directions, Chechen fighters opened fire with machine guns and rocket-propelled grenades, forcing the convoy to retreat from Tsentaroy. One of the marines was killed in the firefight; three others were wounded.

When they returned to the base, it was clear from the glares of the troops that they all blamed Kozyrev for the fiasco, he says, and Gen. Otrakovsky advised the photographer to leave in the morning. “He said it may not be safe anymore for me to stay among his men,” Kozyrev remembers.

The trauma of that incident has lingered, weighing heaviest during his later assignments in Chechnya. Today, the region is ruled by Kadyrov’s son Ramzan, who took over after his father was assassinated in 2004. His native village of Tsentaroy has since enjoyed a generous stream of aid for redevelopment, including the construction of a beautiful mosque dedicated to Ramzan Kadyrov’s mother.

The rest of Chechnya has been rebuilt with similar largesse from Moscow, which has poured billions of dollars into the reconstruction of the cities and towns it had destroyed. When Kozyrev returned to Chechnya in 2009, nearly a decade after the end of the war, he says, “It blew my mind. The place is unrecognizable.”

The Chechen capital of Grozny – which the U.N. deemed “the most destroyed city on earth” in 2003 – is now a gleaming metropolis. Its center is packed with skyscrapers, sporting arenas, shopping plazas and an enormous mosque, the largest in Europe, dedicated to the memory of Akhmad Kadyrov.

His clan now rules the region unchallenged, having sidelined all of its local rivals with Moscow’s unflinching support. Throughout the region, portraits of Putin and the Kadyrovs are now plastered on the facades of buildings and along highways. Among the more ostentatious is a gigantic picture of Akhmad Kadyrov astride a rearing stallion, which adorns a building at the end of the city’s main drag – the Avenue of V.V. Putin.

The strangeness of the transformation, and of its architects, still seems astounding to Kozyrev, who last went on assignment to Chechnya for TIME in April. The trips always remind him of Gen. Otrakovsy, who died of a heart attack while commanding the marines in southern Chechnya, about four months after the young photographer had shown up to ask for his help. The general’s son, whom Kozyrev never did manage to find, went on to become a right-wing politician in Russia with close ties to Orthodox Christian conservative groups.

These were the men who executed the war that helped bring Putin to power. “But it was all the decision of one man to bring Chechnya back under control in ‘99. Putin decided to do that,” Kozyrev says. “And it’s incredible, when you think about it. But the men of Tsentaroy turned out to be his most loyal helpers.”

Yuri Kozyrev is a photojournalist and a TIME contract photographer. He is represented by Noor . In 2000, he received two World Press Photo photojournalism awards for his coverage of the second Chechen war in 1999.

Alice Gabriner , who edited this photo essay, is TIME’s International Photo Editor.

Simon Shuster is a reporter for TIME based in Moscow.

Russian marines repel an attack by Chechen rebels near Tsentaroy, Chechnya, Dec. 1999. In September of that year, Russian forces began military action against separatists. Initial operations were confined to air attacks, but on October 1, 1999, Russian troops entered Chechnya. By the beginning of December, the Russians had surrounded the capital Grozny, which they stormed on Dec. 25, 1999. Yuri Kozyrev—NOOR

More Must-Reads from TIME

Welcome to the Golden Age of Scams
Introducing TIME's 2024 Latino Leaders
How to Make an Argument That’s Actually Persuasive
Did the Pandemic Break Our Brains?
33 True Crime Documentaries That Shaped the Genre
The Ordained Rabbi Who Bought a Porn Company
Why Gut Health Issues Are More Common in Women
The 100 Most Influential People in AI 2024

Chechnya, Russia and 20 years of conflict

How the tiny region shaped post-Soviet Russia on the 20th anniversary of the start of first Chechnya war.

Moscow, Russia – Twenty years ago on Thursday, Moscow started what it thought would be a “blitzkrieg” against secular separatists in Chechnya, a tiny, oil-rich province in Russia’s North Caucasus region that had declared its independence.

But the first Chechen war became Russia’s Vietnam; the second war was declared a victory only in 2009. The two conflicts have reshaped Russia, Chechnya, their rulers – and those who oppose them.

In 1994, s hortly after Moscow invaded Chechnya in an effort to restore its territorial integrity, Akhmad Kadyrov, a bearded, barrel-chested Muslim scholar turned guerrilla commander, declared jihad on all Russians and said each Chechen should kill at least 150 of them.

That was the proportion of the populations on each side of the conflict: some 150 million Russians and less than a million Chechens in a small, landlocked province, which the separatists wanted to carve out of Russia.

Western media and politicians dubbed the Chechens “freedom fighters” – an army of Davids fighting the Russian Goliath.

Moscow was lambasted internationally for disproportionate use of force and rolling back on the democratic freedoms that former leader Boris Yeltsin was so eager to introduce after the 1991 Soviet Union collapse.

Tens of thousands died amid atrocities committed by both sides – and many more were displaced before 1996, when the Russians retreated, leaving Chechnya essentially independent.

Retreating was a humiliation for Russia’s military machine that less than a decade earlier had presented a seemingly formidable threat to the entire Western world.

Chechen against Chechen

Independence did not quite work out for the Chechens. The separatist government based in the ruined capital, Grozny, lost control over the rest of Chechnya.

Feuding field commanders and foreign jihadists, such as the Saudi known as Emir al-Khattab, ruled small districts with their own little armies. Kidnappings for ransom – along with primitive extraction of oil – were their main sources of income.

Many of the foreigners adhered to a puritanical Muslim ideology known as Wahhabism that ran counter to Chechnya’s Sufi traditions.

Akhmad Kadyrov, who was appointed as top Mufti of Chechnya, came into opposition with the puritans and their Chechen supporters, because he saw their extremist views as a threat to the separatist movement. In 1998, Kadyrov openly renounced the Wahhabis – and barely survived the first of many assassination attempts.

Kadyrov soon switched alliances, siding with the people upon whom he had once declared war – the Russians.

A virtually unknown ex-KBG officer, Vladimir Putin became Russia’s new prime minister i n August 1999 and w ithin weeks led a military operation against the Chechen fighters.

RELATED: Timeline: Attacks in Russia

When a series of explosions in apartment buildings in Moscow and two Russian towns killed more than 300 Russians, Moscow blamed Chechen rebels and embarked on an epic “anti-terrorist operation,” which became the second Chechen war.

Putin’s approval ratings skyrocketed, paving the way for his first presidency. A ided by Kadyrov and other Chechen clans who had pledged allegiance to the Kremlin, t he Russian military quickly returned most of Chechnya to Moscow’s control. In 2003, Kadyrov was elected Chechen president.

Russian targets

Cornered in Chechnya, the separatists took the war to Russia.

Attacks throughout the country became a grim reality of the new war and involved explosions in cities and towns, on planes and public transport.

At least two dozen attacks were carried out by female suicide bombers. Dubbed “black widows”, they became a sinister image imprinted on Russia’s collective psyche.

One such attack killed Akhmad Kadyrov in May 2004. His son, 27-year-old Ramzan Kadyrov, was too young to run for president at the time, but as head of his father’s security service, he quickly became Chechnya’s de facto ruler. I n 2007, soon after he turned 30, the younger Kadyrov was elected president.

Four months after his father’s assassination, Chechen separatists seized a public school in the town of Beslan taking more than 1,000 hostages, mostly children. Almost 200 kids died when Russian forces stormed the school. The incident changed the world’s attitude towards the Chechen cause – “freedom fighters” became “Islamic insurgents” in the Western media.

Meanwhile, the media in Russia came under attack.

“The saying was that it was journalists who won the first Chechen war,” says Tatyana Lokshina, deputy director of the Moscow branch of Human Rights Watch, an international rights watchdog.

Moscow used unfavourable media coverage of the war as an excuse to curtail press freedoms. The Kremlin took over all national television networks and most major newspapers.

[AP]

“For years, Vladimir Putin saw the pacification of Chechnya as his main achievement,” says Stanislav Belkovsky, a Moscow-based political analyst . “In that respect, Putin has a colossal psychological dependency on Chechnya and Ramzan Kadyrov who ensured the pacification.”

The Beslan crisis also served as a pretext to tighten political screws in Russia. Putin eliminated regional gubernatorial elections, complicated participation of opposition parties in elections, and limited democratic freedoms.

The public hailed Putin for bringing stability and pacifying Chechnya. The victory revived Moscow’s imperial ambitions – at least in the area of the former Soviet Union.

Shaping today’s Russia

Moscow won the brief 2008 Russo-Georgian war over the breakaway Georgian province of South Ossetia . In March 2014, Russia took over Crimea from Ukraine and helped unleash a civil war between pro-Russian separatists and the central Ukrainian government just a month later.

Both Chechen wars became systemic factors in shaping today's Russia. Instead of peaceful development inside the country we moved to the priority of external expansion by - Stanislav Belkovsky, political analyst

“Both Chechen wars became systemic factors in shaping today’s Russia,” says Belkovsky . “Instead of peaceful development inside the country, we moved to the priority of external expansion.”

Putin declared “the counter-terrorism operation” in Chechnya over in 2009 – just when things in North Caucasus took a turn for the worse.

Dagestan and several other provinces in the region became the new hotbeds of radical Islamism. A new generation of Moscow’s foes did not want secular separation – instead they are fighting to establish a “Caucasus Emirate” that includes adjacent Russian regions with sizable Muslim populations.

At least 529 people were killed and 457 wounded in North Caucasus in 2013, according to Kavkazsky Uzel, a Russian web portal that monitors the situation in the region. The confrontation has turned into “Europe’s most active armed conflict ” , according to the International Crisis Group, a conflict-monitoring organisation.

The insurgency became self-sustaining because of a vicious circle perpetuated by corruption and brutality.

Federal forces and police trigger the violence with extra-judicial killings, arrests, kidnappings and other abuses, according to rights groups and critics. They claim young men have no other options but to join the rebels because corrupt officials blacklist their families to extort bribes.

The fighters, in turn, blackmail corrupt officials who embezzle lavish funds from Moscow. The practise involves “sending a flash card” containing a video message in which bearded men demand a “jihad tax”.

Storming Grozny again

Ramzan Kadyrov was, perhaps, the least attentive man in the crowd of about 1,100 officials in an opulent Kremlin hall on December 4 during Putin’s annual address. The stocky 38-year-old Chechen leader fidgeted in his seat and constantly checked his phone.

Just hours before the Kremlin ceremony, a dozen Islamist fighters attacked Grozny, Chechnya’s newly-rebuilt capital. Shootouts in a publishing house, an empty school, and an office building killed 11 insurgents and 14 law enforcement officers.

A day after the attack, Kadyrov said the attackers’ families should be thrown out of Chechnya, their houses destroyed. At least six houses that belonged to relatives of the Grozny attackers have been burned down by masked men, Lokshina of Human Rights Watch said.

Kadyrov’s threats were not new to Chechens. During the second Chechen war, he led paramilitary squads known as kadyrovtsy that soon gained notoriety for abducting, torturing and killing separatists and civilians suspected of aiding them, according to human rights groups and survivors.

that soon gained notoriety for abducting, torturing and killing separatists and civilians [AFP]

A string of his political enemies and critics, including a former bodyguard, an investigative reporter, and a human rights activist have been gunned down in Chechnya, Moscow, Austria, and Dubai.

Kadyrov denied involvement in the contract-style killings.

Over the years, Kadyrov developed a penchant for luxury – he has a private zoo, race horses, and numerous sports cars. Pop stars, Hollywood actors and sportsmen show up at concerts held on his birthday.

His portraits are seen on billboards, government buildings and schoolchildren’s lapel pins; while streets, schools, mosques and military units are named after his father and mother.

Whatever he does is breaking news on Chechen television – he is shown threatening rebels and corrupt officials, boxing with his ministers, welcoming foreign dignitaries,and bestowing money, apartments and cars upon average Chechens.

Some say Kadyrov’s lifestyle and political ways make him look like an eccentric sovereign, not a public official on the Kremlin payroll.

“Today, Chechnya is a de facto independent state,” says Belkovsky. “Although formally [Kadyrov] shows loyalty to Putin and formally Chechnya is part of Russia.”

IMAGES

Empirical Research: Definition, Methods, Types and Examples
Empirical Research: Definition, Methods, Types and Examples
What Is Empirical Research? Definition, Types & Samples
What Is Empirical Research? Definition, Types & Samples in 2024
Empirical Research: A Comprehensive Guide for Academics
Definition, Types and Examples of Empirical Research

VIDEO

Research Methods
Empirical Research with example
2. Handmade Geiger Counter
Empirical research methods
1. Paper Boomerang
05 2015 How to do empirical research in Economics

COMMENTS

What Is Empirical Research? Definition, Types & Samples in 2024
Traditionally, experimental, laboratory-based research is used to advance knowledge in the physical and life sciences, including psychology. In recent decades, more and more social scientists are also adopting lab experiments (Falk & Heckman, 2009). ... Empirical Research and Writing: A Political Science Student's Practical Guide. Thousand ...
Empirical research
Empirical research is research using empirical evidence. ... researchers combine qualitative and quantitative forms of analysis to better answer questions that cannot be studied in laboratory settings, particularly in the social sciences and in education. In some fields, quantitative research may begin with a research question (e.g., "Does ...
Empirical Research: Definition, Methods, Types and Examples
Empirical research is defined as any research where conclusions of the study is strictly drawn from concretely empirical evidence, and therefore "verifiable" evidence. ... research involving quantitative and qualitative method to better answer questions which cannot be studied in a laboratory setting. LEARN ABOUT: ...
Empirical Research
In empirical research, knowledge is developed from factual experience as opposed to theoretical assumption and usually involved the use of data sources like datasets or fieldwork, but can also be based on observations within a laboratory setting. Testing hypothesis or answering definite questions is a primary feature of empirical research.
Data, measurement and empirical methods in the science of science
The discovery of empirical regularities in science has had a key role in driving conceptual developments and the directions of future research. By observing empirical patterns at scale ...
Empirical Research: Defining, Identifying, & Finding
Empirical research methodologies can be described as quantitative, qualitative, or a mix of both (usually called mixed-methods). Ruane (2016) (UofM login required) gets at the basic differences in approach between quantitative and qualitative research: Quantitative research -- an approach to documenting reality that relies heavily on numbers both for the measurement of variables and for data ...
Empirical Research
Strategies for Empirical Research in Writing is a particularly accessible approach to both qualitative and quantitative empirical research methods, helping novices appreciate the value of empirical research in writing while easing their fears about the research process. This comprehensive book covers research methods ranging from traditional ...
Empirical Research in the Social Sciences and Education
Empirical research is based on observed and measured phenomena and derives knowledge from actual experience rather than from theory or belief. How do you know if a study is empirical? Read the subheadings within the article, book, or report and look for a description of the research "methodology."
What is Empirical Research? Definition, Methods, Examples
Empirical research is the cornerstone of scientific inquiry, providing a systematic and structured approach to investigating the world around us. ... It's highly controlled and often conducted in a laboratory setting. Observational Research: Observational research involves the systematic observation of subjects or phenomena without intervention ...
Designing Empirical Research
Research design can be daunting for all types of researchers. At its heart it might be described as a formalized approach toward problem solving, thinking, and acquiring knowledge-the success of which depends upon clearly defined objectives and appropriate choice of statistical tools, tests, and analysis to meet a project's objectives.
Empirical Research
Empirical research is based on phenomena that can be observed and measured. Empirical research derives knowledge from actual experience rather than from theory or belief. Key characteristics of empirical research include: Specific research questions to be answered; Definitions of the population, behavior, or phenomena being studied;
Empirical Research: What is Empirical Research?
Definition of the population, behavior, or phenomena being studied. Description of the process used to study this population or phenomena, including selection criteria, controls, and testing instruments (such as surveys) Another hint: some scholarly journals use a specific layout, called the "IMRaD" format (Introduction - Method - Results ...
Types of Research Designs Compared
Types of Research Designs Compared | Guide & Examples. Published on June 20, 2019 by Shona McCombes.Revised on June 22, 2023. When you start planning a research project, developing research questions and creating a research design, you will have to make various decisions about the type of research you want to do.. There are many ways to categorize different types of research.
Empirical evidence: A definition
Empirical research is the process of finding empirical evidence. Empirical data is the information that comes from the research. ... The scientific method often involves lab experiments that are ...
LibGuides: Reference Guide: Searching for Empirical Articles
Empirical research is conducted based on observed and measured phenomena and derives knowledge from actual experience, rather than from theory or belief. Empirical research articles are examples of primary research. How do you know if a study is empirical? Read the subheadings within the article, book, or report and look for a description of the research methodology.
(PDF) Laboratory Experiments: The Lab in Relationship to Field
in relationship to empirical research in economics and economic theory. All three empirical techniques, lab experiments, field experiments, and field data, are in principle fully
PDF Writing an Empirical Paper in APA Style
Writing an Empirical Paper in APA Style A lab report is a writeup of an experiment and has the same components as a published research study. This handout provides general tips on how to write a psychology lab report. Course standards vary, so check with your instructor if you are not sure what is required. Using APA Style
Empirical Articles
The authors will have collected data to answer a research question. Empirical research contains observed and measured examples that inform or answer the research question. The data can be collected in a variety of ways such as interviews, surveys, questionnaires, observations, and various other quantitative and qualitative research methods. ...
How do I know if a research article is empirical?
Empirical research articles are considered original, primary research. In these types of articles, readers will generally find the following sections organized by IMRaD format (Introduction, Method, Results, and Discussion). (I)ntroduction: Includes the research hypotheses and the literature review (current research on or related to the topic).
Experiment
In the scientific method, an experiment is an empirical procedure that arbitrates competing models or hypotheses. [2] [3] Researchers also use experimentation to test existing theories or new hypotheses to support or disprove them.[3] [4]An experiment usually tests a hypothesis, which is an expectation about how a particular process or phenomenon works.. However, an experiment may also aim to ...
Chechnya's Paradiplomacy 2000-2020: The Emergence and Evolution of
In an attempt to answer the questions regarding the facilitating conditions for Chechen paradiplomacy, this article pursues two objectives, an empirical one and a conceptual one. First, I trace the evolution of Chechen paradiplomacy to evince the interaction between opportunity structures and Grozny's international agency in the 2000-2020 ...
Understanding the Chechen conflict: Research and reading list
How suicide terrorism as a tactic made its way into Chechnya is the topic of this paper, which provides an analysis of the events concerning the importation of militant ideologies and radical terrorist movements taking place since the Chechen declaration of independence as well as an empirical and theoretical analysis of Chechen suicide ...
Yuri Kozyrev: 15 Years of Chechnya's Troubled History
The first one, fought between 1994 and 1996, had resulted in a humiliating defeat for Russia. But the carnage was far worse when the conflict resumed under Putin in 1999. Arriving in Chechnya that ...
Chechnya, Russia and 20 years of conflict
11 Dec 2014. Moscow, Russia - Twenty years ago on Thursday, Moscow started what it thought would be a "blitzkrieg" against secular separatists in Chechnya, a tiny, oil-rich province in ...

Empirical Research: Definition, Methods, Types and Examples

Empirical research: Definition

Types and methodologies of empirical research

MORE LIKE THIS

QuestionPro: Leading the Charge in Customer Journey Management and Voice of the Customer Platforms

What is Driver Analysis? Importance and Best Practices

Was The Experience Memorable? (Part II) — Tuesday CX Thoughts

Data Discovery: What it is, Importance, Process + Use Cases

Other categories

Empirical Research

Access this chapter

Bibliography

Author information

Corresponding author

Editor information

Rights and permissions

Copyright information

About this entry

Download citation

Share this entry

Data, measurement and empirical methods in the science of science

Similar content being viewed by others

SciSciNet: A large-scale open data lake for the science of science research

A dataset for measuring the impact of research data and their curation

Envisioning a “science diplomacy 2.0”: on data, global challenges, and multi-layered networks

Measurement

Individuals

Institutions

Empirical methods

Descriptive and predictive approaches

Regression analysis

Mechanistic models

Machine learning

Causal approaches

Towards causation

Quasi-experiments

Experiments

Acknowledgements

Author information

Corresponding author

Ethics declarations

Peer review

Additional information

Rights and permissions

About this article

Share this article

This article is cited by

Quantifying attrition in science: a cohort-based, longitudinal study of scientists in 38 OECD countries

Rescaling the disruption index reveals the universality of disruption distributions in science

Signs of Dysconscious Racism and Xenophobiaism in Knowledge Production and the Formation of Academic Researchers: A National Study

Quick links

Empirical Research: Defining, Identifying, & Finding

Characteristics of Empirical Research

Characteristics of Quantitative Research

Characteristics of Qualitative Research

Research: Overview & Approaches

Introduction to Empirical Research

Your Librarian

Penn State University Libraries

Contact the Librarian at your campus for more help!

Introduction: What is Empirical Research?

Reading and Evaluating Scholarly Materials

What is Empirical Research? Definition, Methods, Examples

What is Empirical Research?

Characteristics of Empirical Research

Importance of Empirical Research

How to Conduct Empirical Research?

1. Select a Research Topic

2. Formulate Research Questions

3. Review Existing Literature

4. Define Variables

Empirical Research Design

Types of Empirical Research

Experimental Design

Observational Design

Survey Design

Case Study Design

Qualitative vs. Quantitative Research

Data Collection for Empirical Research

Sampling Methods