experimental study feasibility

• Search Menu
• Sign in through your institution
• Advance articles
• Author Guidelines
• Submission Site
• Open Access
• Why Publish?
• About Research Evaluation
• Editorial Board
• Advertising and Corporate Services
• Journals Career Network
• Self-Archiving Policy
• Dispatch Dates
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

1. the conservatism of peer review, 2. conservatism of peer review and the specific genre of proposal-writing, 3. the empirical material: research initiative and sample of proposals, 4. analytical methods, 4. kinds of evidence for mastering research, 5. differentiation by the kinds of evidence, 6. conclusion.

• < Previous

Evidence of research mastery: How applicants argue the feasibility of their research projects

• Article contents
• Figures & tables
• Supplementary Data

Eva Barlösius, Kristina Blem, Evidence of research mastery: How applicants argue the feasibility of their research projects, Research Evaluation , Volume 30, Issue 4, October 2021, Pages 563–571, https://doi.org/10.1093/reseval/rvab035

• Permissions Icon Permissions

Although many studies have shown that reviewers particularly value the feasibility of a proposed project, very little attention has gone to how applicants try to establish the plausibility of their proposal’s realization. With a sample of 335 proposals, we examined the ways applicants reason the feasibility of their projects and the kinds of evidence they provide to support those assertions. We identified three kinds of evidence for mastering research: the scope of scientific skills, the presence of different assets, and the use of stylistic techniques. Applicants draw on them to align the project with scientific standards, embed it in the current state of research, and meet the scientific field’s expectations of how scientists should conduct a project. These kinds of evidence help substantiate a project’s feasibility and to distinguish the project from other proposals. Such evidence seems to correspond with a project’s positive review and approval (grant success). Evidence of research mastery was cited more often by the authors of the successful (approved) proposals than by the authors of the unsuccessful ones. The applicants of the successful proposals gave details of their planned experiments, emphasized their broad methodological and technical competence, and referred to their own preliminary scientific work.

A general critique of peer review is that it has ‘an inherent conservative bias’ ( Luukkonen 2012 : 48). There is broad consensus that peer reviewers criticize, negatively evaluate, and reject unconventional research ideas and manuscripts treating uncommon topics ( Travis and Collins 1991 ; Laudel 2006 ; Heinze 2008 ; Boudreau et al. 2016 ). Project proposals appear to be ‘pre-structured’ by the ‘conservatism of peer review panels that judge project proposals’ ( Franssen et al. 2018 : 29). This feature of the peer review process is often held responsible for ‘conservative science’ ( O’Connor 2019 ). Studies reveal that the conservatism of peer reviews operates strongly even in the assessment of proposals submitted to funding schemes expressly designed to support groundbreaking and adventurous research ( Luukkonen 2012 ; Laudel and Gläser 2014 ). However, it is too simplistic to assume that the conservatism of peer review leads automatically to conservative proposals. Studies have found that applicants actually do dare submit proposals that pose ‘a radical new idea with polemical references to new findings that question established theories’ ( Philipps and Weißenborn 2019 : 892) or claim to pursue ‘revolutionary’ treatment of a ‘key research question’ ( Barlösius 2019 : 924). The issue that therefore needs addressing is whether there is evidence of conservative scientific practices in the proposals themselves.

Which assessment criteria underlie the conservatism of the peer review process? Overall, the reviewers weigh whether the research project as described in the proposal can be executed as planned: Is the project feasible? Important clues that reviewers seek when considering this crucial matter as they examine a project include evidence that it reflects current research practice, involves up-to-date methodologies, and builds on previous research ( Heinze 2008 ; Luukkonen 2012 ). Most studies on the conservativism of peer review are analyses of the review procedure itself. They are based on interviews with reviewers, participant observation of the process by which a project is selected for approval, or expert interviews with representatives of funding bodies ( Langfeldt 2001 ; Heinze 2008 ; Lamont 2009 ; Luukkonen 2012 ; Laudel and Gläser 2014 ). Such studies have concluded that the conservativism of peer review is due mainly to the way in which the reviewers and panelists conduct the review process and to the criteria they apply when evaluating the research proposals. In particular, they attribute conservatism to the way in which the reviewers analyze the feasibility of a project, namely, by determining whether it follows existing approaches.

One aspect that has received little attention so far, however, is how applicants try to establish the plausibility that their proposed project can be conducted. With that issue in mind, we formulated three research questions: How do the applicants argue their project’s feasibility, especially if they submit their proposal to a funding initiative that invites risky research? Do the applicants tend to favor broadly accepted research, an inclination that could be regarded as conservative? On what kinds of evidence do the applicants claim that their projects are feasible? We analyzed 335 research proposals to find answers.

This article starts with a review of the literature on the conservatism of the peer review process, the concept of scientific conservatism, and the specific genre of proposal-writing. Next, we describe and justify the empirical material and explain our methodological approach. We then show how scientists provide evidence for the feasibility of their projects, and we identify three kinds of evidence of research mastery. Thereafter we analyze how each kind of evidence can be used to argue that a proposed project is indeed workable. In the conclusion, we discuss whether the word conservative is the correct characterization of the way in which applicants show the feasibility of their proposals.

Several studies on the ‘conservative bias of the peer review’ ( Luukkonen 2012 : 49) document the preference reviewers have for conventional research projects that assure a high degree of feasibility (e.g. Chubin and Hackett 1990 ; Travis and Collins 1991 ; Horrobin 1996 ; Berezin 1998 ; Langfeldt 2001 ; Laudel 2006 ). Lamont (2009) analyzed in detail how evaluation panels work, how they come to a consensus, and how disciplinary cultures and definitions of excellence are appraised during the evaluation processes. Luukkonen (2012) , too, studied members of such panels and found that their overarching concern was the feasibility of the project (p. 55). The members of the panels verified feasibility mainly by five criteria: (1) the researchers’ abilities to ‘apply up-to-date methodologies, use required instruments, and carry out the research within the proposed time frame’ ( Luukkonen 2012 : 55); (2) a research plan that is regarded as ‘plausible and workable’ ( Luukkonen 2012 : 55); (3) evidence that the applicants are ‘prepared for any contingency and have an alternative course of action in case the plan does not turn out as expected’ ( Luukkonen 2012 : 55); (4) the presentation of how the project builds on previous research; and (5) an explicit warranty that ‘the needed equipment is available’ ( Luukkonen 2012 : 55).Whereas Luukkonen (2012) asked which criteria reviewers apply to assess the feasibility of proposals, we investigate the manner in which applicants seek to demonstrate it. Luukkonen’s work indicates that projects are subject to certain expectations and that the applicants are likely to be largely aware of them. However, it is possible that the applicants bring a far greater repertoire to bear on their effort to reason their project’s feasibility persuasively.

Some studies have investigated how and why research traditions, personal involvement, and other interests may promote a conservative bias in the peer review process ( Chubin and Hackett 1990 : 62; Travis and Collins 1991 ). Other studies have scrutinized the effects of funding programs, notably those created to sponsor ‘high-risk and outside-the-box research’ ( Heinze 2008 : 303; Laudel and Gläser 2014 ; Van den Besselaar, Sandström and Schiffbaenker 2018 ). The scholars wanted to find out the extent to which particularly heterodox projects benefit from such initiatives. Overall, they concluded that the conservative wariness inherent in peer review appears to apply even to those funding schemes. For example, Heinze identified a ‘tension between plausibility and scientific value of the research’ and ‘its originality’ ( Heinze 2008 : 302). The research proposal’s plausibility is assessed by its ‘conformity with the current scientific practice’ (302), which can be regarded as conservative. The reviewers survey the feasibility of the projects mainly by pointing out direct links to already established research. The great importance that reviewers and panelists attach to the achievability of the project appears to substantiate the conclusion that peer review tends to be conservative. ‘Ensuring the success of the project,’ as Serrano Velarde has shown, is a ‘major concern in applicant rhetoric’ ( Serrano Velarde 2018 : 85).For our analysis, this observation suggests the value of looking at how the applicants argue their projects’ feasibility and at examining the reasons they give for this assertion. However, the researchers may have a different view of how they demonstrate the practicality of their project, so the analysis of proposals should be open to a broad understanding of feasibility.

Currie concluded that funding programs often promote what he called, ‘conservatism-boosting features of scientific practice’ ( Currie 2019 : 5). Like other scholars in this area, he is interested in how scientific conservatism is sustained and how it is reflected in research practices ( Kummerfeld and Zollman2016 : 1058; Stanford 2019 ). Bedessem (2021 : 3) criticized the acquiescence of researchers who ‘accept that current funding arrangements … have conservative effects’ on scientific practice. These studies do not focus specifically on the conservatism of the peer review process but rather on a broad understanding of scientific conservatism. They look at research practices to explain conservatism in science. This approach could be instructive for analyzing research proposals, for scholars generally outline the projected research process. Bedessem followed the discussion on scientific conservatism, which ties in with Kuhn, who addressed ‘essential tensions’ and developed the concept of ‘convergent normal practice’ ( Kuhn 1991 : 146). It is ‘based firmly upon a settled consensus acquired from scientific education’ ( Kuhn 1991 : 140). Taking up Kuhn’s concept, Bedessem (2021 : 3) introduced the notion of ‘practical conservatism’ which covers ‘all the dimensions of scientific activities’ and implies ‘a tendency to congregate on the apparently safer alternatives to solve a given problem’. Practical conservatism prioritizes methods, theories, and practices that are recognized as feasible, plausible, and practicable. He referred to them as a ‘system of practices’ that tends to favor its own stability and prefers such scientific concepts and approaches that ‘were currently being pursued’ ( Bedessem 2021 : 14). These studies on scientific conservatism show how misleading it is to hold the peer review system exclusively accountable for the conservatism of research proposals. Presumably, there is a ‘practical conservatism’ affecting all dimensions of scientific activity. Rather than focusing solely on the matter of feasibility, we should therefore have the present study encompass all scientific activities described in the proposals and analyze the extent to which they are defined as doable.

Proposals constitute a specific genre of academic writing ( Gross 1990 ; Swales 2004 ; Van den Besselaar, Sandström and Schiffbaenker 2018 ). They draw on various rhetorical techniques to underscore the plausibility of a project. Myers (1990) found that the biological research proposals he analyzed were written in a cautious tone and a reserved style and did not harshly criticize existing research. However, other studies have shown that such restraint is not always the case; some applicants do in fact criticize past research directly and sharply ( Philipps and Weißenborn 2019 ; Barlösius 2019 ). Applicants have to create ‘textual evidence’ of their project’s viability ( Myers 1990 : 58), especially to allay the reviewers’ concern that the project might not run as conceptualized in the proposal. Markowitz, for instance, has shown that ‘expression of certainty positively correlated with funding success’ ( Markowitz 2019 : 265). Researchers can resort to the typical rhetoric of the proposal-writing genre to portray themselves as ‘good scientists’ ( Myers 1990 : 59). They also have to demonstrate familiarity with existing research in their field ( Myers 1990 : 58). Furthermore, it is advantageous to adhere to conventional structures by identifying solvable research issues and extensively describing the planned research project ( Connor and Mauranen 1999 ; Connor 2000 ).

Linguistic and rhetorical studies, particularly those that focus on proposals ( Connor and Mauranen 1999 ), are helpful for identifying their passages that contain high ‘textual evidence’ and for practicing a genre-based approach. One of the recognized concepts that these studies have for identifying the organization of texts is move analysis, which is defined as a discursive segment that performs a particular communicative function ( Swales 2004 ). Performing linguistic analyses of grant proposals, Connor and Mauranen (1999) identified 10 moves, such as ‘reporting previous research’, ‘means’, and ‘competence claims’. We decided to use them as searchlights for the empirical analysis of the proposals in our study’s sample.

In summary, this review of the literature reveals that our question has not yet been investigated but that related studies do offer some helpful input for approaching it. From the literature on the conservatism of peer review, we learned to look for how the applicants try to prove the feasibility of their proposals. The studies on scientific conservation showed that we should take into account all dimensions of scientific activities described in the proposals. Lastly, the analysis of the rhetoric of proposals indicated that we should pay special attention to rhetorical techniques for producing textual evidence and emphasizing the plausibility of the proposal.

For the empirical analyses, we sought out a research initiative inviting applicants to submit proposals for a bold research project. The VolkswagenStiftung 1 research initiative entitled ‘Experiment!’ satisfied that condition. 2 It called for ‘fundamentally new research topics’, and the projects were to be ‘unorthodox’ or ‘radically new’. This conceptual orientation of Experiment! led us to expect rich empirical material on how feasibility is presented in the proposals. The proposals were not to exceed 1000 words. This limitation made it possible to analyze the whole text and treat all the described research activities as empirical material, enabling us to remain open for a very broad understanding of feasibility. Proposals, which had to be written in English, were accepted from people with doctorates in science, engineering, or the life sciences (including proximate disciplines in the behavioral sciences). No additional documents were required, such as a list of publications or a curriculum vita, which are usually included to substantiate the applicant’s scientific experience and reputation. Details on the equipment and size of the lab or a list of successfully completed projects were not required, either. The applicants, therefore, could not avail themselves of additional documents to illustrate ‘their own experience-based understanding of practice’ ( Kaltenbrunner and De Rijcke2019 : 863) or to document their own scientific performance through, say, of a publications list. Nor was it necessary to submit a list of projects proving the lab’s use of established methods and existing instruments. To ensure full anonymity of the applicants, the VolkswagenStiftung refrained from asking for these documents. A jury evaluated the proposals in a blind procedure.

When the VolkswagenStiftung set up the research initiative called Experiment!, we were asked whether we would be interested in doing research on it. We would have complete freedom to formulate our research question and determine the methodological approach. For this purpose, the organization facilitated the creation of different samples drawn from the 2,304 research proposals that had been submitted from 2013 through 2016. The proposals are anonymized. Gender, academic status (professor, postdoc) and discipline of the persons in the samples are recorded on a separate list. We have narrowed the pool of applications down to a representative sample of 335 proposals according to the gender, academic rank, and subject area of the applicants, who come from the natural sciences, medicine, and engineering. 3 The main disciplines in the sample are biology (including biophysics and biomedicine), chemistry, engineering, medicine, neurosciences, and physics. The sample also includes a few proposals from other disciplines, such as computer sciences, environmental sciences, geosciences, mathematics, and pharmaceutics. Overall, the representative sample encompasses the breadth of the natural sciences and medical and engineering disciplines. More important, the size of this sample allows in-depth qualitative analyses in order to discern how the applicants argue the feasibility of their research projects. The sample includes 11 successful proposals, which allows a preliminary consideration of whether and how they differ from the non-successful proposals in the way their authors reason the feasibility of their project.

The first reading of the proposals revealed that, despite the thousand-word limit set by the funding body, the applicants had included copious explanations of the planned procedures, elaborate descriptions of the methods and instruments they bring to bear, and detailed presentations of the planned experiments. These descriptions comprised far more than mere statements about the planned projects. Many of the proposals included information about what would be changed if the experiments could not be conducted as initially planned and specified which additional competences, instruments, and experiences were to be tapped for the project. We also found specific stylistic techniques in the proposals. For example, the applicants dialogued hypothetically with the reviewers by raising a potential problem and then immediately solving the issue.

To determine as broadly and specifically as possible how the researchers substantiated the running of their projects, we adopted an open approach oriented to grounded theory ( Schreier 2012 ). This methodological consideration led us to conclude that it was necessary for all discursive segments having a particular communicative function to be coded in a manner that distinguished between explanations of three aspects: (1) why the project would proceed as planned; (2) what the researchers would do if a step could not be taken as anticipated, and (3) which competencies and resources the researchers would rely on if something did not work out as proposed. We also identified instances of stylistic techniques designed to underscore the plausibility of the project. For example, we found that applicants wrote descriptors like ‘simple and elegant’ to justify confidence in their ability to conduct the project successfully. We looked for stylistic techniques that applicants employed to convince the reader that they had thought of every contingency. In summary, we first coded all assertions by the applicants that the project was feasible, that is, assurances that they would master whatever problems might arise and that they had already taken account of all challenges. This approach guarantees the broadest possible range of statements that the project was reasonable.

Next, we selected and minutely analyzed appropriate sections of text, honing our codes through systematic comparison of the passages that spelled out reasons why the project would run successfully. The segments were grouped into categories: ‘the applicant’s own preliminary scientific work’, ‘methods and techniques’, ‘detailed description of the experiment’, ‘technical equipment’, and ‘scientific cooperation’. Three stylistic codes were retained, namely, ‘difficulties and solutions’, ‘different ways of organizing the text’, and the description of the project as ‘simple and elegant’.

The third step of analysis entailed systematizing these codes and assigning them to superordinate codes. To this end, we sought to identify the measure(s) that the applicants relied on for substantiating that their project would proceed as planned. We found that they cited different kinds of resources that they had at their disposal and intended to bring to the project. The applicants adduced three main kinds of resources they would draw on to guarantee the project’s feasibility (scientific skills, different assets, and stylistic techniques). For each of these three resources, we formulated three superordinate codes:

Scientific skills , such as methods and techniques as well as a detailed description of the experiment. Methods and techniques subsumed all statements in which the applicants wrote that they themselves and/or their lab had all necessary methodological and technical competences. The code ‘detailed description of the experiment’ comprises the formulations about particulars of the empirical approach.

Different assets to be brought to the project, including ‘technical equipment’, ‘scientific cooperation’, and the researchers’ ‘own preliminary scientific work’. ‘Technical equipment’ encompasses passages assuring that all required instruments were in place and readily accessible. If the researchers pointed out that they could draw on expertise or instruments from other labs for their project, we coded these segments as ‘scientific cooperation’. The reference to the applicant’s previous work, such as preliminary experiments, data, and proofs, were coded as ‘own preliminary scientific work’.

Stylistic techniques , including ‘difficulties and solutions’, ‘different forms of ordering’, and claims that the project was ‘simple and easy’ to conduct. Under ‘difficulties and solutions’, we summarized all passages in which the applicants employ the stylistic technique of raising a possible problem and then immediately solving it. The code ‘different forms of ordering’ referred to statements about the formal organization of the project. If the applicants made use of characterizations such as elegant , simple , easy , or beautiful to claim that the project would proceed as predicted, these segments were coded with ‘simple and easy’. We coded the entire sample and trained a doctoral student 4 of sociology with several years of experience in qualitative coding to apply our coding rules. She coded 20% of the sample. The total intercoder reliability was 89.49% (see Table A.1 in the Appendix).

4.1 Scientific skills

The first of our three superordinate codes addressed scientific skills. To underline that the project would run as planned, the applicants called attention to their competence and proficiency, especially regarding the method(s) and technique(s) on which they would draw and the expected reliability of that repertoire. Four main arguments of this kind informed the effort to emphasize that the project would turn out well. First, the applicants professed that they were familiar with the full range of ‘standard methods’ and ‘well-established techniques’. The scientists thereby indirectly signaled that they had broad methodological and technical experience. For instance, they stated that the ‘fabrication of [specific] microwave resonators on a semiconductor substrate is now a standard procedure’ for them (Ph_64). 5 The second strategy involved explaining that the applicant was already successfully employing the project’s methods and techniques in his or her lab. They offered this information to mitigate concern about any methodological or technical aspects that might thwart the project. A third line of reasoning was that the project could accommodate more than only one or two methods and techniques and that they had already been tested successfully in the lab and were available. For example, ‘the following features will be analyzed with modern signal-analysis methods developed and tested in [our/my] own previous studies’. This sentence was followed by an enumeration of four quite different methods (Bio_43). 6 The fourth methodological and technical approach to promising that the project would run as planned was to give a detailed description of how the methods and techniques were to be applied, how they would proceed, which adaptions were necessary, and which results would be arrived at. For example, ‘The [specific] technique is relatively simple: while the subject lies in an MRI scanner, her [specific kind of] activity is measured and visualized in real time’. Further, the researchers described in detail what the subject would be asked to do in the scanner, how long it would take until the person knows how to act and what he or she would learn during his or her stay in the scanner. Then came an explanation about which data would be collected and what could be measured with this technique. The detailed explanation concluded with the following sentence: ‘The risks and side effects of the technique are nearly as slight as those of standard fMRI measurements’ (Ne_72). This kind of argumentation transitioned to the next way of assuring that the project would run as planned: demonstrating scientific skills through a comprehensive description of the experiments.

Closely related to methods and techniques was the detailed description of the experiment. In some cases, this description comprised an entire page, although the entire application was not longer than three pages. The description started with formulations that clarified how the applicants aimed to put the research idea into research practice. Typical phrases were, ‘the proposed concept is to utilize’, ‘our concept is to design’, ‘we propose to develop [build]’, ‘a description of the procedure is’, and ‘the research concept starts with exploring’. The applicants would continue by detailing how the project was to proceed, as though the project had already taken place. For instance, they introduced their scientific knowledge and skills, described what the result would be in each part of the experiment, and stated which conclusions they would arrive at. These precise and detailed descriptions demonstrated the applicants’ knowledge and experience, their ability to investigate the research question, the likelihood of reasonable research results, and confidence that they would successfully master the project.

4.2 Different assets

The second superordinate code—different assets—encompassed what the applicants bring to the project to ensure its success. We distinguished between three different assets: the applicants’ ‘own preliminary work/studies’, ‘technical equipment’, and ‘scientific cooperation’. The first of these three categories consisted of explanations of the scientific results the applicants had already achieved as a basis on which the project was to build and by which its feasibility had been proven. The applicants pointed out that they had already prepared the planned experiments so that they could start immediately. For example: ‘These setups were developed in previous projects at [name of an institution] and are available for the investigations planned here’ (En_35), and ‘Meeting the first objective is feasible, as our institute [has] built several trackers over the last couple of years’ (Ph_96). Second, the scientists also emphasized that they already had evidence of their project’s practicality. They referred to ‘initial tests’, ‘pilot studies’, or a ‘first proof of their concept’: ‘In a pilot study we have investigated chromosome segregation in young [animals]’ (Bio_28). ‘The project is based on a pilot study currently underway [experiment], and we seek to validate the results with the help of a [specific type of] study’ (Me_95). ‘In a proof of principle experiment, I managed to show the biological feasibility of the proposed concept’ (Me_98). Third, the applicants cited outcomes of their own research to substantiate the correctness of their research hypothesis. They referred to ‘recently published results’, ‘[my/our] own unpublished results or data’, and ‘results of my own doctoral thesis’. Typical statements were, ‘We base our concept on a recent observation from our lab [unpublished results]’ (Ph_85), and ‘Our preliminary data shows that in [animal], high-salt diet impacts’ (Biome_52). These three variants of references to the applicants’ own preliminary work or studies addressed different concerns about the functioning of the planned projects—the questions of whether the scientists would succeed in setting up and running the experiment, whether it would be possible to test the hypotheses empirically, and whether the experiment would yield any relevant results. The applicants tried to discount all these potential research problems by stressing that they had already successfully dealt with them.

The second asset, technical equipment, is understood broadly to mean necessary research infrastructure. The applicants informed the reviewers that their labs are well equipped. They did so without referring to specific technical needs of the planned project. Then they reported having ‘access to the full range of facilities, technologies, and expertise required for this experiment’ (Ph_24). Often, the authors of the proposals underscored that project-related technical devices and specific objects exist in their laboratory: ‘Our lab is fully equipped to carry out the majority of the experiments and analysis’ (Bio_85), and ‘All necessary prerequisites (micro-fabrication, advanced imaging techniques, [animal] cell culture, and initial observations of directed polarized motion in narrow channels) are well established in our lab’ (Ph_85). Such confirmations of access to all the technical devices needed for the project were meant to allay the concerns about this matter.

The third asset to which the scientists referred, scientific cooperation, related to the external expertise and equipment that they would draw on for the project. Again, three explanatory variants surfaced. As with the first variant pertaining to the second asset (technical equipment), the applicants spoke in general terms of abundance—in this case, of having comprehensive scientific cooperation outside their lab, though they did not indicate the part of the project for which these resources would be necessary. By mentioning their interinstitutional contacts, the applicants showed their integration into the scientific community and the fact that their scientific performance is appreciated by the scientific community. An example of this kind of general statement was, ‘collaboration for exploring this research area has already been formed’ (Ph_24). The second variant was emphasis on contacts or cooperation that the applicants had with labs and scientists who would bring in expertise or equipment that would otherwise be lacking or unaffordable. Two examples were ‘We will overcome this obstacle through close cooperation with our partner institutes’ (En_11), and ‘To attain this goal I will collaborate with a colleague from social sciences renowned for her studies on the quality and structure of online social content’ (Ne_25). A third variant consisted of assurance that the applicants were engaging in research cooperation, meeting the expectation of good scientific practice. The analyzed proposals contained statements that organizing scientific exchange would be an important part of the project. For instance, ‘By collaborating with other scientific groups that use [name of equipment], the value of this novel approach would increase dramatically’ (Me_16). By emphasizing the intention to have the project’s procedures include external expertise and equipment, the scientists asserted their awareness that scientific cooperation—more generally, scientific networking—is considered to an asset needed not only for successful research but also for recognition as a good scientist. Specifying the three assets subsumed in our second superordinate code enabled the applicants to confirm that they had what was needed to meet the scientific objective they had set.

4.3 Stylistic techniques

We come now to our third superordinate code: the stylistic techniques that shaped the research proposals. They did not originate in the field of science; they were applied rather generally to elicit approval, convince other people, and generate textual evidence ( Myers 1990 ). One of the patterns that we identified as having a stylistic character was what we named ‘difficulties and solutions’. The scientists described a difficulty they might encounter, then immediately offered the solution to it. These text sequences were constructed like a dialogue between the writer of the proposal and the reviewer. They varied in length and appeared mostly in the passages detailing the experiment. The presentation of the possible problem often began with a formulation such as ‘the obstacle’, ‘it could [will] be critical’, and many of the ripostes began with expressions such as ‘therefore’, ‘thus’, ‘to address these issues [this problem]’, ‘to tackle … this possibility’, and ‘we will modify [the procedure] if this is the case’. In the following two examples, the first ellipsis marks the shift of focus from the potential problem to its solution: ‘The major problem of our approach is that we don’t know yet … We believe that this can be clarified with …’ (Bio_35). ‘One possible obstacle to overcome during the project is … One strategy is to substitute …’ (Ch_89). Often, the applicants did not introduce a real research problem. Instead, they demonstrated that they had thought about possible objections from the reviewer. This stylistic figure addressed foreseeable doubts about the practicality of the project. Generally, these concerns related to the reasonableness and feasibility of the research approach.

‘Different forms of ordering’, too, informed the presentation of the experiment or the overall project. The applicants resorted primarily to four formats for this purpose. The first was chronological order, often signaled by expressions ‘at first … secondly’, ‘first … second step [phase, stage]; or similar ordinal formulations’. The second format consisted of alphabetical sequencing: a, b, c. The third format consisted of a Roman or Arabic cardinal numbering scheme (I, II, III or 1, 2, 3). Lastly, the applicants organized their research process in different units by either defining elements like work packages and milestones or by inserting typographical symbols such as bullet points or dashes to demarcate the sections. Two organizational approaches were commonly coexisted. They were generally combined with an outline of the working process, sometimes with a project schedule. This practice can be regarded as a desire to show that the research processes were well conceived. These ways of conveying the research process draw on principles of organization from outside the field of research. They help systematize the research process and are broadly recognized as applicable to the organization and planning of operations.

Another stylistic device that the scientists turned to was the depiction of the entire project or aspects of it as ‘simple’, ‘elegant’, ‘easy’, ‘robust’, and the like. In some proposals, the applicants wrote that they were posing ‘a few radically simple and new questions’ (Ch_87) or described their research idea as ‘simple’ (Biome_17). Others stated that the ‘procedure is quite easy’ (Ne_72) or that the model or the techniques were simple and easy. In general, this argument surfaced where elegance, simplicity, and functionality serve as proof that the research idea is realizable. This aspiration harks back to the assumption that a true statement is often simple and beautiful ( Ivanova 2017 ). Scientists often choose a theory for its elegance and simplicity ( Gross 1990 : 16). Gross recommends that one ‘not … deny that there is an aesthetic dimension to science’ (5). Ivanova agrees, underlining that ‘aesthetic judgements are an integral part of scientific practice’ ( Ivanova 2017 : 2582) and that simplicity ‘aids the development of hypotheses, our choice of hypotheses, and ultimately guides our choice between theories that equally fit the data’ ( Ivanova 2017 : 2587).

References to the three resources served to assure the reviewer that the project would be successfully conducted as described in the proposal. Each type of resource represents a particular way of presenting evidence that the scientists will master the research. Accordingly, we call them ‘kinds of evidence of research mastery.’ We propose this expression rather than import the linguistic term move used by Connor and Mauranen (1999) , because research mastery fits best to the nature of research ventures. Table  1 provides an overview of the various kinds of research mastery we have identified.

Kinds of evidence of research mastery

As noted in the literature review, feasibility is closely connected with methodological and technical aspects of the research process. Our empirical inquiry has disclosed an understanding of feasibility that encompasses the entire research process and the gamut of scientific practices and competences. Significantly, this study also includes the self-presentation of the applicants as experienced, competent, and integrated into the scientific community. The first kind of evidence calls attention to the broad scientific skills the researchers would tap to meet the expectations the scientific field has of a good scientist. The applicants cited their familiarity with the established knowledge and customary skills recognized as a manifest indication of a ‘great mastery’ ( Bourdieu 2004 : 38) of research practices.

The second kind of evidence stresses that all of the project’s necessary assets will be in place. The applicants demonstrated their awareness of the assets needed to guarantee the practicability of the project. In addition, they described their lab’s quality equipment, a strategy that also sets that facility apart from other labs without the applicants needing to blow their own horn. The applicants sought to substantiate their project’s feasibility by describing their entire range of available technical equipment that past research had proven to be essential.

This interpretation is supported by the third kind of evidence—stylistic techniques—that the applicants marshaled to prove their research mastery in the proposals, which exemplify the expected modes of organizing research projects and their processes. Availing themselves of these modes, the applicants adopted the expected proposal-writing style for generating textual evidence ( Myers 1990 ). Similarly, portraying the research idea as ‘simple’ and ‘elegant’ is broadly recognized as a criterion that validates scientific hypotheses. Appropriate stylistic techniques thereby help communicate that the applicants know and accept how science is and should be pursued. Through effective writing style, scientists exhibit their mastery of the discipline.

In summary, the three kinds of evidence of research mastery were cited by the applicants to align their projects with scientific standards, embed them in the current state of research, and meet the scientific field’s expectations of how scientists should conduct a research project. The applicants thereby increased the chances that their projects would proceed successfully to their conclusion. One could say that the three kinds of evidence refer to the way the scientific field describes and understands itself ( Bourdieu 2004 ).

The three kinds of evidence of research mastery we have distilled in our inquiry have some similarities with those studied by Luukkonen (2012) , but ours include a number of new aspects as well. Like Luukkonen (2012) , we found strong reference to the applicants’ own preliminary scientific work and to the availability of the needed technical equipment. Luukkonen’s study and ours also align in the finding that reviewers are keen on seeing evidence that the researcher is ready for all eventualities. Indeed, we noted that the applicants presented themselves as being prepared for any contingency and offered alternative solutions. By contrast, our study enabled us to give a detailed account of how the researchers substantiated their preparedness, namely, by highlighting their vast repertoire of methods and techniques and by minutely describing their experiments. This comparative advantage of our study is highly relevant for demonstrating feasibility (see the next section). We not only learned that research plans and timetables were used to promote feasibility but also showed how they were enriched with different stylistic techniques to increase the textual evidence of the applicant’s competence. Luukkonen (2012) did not take such stylistic techniques into account. Parsing the elements of our sample’s research proposals into three kinds of evidence that substantiate research mastery was yet another innovation making it explicit that the applicants draw on different kinds of resources for emphasizing their projects’ feasibility.

At first glance, the three kinds of evidence of research mastery seem to codify nothing more than fulfillment of scientific expectations of how research must be conducted and of what a successful research process requires. In principle, all researchers should have access to these resources. At second glance, however, it becomes clear that the three kinds of evidence of research mastery serve not only to bear out a project’s feasibility but also to distinguish the project from other proposals. We can assume that not all researchers have the same capacity to offer the three kinds of evidence in the same way. Experienced researchers and those who have the privilege of working in a large lab are more likely than others to be in a position to verify that they will bring broad methodical and technical experience to their project ( Hackett 2005 ). Nor will researchers be equally able to give a detailed description of an experiment to signal that it has already taken place as a means of demonstrating its practicability. It requires a ‘practical mastery’, ‘a kind of “connoisseurship”’ ( Bourdieu 2004 : 38), to design experimental setups that yield a persuasive description of the experiment—particularly when the experiment is still on the drawing board.

In general, it can be assumed that researchers who have already successfully mastered the first research steps surpass junior researchers in opportunities to point to their own preliminary scholarship, such as publications or self-developed proofs of a concept. Usually, junior scientists do not have enough of their own preliminary work to cite; such assets ‘focus on prior merits’ ( Langfeldt 2001 : 836). Nor do researchers all have equal access to a broad range of equipment. Applicants who work at particularly well-equipped labs ( Hackett 2005 ) can refer to the availability of the full range of requisite facilities and technologies and thereby set their project apart from others. Such disparity in opportunities also applies to references to impressive scientific cooperation to which the applicant has access, with the odds favoring established researchers and scholars in well-known scientific institutions. By the same token, some applicants find it easier than others to use the stylistic techniques in ways considered appropriate for the specific genre of academic writing. Linguistic studies on proposal-writing have shown that senior researchers are significantly more experienced than their younger colleagues in that skill (e.g. Urquhart-Cronish and Otto 2019 ). This finding suggests that experienced researchers in our sample, too, are more likely than junior applicants to have mastered the stylistic techniques we have identified.

Overall, we find that senior researchers and applicants from relatively large and well-known labs can provide the three kinds of evidence of research mastery not only to demonstrate that their projects meet the scientific field’s expectations of good research but also to convince reviewers that it is highly likely achievable as planned. Moreover, they can do so to substantiate the practicability of their proposals by virtue of their excellently equipped labs, extensive previous work and experience, and valuable scientific cooperation. They are able to use institutional attributes to confirm themselves as researchers and can point to their lab to indicate their merits and stress that their proposal is feasible. Scientists can be at a disadvantage if they have little or no access to this institutional asset. In addition, studies on gender-specific differences suggest that female researchers tend to be more cautious than male researchers in academic writing ( Ramnial, Panchoo and Pudaruth 2016 ; Lerchenmueller, Sorenson and Jena2019 ). Accordingly, it may well be that the resources we have identified are exploited less by female than by male researchers in their respective proposals.

This factor is particularly important in the research initiative that we studied, for the funding body specifically intended to level the playing field by excluding additional documents such as CVs and lists of publications. Analyses have shown that such documents are used by reviewers to ‘provide a meaningful basis for judging scientific potential’ ( Kaltenbrunner and De Rijcke 2019 : 871). Our study has found that even anonymized proposal processes enable the applicants to use the three kinds of evidence of research mastery to substantiate their scientific potential. It is therefore not surprising that many of the proposals we examined encompassed far more than the conception of the research project. For example, they referred to scientific skills, assets, and stylistic techniques that have no direct bearing on the planned project—such as vast experience accumulated in other projects and a sophisticated outline for the proposal. These forms of substantiation show that proposals cannot be examined only as a specific genre of academic writing. Studies should also analyze which kinds of substantiation are used in the proposals and how they differentiate proposals. Such work should focus on comparing successful with unsuccessful proposals, for our results, as we show below, suggest a link between use of the three kinds of evidence of research mastery and an increase in the odds of success.

Our sample of research proposals allowed us to study how applicants can argue the feasibility of their proposed project. We identified three kinds of evidence of research mastery that encompass the planned project’s entire research process: scientific skills, different assets, and stylistic techniques. The sample consisted of 11 successful proposals, that is, they had been approved after having received a positive review. We studied whether the separate categories of resources and the three resources themselves had been used more frequently in those proposals than in the proposals that had not been reviewed positively or approved. Because of the small number of our sample’s successful proposals, our results are not conclusive, but some differences are striking. Except for ‘simple’ and ‘easy’—the only codes for which the successful and unsuccessful proposals did not differ—all the codes were present more often in the successful proposals, with ‘difficulties and solutions’, ‘technical equipment’, and ‘scientific cooperation’ being slightly in the lead. The small absolute number of cases warrants caution in the interpretation of these differences, however. We identified rather large divergence (from 25 to 28%) in the number of times the applicants made use of methods and techniques, their own preliminary work, and different ways of organizing the text. The number of detailed descriptions of a given experiment was fully 40% higher in the successful proposals than in the unsuccessful ones—a conspicuously great difference. Overall, it seems unsurprising that a proposal’s convincing presentation of the applicant’s scientific skills augured particularly well for the proposal’s success. This finding plainly illustrates that this kind of evidence of research mastery is what is crucial for making credible claims about the project’s feasibility. There is reason to believe it is also important to point out one’s own preliminary work and to carefully organize the proposal’s text. Affirming the availability of the needed technical equipment and the existence of scientific cooperation seemed less central to success that these other codes did. We reiterate, however, that the proposals under review had been anonymized. It could be that the reviewers very likely consider these two categories when reading applications that have not been anonymized.

Our results show that even a short proposal can be formulated to portray oneself as a skilled scientist and demonstrate one’s practical mastery at designing experimental setups. The three kinds of evidence of research mastery serve to signal that the project meets the scientific field’s expectations of how a scientist should conduct a research project. In other words, they correspond to the self-image of the scientific field. They clearly have the character of an authoritative, binding standard, especially, it seems, for the categories from which the authors of positively reviewed (and thus successful) proposals had drawn more liberally than had the authors of unsuccessful proposals in the effort to underscore the feasibility of their respective projects.

The question remains as to whether three kinds of evidence of research mastery can be considered an expression of scientific conservatism. One can certainly interpret scientific skills and different assets—the two kinds of evidence that meet the expectations, requirements, and standards of the scientific field—as examples of a ‘practical conservatism’, for they build largely on existing research. However, it would be misleading to conclude from this point that the entire proposal is conservative and representative of conventional research. Recent studies ( Barlösius 2019 ; Philipps and Weißenborn 2019 ) have shown that proposals can indeed contain revolutionary and radical research ideas. It could be useful to have the analysis of research proposals distinguish the research ideas, which may be revolutionary or conventional, from the presentation of the project’s feasibility, which usually cites existing practices and methods. This aspect can be considered practical conservatism, but thorough examination of this differentiation was not possible in the present study. That engaging topic invites future research. It would also be instructive to clarify how the third resource—the stylistic techniques used by the writers of successful research proposals—could be linked with the moves identified by Connor and Mauranen (1999) .

A limitation of this study is that it was based on relatively brief proposals, although this length made it possible to analyze each document in its entirety. A helpful contribution for future research would be a study of whether long proposals harbor additional kinds of evidence of research mastery. We bear in mind, too, that the research initiative Experiment! only funds proposals from the natural sciences and engineering, and it would be enlightening to learn how the feasibility of projects is argued in the social sciences and the humanities and whether similar kinds of evidence of research mastery surface there as well in principle. The key question resulting from our analysis is whether additional studies will confirm our initial results. After all, successful proposals are characterized by an especially liberal use of the three kinds of evidence of research mastery, especially references to the extensive competence the applicants have in methods and technology, the preliminary work of those researchers, and a detailed description of the experiment.

The VolkswagenStiftung is the largest private research foundation in Germany.

https://www.volkswagenstiftung.de/en/funding/our-funding-portfolio-at-a-glance/experiment

We thank Axel Philipps for preparing the representative sample.

We thank Johanna Johannsen for her help with coding the proposals.

The names of the disciplines represented in our representative sample are abbreviated in the remaining text: Bio: biology; Bioch: biochemistry; Bioph: biophysics; Biome: biomedicine; Ch: chemistry; En: mechanical engineering; Me: medicine; Ne: neurosciences; Ph: physics; and Psy: psychology.

To preserve the anonymity of the applicants, passages can no longer be quoted.

Barlösius E. ( 2019 ) ‘ Concepts of originality in the natural science, medical, and engineering disciplines: An analysis of research proposals, Science, Technology ’, Human Values , 44 : 915 – 937 .

Google Scholar

Bedessem B. ( 2021 ) ‘ Two Conceptions of the Sources of Conservatism in Scientific Research ’, Synthese , 198 : 6597 – 6614 .

Berezin A. ( 1998 ) ‘ The Perils of Centralized Research Funding Systems ’, Knowledge, Technology & Policy , 11 : 5 – 26 .

Boudreau K. J. , Guinan E. C. , Lakhani K. R. , Riedl C. ( 2016 ) ‘ Looking across and Looking beyond the Knowledge Frontier: Intellectual Distance and Resource Allocation in Science ’, Management Science , 62 : 2765 – 83 .

Bourdieu P. ( 2004 ) Science of Science and Reflexivity . Chicago : University of Chicago Press .

Google Preview

Chubin D. E. , Hackett E. J. ( 1990 ) Peerless Science: Peer Review and U.S. Science Policy . Albany : State University of New York Press .

Connor U. ( 2000 ) ‘ Variation in Rhetorical Moves in Grant Proposals of US Humanists and Scientists ’, Interdisciplinary Journal for the Study of Discourse , 20 : 1 – 28 .

Connor U. , Mauranen A. ( 1999 ) ‘ Linguistic Analysis of Grant Proposals: European Union Research Grants ’, English for Specific Purposes , 18 : 47 – 62 .

Currie A. ( 2019 ) ‘Introduction: Creativity, Conservatism & the Social Epistemology of Science’. < http://hdl.handle.net/10871/35767 > accessed 26 Apr 2021.

Franssen T. , Scholten W. , Hessels L. K. , De Rijcke S. ( 2018 ) ‘ The Drawbacks of Project Funding for Epistemic Innovation: Comparing Institutional Affordances and Constraints of Different Types of Research Funding ’, Minerva , 56 : 11 – 33 .

Gross A. G. ( 1990 ) The Rhetoric of Science . Cambridge, MA : Harvard University Press .

Hackett E. J. ( 2005 ) ‘ Essential Tensions: Identity, Control, and Risk in Research ’, Social Studies of Science , 35 : 787 – 826 .

Heinze T. ( 2008 ) ‘ How to Sponsor Ground-Breaking Research: A Comparison of Funding Schemes ’, Science and Public Policy , 35 : 302 – 18 .

Horrobin D. F. ( 1996 ) ‘ Peer Review of Grant Applications: A Harbinger for Mediocrity in Clinical Research? ’, The Lancet , 348 : 1293 – 5 .

Ivanova M. ( 2017 ) ‘ Poincaré’s Aesthetics of Science ’, Synthese , 194 : 2581 – 94 .

Kaltenbrunner W. , De Rijcke S. ( 2019 ) ‘ Filling in the Gaps: The Interpretation of Curricula Vitae in Peer Review ’, Social Studies of Science , 49 : 863 – 83 .

Kuhn T. ( 1991 ) ‘Chapter 7: Scientific Revolutions—A. The Essential Tension: Tradition and Innovation in Scientific Research’, in Boyd R. , Gasper P. , Trout J. D. (eds) The Philosophy of Science , pp. 140 – 7 . Cambridge : MIT Press .

Kummerfeld E. , Zollman K. J. S. ( 2016 ) ‘ Conservatism and the Scientific State of Nature ’, The British Journal for the Philosophy of Science , 67 : 1057 – 76 .

Lamont M. ( 2009 ) How Professors Think: Inside the Curious World of Academic Judgement . Cambridge : Harvard University Press .

Langfeldt L. ( 2001 ) ‘ The Decision-Making Constraints and Processes of Grant Peer Review, and Their Effects on the Review Outcome ’, Social Studies of Science , 31 : 820 – 41 .

Laudel G. ( 2006 ) ‘ The Art of Getting Funded: How Scientists Adapt to Their Funding Conditions ’, Science and Public Policy , 33 : 489 – 504 .

Laudel G. , Gläser J. ( 2014 ) ‘ Beyond Breakthrough Research: Epistemic Properties of Research and Their Consequences for Research Funding ’, Research Policy , 43 : 1204 – 16 .

Lerchenmueller M. J. , Sorenson O. , Jena A. B. ( 2019 ) ‘ Gender Differences in How Scientists Present the Importance of Their Research: Observational Study ’, BMJ , 367 : l6573 .

Luukkonen T. ( 2012 ) ‘ Conservatism and Risk-Taking in Peer Review: Emerging ERC Practices ’, Research Evaluation , 21 : 48 – 60 .

Markowitz D. M. ( 2019 ) ‘ What Words Are Worth: National Science Foundation Grant Abstracts Indicate Award Funding ’, Journal of Language and Social Psychology , 38 : 264 – 82 .

Myers G. ( 1990 ) Writing Biology: Texts in the Social Construction of Scientific Knowledge. Science and Literature Series . Madison : University of Wisconsin .

O’Connor C. ( 2019 ) ‘ The Natural Selection of Conservative Science ’, Studies in History and Philosophy of Science Part A , 76 : 24 – 9 .

Philipps A. , Weißenborn L. ( 2019 ) ‘ Unconventional Ideas Conventionally Arranged: A Study of Grant Proposals for Exceptional Research ’, Social Studies of Science , 49 : 884 – 97 .

Ramnial H. , Panchoo S. , Pudaruth S. ( 2016 ) ‘Gender Profiling from PhD Theses Using k-Nearest Neighbour and Sequential Minimal Optimisation’, in Berretti S. , Thampi S. , Dasgupta S. (eds), Intelligent Systems Technologies and Applications: Advances in Intelligent Systems and Computing , vol 385 . Cham : Springer .

Schreier M. ( 2012 ) Qualitative Content Analysis in Practice . London : Sage .

Serrano Velarde K. ( 2018 ) ‘ The Way we Ask for Money…the Emergence and Institutionalization of Grant-Writing Practices in Academia ’, Minerva , 56 : 85 – 107 .

Stanford P. K. ( 2019 ) ‘ Unconceived Alternatives and Conservatism in Science: The Impact of Professionalization, Peer-Review, and Big Science ’, Synthese , 196 : 3915 – 32 .

Swales J. M. ( 2004 ) Research Genres: Explorations and Applications . Cambridge, UK : University Press .

Travis G. D. L. , Collins H. M. ( 1991 ) ‘ New Light on Old Boys: Cognitive and Institutional Particularism in the Peer Review System ’, Science, Technology and Human Values , 16 : 322 – 41 .

Urquhart-Cronish M. , Otto S. P. ( 2019 ) ‘ Gender and Language Use in Scientific Grant Writing ’, FACETS , 4 : 442 – 58 .

Van den Besselaar P. , Sandström U. , Schiffbaenker H. ( 2018 ) ‘ Studying Grant Decision-Making: A Linguistic Analysis of Review Reports ’, Scientometrics , 117 : 313 – 29 .

Barlösius, E. ( 2019 ). ‘Concepts of originality in the natural science, medical, and engineering disciplines: An analysis of research proposals', Science, Technology, & Human Values , 44 : 915 – 37 .

Intercoder reliability between the authors and doctoral assistant

a The proposals coded by the authors and the doctoral assistant encompassed 390 codings, of which 349 matched and 41 diverged.

Email alerts

Citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 1471-5449
• Print ISSN 0958-2029
• Copyright © 2024 Oxford University Press
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

An Experimental Feasibility Study Evaluating the Adequacy of a Sportswear-Type Wearable for Recording Exercise Intensity

Affiliations.

• 1 Graduate School of Sport and Exercise Sciences, Osaka University of Health and Sport Sciences, Kumatori 590-0496, Osaka, Japan.
• 2 Department of Health and Sport Sciences, Osaka University Graduate School of Medicine, Toyonaka 560-0043, Osaka, Japan.
• 3 Department of Sports Medical Biomechanics, Osaka University Graduate School of Medicine, Suita 565-0871, Osaka, Japan.
• PMID: 35408192
• PMCID: PMC9003462
• DOI: 10.3390/s22072577

Sportswear-type wearables with integrated inertial sensors and electrocardiogram (ECG) electrodes have been commercially developed. We evaluated the feasibility of using a sportswear-type wearable with integrated inertial sensors and electrocardiogram (ECG) electrodes for evaluating exercise intensity within a controlled laboratory setting. Six male college athletes were asked to wear a sportswear-type wearable while performing a treadmill test that reached up to 20 km/h. The magnitude of the filtered tri-axial acceleration signal, recorded by the inertial sensor, was used to calculate the acceleration index. The R-R intervals of the ECG were used to determine heart rate; the external validity of the heart rate was then evaluated according to oxygen uptake, which is the gold standard for physiological exercise intensity. Single regression analysis between treadmill speed and the acceleration index in each participant showed that the slope of the regression line was significantly greater than zero with a high coefficient of determination (walking, 0.95; jogging, 0.96; running, 0.90). Another single regression analysis between heart rate and oxygen uptake showed that the slope of the regression line was significantly greater than zero, with a high coefficient of determination (0.96). Together, these results indicate that the sportswear-type wearable evaluated in this study is a feasible technology for evaluating physical and physiological exercise intensity across a wide range of physical activities and sport performances.

Keywords: acceleration; electrocardiogram; exercise intensity; heart rate; wearable sensor.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Sportswear-type wearable examined in this…

Sportswear-type wearable examined in this study. ECG electrodes are located under each armpit,…

ECG waveforms recorded by the…

ECG waveforms recorded by the sportswear-type wearable: ( a ) at rest (quiet…

Mean and standard deviation of…

Mean and standard deviation of ( a ) heart rate (bpm), ( b…

Regression lines of the heart…

Regression lines of the heart rate and oxygen uptake comparison of all participants.…

Regression lines of the acceleration…

Regression lines of the acceleration index and treadmill speed comparison of all participants.…

Similar articles

• A Wearable System for Heart Rate Recovery Evaluation with Real-Time Classification on Exercise Condition. Kim Y, Jeung J, Song Y, Ko H, Park S, Park H, Jeon G, Chung Y. Kim Y, et al. Annu Int Conf IEEE Eng Med Biol Soc. 2021 Nov;2021:7609-7612. doi: 10.1109/EMBC46164.2021.9629673. Annu Int Conf IEEE Eng Med Biol Soc. 2021. PMID: 34892851
• Multichannel ECG recording from waist using textile sensors. Alizadeh Meghrazi M, Tian Y, Mahnam A, Bhattachan P, Eskandarian L, Taghizadeh Kakhki S, Popovic MR, Lankarany M. Alizadeh Meghrazi M, et al. Biomed Eng Online. 2020 Jun 16;19(1):48. doi: 10.1186/s12938-020-00788-x. Biomed Eng Online. 2020. PMID: 32546233 Free PMC article.
• Accuracy of Consumer Wearable Heart Rate Measurement During an Ecologically Valid 24-Hour Period: Intraindividual Validation Study. Nelson BW, Allen NB. Nelson BW, et al. JMIR Mhealth Uhealth. 2019 Mar 11;7(3):e10828. doi: 10.2196/10828. JMIR Mhealth Uhealth. 2019. PMID: 30855232 Free PMC article.
• Evaluating the Validity and Utility of Wearable Technology for Continuously Monitoring Patients in a Hospital Setting: Systematic Review. Patel V, Orchanian-Cheff A, Wu R. Patel V, et al. JMIR Mhealth Uhealth. 2021 Aug 18;9(8):e17411. doi: 10.2196/17411. JMIR Mhealth Uhealth. 2021. PMID: 34406121 Free PMC article. Review.
• Review on Wearable Technology Sensors Used in Consumer Sport Applications. Aroganam G, Manivannan N, Harrison D. Aroganam G, et al. Sensors (Basel). 2019 Apr 28;19(9):1983. doi: 10.3390/s19091983. Sensors (Basel). 2019. PMID: 31035333 Free PMC article. Review.
• Gaussian mixture modeling of acceleration-derived signal for monitoring external physical load of tennis player. Marutani Y, Konda S, Ogasawara I, Yamasaki K, Yokoyama T, Maeshima E, Nakata K. Marutani Y, et al. Front Physiol. 2023 Mar 24;14:1161182. doi: 10.3389/fphys.2023.1161182. eCollection 2023. Front Physiol. 2023. PMID: 37035679 Free PMC article.
• Heartbeat detector from ECG and PPG signals based on wavelet transform and upper envelopes. Merino-Monge M, Castro-García JA, Lebrato-Vázquez C, Gómez-González IM, Molina-Cantero AJ. Merino-Monge M, et al. Phys Eng Sci Med. 2023 Jun;46(2):597-608. doi: 10.1007/s13246-023-01235-6. Epub 2023 Mar 6. Phys Eng Sci Med. 2023. PMID: 36877361 Free PMC article.
• Are Activity Wrist-Worn Devices Accurate for Determining Heart Rate during Intense Exercise? Martín-Escudero P, Cabanas AM, Dotor-Castilla ML, Galindo-Canales M, Miguel-Tobal F, Fernández-Pérez C, Fuentes-Ferrer M, Giannetti R. Martín-Escudero P, et al. Bioengineering (Basel). 2023 Feb 15;10(2):254. doi: 10.3390/bioengineering10020254. Bioengineering (Basel). 2023. PMID: 36829748 Free PMC article.
• Aroganam G., Manivannan N., Harrison D. Review on Wearable Technology Sensors Used in Consumer Sport Applications. Sensors. 2019;19:1983. doi: 10.3390/s19091983. - DOI - PMC - PubMed
• Iqbal M.H., Aydin A., Brunckhorst O., Dasgupta P., Ahmed K. A Review of Wearable Technology in Medicine. J. R. Soc. Med. 2016;109:372–380. doi: 10.1177/0141076816663560. - DOI - PMC - PubMed
• Camomilla V., Bergamini E., Fantozzi S., Vannozzi G. Trends Supporting the in-Field Use of Wearable Inertial Sensors for Sport Performance Evaluation: A Systematic Review. Sensors. 2018;18:873. doi: 10.3390/s18030873. - DOI - PMC - PubMed
• Patel M.S., Asch D.A., Volpp K.G. Wearable Devices as Facilitators, Not Drivers, of Health Behavior Change. JAMA. 2015;313:459–460. doi: 10.1001/jama.2014.14781. - DOI - PubMed
• Cross M.J., Williams S., Trewartha G., Kemp S.P., Stokes K.A. The Influence of in-Season Training Loads on Injury Risk in Professional Rugby Union. Int. J. Sports Physiol. Perform. 2016;11:350–355. doi: 10.1123/ijspp.2015-0187. - DOI - PubMed
• Search in MeSH

Related information

• PubChem Compound (MeSH Keyword)

Grants and funding

• Sports Research Innovation Project (SRIP) grant/Japan Sports Agency
• &quot;Innovation Platform for Society 5.0&quot; Program/Ministry of Education, Culture, Sports, Science and Technology

LinkOut - more resources

Full text sources.

• Europe PubMed Central
• PubMed Central

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List

Defining Feasibility and Pilot Studies in Preparation for Randomised Controlled Trials: Development of a Conceptual Framework

Sandra m. eldridge.

1 Centre for Primary Care and Public Health, Queen Mary University of London, London, United Kingdom

Gillian A. Lancaster

2 Department of Mathematics and Statistics, Lancaster University, Lancaster, Lancashire, United Kingdom

Michael J. Campbell

3 School of Health and Related Research, University of Sheffield, Sheffield, South Yorkshire, United Kingdom

Lehana Thabane

4 Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada

Sally Hopewell

5 Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, Oxfordshire, United Kingdom

Claire L. Coleman

Christine m. bond.

6 Centre of Academic Primary Care, University of Aberdeen, Aberdeen, Scotland, United Kingdom

Conceived and designed the experiments: SE GL MC LT SH CB. Performed the experiments: SE GL MC LT SH CB CC. Analyzed the data: SE GL MC LT SH CB CC. Contributed reagents/materials/analysis tools: SE GL MC LT SH CB. Wrote the paper: SE GL MC LT SH CB CC.

Associated Data

Due to a requirement by the ethics committee that the authors specified when the data will be destroyed, the authors are not able to give unlimited access to the Delphi study quantitative data. These data are available from Professor Sandra Eldridge. Data will be available upon request to all interested researchers. Qualitative data from the Delphi study are not available because the authors do not have consent from participants for wider distribution of this more sensitive data.

We describe a framework for defining pilot and feasibility studies focusing on studies conducted in preparation for a randomised controlled trial. To develop the framework, we undertook a Delphi survey; ran an open meeting at a trial methodology conference; conducted a review of definitions outside the health research context; consulted experts at an international consensus meeting; and reviewed 27 empirical pilot or feasibility studies. We initially adopted mutually exclusive definitions of pilot and feasibility studies. However, some Delphi survey respondents and the majority of open meeting attendees disagreed with the idea of mutually exclusive definitions. Their viewpoint was supported by definitions outside the health research context, the use of the terms ‘pilot’ and ‘feasibility’ in the literature, and participants at the international consensus meeting. In our framework, pilot studies are a subset of feasibility studies, rather than the two being mutually exclusive. A feasibility study asks whether something can be done, should we proceed with it, and if so, how. A pilot study asks the same questions but also has a specific design feature: in a pilot study a future study, or part of a future study, is conducted on a smaller scale. We suggest that to facilitate their identification, these studies should be clearly identified using the terms ‘feasibility’ or ‘pilot’ as appropriate. This should include feasibility studies that are largely qualitative; we found these difficult to identify in electronic searches because researchers rarely used the term ‘feasibility’ in the title or abstract of such studies. Investigators should also report appropriate objectives and methods related to feasibility; and give clear confirmation that their study is in preparation for a future randomised controlled trial designed to assess the effect of an intervention.

Introduction

There is a large and growing number of studies in the literature that authors describe as feasibility or pilot studies. In this paper we focus on feasibility and pilot studies conducted in preparation for a future definitive randomised controlled trial (RCT) that aims to assess the effect of an intervention. We are primarily concerned with stand-alone studies that are completed before the start of such a definitive RCT, and do not specifically cover internal pilot studies which are designed as the early stage of a definitive RCT; work on the conduct of internal pilot studies is currently being carried out by the UK MRC Network of Hubs for Trials Methodology Research. One motivating factor for the work reported in this paper was the inconsistent use of terms. For example, in the context of RCTs ‘pilot study’ is sometimes used to refer to a study addressing feasibility in preparation for a larger RCT, but at other times it is used to refer to a small scale, often opportunistic, RCT which assesses efficacy or effectiveness.

A second, related, motivating factor was the lack of agreement in the research community about the use of the terms ‘pilot’ and ‘feasibility’ in relation to studies conducted in preparation for a future definitive RCT. In a seminal paper in 2004 reviewing the literature in relation to pilot and feasibility studies conducted in preparation for an RCT [ 1 ], Lancaster et al reported that they could find no formal guidance as to what constituted a pilot study. In the updated UK Medical Research Council (MRC) guidance on designing and evaluating complex interventions published four years later, feasibility and pilot studies are explicitly recommended, particularly in relation to identifying problems that might occur in an ensuing RCT of a complex intervention [ 2 ]. However, while the guidance suggests possible aims of such studies, for example, testing procedures for their acceptability, estimating the likely rates of recruitment and retention of subjects, and the calculation of appropriate sample sizes, no explicit definitions of a ‘pilot study’ or ‘feasibility study’ are provided. In 2010, Thabane and colleagues presented a number of definitions of pilot studies taken from various health related websites [ 3 ]. While these definitions vary, most have in common the idea of conducting a study in advance of a larger, more comprehensive, investigation. Thabane et al also considered the relationship between pilot and feasibility, suggesting that feasibility should be the main emphasis of a pilot study and that ‘a pilot study is synonymous with a feasibility study intended to guide the planning of a large scale investigation’. However, at about the same time, the UK National Institute for Health Research (NIHR) developed definitions of pilot and feasibility studies that are mutually exclusive, suggesting that feasibility studies occurred slightly earlier in the research process and that pilot studies are ‘a version of the main study that is run in miniature to test whether the components of the main study can all work together’. Arain et al . felt that the NIHR definitions were helpful, and showed that studies identified using the keyword ‘feasibility’ had different characteristics from those identified as ‘pilot’ studies [ 4 ]. The NIHR wording for pilot studies has been changed more recently to ‘a smaller version of the main study used to test whether the components of the main study can all work together’ ( Fig 1 ). Nevertheless, it still contrasts with the MRC framework guidance that explicitly states: ‘A pilot study need not be a “scale model” of the planned main-stage evaluation, but should address the main uncertainties that have been identified in the development work’ [ 2 ]. These various, sometimes conflicting, approaches to the interpretation of the terms ‘pilot’ and ‘feasibility’ exemplify differences in current usage and opinion in the research community.

While lack of agreement about definitions may not necessarily affect research quality, it can become problematic when trying to develop guidance for research conduct because of the need for clarity over what the guidance applies to and therefore what it should contain. Previous research has identified weaknesses in the reporting and conduct of pilot and feasibility studies [ 1 , 3 , 4 , 7 ], particularly in relation to studies conducted in preparation for a future definitive RCT assessing the effect of an intervention or therapy. While undertaking research to develop guidance to address some of the weaknesses in reporting these studies, we became convinced by the current interest in this area, the lack of clarity, and the differences of opinion in the research community, that a re-evaluation of the definitions of pilot and feasibility studies was needed. This paper describes the process and results of this re-evaluation and suggests a conceptual framework within which researchers can operate when designing and reporting pilot/feasibility studies. Since our work on reporting guidelines focused specifically on pilot and feasibility studies in preparation for an RCT assessing the effect of some intervention or therapy, we restrict our re-evaluation to these types of pilot and feasibility studies.

The process of developing and validating the conceptual framework for defining pilot and feasibility studies was, to a large extent, integral to the development of our reporting guidelines, the core components of which were a large Delphi study and an international expert consensus meeting focused on developing an extension of the 2010 CONSORT statement for RCTs [ 8 ] to randomised pilot studies. The reporting guidelines, Delphi study and consensus meeting are therefore referred to in this paper. However, the reporting guidelines will be reported separately; this paper focuses on our conceptual framework.

Developing a conceptual framework—Delphi study

Following research team discussion of our previous experience with, and research on, pilot and feasibility studies we initially produced mutually exclusive definitions of pilot and feasibility studies based on, but not identical to, the definitions used by the NIHR. We drew up two draft reporting checklists based on the 2010 CONSORT statement [ 8 ], one for what we had defined as feasibility studies and one for what we had defined as pilot studies. We constructed a Delphi survey, administered on-line by Clinvivo [ 9 ], to obtain consensus on checklist items for inclusion in a reporting guideline, and views on the definitions. Following user-testing of a draft version of the survey with a purposive sample of researchers active in the field of trials and pilot studies, and a workshop at the 2013 Society for Clinical Trials Conference in Boston, we further refined the definitions, checklists, survey introduction and added additional questions.

The first round of the main Delphi survey included: a description and explanation of our definitions of pilot and feasibility studies including examples (Figs ​ (Figs2 2 and ​ and3); 3 ); questions about participants’ characteristics; 67 proposed items for the two checklists and questions about overall appropriateness of the guidelines for feasibility or pilot studies; and four questions related to the definitions of feasibility and pilot studies: How appropriate do you think our definition for a pilot study conducted in preparation for an RCT is ? How appropriate do you think our definition for a feasibility study conducted in preparation for an RCT is ? How appropriate is the way we have distinguished between two different types of study conducted in preparation for an RCT ? How appropriate are the labels ‘pilot’ and ‘feasibility’ for the two types of study we have distinguished ? Participants were asked to rate their answers to the four questions on a nine-point scale from ‘not at all appropriate’ to ‘completely appropriate’. There was also a space for open comments about the definitions. The second round included results from the first round and again asked for further comments about the definitions.

Participants for the main survey were identified as likely users of the checklist including trialists, methodologists, statisticians, funders and journal editors. Three hundred and seventy potential participants were approached by email from the project team or directly from Clinvivo. These were individuals identified based on personal networks, authors of relevant studies in the literature, members of the Canadian Institute of Health Research, Biostatistics section of Statistics Society of Canada, and the American Statistical Society. The International Society for Clinical Biostatistics and the Society for Clinical Trials kindly forwarded our email to their entire membership. There was a link within the email to the on-line questionnaire. Each round lasted three weeks and participants were sent one reminder a week before the closure of each survey. The survey took place between August and October 2013. Ethical approval was granted by the ScHARR research ethics committee at the University of Sheffield.

Developing a conceptual framework—Open meeting and research team meetings

The results of the Delphi survey pertaining to the definitions of feasibility and pilot studies were presented to an open meeting at the 2 nd UK MRC Trials Methodology Conference in Edinburgh in November 2013 [ 13 ]. Attendees chose their preferred proposition from four propositions regarding the definitions, based variously on our original definitions, the NIHR and MRC views of pilot and feasibility studies and different views expressed in the Delphi survey. At a subsequent two-day research team meeting we collated the findings from the Delphi survey and the open meeting, and considered definitions of piloting and feasiblity outside the health research context found from on-line searches using the terms ‘pilot definition’, ‘feasiblity definition’, ‘pilot study definition’ and ‘feasibility study definition’ in Google. We expected all searches to give a very large number of hits and examined the first two pages of hits only from each search. From this, we developed a conceptual framework reflecting consensus about the definitions, types and roles of feasibility and pilot studies conducted in preparation for an RCT evaluating the effect of an intervention or therapy. To ensure we incorporated the views of all researchers likely to be conducting pilot/feasiblity studies, two qualitative researchers joined the second day of the meeting which focused on agreeing this framework. Throughout this process we continually referred back to examples that we had identified to check that our emerging definitions were workable.

Validating the conceptual framework—systematic review

To validate the proposed conceptual framework, we identified a selection of recently reported studies that fitted our definition of pilot and feasibility studies, and tested a number of hypotheses in relation to these studies. We expected that approximately 30 reports would be sufficient to test the hypotheses. We conducted a systematic review to identify studies that authors described as pilot or feasibility studies, by searching Medline via PubMed for studies that had the words ‘pilot’ or ‘feasibility’ in the title. To increase the likelihood that the studies would be those conducted in preparation for a randomised controlled trial of the effect of a therapy or intervention we limited our search to those that contained the word ‘trial’ in the title or abstract. For full details of the search strategy see S1 Fig .

To focus on current practice, we selected the 150 most recent studies from those identified by the electronic search. We did not exclude protocols since we were primarily interested in identifying the way researchers characterised their study and any possible future study and the relationship between them; we expected investigators to describe these aspects of their studies in a similar way in protocols and reports of findings. Two research team members independently reviewed study abstracts to assess whether each study fitted our working definition of a pilot or feasibility study in preparation for an RCT evaluating the effect of an intervention or therapy. Where reviewers disagreed, studies were classed as ‘possible inclusions’ and disagreements resolved by discussion with referral to the full text of the paper as necessary. Given the difficulty of interpreting some reports and to ensure that all research team members agreed on inclusion, the whole team then reviewed relevant extracted sections of the papers provisionally agreed for inclusion. We recognised that abstracts of some studies might not include appropriate information, and therefore that our initial abstract review could have excluded some relevant studies; we explored the extent of this potential omission of studies by reviewing the full texts of a random sample of 30 studies from the original 150. Since our prime goal was to identify a manageable number of relevant studies in order to test our hypotheses rather than identify all possible relevant studies we did not include any additional studies as a result of this exploratory study.

We postulated that the following hypotheses would support our conceptual framework:

• The words ‘pilot’ and ‘feasibility’ are both used in the literature to describe studies undertaken in preparation for an RCT evaluating the effect of an intervention or therapy
• It is possible to identify a subset of studies within the literature that are RCTs conducted in preparation for a larger RCT which evaluates the effect of an intervention or therapy. Authors do not use the term ‘pilot trial’ consistently in relation to these studies.
• Within the literature it is not possible to apply unique mutually exclusive definitions of pilot and feasibility studies in preparation for an RCT evaluating the effect of an intervention or therapy that are consistent with the way authors describe their studies.
• Amongst feasibility studies in preparation for an RCT which evaluates the effect of an intervention or therapy it is possible to identify some studies that are not pilot studies as defined within our conceptual framework, but are studies that acquire information about the feasibility of applying an intervention in a future study.

In order to explore these hypotheses, we categorised included studies into three groups that tallied with our framework (see results for details): randomised pilot studies, non-randomised pilot studies, feasibility studies that are not pilot studies. We also extracted data on objectives, and the phrases that indicated that the studies were conducted in preparation for a subsequent RCT.

Validating the conceptual framework—Consensus meeting

We also took an explanation and visual representation of our framework to an international consensus meeting primarily designed to reach consensus on an extension of the 2010 CONSORT statement to randomised pilot studies. There were 19 invited participants with known expertise, experience, or interest in pilot and feasibility studies, including representatives of CONSORT, funders, journal editors, and those who had been involved in writing the NIHR definitions of pilot and feasibility studies and the MRC guidance on designing and evaluating complex interventions. Thus this was an ideal forum in which to discuss the framework also. This project was not concerned with any specific disease, and was methodological in design; no patients or public were involved.

Ninety-three individuals, including chief investigators, statisticians, trial managers, clinicians, research assistants and a funder, participated in the first round of the Delphi survey and 79 in the second round. Over 70% of participants in the first round felt that our definitions, the way we had distinguished between pilot and feasibility studies, and the labels ‘pilot’ and ‘feasibility’ were appropriate. However, these four items had some of the lowest appropriateness ratings in the survey and there were a large number of comments both in direct response to our four survey items related to appropriateness of definitions, and in open comment boxes elsewhere in the survey. Some of these comments are presented in Fig 4 . Some participants commented favourably on the definitions we had drawn up (quote 1) but others were confused by them (quote 2). Several compared our definitions to the NIHR definitions pointing out the differences (quote 3) and suggesting this might make it particularly difficult for the research community to understand our definitions (quote 4). Some expressed their own views about the definitions (quote 5); largely these tallied with the NIHR definitions. Others noted that both the concept of feasibility and the word itself were often used in relation to studies which investigators referred to as pilot studies (quote 6). Others questioned whether it was practically and/or theoretically possible to make a distinction between pilot and feasibility studies (quote 6, quote 7), suggesting that the two terms are not mutually exclusive and that feasibility was more of an umbrella term for studies conducted prior to the main trial. Some participants felt that, using our definitions, feasibility studies would be less structured and more variable and therefore their quality would be less appropriately assessed via a checklist (quote 8). These responses regarding definitions mirrored what we had found in the user-testing of the Delphi survey, the Society for Clinical Trials workshop, and differences of opinion already apparent in the literature. In the second round of the survey there were few comments about definitions.

There was a wide range of participants in the open meeting, including senior quantitative and qualitative methodologists, and a funding body representative. The four propositions we devised to cover different views about definitions of pilot and feasibility studies are shown in Fig 5 . Fourteen out of the fifteen attendees who voted on these propositions preferred propositions 3 or 4, based on comments from the Delphi survey and the MRC guidance on designing and evaluating complex interventions respectively. Neither of these propositions implied mutually exclusive definitions of pilot and feasibility studies.

Definitions of feasibility outside the health research context focus on the likelihood of being able to do something. For example, the Oxford on-line dictionary defines feasibility as: ‘The state or degree of being easily or conveniently done’ [ 14 ] and a feasibility study as: ‘An assessment of the practicality of a proposed plan or method’ [ 15 ]. Some definitions also suggest that a feasibility study should help with decision making, for example [ 16 ]: ‘The feasibility study is an evaluation and analysis of the potential of a proposed project. It is based on extensive investigation and research to support the process of decision making’. Outside the health research context the word ‘pilot’ has several different meanings but definitions of pilot studies usually focus on an experiment, project or development undertaken in advance of a future wider experiment, project or development. For example the Oxford on-line dictionary describes a pilot study as: ‘Done as an experiment or test before being introduced more widely’ [ 17 ]. Several definitions carry with them ideas that the purpose of a pilot study is also to facilitate decision making, for example ‘a small-scale experiment or set of observations undertaken to decide how and whether to launch a full-scale project’ [ 18 ] and some definitions specifically mention feasibility, for example: ‘a small scale preliminary study conducted in order to evaluate feasibility’ [ 19 ].

In keeping with these definitions not directly related to the health research context, we agreed that feasiblity is a concept encapsulating ideas about whether it is possible to do something and that a feasibility study asks whether something can be done , should we proceed with it , and if so , how . While piloting is also concerned with whether something can be done and whether and how we should proceed with it, it has a further dimension; piloting is implementing something, or part of something, in a way you intend to do it in future to see whether it can be done in practice. We therefore agreed that a pilot study is a study in which a future study or part of a future study , is conducted on a smaller scale to ask the question whether something can be done , should we proceed with it , and if so , how . The corollary of these definitions is that all pilot studies are feasibility studies but not all feasibility studies are pilot studies. Within the context of RCTs, the focus of our research, the ‘something’ in the definitions can be replaced with ‘a future RCT evaluating the effect of an intervention or therapy’. Studies that address the question of whether the RCT can be done, should we proceed with it and if so how, can then be classed as feasibility or pilot studies. Some of these studies may, of course, have other objectives but if they are mainly focusing on feasiblity of the future RCT we would include them as feasiblity studies. All three studies used as examples in our Delphi survey [ 10 – 12 ] satisfy the definition of a feasiblity study. However, a study by Piot et al , that we encountered while developing the Delphi study, does not. This study is described as a pilot trial in the abstract but the authors present only data on effectiveness and although they state that their results require confirmation in a larger study it is not clear that their pilot study was conducted in preparation for such a larger study [ 20 ]. On the other hand, Palmer et al ‘performed a feasibility study to determine whether patient and surgeon opinion was permissive for a Randomised Controlled Trial (RCT) comparing operative with non-operative treatment for FAI [femoroacetabular impingement]’ [ 12 ]. Heazell et al describe the aim of their randomised study as ‘to address whether a randomised controlled trial (RCT) of the management of RFM [reduced fetal movement] was feasible’ [ 10 ]. Their study was piloting many of the aspects they hoped to implement in a larger trial of RFM, thus making this also a pilot study, whereas the study conducted by Palmer et al , which comprised a questionnare to clinicians and seeking patient opinion, is not a pilot study but is a feasibility study.

Within our framework, some important studies conducted in advance of a future RCT to evaluate the effect of a therapy or intervention are not feasibility studies. For example, a systematic review, usually an essential pre-requisite for such an RCT, normally addresses whether the future RCT is necessary or desirable , not whether it is feasible . To reflect this, we developed a comprehensive diagrammatical representation of our framework for studies conducted in preparation for an RCT which, for completeness, includes, on the left hand side, early studies that are not pilot and feasibility studies, such as systematic reviews and, along the bottom, details of existing or planned reporting guidelines for different types of study ( S2 Fig ).

Validating the conceptual framework—Systematic review

From the 150 most recent studies identified by our electronic search, we identified 27 eligible reports ( Fig 6 ). In keeping with our working definition of a pilot or feasibility study, to be included the reports had to show evidence that investigators were addressing at least some feasibility objectives and that the study was in preparation for a future RCT evaluating the effect of an intervention. Ideally we would have stipulated that the primary objective of the study should be a feasibility objective but, given the nature of the reporting of most of these studies, we felt this would be too restrictive.

The 27 studies are reported in Table 1 and results relating to terminology that authors used summarised in Table 2 . Results in Table 2 support our first hypothesis that the words ‘pilot’ and ‘feasibility’ are both used in the literature to describe studies undertaken in preparation for a randomised controlled trial of effectiveness; 63% (17/27) used both terms somewhere in the title or abstract. The table also supports our second hypothesis that amongst the subset of feasibility studies in preparation for an RCT that are themselves RCTs, authors do not use the term ‘pilot trial’ consistently in relation to these studies; of the 18 randomised studies only eight contained the words ‘pilot’ and ‘trial’ in the title. Our third hypothesis, namely that it is not possible to apply unique mutually exclusive definitions of pilot and feasibility studies in preparation for an RCT that are consistent with the way authors describe their studies, is supported by the characteristics of studies presented in Table 1 and summarised in Table 2 . We could find no design or other features (such as randomisation or presence of a control group) that distinguished between those that investigators called feasibility studies and those that they called pilot studies. However, the fourth hypothesis, that amongst studies in preparation for an RCT evaluating the effect of an intervention or therapy it is possible to identify some studies that explore the feasibility of a certain intervention or acquire related information about the feasibility of applying an intervention in a future study but are not pilot studies, was not supported; we identified no such studies amongst those reported in Table 1 . Nevertheless, we had identified two prior to carrying out the review [ 10 , 15 ].

Out of our exploratory sample of 30 study reports for which we reviewed full texts rather than only titles and abstracts, we identified 10 that could be classed as pilot or feasibility studies using our framework. We had already identified four of these in our sample reported in Table 1 , but had failed to identify the other six. As expected, this was because key information to identify them as pilot or feasiblity studies such as the fact that they were in preparation for a larger RCT, or that the main objectives were to do with feasiblity were not included in the abstract. Thus our assumption that an initial screen using only abstracts resulted in the omission of some pilot and feasiblity studies was correct.

International consensus meeting participants agreed with the general tenets of our conceptual framework including the ideas that all pilot studies are feasibility studies but that some feasibility studies are not pilot studies. They suggested that any definitive diagrammatic representation should more strongly reflect non-linearity in the ordering of feasibility studies. As a result of their input we produced a new, simplified, diagrammatical representation of the framework ( Fig 7 ) which focuses on the key elements represented inside an oval shape on our original diagram, omits the wider context outside this shape, and highlights some features, including the non-linearity, more clearly.

The finalised framework

Fig 7 represents the framework. The figure indicates that where there is uncertainty about future RCT feasibility, a feasibility study is appropriate. Feasibility is thus an overarching concept within which we distinguish between three distinct types of study. Randomised pilot studies are those studies in which the future RCT, or parts of it, including the randomisation of participants, is conducted on a smaller scale (piloted) to see if it can be done. Thus randomised pilot studies can include studies that for the most part reflect the design of a future definitive trial but, if necessary due to remaining uncertainty, may involve trying out alternative strategies, for example, collecting an outcome variable via telephone for some participants and on-line for others. Within the framework randomised pilot studies could also legitimately be called randomised feasibility studies. Two-thirds of the studies presented in Table 1 are of this type.

Non-randomised pilot studies are similar to randomised pilot studies; they are studies in which all or part of the intervention to be evaluated and other processes to be undertaken in a future trial is/are carried out (piloted) but without randomisation of participants. These could also legitimately be called by the umbrella term, feasibility study. These studies cover a wide range from those that are very similar to randomised pilot studies except that the intervention and control groups have not been randomised, to those in which only the intervention, and no other trial processes, are piloted. One-third of studies presented in Table 1 are of this type.

Feasibility studies that are not pilot studies are those in which investigators attempt to answer a question about whether some element of the future trial can be done but do not implement the intervention to be evaluated or other processes to be undertaken in a future trial, though they may be addressing intervention development in some way. Such studies are rarer than the other types of feasibility study and, in fact, none of the studies in Table 1 were of this type. Nevertheless, we include these studies within the framework because they do exist; the Palmer study [ 15 ] in which surgeons and patients were asked about the feasibility of randomisation is one such example. Other examples might be interviews to ascertain the acceptability of an intervention, or questionnaires to assess the types of outcomes participants might think important. Within the framework these studies can be called feasibility studies but cannot be called pilot studies since no part of the future randomised controlled trial is being conducted on a smaller scale.

Investigators may conduct a number of studies to assess feasibility of an RCT to test the effect of any intervention or therapy. While it may be most common to carry out what we have referred to as feasibility studies that are not pilot studies before non-randomised pilot studies , and non-randomised pilot studies prior to randomised pilot studies , the process of feasibility work is not necessarily linear and such studies can in fact be conducted in any order. For completeness the diagram indicates the location of internal pilot studies.

There are diverse views about the definitions of pilot and feasibility studies within the research community. We reached consensus over a conceptual framework for the definitions of these studies in which feasibility is an overarching concept for studies assessing whether a future study, project or development can be done. For studies conducted in preparation for a RCT assessing the effect of a therapy or intervention, three distinct types of study come under the umbrella of feasibility studies: randomised pilot studies, non-randomised pilot studies, feasibility studies that are not pilot studies. Thus pilot studies are a subset of feasibility studies. A review of the literature confirmed that it is not possible to apply mutually exclusive definitions of pilot and feasibility studies in preparation for such an RCT that are consistent with the way authors describe their studies. For example Lee et al [ 31 ], Boogerd et al [ 22 ] and Wolf et al [ 38 ] all describe randomised studies exploring the feasibility of introducing new systems (brain computer interface memory training game, on-line interactive treatment environment, bed-exit alarm respectively) but Lee et al describe their study as a ‘A Randomized Control Pilot Study’, with the word ‘feasibility’ used in the abstract and text, while the study by Boogerd et al . is titled ‘Teaming up: feasibility of an online treatment environment for adolescents with type 1 diabetes’, and Wolf at al describe their study as a pilot study without using the word ‘feasibility’.

Our re-evaluation of the definitions of pilot and feasibility studies was conducted over a period of time with input via a variety of media by multi-disciplinary and international researchers, publishers, editors and funders. It was to some extent a by-product of our work developing reporting guidelines for such studies. Nevertheless, we were able to gather a wide range of expert views, and the iterative nature of the development of our thinking has been an important part of obtaining consensus. Other parallel developments, including the recent establishment of the new Pilot and Feasibility Studies journal [ 48 ], suggest that our work is, indeed, timely. We encountered several difficulties in reviewing empirical study reports. Firstly, it was sometimes hard to assess whether studies were planned in preparation for an RCT or whether the authors were conducting a small study and simply commenting on the fact that a larger RCT would be useful. Secondly, objectives were sometimes unclear, and/or effectiveness objectives were often emphasised in spite of recommendations that pilot and feasibility studies should not be focusing on effectiveness [ 1 , 4 ]. In identifying relevant studies we erred on the side of inclusiveness, acknowledging that getting these studies published is not easy and that there are, as yet, no definitive reporting guidelines for investigators to follow. Lastly, our electronic search was unable to identify any feasibility studies that were not pilot studies according to our definitions. Subsequent discussion with qualitative researchers suggested that this is because such studies are often not described as feasibility studies in the title or abstract.

Our framework is compatible with the UK MRC guidance on complex interventions which suggests a ‘feasibility and piloting’ phase as part of the work to design and evaluate such interventions without any explicit distinction between pilot and feasibility studies. In addition, although our framework has a different underlying principle from that adopted by UK NIHR, the NIHR definition of a pilot study is not far from the subset of studies we have described as randomised pilot studies. Although there appears to be increasing interest in pilot and feasibility studies, as far as we are aware no other funding bodies specifically address the nature of such studies. The National Institute for Health in the USA does, however, routinely require published pilot studies before considering funding applications for certain streams, and the Canadian Institutes of Health Research routinely have calls for pilot or feasibility studies in different clinical areas to gather evidence necessary to determine the viability of new research directions determined by their strategic funding plans. These approaches highlight the need for clarity regarding what constitutes a pilot study.

There are several previous reviews of empirical pilot and feasibility studies [ 1 , 4 , 7 ]. In the most recent, reviewing studies published between 2000 and 2009 [ 7 ], the authors identified a large number of studies, described similar difficulty in identifying whether a larger study was actually being planned, and similar lack of consistency in the way the terms ‘pilot’ and ‘feasibility’ are used. Nevertheless, in methodological work, many researchers have adopted fairly rigid definitions of pilot and feasibility studies. For example, Bugge et al in developing the ADEPT framework refer to the NIHR definitions and suggest that feasibility studies ask questions about ‘whether the study can be done’ while pilot trials are ‘(a miniature version of the main trial), which aim to test aspects of study design and processes for the implementation of a larger main trial in the future’ [ 49 ]. Although not explicitly stated, the text seems to suggest that pilot and feasibility studies are mutually exclusive. Our work indicates that this is neither necessary nor desirable. There is, however, general agreement in the literature about the purpose of pilot and feasibility studies. For example, pilot trials are ‘to provide sufficient assurance to enable a larger definitive trial to be undertaken’ [ 50 ], and pilot studies are ‘designed to test the performance characteristics and capabilities of study designs, measures, procedures, recruitment criteria, and operational strategies that are under consideration for use in a subsequent, often larger, study’ [ 51 ], and ‘play a pivotal role in the planning of large-scale and often expensive investigations’ [ 52 ]. Within our framework we define all studies aiming to assess whether a future RCT is do-able as ‘feasibility studies’. Some might argue that the focus of their study in preparation for a future RCT is acceptability rather than feasibility, and indeed, in other frameworks, such as the RE-AIM framework [ 53 ], feasibility and acceptability are seen as two different concepts. However, it is perfectly possible to explore the acceptability of an intervention, of a data collection process or of randomisation in order to determine the feasibility of a putative larger RCT. Thus the use of the term ‘feasibility study’ for a study in preparation for a future RCT is not incompatible with the exploration of issues other than feasibility within the study itself.

There are numerous previous studies in which the investigators review the literature and seek the counsel of experts to develop definitions and clarify terminology. Most of these relate to clinical or physiological definitions [ 54 – 56 ]. A few explorations of definitions relate to concepts such as quality of life [ 57 ]. Implicit in much of this work is that from time to time definitions need rethinking as knowledge and practice moves on. From an etymological point of view this makes sense. In fact, the use of the word ‘pilot’ to mean something that is a prototype of something else only appears to emerge in the middle of the twentieth century and the first use of the word in relation to research design that we could find was in 1947—a pilot survey [ 58 ]. Thus we do not have to look very far back to see changes in the use of one of the words we have been dealing with in developing our conceptual framework. We hope what we are proposing here is helpful in the early twenty-first century to clarify the use of the words ‘pilot’ and ‘feasibility’ in a health research context.

We suggest that researchers view feasibility as an overarching concept, with all studies done in preparation for a main study open to being called feasibility studies, and with pilot studies as a subset of feasibility studies. All such studies should be labelled ‘pilot’ and/or ‘feasibility’ as appropriate, preferably in the title of a report, but if not certainly in the abstract. This recommendation applies to all studies that contribute to an assessment of the feasibility of an RCT evaluating the effect of an intervention. Using either of the terms in the title will be most helpful for those conducting future electronic searches. However, we recognise that for qualitative studies, authors may find it convenient to use the terms in the abstract rather than the title. Authors also need to describe objectives and methods well, reporting clearly if their study is in preparation for a future RCT to evaluate the effect of an intervention or therapy.

Though the focus of this work was on the definitions of pilot and feasibility studies and extensive recommendations for the conduct of these studies is outside its scope, we suggest that in choosing what type of feasibility study to conduct investigators should pay close attention to the major uncertainties that exist in relation to trial or intervention. A randomised pilot study may not be necessary to address these; in some cases it may not even be necessary to implement an intervention at all. Similarly, funders should look for a justification for the type of feasibility study that investigators propose. We have has also highlighted the need for better reporting of these studies. The CONSORT extension for randomised pilot studies that our group has developed are important in helping to address this need and will be reported separately. Nevertheless, further work will be necessary to extend or adapt these reporting guidelines for use for non-randomised pilot studies and for feasibility studies that are not pilot studies. There is also more work to be done in developing good practice guidance for the conduct of pilot and feasibility studies.

Supporting Information

Acknowledgments.

We thank Alicia O’Cathain and Pat Hoddinot for discussions about the reporting of qualitative studies, and consensus participants for their views on our developing framework. Claire Coleman was funded by a National Institute for Health Research (NIHR) Research Methods Fellowship. This article presents independent research funded by the NIHR. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.

Funding Statement

The authors received small grants from Queen Mary University of London (£7495), University of Sheffield (£8000), NIHR RDS London (£2000), NIHR RDS South East (£2400), Chief Scientist Office Scotland (£1000). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Data Availability

General guidance - Innovate UK

Categories of research and development.

The competition scope will specify the category of research and development (R&D) activity for that particular competition.

Innovate UK supports the following R&D categories:

• fundamental research
• feasibility studies
• industrial research
• experimental development

Fundamental research

This means experimental or theoretical work primarily to gain new knowledge of underlying phenomena and visible facts, without any direct practical application or usage. This type of research is usually undertaken by a research organisation.

Feasibility studies

This means analysis and evaluation of a project’s potential, aimed at supporting the process of decision making.

This is achieved by uncovering its strengths, weaknesses, opportunities and threats as well as identifying the resources needed and the prospects for success.

Feasibility studies will usually help businesses decide to work either individually or collaboratively with other industrial or research organisations, before conducting a subsequent larger project.

Individual competition scopes will define their own requirements for feasibility studies in terms of project size and length.

Industrial research

This means planned research or critical investigation to gain new knowledge and skills. This should be for the purpose of product development, processes or services that lead to an improvement in existing products, processes or services.

It can include the creation of component parts to complex systems and may include prototypes in a laboratory or environment with simulated interfaces to existing systems, particularly for generic technology validation.

Experimental development

‘Experimental development’ means acquiring, combining, shaping and using existing scientific, technological, business and other relevant knowledge and skills with the aim of developing new or improved products, processes or services.

This may also include, for example, activities aimed at the conceptual definition, planning and documentation of new products, processes or services.

Experimental development may comprise prototyping, demonstrating, piloting, testing and validation of new or improved products, processes or services in environments representative of real life operating conditions.

The primary objective is to make further technical improvements on products, processes or services that are not substantially set.

This may include the development of a commercially usable prototype or pilot which is not necessarily the final commercial product and which is too expensive to produce for it to be used only for demonstration and validation purposes.

Experimental development does not include routine or periodic changes made to existing products, production lines, manufacturing processes, services and other operations in progress, even if those changes may represent improvements.

Projects that span more than one category of research

In your application enter the research category that reflects the majority of your work. If your work is going to be evenly split across categories, please email [email protected]

Last updated: 24 November 2023

This is the website for UKRI: our seven research councils, Research England and Innovate UK. Let us know if you have feedback or would like to help improve our online products and services .

• Open access
• Published: 23 August 2024

Modeling of the native knee with kinematic data derived from experiments using the VIVO™ joint simulator: a feasibility study

• Paul Henke 1   na1 ,
• Johanna Meier 1   na1 ,
• Leo Ruehrmund 1 ,
• Saskia A. Brendle 2 , 3 ,
• Sven Krueger 2 ,
• Thomas M. Grupp 2 , 3 ,
• Christoph Lutter 1 ,
• Christoph Woernle 4 ,
• Rainer Bader 1 &
• Maeruan Kebbach 1  

BioMedical Engineering OnLine volume  23 , Article number:  85 ( 2024 ) Cite this article

Metrics details

Despite advances in total knee arthroplasty, many patients are still unsatisfied with the functional outcome. Multibody simulations enable a more efficient exploration of independent variables compared to experimental studies. However, to what extent numerical models can fully reproduce knee joint kinematics is still unclear. Hence, models must be validated with different test scenarios before being applied to biomechanical questions.

In our feasibility study, we analyzed a human knee specimen on a six degree of freedom joint simulator, applying a passive flexion and different laxity tests with sequential states of ligament resection while recording the joint kinematics. Simultaneously, we generated a subject-specific multibody model of the native tibiofemoral joint considering ligaments and contact between articulating cartilage surfaces.

Our experimental data on the sequential states of ligament resection aligned well with the literature. The model-based knee joint kinematics during passive flexion showed good agreement with the experiment, with root-mean-square errors of less than 1.61 mm for translations and 2.1° for knee joint rotations. During laxity tests, the experiment measured up to 8 mm of anteroposterior laxity, while the numerical model allowed less than 3 mm.

Although the multibody model showed good agreement to the experimental kinematics during passive flexion, the validation showed that ligament parameters used in this feasibility study are too stiff to replicate experimental laxity tests correctly. Hence, more precise subject-specific ligament parameters have to be identified in the future through model optimization.

Despite tremendous advances in total knee arthroplasty (TKA), only 80%–85% of patients are satisfied with the postoperative outcome [ 1 , 2 ]. Biomechanical studies are conducted to better understand knee joint kinematics before and after TKA [ 3 , 4 , 5 ]. Such studies are performed experimentally or numerically, while computer-based studies need experimental data to be validated [ 6 , 7 , 8 ].

On the other hand, studies on patients enable the investigation of the native knee joint and functionality of TKA, considering the soft tissue structures [ 9 ]. However, extensive parameter studies are not possible due to ethical aspects. In addition, investigating different motion sequences is technically challenging, and the measurement of kinematics on the patient is partially prone to errors [ 10 ]. Contrarily, experimental setups with human specimens allow for examining the knee joint at different complexity scales. For this purpose, mechanical knee rigs [ 11 , 12 ] and industrial robots [ 13 , 14 , 15 ] have been used. Although these setups provide valuable insights into the knee joint dynamics, test rigs usually cannot apply defined loads and torques about all three spatial axes, and the usage of robots can be very complex and time-consuming. To overcome these limitations, the six degrees of freedom joint simulator VIVO™ (Advanced Mechanical Technology, Inc., Watertown, USA) is used to study the kinematics of the native human knee joint and the contribution of specific ligaments to overall joint stability [ 16 , 17 ]. It can also be used for testing implant components with virtual ligaments represented by numerical strain-force laws [ 18 , 19 ].

Based on the experimental data, multibody simulations are used to noninvasively analyze the knee joint kinematics and serve as a tool for virtual tests of different implant designs [ 20 , 21 , 22 , 23 ]. However, the quality of the simulation results highly depends on the boundary conditions used, and it is still unclear to what extent they fully reproduce the kinematics of the human knee joint [ 24 ]. Especially the parameters used to model the ligaments are described with a broad range in the literature and significantly contribute to joint kinematics [ 25 ].

Therefore, the aim of our present study is to generate a subject-specific multibody model of the right knee joint that mimics an experimental setup for investigating the human knee joint kinematics. For this setup, a knee specimen is experimentally examined on the VIVO™ joint simulator using different load cases, i.e., passive knee flexion and various laxity tests, to characterize knee joint kinematics. Additionally, tests are repeated with a sequential resection strategy of the knee ligaments. Based on medical images of the human specimen, a complex subject-specific multibody model comprising ligament structures and contact between articulating cartilage surfaces is generated to mimic the experimental tests. A more physiological simulation of the human knee joint kinematics will significantly contribute to several biomechanical questions, such as soft tissue management during TKA [ 26 ].

Experimentally derived knee kinematics

Passive flexion.

Figure  1 shows the kinematics over time during the passive knee flexion for the different resection states of the ligaments. Results were evaluated for the tibia coordinate system with respect to the femur coordinate system according to the Grood and Suntay convention [ 27 ].

Results of the anteroposterior translation, the varus/valgus rotation, and the external/internal rotation for different resection states during passive flexion and extension. The experimental data shows the position of the tibia coordinate system with respect to the femur coordinate system in Grood and Suntay convention [ 27 ]

Starting from initial values of 19–22 mm, the anterior translations for all resection states peaked approximately near the highest flexion value at 12.5 s with maximum values between 27 and 33 mm before returning to the original initial values with the shift back to extension. Resection of the patella slightly reduced anterior translation during the transition from flexion to extension, whereas the reverse was unaffected.

The varus/valgus curve showed a similar trend for all resection stages. Starting from a slight valgus position between − 2.4° and − 1.5°, the varus angle constantly decreased until midflexion (30–45° flexion) before it rose to its original value. While resection of the patella and lateral collateral ligament (LCL) decreased the varus/valgus rotation, removal of the medial collateral ligament (MCL) increased the rotation by up to 0.5°.

The course of external/internal rotation was similar for all resection states. It started at the same point at about 1° external rotation, then climbed to a maximum during midflexion before returning to its initial value. The rotation depends on whether it goes into or comes out of flexion. Patellar resection reduced external rotation by up to 3°. Resection of the anterior cruciate ligament (ACL) and especially the posterior cruciate ligament (PCL) further reduced the range of external/internal rotation.

Laxity tests

All three translational and rotational degrees of freedom during loading in anteroposterior, varus/valgus, and external/internal directions were recorded over 10 s and averaged to evaluate the laxity tests. The range of movement is shown via bars in Fig.  2 . The upper end of the bars represents the position for 40 N posterior/5 Nm varus/2.5 Nm internal loading. A line highlights the position during passive flexion, while the lower end of the bars represents the position for 40 N anterior/5 Nm valgus/2.5 Nm external loading. Resection of the patella had no considerable influence on the knee laxity results. With resection of the ACL, laxity in the anterior direction increased by up to 5 mm for flexion angles of 30° and 60°. At 90° flexion, the load had to be reduced to 10 N due to the limitation of the working space of the joint simulator. These cases are marked with a triangle in Fig.  2 . Laxity during anterior loading was not changed by the resection of the PCL and LCL at 30° and 60° flexion. At 90° flexion, the maximum load could not be applied during the working space's limitation. Laxity during posterior loading increased by approximately 5 mm after resecting the PCL, 5 mm after resecting the LCL, and 2.5 mm after resection of the MCL at a flexion angle of 30°. The varus/valgus rotation limits at 5 Nm load showed almost no influence on the patella, ACL, and PCL resection for 30° and 60° flexion angles. However, the resection of the LCL and MCL had a distinct influence on valgus/varus rotation. Some load cases could not be included due to the specimen's instability and are marked with a rhombus in Fig.  2 . External and internal rotation limits were not reasonably changed by the patella, ACL, and PCL resection at a 30° flexion angle. For 60° and 90° flexion angles, resecting the PCL increased the internal laxity by about 5°. Resection of the LCL resulted in higher external rotation for all flexion angles, whereas resecting the MCL led to higher external and internal rotation at all flexion angles.

Results of the experimental laxity tests, where the upper end of the bars represents the position of the specimen for 40 N anterior/5 Nm varus/2.5 Nm external load. The lower end represents the position for 40 N posterior/5 Nm valgus/2.5 Nm internal load. The position during passive flexion is marked in between. The loads that had to be reduced to 10 N instead of 40 N due to the limitation of the working space of the joint simulator are marked with a triangle. The trials marked with a rhombus could not be carried out for the same reason

Numerically derived kinematics of passive flexion

The root mean square error (RMSE) was calculated for the kinematic parameters obtained from the multibody simulations of the passive flexion cycle to compare the numerical data with the experimental results. In the case of translational kinematics, the RMSE was 1.32 mm in the mediolateral direction, 1.61 mm in the anteroposterior direction, and 1.34 mm in the superoinferior direction. For the rotational kinematics, the RMSE was 0.84° for varus/valgus and 2.12° for external/internal rotation.

The data of anterior translation of the tibia with increasing flexion angles derived from the multibody modeling agreed with the experimental kinematics (Fig.  3 ). The varus/valgus rotation displayed the same progression with increasing flexion angle but slightly different start values and positions during 90° of flexion. For external and internal rotation, numerical and experimental kinematics showed the maximum external rotation at 60° flexion and an external rotation when reaching 90° extension and the end of the motion. The amplitude of the rotations was in the same range, whereby the numerical kinematics showed more deviations in the rotation angle.

Numerically derived data of the anteroposterior translation, the varus/valgus rotation, and the external/internal rotation compared to experimental results during passive flexion and extension. The experimental results refer to the resection state without the patella but with all ligaments. Data show the position of the tibia coordinate system with respect to the femur coordinate system in Grood and Suntay convention [ 27 ]

A maximum contact force during passive flexion of 1000 N was calculated from the multibody model at a flexion angle of 90°. In general, numerical modeling resulted in lower laxity of the ligaments (Fig.  4 ). While the experiments showed a range of laxity between 6 and 8 mm in anteroposterior direction for different flexion angles when all ligaments were intact, the numerical model only allowed 2–3 mm of movement when applying the same loads. For varus/valgus laxity, the experiments showed a laxity of about 10°, while in the numerical model, the laxity was reduced to approximately 1.5° for all flexion angles. Laxity for external and internal rotation was also reduced in the numerical model from approximately 25° in the experiments to approximately 10° in the multibody simulation. The influence of sequential resection could not be further investigated in the numerical model, as the resection of individual ligaments led to significant instability.

Results of the simulated laxity tests with all ligaments intact. The upper end of the bars represents the position of the specimen for 40 N anterior/5 Nm varus/2.5 Nm external load. The lower end represents the position for 40 N posterior/5 Nm valgus/2.5 Nm internal load. The position during passive flexion is marked in between

Ligament forces calculated by the numerical model varied between 300 and 550 N during passive flexion for the ACL and between 0 and 420 N for the PCL. The progression of the forces in the cruciate ligaments is shown in Fig.  5 . The force in the ACL decreased with flexion angle, while the PCL force increased with flexion angle. Results for the LCL and MCL forces are available as Additional file 1 in the Supplementary Information.

The calculated ACL and PCL ligament forces derived from the multibody model for the passive flexion over time

This feasibility study aimed to investigate individual knee kinematics during different loading scenarios by generating a subject-specific multibody model that mimics the experimental data. The measured kinematics during passive flexion and different laxity measurements were in good agreement with the literature for both native and resected states. Nevertheless, it has to be noted that the direct comparison with other studies is only of limited value due to the different and often insufficiently documented positioning of the joint coordinate systems. In future work the coordinate systems should be defined as standardized as possible (e.g. using 'REFRAME' [ 28 ]) to ensure better comparability with studies of other groups.

While the kinematics of passive flexion of the experimental testing could be well reproduced with the multibody model, it showed a reduced range of laxity compared to the experimental tests. We conclude that the ligament parameters from the literature used in our study are too stiff to fully represent our experimental test results.

The test results considering tensile loading of the quadriceps femoris tendon showed increased anterior translation of the tibia during passive flexion. It has been demonstrated in a previous study that isolated muscle loading affects tibiofemoral kinematics [ 29 ]. Passive flexion induced an anterior displacement of the tibia, primarily caused by the tension of the PCL with increasing knee flexion [ 30 ]. This displacement could be seen in the experimentally recorded data and was comparable to data from other groups [ 25 , 29 , 31 ]. After PCL resection, the translation was less consistent with the literature [ 25 , 32 ]. In contrast, removing the ACL increased anterior translation of the tibia, especially in early flexion.

The patella or the ligaments' resection did not considerably affect the varus/valgus rotation during passive flexion. There was a slight increase in valgus rotation after the resection of the LCL and a slight rise in varus rotation after the resection of the MCL, corresponding to the ligaments' physiological directions of action. The resections of the collateral ligaments resulted in increased valgus and varus rotation, as both the MCL [ 33 ] and the LCL [ 34 ] function as stabilisers for varus/valgus rotations in the human knee joint. This agrees with the literature, where the varus/valgus rotation does not depend on the flexion angle [ 35 ].

External and internal rotation showed the screw-home mechanism when reaching extension in experimental and numerical data, which is well documented in the literature [ 25 , 31 , 32 , 36 ]. When full flexion of 90° was achieved, an internal rotation of approximately 4° was also evident in the experimental results. During the last 10° of extension, a final rotation of approximately 2° could be observed. Resection of the ACL and PCL reduced the changes in external/internal rotation angle during flexion due to missing wrapping of the cruciate ligaments. In general, rotation was more internally rotated after the resection of the ACL, which can be explained physiologically by the lack of wrapping of the cruciate ligaments around each other. Similar observations were documented by Markolf et al. [ 31 ]. Resection of the collateral ligaments led to a further reduction in tibial rotation during passive flexion, as the tibial torque was reduced since at least one of the ligaments is usually under tension throughout flexion movements [ 37 ].

Liu-Barba et al. [ 38 ] described a correlation between the applied compression force and the resulting translations and rotations. Different amounts of applied compression force complicate the comparability between the studies. Increased internal rotation during extension compared with flexion has been similarly documented by Markolf et al. [ 31 ]. Testing with the patella and loaded quadriceps femoris tendon also showed increased internal rotation, which may be explained by the individually acting lever arms of the muscle forces and cannot be evaluated with sufficient accuracy without considering a force effect in the hamstring muscles.

Considering the resection states, the results of the laxity tests agreed with the literature data [ 8 ]. An almost identical AP laxity during the tests with and without the patella supports the assumption that the differences in kinematics with and without the patella during passive flexion were due to the load applied to the tendon of the quadriceps femoris muscle. Nevertheless, further studies should be performed to verify this assumption. Resection of the ACL increased the possible translation in the anterior direction, and resection of the PCL showed a more posterior translation of the tibia as supported by Willinger et al. [ 39 ] and Kennedy et al. [ 40 ]. In addition, the possible internal rotation during laxity tests after resection of the ACL and PCL was higher, especially at large flexion angles, which is plausible since the wrapping of the cruciate ligaments no longer restricts internal rotation. Resection of the LCL essentially allowed more valgus rotation, while removal of the MCL increased varus rotation. In addition, more external rotation was possible after the resection of the LCL and more rotation in general after the resection of the MCL, which can explain the instability of the knee joint. The translation values during anterior and posterior loading of the intact preparation were lower than those of Moslemian et al. [ 36 ]. However, they agreed well with the data of Pedersen et al. [ 8 ]. On the other hand, the values of varus/valgus and external/internal rotations were comparable to the results of Moslemian et al. [ 36 ].

Nevertheless, the comparison is limited because the applied loads and the definition of the coordinate systems may differ in several studies. More precisely, the loads applied in this study were reduced by 50% compared to those documented in previous literature [ 8 , 16 , 17 ]. This modification was necessary as initial trials with increased loads exceeded the machine's working space after multiple resections due to rising laxity.

A multibody model of the tibiofemoral joint has been created to mimic the experimental kinematics during passive flexion and laxity tests. The RMSE was considerably low for all degrees of freedom. Hence, a good agreement was reached between experimental and numerical kinematics. The range of motion of the tibial anteroposterior translation in the simulation was slightly lower compared to the experiment. This could be due to stiffer cruciate ligaments in the multibody model restricting the range of motion.

The varus/valgus curve also showed a similar curve in the multibody simulation. However, it started with about 1.5° more valgus rotation. Contrary to the experimentally measured data, whether the flexion angle was approached from flexion or extension was also not essential in the multibody model, as the values do not differ. This is a general difference in comparing the kinematics between the experiment and the multibody model, which becomes even more apparent when considering the external/internal rotation. While the numerical curves show a symmetry axis in the middle of the load cycle (at 12.5 s), the experiment's results differ between the extension and flexion phases. This may be due to the neglected time dependency of the ligament properties. The reduced range of motion in rotation in comparison to the literature was also reflected by the model. The resection of a single ligament led to a considerable imbalance in the simulation, which the remaining ligaments could not compensate for.

While the same progression of change in the ACL and PCL force over passive flexion was shown in other models [ 25 , 41 , 42 , 43 ] and experimental studies [ 43 ], the value of the forces in the simulation is considerably higher. The decreased range of movement in simulated laxity tests indicates that ligament forces in the model are much higher than in the experiment. Too high ligament stiffness is probably the reason for model instabilities in the simulation of the individual resection stages. This points out the necessity for further optimization of ligament modeling.

Some limitations of our feasibility study have to be taken into consideration. First, only one human specimen was tested, which does not provide a statistical indication of the reproducibility of the tests. This specimen was preconditioned before the start and rewetted at regular intervals between the experiments to ensure the same test conditions before each test. Nevertheless, it can not be excluded that the physiology changed over the testing period of eight hours. The friction properties of the contact surfaces on the preparation may also have changed over time. Physiological lubrication conditions were assumed in the tests on the intact specimen. After patellar resection, the contact surfaces were continuously wetted with sodium chloride to prevent them from drying out. However, this may also have an influence on the friction properties of the preparation due to different properties compared to human synovial fluid [ 44 ]. In the experimental setup, when testing the intact joint, only the quadriceps muscles were loaded over the patella and only with a constant force, while the posterior muscles were neglected. When performing the tests, only the joint reaction forces were controlled. However, we do not have any information about the actual contact forces acting in the joint, which would have been beneficial for the subsequent validation of the multibody model.

Regarding the multibody model, limitations were mainly associated with ligament modeling and tibiofemoral contact. While being three-dimensional organic structures, ligaments were modeled as one-dimensional force spring elements. However, according to the literature [ 37 ], this assumption is legitimate to simulate the ligaments accurately. The multibody model does not use time-dependent force elements for the ligaments, whereas in-vivo, this property can be attributed to the viscoelasticity of the ligaments and the cartilage [ 45 ]. As this may be a reason for differences in the symmetry of the kinematic, the influence of time dependence when modeling the ligaments should be investigated in the future. Wrapping the ligament around bones or other soft tissues was omitted. In future work, the virtual ligament model of the joint simulator shall be used, which does not provide such wrapping around obstacles [ 46 , 47 , 48 ]. In addition, previous studies [ 37 , 49 ] showed that the absence of wrapping is essential for non-physiological loading and movement scenarios, which were not investigated in this work. Moreover, identifying suitable ligament parameters is still challenging as the reported parameters in the literature are inconsistent [ 37 ]. The results of the numerical laxity tests indicate that the entire ligament apparatus is slightly too stiff to reproduce the experimental investigations more accurately. The computation of ligament force as Wismans' method, which does not consider viscoelasticity, is also predefined by the virtual ligament modeling used in the joint simulator. This factor could explain the observed asymmetry in the experimental studies, which was not replicated numerically. The current model is also limited by the contact implementation, where the menisci are neglected, and viscoelasticity is missing at the contact element between the femoral and tibial cartilage. In addition, fluid effects, which have an influence on contact mechanics in numerical studies, are not considered by the contact element used [ 50 , 51 ]. Furthermore, the parametrization of contact modeling is always a challenge in biomechanical simulations [ 37 ].

Conclusions

In our feasibility study, a human fresh-frozen specimen was successfully investigated experimentally on a complex six-degree-of-freedom joint simulator with passive flexion and different laxity tests under sequential ligament resections. A subject-specific multibody model of the tibiofemoral joint was generated and showed good agreement with the experimental kinematics during passive flexion. Differences in the laxity tests between the multibody model and the experimental investigations suggest that ligament parameterization is still challenging and needs further improvement. Even with a good kinematic agreement between experiment and simulation for the passive flexion, it is necessary to validate multibody models with different load cases that especially consider the stiffness of specific ligaments, such as laxity tests. Currently, a multibody optimization algorithm is developed to determine subject-specific ligament parameters using the experimental results obtained in this work. By coupling the multibody model with an optimisation algorithm, the ligament parameters will be iterated until the error between the experimentally determined kinematics and the calculated model kinematics is minimized [ 14 ]. These ligament parameters can be used in future experimental studies using the joint simulator and in computational investigations of knee joint dynamics under different loading scenarios.

Experimental testing

Ethical approval was obtained from the Institutional Review Board (A 2023-0055) for our experimental testing. Before preceding the experiments on the VIVO™ joint simulator, a fresh-frozen human specimen (male, 76 years, 56.7 kg) of the right knee was analyzed by computed tomography (SOMATOM Perspective, Siemens Healthineers, Erlangen, Germany). Based on these data, bone structures of the fibula, tibia, femur as well as the femoral cartilage structures were segmented and reconstructed with the software Amira 5.4.1 (Thermo Fisher Scientific, Waltham, Massachusetts, USA and Zuse Institute Berlin, Berlin, Germany). The tibial cartilage was measured and digitised with an Artec Space Spider 3D scanner (Artec 3D, Senningerberg, Luxembourg), as it was not possible to clearly distinguish between tibial cartilage and menisci during the segmentation process using the CT scan of the frozen specimen. All cartilage surfaces were processed in Geomagic Studio 2013 (3D Systems, Rock Hill, South Carolina, USA). Joint coordinate systems were defined according to the literature [ 52 , 53 , 54 ]. Relative motion between the femur and tibia was defined in Grood and Suntay coordinates [ 27 ]. In Fig.  6 , a brief overview of the convention of Grood and Suntay is depicted based on [ 27 ].

Depiction of the femoral and tibial joint coordinate systems, with the flexion/extension axis fixed to the femur coordinate system, the external/internal rotation axis fixed to the tibia coordinate system and the varus/valgus rotation axis defined as a floating axis perpendicular to these two according to Grood and Suntay [ 27 ]

Two actuators realize the movement of the joint simulator. The upper actuator consists of an abduction arm coupled with a flexion arm, thus enabling two rotations (flexion/extension, varus/valgus-rotation). The lower actuator, the xyz table, can perform three translations and one rotation around the vertical axis (internal/external). During the test, a kinematic loop equation maps the movements of the machine's hydraulic actuators to the Grood and Suntay coordinates in real time [ 55 ]. The knee was aligned in the joint simulator to match the previously defined femoral and tibial joint coordinate systems. Thus, the knee joint flexion axis corresponded to the joint simulator's flexion axis. For fixation, the femoral and tibial bones were resected proximally and distally and embedded in fast-curing resin GP 010 (Gößl und Pfaff, Brautlach, Germany) to ensure the correct positioning (Fig.  7 ). VIVO TM s degrees of freedom, as well as the experimental setup, are depicted in Fig.  7 .

Illustration of the various degrees of freedom of the VIVO™ simulator (left) with the upper actuator enabling two rotations, one with the flexion arm (red) and one with the abduction arm (blue). The lower actuator has one rotational and three translational degrees of freedom (green). The experimental setup (right) has the femur and tibia attached to the upper and lower actuators of the VIVO™ joint simulator. The constant tensile load was applied to the patella via a cable

Regarding the experimental protocol, the human specimen was thawed for twelve hours before testing. Afterward, the specimen was moved with 20 cycles of passive flexion to precondition the soft tissues [ 55 ]. Different resection states were performed, and simultaneous knee joint kinematics were recorded during the passive flexion and laxity tests. Before each recording, the specimen was moved with at least two cycles of passive flexion for preconditioning purposes. To simulate passive flexion, the flexion axis of the simulator was position-controlled with a prescribed flexion angle as a sine wave with a period 25 s for a cycle between full extension (0°) and flexion (90°). The two other rotational degrees of freedom, the varus/valgus rotation and the tibial internal/external rotation, were torque-controlled to a constant value of 0 Nm.

In comparison, the three translation axes were force-controlled to 0 N for the two horizontal axes and to 50 N for the vertical axis representing a compressive load. The same compressive load was applied for the laxity tests, while flexion angles of 30°, 60° and 90° were prescribed. Precisely, loads in anterior/posterior, varus/valgus, and external/internal directions were applied for 10 s, according to Table  1 . At the same time, forces and torques of all other degrees of freedom were force- and torque-controlled to 0 N and 0 Nm, respectively. The data were generated with 100 Hz frequency and smoothed using a four-pole Butterworth filter. The applied loads were adapted from the literature [ 8 , 16 , 17 ].

For all tests of the knee specimen before the resection states, the patella was loaded via the quadriceps tendon with a constant tensile force of 20 N to stabilize the patella during testing [ 56 ]. The first resection state included resecting the patella and, thus, the quadriceps load. After repeated testing, including passive flexion and laxity tests according to Table  1 , further resection states include the anterior cruciate ligament (ACL), the posterior cruciate ligament (PCL), the lateral collateral ligament (LCL) and the medial collateral ligament (MCL) (Fig.  8 ).

Protocol for the sequential resection states during the experiment with passive flexion and laxity tests performed. Knee joint kinematics was measured at every resection state (except where the joint became too unstable or exceeded the machine's workspace)

Multibody model

The multibody model of the tibiofemoral joint was generated in the software Simpack (v2022, Dassault Systèmes, Vélizy-Villacoublay, France) to mimic the previously described experimental setup. The reconstructions of the CT dataset were used to set up a multibody topology with the femur and tibia. The multibody simulation did not include the patellofemoral joint, as the additional contact considerably increases the complexity. The femoral and tibial coordinate systems were implemented according to the experiment.

Regarding the multibody topology, the femur was fixed in the world coordinate system. A five degrees of freedom joint with contact between femoral and tibial cartilage was defined to connect the tibia to the femur so that the flexion could be specified at any time. The five degrees of freedom were set according to their state of equilibrium depending on internal (contact) and external (ligaments) forces. Ligaments were modeled as non-linear spring force elements according to Wismans et al. [ 57 ], and Blankevoort et al. [ 58 ], with the exerting force defined by

where \(\varepsilon\) is the strain, \(k\) the stiffness and \({\varepsilon }_{l}\) a constant set to 0.03 [ 59 ]. The strain can be calculated with

where l is the actual length of the ligament and \(l_{0}\) is the slack length of the ligament. Accordingly, the slack length can be calculated from the reference length \(l_{r}\) and the reference strain \(\varepsilon_{r} ,\)

Regarding the multibody model, the ACL and PCL, the LCL, and the MCL are divided into a superficial and deep medial collateral ligament (sMCL, dMCL) and the oblique popliteal ligament (OPL) consisting of two bundles each (anterior and posterior bundle). Additionally, one bundle each was added for the posterior oblique ligament (POL), the arcuate popliteal ligament (APL), and medial and lateral posterior capsular structures (mCAP, lCAP). Ligament insertion points were identified using the CT data, literature [ 60 , 61 , 62 , 63 ] and bony landmarks and were approved by an experienced board-certified orthopedic surgeon. The mechanical parameters of the ligaments used in the simulation are based on a previous study [ 20 ] and are summarized in Table  2 .

To consider the interaction of the articulating surfaces, a contact force element between femoral and tibial cartilage was defined using an elastic foundation model according to Hippmann et al. [ 64 ]. The compressive force of 50 N in the experiment and the loads required for the laxity tests were applied via external force elements in the multibody model. Gravity was neglected to mimic the experimental setup. Figure  9 shows the multibody model of the right knee with the prescribed flexion angle for passive flexion.

Multibody model of the right knee joint after patella resection during different phases of passive flexion for 0°, 30°, 60°, and 90° with ligaments in yellow and contact surfaces in blue. A five degree-of-freedom rheonomic joint forces a rotation of the tibia around the femoral flexion axis (x-axis). In addition to contact and ligament force elements, a constant compressive force (F) of 50 N was applied

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Actual ligament length

Slack length of the ligament

Reference length of the ligament

Ligament stiffness

Actual strain of the ligament

Transition strain—transition from quadratic region to linear region for 2 ε l

Reference strain

Total knee arthroplasty

Anterior cruciate ligament

Posterior cruciate ligament

Lateral collateral ligament

Medial collateral ligament

Superficial medial collateral ligament

Deep medial collateral ligament

Posterior oblique ligament

Arcuate popliteal ligament

Medial posterior capsular structures

Lateral posterior capsular structures

Root mean square error

Bourne RB, Chesworth BM, Davis AM, Mahomed NN, Charron KDJ. Patient satisfaction after total knee arthroplasty: who is satisfied and who is not? Clin Orthop Relat Res. 2010;468(1):57–63. https://doi.org/10.1007/s11999-009-1119-9 .

Article   Google Scholar  

Gibon E, Amanatullah DF, Loi F, Pajarinen J, Nabeshima A, Yao Z, Hamadouche M, Goodman SB. The biological response to orthopaedic implants for joint replacement: part i: metals. J Biomed Mater Res B Appl Biomater. 2017;105(7):2162–73. https://doi.org/10.1002/jbm.b.33734 .

Sharma GB, Kuntze G, Kukulski D, Ronsky JL. Validating dual fluoroscopy system capabilities for determining in-vivo knee joint soft tissue deformation: a strategy for registration error management. J Biomech. 2015;48(10):2181–5. https://doi.org/10.1016/j.jbiomech.2015.04.045 .

Lee TQ. Biomechanics of hyperflexion and kneeling before and after total knee arthroplasty. Clin Orthop Surg. 2014;6(2):117–26. https://doi.org/10.4055/cios.2014.6.2.117 .

Koh IJ, Lin CC, Patel NA, Chalmers CE, Maniglio M, Han SB, McGarry MH, Lee TQ. Kinematically aligned total knee arthroplasty reproduces more native rollback and laxity than mechanically aligned total knee arthroplasty: a matched pair cadaveric study. Orthop Traumatol Surg Res. 2019;105(4):605–11. https://doi.org/10.1016/j.otsr.2019.03.011 .

Uzuner S, Kuntze G, Li LP, Ronsky JL, Kucuk S. Creep behavior of human knee joint determined with high-speed biplanar video-radiography and finite element simulation. J Mech Behav Biomed Mater. 2022;125: 104905. https://doi.org/10.1016/j.jmbbm.2021.104905 .

Blankevoort L, Huiskes R. Validation of a three-dimensional model of the knee. J Biomech. 1996;29(7):955–61. https://doi.org/10.1016/0021-9290(95)00149-2 .

Pedersen D, Vanheule V, Wirix-Speetjens R, Taylan O, Delport HP, Scheys L, Andersen MS. A novel non-invasive method for measuring knee joint laxity in four dof: In vitro proof-of-concept and validation. J Biomech. 2019;82:62–9. https://doi.org/10.1016/j.jbiomech.2018.10.016 .

Ewing JA, Kaufman MK, Hutter EE, Granger JF, Beal MD, Piazza SJ, Siston RA. Estimating patient-specific soft-tissue properties in a TKA knee. J Orthop Res. 2016;34(3):435–43. https://doi.org/10.1002/jor.23032 .

Andersen MS, Dzialo CM, Marra MA, Pedersen D. A methodology to evaluate the effects of kinematic measurement uncertainties on knee ligament properties estimated from laxity measurements. J Biomech Eng. 2021;143(6):061003. https://doi.org/10.1115/1.4050027

Bloemker KH, Guess TM, Maletsky L, Dodd K. Computational knee ligament modeling using experimentally determined zero-load lengths. Open Biomed Eng J. 2012;6:33–41. https://doi.org/10.2174/1874230001206010033 .

Steinbrück A, Schröder C, Woiczinski M, Fottner A, Müller PE, Jansson V. Patellofemoral contact patterns before and after total knee arthroplasty: an in vitro measurement. Biomed Eng Online. 2013;12:58. https://doi.org/10.1186/1475-925X-12-58 .

Vap AR, Schon JM, Moatshe G, Cruz RS, Brady AW, Dornan GJ, Turnbull TL, Laprade RF. The role of the peripheral passive rotation stabilizers of the knee with intact collateral and cruciate ligaments: a biomechanical study. Orthop J Sports Med. 2017;5(5):2325967117708190. https://doi.org/10.1177/2325967117708190 .

Winter E, Schröder I, Bader R, Woernle C. Influence of ligament modelling on knee joint kinematics with respect to multibody optimisation. In the 5th joint international conference on multibody system dynamics. 2018.

Kebbach M, Grawe R, Geier A, Winter E, Bergschmidt P, Kluess D, D’Lima D, Woernle C, Bader R. Effect of surgical parameters on the biomechanical behaviour of bicondylar total knee endoprostheses—a robot-assisted test method based on a musculoskeletal model. Sci Rep. 2019;9(1):14504. https://doi.org/10.1038/s41598-019-50399-3 .

Sarpong NO, Sonnenfeld JJ, LiArno S, Rajaravivarma R, Donde S, Sneddon E, Kaverina T, Cooper HJ, Shah RP, Geller JA. Virtual reconstruction of the posterior cruciate ligament for mechanical testing of total knee arthroplasty implants. Knee. 2020;27(1):151–6. https://doi.org/10.1016/j.knee.2019.10.023 .

Moslemian A, Arakgi ME, Roessler PP, Sidhu RS, Degen RM, Willing R, Getgood AMJ. The Medial structures of the knee have a significant contribution to posteromedial rotational laxity control in the PCL-deficient knee. Knee Surg Sports Traumatol Arthrosc. 2021;29(12):4172–81. https://doi.org/10.1007/s00167-021-06483-1 .

Vakili S, Lanting B, Getgood A, Willing R. Development of multibundle virtual ligaments to simulate knee mechanics after total knee arthroplasty. J Biomech Eng. 2023;145(9):091003. https://doi.org/10.1115/1.4062421

Sekeitto AR, McGale JG, Montgomery LA, Vasarhelyi EM, Willing R, Lanting BA. Posterior-stabilized total knee arthroplasty kinematics and joint laxity: a hybrid biomechanical study. Arthroplasty (London, England). 2022;4(1):53. https://doi.org/10.1186/s42836-022-00153-4 .

Kebbach M, Darowski M, Krueger S, Schilling C, Grupp TM, Bader R, Geier A. Musculoskeletal multibody simulation analysis on the impact of patellar component design and positioning on joint dynamics after unconstrained total knee arthroplasty. Materials. 2020;13(10):2365. https://doi.org/10.3390/ma13102365 .

Marra MA, Vanheule V, Fluit R, Koopman BH, Rasmussen J, Verdonschot N, Andersen MS. A subject-specific musculoskeletal modeling framework to predict in vivo mechanics of total knee arthroplasty. J Biomech Eng. 2015;137(2):020904. https://doi.org/10.1115/1.4029258

Kebbach M, Geier A, Darowski M, Krueger S, Schilling C, Grupp TM, Bader R. Computer-based analysis of different component positions and insert thicknesses on tibio-femoral and patello-femoral joint dynamics after cruciate-retaining total knee replacement. Knee. 2023;40:152–65. https://doi.org/10.1016/j.knee.2022.11.010 .

Tzanetis P, Fluit R, de Souza K, Robertson S, Koopman B, Verdonschot N. Pre-planning the surgical target for optimal implant positioning in robotic-assisted total knee arthroplasty. Bioengineering. 2023;10(5):543. https://doi.org/10.3390/bioengineering10050543

Wang L, Wang CJ, Berryman F. Modeling of the knee joint. In Computational modelling of biomechanics and biotribology in the musculoskeletal system. Woodhead Publishing; 2021. p. 437–72.

Kia M, Schafer K, Lipman J, Cross M, Mayman D, Pearle A, Wickiewicz T, Imhauser C. A multibody knee model corroborates subject-specific experimental measurements of low ligament forces and kinematic coupling during passive flexion. J Biomech Eng. 2016;138(5):051010. https://doi.org/10.1115/1.4032850

Johnston H, Abdelgaied A, Pandit H, Fisher J, Jennings LM. The effect of surgical alignment and soft tissue conditions on the kinematics and wear of a fixed bearing total knee replacement. J Mech Behav Biomed Mater. 2019;100: 103386. https://doi.org/10.1016/j.jmbbm.2019.103386 .

Grood ES, Suntay WJ. A joint coordinate system for the clinical description of three-dimensional motions: application to the knee. J Biomech Eng. 1983;105(2):136–44. https://doi.org/10.1115/1.3138397 .

Vasquez AO, Taylor WR, Postolka B, Schütz P, Maas A, Woiczinski M, Sauer A. A reproducible and robust representation of tibiofemoral kinematics of the healthy knee joint during stair descent using REFRAME—Part I: REFRAME foundations and validation. 2024.

Victor J, Labey L, Wong P, Innocenti B, Bellemans J. The influence of muscle load on tibiofemoral knee kinematics. J Orthop Res. 2010;28(4):419–28. https://doi.org/10.1002/jor.21019 .

Aumüller G, Aust G, Engele J, Maio G, Kirsch J. Anatomie. Stuttgart: Thieme; 2020.

Google Scholar  

Markolf KL, Jackson SR, Foster B, McAllister DR. ACL forces and knee kinematics produced by axial tibial compression during a passive flexion-extension cycle. J Orthop Res. 2014;32(1):89–95. https://doi.org/10.1002/jor.22476 .

Akbari Shandiz M, Boulos P, Saevarsson SK, Yoo S, Miller S, Anglin C. Changes in knee kinematics following total knee arthroplasty. Proc Inst Mech Eng Part H J Eng Med. 2016;230(4):265–78. https://doi.org/10.1177/0954411916632491

Matsumoto H, Suda Y, Otani T, Niki Y, Seedhom BB, Fujikawa K. Roles of the anterior cruciate ligament and the medial collateral ligament in preventing valgus instability. J Orthop Sci. 2001;6(1):28–32. https://doi.org/10.1007/s007760170021 .

Yaras RJ, O'Neill N, Yaish AM. Lateral collateral ligament knee injury. In: StatPearls [Internet] 2022 May 20. StatPearls Publishing.

Wilson D, Feikes J, Zavatsky A, O’Connor J. The components of passive knee movement are coupled to flexion angle. J Biomech. 2000;33(4):465–73. https://doi.org/10.1016/S0021-9290(99)00206-7 .

Moslemian A, Sidhu R, Roessler P, Wood R, Degen R, Getgood A, Willing R. Influence of the posterior cruciate ligament on kinematics of the knee during experimentally simulated clinical tests and activities of daily living. J Biomech. 2021;115:110133. https://doi.org/10.1016/j.jbiomech.2020.110133 .

Farshidfar SS, Cadman J, Deng D, Appleyard R, Dabirrahmani D. The effect of modelling parameters in the development and validation of knee joint models on ligament mechanics: a systematic review. PLoS ONE. 2022;17(1): e0262684. https://doi.org/10.1371/journal.pone.0262684 .

Liu-Barba D, Hull ML, Howell SM. Coupled motions under compressive load in intact and ACL-deficient knees: a cadaveric study. J Biomech Eng. 2007;129(6):818–24. https://doi.org/10.1115/1.2800762 .

Willinger L, Athwal KK, Williams A, Amis AA. An anterior cruciate ligament in vitro rupture model based on clinical imaging. Am J Sports Med. 2021;49(9):2387–95. https://doi.org/10.1177/03635465211017145 .

Kennedy NI, Wijdicks CA, Goldsmith MT, Michalski MP, Devitt BM, Årøen A, Engebretsen L, Laprade RF. Kinematic analysis of the posterior cruciate ligament, part 1: the individual and collective function of the anterolateral and posteromedial bundles. Am J Sports Med. 2013;41(12):2828–38. https://doi.org/10.1177/0363546513504287 .

Ali AA, Harris MD, Shalhoub S, Maletsky LP, Rullkoetter PJ, Shelburne KB. Combined measurement and modeling of specimen-specific knee mechanics for healthy and ACL-deficient conditions. J Biomech. 2017;57:117–24. https://doi.org/10.1016/j.jbiomech.2017.04.008 .

Shelburne KB, Pandy MG, Anderson FC, Torry MR. Pattern of anterior cruciate ligament force in normal walking. J Biomech. 2004;37(6):797–805. https://doi.org/10.1016/j.jbiomech.2003.10.010 .

Hosseini Nasab SH, List R, Oberhofer K, Fucentese SF, Snedeker JG, Taylor WR. Loading patterns of the posterior cruciate ligament in the healthy knee: a systematic review. PloS One. 2016;11(11):e0167106. https://doi.org/10.1371/journal.pone.0167106

Bortel E, Charbonnier B, Heuberger R. Development of a synthetic synovial fluid for tribological testing. Lubricants. 2015;3(4):664–86. https://doi.org/10.3390/lubricants3040664 .

Robi K, Jakob N, Matevz K, Matjaz V. The physiology of sports injuries and repair processes. Curr Issues Sports Exerc Med. 2013.

Charlton IW, Johnson GR. Application of spherical and cylindrical wrapping algorithms in a musculoskeletal model of the upper limb. J Biomech. 2001;34(9):1209–16. https://doi.org/10.1016/S0021-9290(01)00074-4 .

Zarifi O, Stavness I. Muscle wrapping on arbitrary meshes with the heat method. Comput Methods Biomech Biomed Engin. 2017;20(2):119–29. https://doi.org/10.1080/10255842.2016.1205043 .

Lloyd JE, Roewer-Despres F, Stavness I. Muscle path wrapping on arbitrary surfaces. IEEE Trans Biomed Eng. 2021;68(2):628–38. https://doi.org/10.1109/TBME.2020.3009922 .

Montgomery L. Biomechanical analysis of ligament modelling techniques and femoral component malrotation following TKA. The University of Western Ontario. https://ir.lib.uwo.ca/etd/8331

Uzuner S, Li LP. Alteration in ACL loading after total and partial medial meniscectomy. BMC Musculoskelet Disord. 2024;25(1):94. https://doi.org/10.1186/s12891-024-07201-x .

Uzuner S. Numerical analysis of a poroelastic cartilage model: Investigating the influence of changing material properties in osteoarthritis. Proc Inst Mech Eng Part E J Proc Mech Eng. 2024. https://doi.org/10.1177/09544089241248147 .

Yin L, Chen K, Guo L, Cheng L, Wang F, Yang L. Identifying the functional flexion-extension axis of the knee: an in-vivo kinematics study. PLoS ONE. 2015;10(6): e0128877. https://doi.org/10.1371/journal.pone.0128877 .

Victor J, Van Doninck D, Labey L, Van Glabbeek F, Parizel P, Bellemans J. A common reference frame for describing rotation of the distal femur: a ct-based kinematic study using cadavers. J Bone Jt Surg Br Vol. 2009;91(5):683–90. https://doi.org/10.1302/0301-620X.91B5.21827

Kahn TL, Soheili A, Schwarzkopf R. Outcomes of total knee arthroplasty in relation to preoperative patient-reported and radiographic measures: data from the osteoarthritis initiative. Geriatr Orthop Surg Rehabil. 2013;4(4):117–26. https://doi.org/10.1177/2151458514520634 .

Henke P, Ruehrmund L, Bader R, Kebbach M. Exploration of the Advanced VIVOTM Joint Simulator: An In-Depth Analysis of Opportunities and Limitations Demonstrated by the Artificial Knee Joint. Bioengineering (Basel). 2024;11(2):178. https://doi.org/10.3390/bioengineering11020178 .

Naghibi H, Mazzoli V, Gijsbertse K, Hannink G, Sprengers A, Janssen D, van den Boogaard T, Verdonschot N. A noninvasive MRI based approach to estimate the mechanical properties of human knee ligaments. J Mech Behav Biomed Mater. 2019;93:43–51. https://doi.org/10.1016/j.jmbbm.2019.01.022 .

Wismans J, Veldpaus F, Janssen J, Huson A, Struben P. A three-dimensional mathematical model of the knee-joint. J Biomech. 1980;13(8):677–85. https://doi.org/10.1016/0021-9290(80)90354-1 .

Blankevoort L, Kuiper JH, Huiskes R, Grootenboer HJ. Articular contact in a three-dimensional model of the knee. J Biomech. 1991;24(11):1019–31. https://doi.org/10.1016/0021-9290(91)90019-J .

Butler DL, Kay MD, Stouffer DC. Comparison of material properties in fascicle-bone units from human patellar tendon and knee ligaments. J Biomech. 1986;19(6):425–32. https://doi.org/10.1016/0021-9290(86)90019-9 .

Tillmann B. Atlas der Anatomie des Menschen: Mit Muskeltabellen. Berlin: Springer; 2017.

Laprade RF, Ly TV, Wentorf FA, Engebretsen L. The posterolateral attachments of the knee: a qualitative and quantitative morphologic analysis of the fibular collateral ligament, popliteus tendon, popliteofibular ligament, and lateral gastrocnemius tendon. Am J Sports Med. 2003;31(6):854–60. https://doi.org/10.1177/03635465030310062101 .

LaPrade RF, Morgan PM, Wentorf FA, Johansen S, Engebretsen L. The anatomy of the posterior aspect of the knee: an anatomic study. Jbjs. 2007;89(4):758–64. https://doi.org/10.2106/JBJS.F.00120

Athwal KK, Willinger L, Shinohara S, Ball S, Williams A, Amis AA. The bone attachments of the medial collateral and posterior oblique ligaments are defined anatomically and radiographically. Knee Surg Sports Traumatol Arthrosc. 2020;28(12):3709–19. https://doi.org/10.1007/s00167-020-06139-6 .

Hippmann G. An algorithm for compliant contact between complexly shaped bodies. Multibody SysDyn. 2004;12(4):345–62. https://doi.org/10.1007/s11044-004-2513-4 .

Article   MathSciNet   Google Scholar  

Download references

Acknowledgements

We thank Karl Lippert for providing the 3D scanner and helping scan the cartilage surface of the human specimen.

Open Access funding enabled and organized by Projekt DEAL. This work was supported by the German Research Foundation DFG (grant INST 2268/17-1 FUGG) and carried out as part of a project supported by Aesculap AG, Research and Development.

Author information

Paul Henke and Johanna Meier have shared first authorship and equally contributed.

Authors and Affiliations

Department of Orthopaedics, Rostock University Medical Center, Doberaner Straße 142, 18057, Rostock, Germany

Paul Henke, Johanna Meier, Leo Ruehrmund, Christoph Lutter, Rainer Bader & Maeruan Kebbach

Research and Development, Aesculap AG, Am Aesculap-Platz, 8532, Tuttlingen, Germany

Saskia A. Brendle, Sven Krueger & Thomas M. Grupp

Department of Orthopaedic and Trauma Surgery, Musculoskeletal University Center Munich (MUM), Campus Grosshadern, Ludwig Maximilians University, Munich, Germany

Saskia A. Brendle & Thomas M. Grupp

Chair of Technical Mechanics, University of Rostock, Justus-Von-Liebig-Weg 6, 18059, Rostock, Germany

Christoph Woernle

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization, PH, JM, LR, SB, SK, TMG, CL, CW, RB and MK; methodology, PH, JM, LR, SB, SK, RB and MK; computer simulation, JM; data curation, JM; writing—original draft preparation, PH, JM; writing—review and editing PH, JM, LR, SB, SK, TMG, CL, CW, RB and MK; supervision, TMG, RB and MK; visualization, PH, JM, LR; project administration, RB and MK; funding acquisition, RB and MK; All authors have read and agreed to the published version of the manuscript.

Corresponding authors

Correspondence to Paul Henke or Johanna Meier .

Ethics declarations

Ethics approval and consent to participate.

Ethical approval was granted by the Ethics Committee of the University Medical Center Rostock according to A 2023-0055.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1..

Predicted LCL and MCL forces of the multibody model.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Henke, P., Meier, J., Ruehrmund, L. et al. Modeling of the native knee with kinematic data derived from experiments using the VIVO™ joint simulator: a feasibility study. BioMed Eng OnLine 23 , 85 (2024). https://doi.org/10.1186/s12938-024-01279-z

Download citation

Received : 15 April 2024

Accepted : 06 August 2024

Published : 23 August 2024

DOI : https://doi.org/10.1186/s12938-024-01279-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Human knee joint
• Musculoskeletal modeling
• Joint simulator
• Cadaveric testing
• Ligament resection

BioMedical Engineering OnLine

ISSN: 1475-925X

• Submission enquiries: [email protected]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Published: 19 August 2024

Chronic adaptive deep brain stimulation versus conventional stimulation in Parkinson’s disease: a blinded randomized feasibility trial

• Carina R. Oehrn   ORCID: orcid.org/0000-0001-8451-7960 1   na1 ,
• Stephanie Cernera 1   na1 ,
• Lauren H. Hammer 2   na1 ,
• Maria Shcherbakova 1 ,
• Jiaang Yao   ORCID: orcid.org/0000-0001-7062-2508 1 , 3 ,
• Amelia Hahn 1 ,
• Sarah Wang 2 , 4 ,
• Jill L. Ostrem 2 , 4 ,
• Simon Little   ORCID: orcid.org/0000-0001-6249-6230 2 , 3 , 4   na2 &
• Philip A. Starr   ORCID: orcid.org/0000-0003-2733-4003 1 , 3 , 4   na2  

Nature Medicine ( 2024 ) Cite this article

2157 Accesses

707 Altmetric

Metrics details

• Biomedical engineering
• Neurophysiology
• Parkinson's disease

Deep brain stimulation (DBS) is a widely used therapy for Parkinson’s disease (PD) but lacks dynamic responsiveness to changing clinical and neural states. Feedback control might improve therapeutic effectiveness, but the optimal control strategy and additional benefits of ‘adaptive’ neurostimulation are unclear. Here we present the results of a blinded randomized cross-over pilot trial aimed at determining the neural correlates of specific motor signs in individuals with PD and the feasibility of using these signals to drive adaptive DBS. Four male patients with PD were recruited from a population undergoing DBS implantation for motor fluctuations, with each patient receiving adaptive DBS and continuous DBS. We identified stimulation-entrained gamma oscillations in the subthalamic nucleus or motor cortex as optimal markers of high versus low dopaminergic states and their associated residual motor signs in all four patients. We then demonstrated improved motor symptoms and quality of life with adaptive compared to clinically optimized standard stimulation. The results of this pilot trial highlight the promise of personalized adaptive neurostimulation in PD based on data-driven selection of neural signals. Furthermore, these findings provide the foundation for further larger clinical trials to evaluate the efficacy of personalized adaptive neurostimulation in PD and other neurological disorders. ClinicalTrials.gov registration: NCT03582891 .

This is a preview of subscription content, access via your institution

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 12 print issues and online access

195,33 € per year

only 16,28 € per issue

Buy this article

• Purchase on SpringerLink
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Emerging technologies for improved deep brain stimulation

Eight-hours conventional versus adaptive deep brain stimulation of the subthalamic nucleus in Parkinson’s disease

Modulation of subthalamic beta oscillations by movement, dopamine, and deep brain stimulation in Parkinson’s disease

Data availability.

De-identified individual participant data, including neural, wearable and digital diary data, are shared on the Data Archive for the BRAIN Initiative website ( https://dabi.loni.usc.edu/ ; https://doi.org/10.18120/cq9c-d057 ). The study protocol is provided in the Supplementary Information . The Food and Drug Administration investigational device exemption is available on the Open Mind website ( https://osf.io/cmndq/ ). Data will be available permanently with no restrictions, for purposes of replicating the findings or conducting meta-analyses.

Code availability

Code written in C# and MATLAB, which operates the investigational device and extracts raw neural data, is available on the Open Mind GitHub platform ( https://openmind-consortium.github.io ). The code for biomarker identification implemented in MATLAB is available in the repository Code Ocean, without restrictions 59 , except for code related to linear discriminant analysis (Fig. 4c–e ), which will be made available after publication of a subsequent manuscript (currently in preparation) that uses this code.

Lozano, A. M. et al. Deep brain stimulation: current challenges and future directions. Nat. Rev. Neurol. 15 , 148–160 (2019).

Article   PubMed   PubMed Central   Google Scholar  

Neumann, W. -J., Gilron, R., Little, S. & Tinkhauser, G. Adaptive deep brain stimulation: from experimental evidence toward practical implementation. Mov. Disord . https://doi.org/10.1002/mds.29415 (2023).

Marceglia, S. et al. Deep brain stimulation: is it time to change gears by closing the loop? J. Neural Eng. 18 , 061001 (2021).

Article   Google Scholar  

Stanslaski, S. et al. Design and validation of a fully implantable, chronic, closed-loop neuromodulation device with concurrent sensing and stimulation. IEEE Trans. Neural Syst. Rehabil. Eng. 20 , 410–421 (2012).

Article   PubMed   Google Scholar  

Stanslaski, S. et al. A chronically implantable neural coprocessor for investigating the treatment of neurological disorders. IEEE Trans. Biomed. Circuits Syst. 12 , 1230–1245 (2018).

Thenaisie, Y. et al. Towards adaptive deep brain stimulation: clinical and technical notes on a novel commercial device for chronic brain sensing. J. Neural Eng. 18 , 042002 (2021).

Starr, P. A. Totally implantable bidirectional neural prostheses: a flexible platform for innovation in neuromodulation. Front. Neurosci. 12 , 619 (2018).

Nakajima, A. et al. Case report: chronic adaptive deep brain stimulation personalizing therapy based on Parkinsonian state. Front. Hum. Neurosci. 15 , 702961 (2021).

Gilron, R. et al. Long-term wireless streaming of neural recordings for circuit discovery and adaptive stimulation in individuals with Parkinson’s disease. Nat. Biotechnol. 39 , 1078–1085 (2021).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Little, S. & Brown, P. Debugging adaptive deep brain stimulation for Parkinson’s disease. Mov. Disord. 35 , 555–561 (2020).

Wilkins, K. B., Melbourne, J. A., Akella, P. & Bronte-Stewart, H. M. Unraveling the complexities of programming neural adaptive deep brain stimulation in Parkinson’s disease. Front. Hum. Neurosci. 17 , 1310393 (2023).

Ansó, J. et al. Concurrent stimulation and sensing in bi-directional brain interfaces: a multi-site translational experience. J. Neural Eng. 19 , 026025 (2022).

Ascherio, A. & Schwarzschild, M. A. The epidemiology of Parkinson’s disease: risk factors and prevention. Lancet Neurol. 15 , 1257–1272 (2016).

Vitek, J. L. et al. Subthalamic nucleus deep brain stimulation with a multiple independent constant current-controlled device in Parkinson’s disease (INTREPID): a multicentre, double-blind, randomised, sham-controlled study. Lancet Neurol. 19 , 491–501 (2020).

Article   CAS   PubMed   Google Scholar  

Okun, M. S. et al. Subthalamic deep brain stimulation with a constant-current device in Parkinson’s disease: an open-label randomised controlled trial. Lancet Neurol. 11 , 140–149 (2012).

Weaver, F. M. et al. Bilateral deep brain stimulation vs best medical therapy for patients with advanced Parkinson disease: a randomized controlled trial. JAMA 301 , 63–73 (2009).

Deuschl, G. et al. A randomized trial of deep-brain stimulation for Parkinson’s disease. N. Engl. J. Med. 355 , 896–908 (2006).

Follett, K. A. et al. Pallidal versus subthalamic deep-brain stimulation for Parkinson’s disease. N. Engl. J. Med. 362 , 2077–2091 (2010).

Odekerken, V. J. et al. Subthalamic nucleus versus globus pallidus bilateral deep brain stimulation for advanced Parkinson’s disease (NSTAPS study): a randomised controlled trial. Lancet Neurol. 12 , 37–44 (2013).

Bronte-Stewart, H. et al. Adaptive DBS Algorithm for Personalized Therapy in Parkinson’s Disease: ADAPT-PD clinical trial methodology and early data (P1-11.002). Neurology https://doi.org/10.1212/WNL.0000000000203099 (2023).

Marceglia, S. et al. Double-blind cross-over pilot trial protocol to evaluate the safety and preliminary efficacy of long-term adaptive deep brain stimulation in patients with Parkinson’s disease. BMJ Open 12 , e049955 (2022).

Kühn, A. A., Kupsch, A., Schneider, G.-H. & Brown, P. Reduction in subthalamic 8-35 Hz oscillatory activity correlates with clinical improvement in Parkinson’s disease. Eur. J. Neurosci. 23 , 1956–1960 (2006).

Kühn, A. A. et al. High-frequency stimulation of the subthalamic nucleus suppresses oscillatory β activity in patients with Parkinson’s disease in parallel with improvement in motor performance. J. Neurosci. 28 , 6165–6173 (2008).

Little, S. et al. Adaptive deep brain stimulation in advanced Parkinson disease. Ann. Neurol. 74 , 449–457 (2013).

Velisar, A. et al. Dual threshold neural closed loop deep brain stimulation in Parkinson disease patients. Brain Stimul. 12 , 868–876 (2019).

Bocci, T. et al. Eight-hours conventional versus adaptive deep brain stimulation of the subthalamic nucleus in Parkinson’s disease. NPJ Park. Dis. 7 , 88 (2021).

Article   CAS   Google Scholar  

Tinkhauser, G. et al. The modulatory effect of adaptive deep brain stimulation on beta bursts in Parkinson’s disease. Brain J. Neurol. 140 , 1053–1067 (2017).

Bronstein, J. M. et al. Deep brain stimulation for Parkinson disease: an expert consensus and review of key issues. Arch. Neurol. 68 , 165 (2011).

Swann, N. C. et al. Gamma oscillations in the hyperkinetic state detected with chronic human brain recordings in Parkinson’s disease. J. Neurosci. 36 , 6445–6458 (2016).

Swann, N. C. et al. Adaptive deep brain stimulation for Parkinson’s disease using motor cortex sensing. J. Neural Eng. 15 , 046006 (2018).

Bove, F., Genovese, D. & Moro, E. Developments in the mechanistic understanding and clinical application of deep brain stimulation for Parkinson’s disease. Expert Rev. Neurother. 22 , 789–803 (2022).

Wiest, C. et al. Finely-tuned gamma oscillations: spectral characteristics and links to dyskinesia. Exp. Neurol. 351 , 113999 (2022).

Sermon, J. J. et al. Sub-harmonic entrainment of cortical gamma oscillations to deep brain stimulation in Parkinson’s disease: model based predictions and validation in three human subjects. Brain Stimul. 16 , 1412–1424 (2023).

Olaru, M. et al. Motor network gamma oscillations in chronic home recordings predict dyskinesia in Parkinson’s disease. Brain J. Neurol . https://doi.org/10.1093/brain/awae004 (2024).

Herdman, M. et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual. Life Res. 20 , 1727–1736 (2011).

Horne, M. K., McGregor, S. & Bergquist, F. An objective fluctuation score for Parkinson’s disease. PLoS ONE 10 , e0124522 (2015).

Nutt, J. G., Woodward, W. R., Hammerstad, J. P., Carter, J. H. & Anderson, J. L. The “on–off” phenomenon in Parkinson’s disease: relation to levodopa absorption and transport. N. Engl. J. Med. 310 , 483–488 (1984).

van Rheede, J. J. et al. Diurnal modulation of subthalamic beta oscillatory power in Parkinson’s disease patients during deep brain stimulation. NPJ Parkinsons Dis. 8 , 88 (2022).

Tinkhauser, G. & Moraud, E. M. Controlling clinical states governed by different temporal dynamics with closed-loop deep brain stimulation: a principled framework. Front. Neurosci. 15 , 734186 (2021).

Alagapan, S. et al. Cingulate dynamics track depression recovery with deep brain stimulation. Nature 622 , 130–138 (2023).

Heck, C. N. et al. Two-year seizure reduction in adults with medically intractable partial onset epilepsy treated with responsive neurostimulation: final results of the RNS System Pivotal trial. Epilepsia 55 , 432–441 (2014).

Scangos, K. W. et al. Closed-loop neuromodulation in an individual with treatment-resistant depression. Nat. Med. 27 , 1696–1700 (2021).

Vizcarra, J. A. et al. Subthalamic deep brain stimulation and levodopa in Parkinson’s disease: a meta-analysis of combined effects. J. Neurol. 266 , 289–297 (2019).

Brown, P. et al. Dopamine dependency of oscillations between subthalamic nucleus and pallidum in Parkinson’s disease. J. Neurosci. 21 , 1033–1038 (2001).

Halje, P. et al. Levodopa-induced dyskinesia is strongly associated with resonant cortical oscillations. J. Neurosci. 32 , 16541–16551 (2012).

Wiest, C. et al. Subthalamic deep brain stimulation induces finely-tuned gamma oscillations in the absence of levodopa. Neurobiol. Dis. 152 , 105287 (2021).

Arlotti, M. et al. Eight-hours adaptive deep brain stimulation in patients with Parkinson disease. Neurology 90 , e971–e976 (2018).

Foffani, G. & Alegre, M. Brain oscillations and Parkinson disease. Handb. Clin. Neurol. 184 , 259–271 (2022).

Feldmann, L. K. et al. Toward therapeutic electrophysiology: beta-band suppression as a biomarker in chronic local field potential recordings. NPJ Parkinsons Dis. 8 , 44 (2022).

Chen, Y. et al. Neuromodulation effects of deep brain stimulation on beta rhythm: a longitudinal local field potential study. Brain Stimul. 13 , 1784–1792 (2020).

Olson, J. D. et al. Comparison of subdural and subgaleal recordings of cortical high-gamma activity in humans. Clin. Neurophysiol. 127 , 277–284 (2016).

Piña-Fuentes, D. et al. Acute effects of adaptive deep brain stimulation in Parkinson’s disease. Brain Stimul. 13 , 1507–1516 (2020).

Busch, J. L. et al. Single threshold adaptive deep brain stimulation in Parkinson’s disease depends on parameter selection, movement state and controllability of subthalamic beta activity. Brain Stimul. 17 , 125–133 (2024).

Merk, T. et al. Machine learning based brain signal decoding for intelligent adaptive deep brain stimulation. Exp. Neurol. 351 , 113993 (2022).

Davis, T. S. et al. LeGUI: a fast and accurate graphical user interface for automated detection and anatomical localization of intracranial electrodes. Front. Neurosci. 15 , 769872 (2021).

Horn, A. et al. Lead-DBS v2: towards a comprehensive pipeline for deep brain stimulation imaging. NeuroImage 184 , 293–316 (2019).

Oehrn, C. R. et al. Chronic adaptive deep brain stimulation is superior to conventional stimulation in Parkinson's disease: a blinded randomized feasibility trial [Source Data]. Data Archive for the Brain Initiative https://doi.org/10.18120/cq9c-d057 (2024).

Oostenveld, R., Fries, P., Maris, E. & Schoffelen, J. -M. FieldTrip: open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput. Intell. Neurosci. 2011 , 156869 (2011).

Oehrn, C. R. et al. Chronic adaptive deep brain stimulation is superior to conventional stimulation in Parkinson's disease: a blinded randomized feasibility trial. Code Ocean . https://doi.org/10.24433/CO.5656158.v1 (2024).

Oehrn, C. R. et al. Direct electrophysiological evidence for prefrontal control of hippocampal processing during voluntary forgetting. Curr. Biol. 28 , 3016–3022 (2018).

Maris, E. & Oostenveld, R. Nonparametric statistical testing of EEG- and MEG-data. J. Neurosci. Methods 164 , 177–190 (2007).

Gilron, R. et al. Sleep-aware adaptive deep brain stimulation control: chronic use at home with dual independent linear discriminate detectors. Front. Neurosci. 15 , 732499 (2021).

Cernera, S. et al. Wearable sensor-driven responsive deep brain stimulation for essential tremor. Brain Stimul. 14 , 1434–1443 (2021).

Hammer, L. H., Kochanski, R. B., Starr, P. A. & Little, S. Artifact characterization and a multipurpose template-based offline removal solution for a sensing-enabled deep brain stimulation device. Stereotact. Funct. Neurosurg. 100 , 168–183 (2022).

Neumann, W. -J. et al. The sensitivity of ECG contamination to surgical implantation site in brain computer interfaces. Brain Stimul. 14 , 1301–1306 (2021).

Goetz, C. G. et al. Movement Disorder Society-sponsored revision of the Unified Parkinson’s Disease Rating Scale (MDS-UPDRS): process, format, and clinimetric testing plan. Mov. Disord. 22 , 41–47 (2007).

McAuley, M. D. Incorrect calculation of total electrical energy delivered by a deep brain stimulator. Brain Stimul. 13 , 1414–1415 (2020).

Download references

Acknowledgements

The study was supported by National Institute of Neurological Disorders and Stroke (NINDS) UH3NS100544 (to P.A.S.), the Parkinson Fellowship of the Thiemann Foundation (to C.R.O.), NINDS F32NS129627 (to S.C.), NINDS R25NS070680 (to L.H.H.) and TUYF Charitable Trust Fund (to J.Y.). Research reported in this publication was also partly supported by R01 NS090913 (to P.A.S.), NINDS K23NS120037 (to S.L.) and R01 NS131405 (to S.L.). Investigational devices were provided at no charge by the manufacturer, but the manufacturer had no role in the conduct, analysis or interpretation of the study. The Open Mind consortium for technology dissemination, funded by NINDS U24 NS113637 (to P.A.S.), provided technical resources for the use of the Summit RC+S neural interface. We thank T. Wozny for lead localization, W. Chiong for neuroethical input, C. Smyth, R. Gilron, R. Wilt and C. de Hemptinne for technical contributions and K. Probst for medical art (Fig. 1a ). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Author information

These authors contributed equally: Carina R. Oehrn, Stephanie Cernera, Lauren H. Hammer.

These authors jointly supervised this work: Simon Little, Philip A Starr.

Authors and Affiliations

Department of Neurological Surgery, University of California, San Francisco, San Francisco, CA, USA

Carina R. Oehrn, Stephanie Cernera, Maria Shcherbakova, Jiaang Yao, Amelia Hahn & Philip A. Starr

Department of Neurology, University of California, San Francisco, San Francisco, CA, USA

Lauren H. Hammer, Sarah Wang, Jill L. Ostrem & Simon Little

Graduate Program in Bioengineering, University of California, Berkeley and University of California, San Francisco, San Francisco, CA, USA

Jiaang Yao, Simon Little & Philip A. Starr

Weill Institute for Neurosciences, University of California, San Francisco, San Francisco, CA, USA

Sarah Wang, Jill L. Ostrem, Simon Little & Philip A. Starr

You can also search for this author in PubMed   Google Scholar

Contributions

P.A.S., S.L., J.L.O., C.R.O., S.C. and L.H.H. designed the study and analysis pipeline. C.R.O., S.C., L.H.H., M.S. and J.Y. collected and analyzed the data. A.H. facilitated patient communication and coordination throughout the study. S.W. oversaw study administration, including institutional review board approval and regulatory compliance. C.R.O., S.C., L.H.H., S.L. and P.A.S. drafted the manuscript, and all authors reviewed, commented on and approved the final version.

Corresponding author

Correspondence to Carina R. Oehrn .

Ethics declarations

Competing interests.

S.L. consults for Iota Biosciences. J.L.O. reports support from Medtronic and Boston Scientific for research and education and consults for AbbVie and Rune Labs. P.A.S. receives support from Medtronic and Boston Scientific for fellowship education. C.R.O., S.C., L.H.H., M.S., J.Y., A.H. and S.W. declare no competing interests.

Peer review

Peer review information.

Nature Medicine thanks Jaimie Henderson, Andrea Kühn and Theoden Netoff for their contribution to the peer review of this work. Primary Handling Editor: Jerome Staal, in collaboration with the Nature Medicine team.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended data fig. 1 localization of leads over sensorimotor cortex and within subthalamic nucleus in native space..

a–d , Example localization of cortical and subcortical leads in patient 2, generated by fusing postoperative CT with preoperative MRI scans. Contacts appear as white CT artifacts due to metal content and are labeled with red arrows. a , Cortical leads on axial T1-weighted MRI through the vertex. b , STN leads on axial T2-weighted MRI through the region of the dorsal STN, 3 mm inferior to the intercommissural plane. c,d , Cortical leads on oblique sagittal T1-weighted MRI passing through the long axis of the lead array in left (c) and right (d) hemispheres, respectively. e–h , Location of cortical leads for each patient overlayed on 3D reconstruction of cortex rendered using the Locate Electrodes Graphical User Interface (LeGUI). Electrodes used in the anterior and posterior cortical montages are shown in cyan and yellow, respectively. For patient 1 (e) , 2 (f) and 4 (h) , anterior and posterior montages covered the pre- and postcentral gyrus, respectively. For patient 3, right side (g) , the anterior montage included one electrode on the middle frontal and one on the precentral gyrus. The posterior montage comprised one pre- and one postcentral electrode. In all figures, red arrows indicate the location of the central sulcus.

Extended Data Fig. 2 Initial and finalized adaptive stimulation parameters and example adaptive control policies.

a , Suggested initial parameters for algorithms developed for time scales of minutes to hours, as identified during steps 5 and 6 of the pipeline. An update rate of 10 s typically provided a signal to noise ratio that allowed for adequate discrimination between the presence and absence of the most bothersome symptom, and this could often be improved with a further increase in update rate. The ramp rate chosen for each patient depended on the results of step 5 (we chose an example of 1 mA/s). b , Detailed final adaptive stimulation parameters including control signals, thresholds, FFT interval, update rates, blanking periods, onset and termination duration, and ramp rates used for each patient and hemisphere. c–e , Examples of potential control policies that can be used for an adaptive algorithm, using artificial data. The upper subpanels of each subfigure illustrate an on-state biomarker (blue), as used in our study, along with thresholds (red). Lower subpanels demonstrate the adjustment of stimulation amplitude based on the relationship of the neural signal to the thresholds. c , A single threshold control policy with two stimulation amplitudes. When the biomarker is above the threshold, stimulation amplitude decreases and once below threshold, stimulation amplitude increases. d , A dual threshold control policy with three stimulation amplitudes (not used in this study), which may be applied to address three symptom states. When the neural signal is below both thresholds, the stimulation amplitude is high (for example, 4 mA). When the biomarker is between the two thresholds, stimulation adjusts to a middle amplitude (for example, 3 mA). When the biomarker exceeds the second threshold, stimulation decreases to the low amplitude (for example, 2 mA). e , A control policy utilizing a middle state as noise buffer. Stimulation is high when the control signal is below the bottom threshold and stimulation is low when the control signal is above the top threshold. When the control signal is between the two thresholds, it remains at the level of the stimulation amplitude prior to crossing the threshold (that is, no changes are made).

Extended Data Fig. 3 Neural biomarkers of medication effects identified in-clinic.

a,b , All tables show the results from our within-patient non-parametric cluster-based permutation analyses using in-clinic recordings during two medication states (off vs. on) and stimulation conditions (low vs. high stimulation amplitude). P -values were Bonferroni-corrected for multiple comparisons. Note that p < 10 −3 indicates that the cluster was found in all 1000 permutations. This means the probability of observing this effect by chance is less than 1 in 1000. a , Statistics for the largest main effect of medication, stimulation, and their interaction for each patient and hemisphere when searching the whole frequency space (2–100 Hz) across brain regions. Frequencies represent the center frequency of 1-Hz wide power spectral density bins. For all four patients (five out of six hemispheres), we found that gamma power (specifically, stimulation-entrained gamma in four hemispheres) in the STN or cortex was the best predictor of medication state (in pat-3L, there was no significant effect of medication in any frequency band in clinic, but at home symptom monitoring identified cortical stimulation-entrained gamma power as neural biomarker; Extended Data Fig. 4 ). Positive Cohen’s d values for the medication effect highlight that the neural biomarker was higher during on-medication states. Positive Cohen’s d values for the stimulation effect indicate that the neural biomarker was higher during on-stimulation states (independent of medication), which could result in undesirable self-triggering of the algorithm (threshold crossing of the neural biomarker linked to stimulation change itself, rather than true fluctuations of medication states and symptoms). Therefore, for patient 1, we excluded 63 and 67 Hz from the subsequently used control signal (positive Cohen’s d main effect of stimulation). For patients 2, 3 and 4, we did not find stimulation effects that positively modulated biomarkers and therefore were unrestricted in biomarker selection. b , When constraining the anatomic location and frequency space to STN beta oscillations (13–30 Hz), STN spectral beta power was only predictive for medication state in two hemispheres (pat-2R and pat-4) and smaller in effect size than cortical/STN stimulation-entrained gamma oscillations for all patients.

Extended Data Fig. 4 Neural biomarkers of symptoms identified at-home.

We identified predictors of the most bothersome symptom (pat-1: bradykinesia, pat-2: lower limb dystonia), or the opposite symptom that limits the therapeutic window (pat-3 and pat-4: dyskinesia). a , Heatmaps of t -values derived from stepwise linear regressions using 1 Hz power bands between 2–100 Hz in the STN (left), anterior cortical montage (middle) and posterior cortical montage (right) to predict symptoms continuously measured with upper extremity wearable monitors for patients 1, 3 and 4 (patient 2’s bothersome symptom did not involve the upper extremity). b–d , Results from the linear regression (left) and linear discriminant analysis (LDA; right). P-values were Bonferroni-corrected for multiple comparisons (289 predictors). b , Both methods provide converging evidence that stimulation-entrained gamma power centered at half the stimulation frequency (65 Hz) in the STN and cortex optimally distinguishes hypo- and hyperkinetic symptoms. c , When constraining the anatomic location and frequency space to STN beta oscillations (13–30 Hz), frequency bands identified as most predictive were less discriminative than cortical/STN stimulation-entrained gamma oscillations (LDA: AUC < 0.7). Regression models resulted in smaller magnitude coefficients, with only one hemisphere demonstrating a significant negative association with hyperkinetic symptoms (pat-3L). d , STN beta frequency bands were also poorly predictive of wearable bradykinesia scores (AUC < 0.6), again with only one hemisphere demonstrating a significant effect in the regression model (corresponding to a positive relationship with hypokinetic symptoms; pat-3L). e , Comparison of LDA results for STN and cortical gamma activity in predicting bothersome symptoms. Neural signals selected for adaptive stimulation are shaded in grey. In three out of six hemispheres (pat-2L, pat-2R, pat-4), stimulation-entrained gamma activity in the STN distinguished between hypo- and hyperkinetic symptoms. For pat-2, STN stimulation-entrained spectral gamma power was the optimal biomarker used for aDBS in both hemispheres. In pat-4, stimulation-entrained gamma activity in the STN was a strong predictor of residual motor signs but slightly underperformed compared to cortical signals. f , Visual illustration of AUC values comparing STN and cortical gamma activity in predicting bothersome symptoms. For pat-4, the predictive value of stimulation-entrained spectral gamma power was only slightly reduced compared to cortical signals.

Extended Data Fig. 5 Beta oscillations in the STN.

a , Power spectral density in the STN based on in-clinic recordings off medication and off stimulation for all six hemispheres. All but one hemisphere (pat-1) exhibited a peak in the beta frequency band (illustrated in yellow). b , Example of the suppressive effect of DBS on STN beta oscillations precluding use of beta band activity as a biomarker of medication state during active stimulation (pat-2L, all data collected during the same in-clinic recording session). Off stimulation, the spectral peak in the beta frequency range was suppressed by medication (13–21 Hz, Cohens’ d  = −1.09, p < 10 −3 ). However, this medication effect diminished during active stimulation, even at low stimulation amplitudes (1.8 mA, largest effect in the beta band: 15–18 Hz, Cohens’ d  = 0.31, p  = 0.026). Data are corrected for stimulation-induced broadband shifts.

Extended Data Fig. 6 Effects of aDBS and cDBS on most bothersome symptom severity, additional motor symptoms, and sleep quality.

a–j , Bar plots illustrating the mean (±s.e.m.) self-reported symptoms, aside from the most bothersome symptoms, across testing days. Each dot represents the rating for one testing day (blue: cDBS, red: aDBS). These ratings constituted secondary outcome measures to ensure that we are not aggravating other motor and non-motor symptoms. a,b , Patient self-reported motor symptom severity from daily questionnaires (1 = least severe, 10 = most severe). Note that patients rated symptom severity (shown here) independently of symptom duration ; bar graphs for the latter are in Fig. 5a,b . Patient 3 did not record ratings within the instructed range of 1–10 and their data are therefore not reported. a , In addition to a decrease in the amount of daily hours with the most bothersome symptom (symptom duration , shown in Fig. 5a ), patients 1, 2, and 4 also experienced a significant improvement of symptom severity (pat-1: p < 10 −5 , pat-2: p  = 0.018, pat = 4: p  = 0.003). b , No subject reported worsened severity of their opposite symptom (pat-1: p  = 0.18, pat-2: p  = 1, pat-4: p  = 0.19). c–h , Comprehensive list of the self-reported duration of motor symptoms from daily questionnaires. These bar graphs illustrate only symptoms that were not identified by the patient as the most bothersome or as the opposite symptom. For each patient’s most bothersome symptom, results are displayed in Fig. 5a and panel a of this figure; and are labeled in c–h as not applicable (n/a). None of these “other” motor symptoms were worsened by aDBS, and patient 2 demonstrated significant improvement in the percentage of waking hours with dyskinesia ( d , p = 0.044) and gait disturbance ( h , p < 10 −4 ). i,j , Self-reported sleep quality (1 = poorest sleep, 10 = best sleep) and duration from daily questionnaires. aDBS provided no change in patients’ sleep characteristics. The number of testing days for each patient and condition used for statistical tests are summarized in Fig. 6a . Asterisks illustrate results from two-sided Wilcoxon rank sum tests. P-values for all within-subject control analyses were adjusted for multiple comparisons using the false discovery rate procedure and are indicated as: *p < 0.05, **p < 0.01, ***p < 0.001.

Extended Data Fig. 7 aDBS algorithm dynamics during nighttime.

a , Percent time spent at each stimulation amplitude during the night. Each dot represents the mean values of one night of aDBS testing across high stimulation states (orange) and low stimulation states (blue) in one hemisphere. Graphs are standard box plots (center: median; box limits: upper and lower quartiles; whiskers: minima = 25th percentile-1.5 times the interquartile range, maxima = 75th percentile+1.5 times the interquartile range). Each patient spent most of the night in the high stimulation state. b , Mean (±s.e.m.) total electrical energy delivered (TEED) during aDBS and cDBS overnight, showing increased TEED during aDBS, similar to daytime analyses (stimulation main effect: β  = 27.7, p  < 10 −25 , time main effect: β  = 0.05, p  = 0.377). Individually, TEED was increased in all hemispheres during aDBS (two-sided, one-sample Wilcoxon signed rank test, pat-1: p < 10 −6 , pat-2R: p < 10 −5 , pat-2L: p < 10 −5 , pat-3R: p < 10 −6 , pat-3L: p < 10 −6 , pat-4: p < 10 −4 ). The number of testing nights for each patient and condition used for both illustrations are stated in Fig. 6a and are equivalent to the testing days. Asterisks illustrate results from two-sided one-sample Wilcoxon signed rank tests. P-values for TEED evaluations were adjusted for multiple comparisons using the false discovery rate procedure and are indicated as: *p < 0.05, **p < 0.01, ***p < 0.001.

Extended Data Fig. 8 Flowchart of biomarker identification analyses.

We identified neural biomarkers using standardized in-clinic and at-home recordings in patients’ naturalistic environments. Non-parametric cluster-based permutation analysis identified candidate spectral biomarkers from in-clinic data by assessing main effects of medication state, stimulation amplitude, and the interaction. Next, the predictability of neural biomarkers as robust aDBS control signals of symptom state was tested using at-home recordings. For patients where the most bothersome symptom was monitored by a wearable device (for example, upper extremity bradykinesia or dyskinesia), linear stepwise regression was used to take advantage of the continuous nature of the symptom measurements. The most predictive frequency bands and recording sites were selected based on t -values. If the patient’s most bothersome symptom could not be captured by wearable monitors, the patient’s motor diaries and streaming app entries instead labeled the presence of symptoms. A linear discriminant analysis (LDA) based method identified the most predictive frequency band and recording site from these discretely labeled neural signal data, as measured by the area under the receiver operating curve (AUC). We also applied the LDA-based approach to symptoms measured by wearable monitors by mapping the continuous wearable scores to discrete symptom labels using a patient-specific dichotomization. This dichotomization allowed for subsequent offline assessment of the prediction accuracy based on multiple neural biomarkers combined as shown in Fig. 4e (note for online aDBS only single power band classifiers were implemented, as multiple power band classifiers were not found to be superior).

Supplementary information

Supplementary information.

Supplementary Methods, Tables 1 and 2 and References.

Reporting Summary

Rights and permissions.

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cite this article.

Oehrn, C.R., Cernera, S., Hammer, L.H. et al. Chronic adaptive deep brain stimulation versus conventional stimulation in Parkinson’s disease: a blinded randomized feasibility trial. Nat Med (2024). https://doi.org/10.1038/s41591-024-03196-z

Download citation

Received : 04 January 2024

Accepted : 15 July 2024

Published : 19 August 2024

DOI : https://doi.org/10.1038/s41591-024-03196-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

• Open access
• Published: 06 October 2019

Guidelines for reporting non-randomised pilot and feasibility studies

• Gillian A. Lancaster   ORCID: orcid.org/0000-0002-6957-6177 1 &
• Lehana Thabane 1  

Pilot and Feasibility Studies volume  5 , Article number:  114 ( 2019 ) Cite this article

49k Accesses

187 Citations

35 Altmetric

Metrics details

As the number of submissions to Pilot and Feasibility Studies increases, there is a need for good quality reporting guidelines to help researchers tailor their reports in a way that is consistent and helpful to other readers. The publication in 2016 of the CONSORT extension to pilot and feasibility trials filled a much-needed gap, but there still remains some uncertainty as to how to report pilot and feasibility studies that are not randomised. This editorial aims to provide some general guidance on how to report the most common types of non-randomised pilot and feasibility studies that are submitted to the journal. We recommend using the CONSORT extension to pilot and feasibility trials as the main reference document—it includes detailed elaboration and explanation of each item, and in most cases, simple adaptation, or non-use of items that are not applicable, will suffice. Several checklists found on the Equator website may provide helpful supplementary guidance, when used alongside the CONSORT extension, and we give some examples.

Peer Review reports

Introduction

Since the inception of the BMC journal Pilot and Feasibility Studies in 2015 [ 1 ], the number of published studies has risen sharply each year, totalling 379 by the end of 2018. In 2016, the Consolidated Standards of Reporting Trials (CONSORT) extension to randomised pilot and feasibility trials and two related methodology papers were published by the Pilot and Feasibility Studies (PAFS) Working Group (see the “Acknowledgements” section) to aid researchers in the planning and reporting of these types of studies [ 2 , 3 , 4 , 5 ]. An associated PAFS website was created as a point of reference for information about pilot and feasibility studies and associated events ( https://pilotandfeasibilitystudies.qmul.ac.uk/ ). Recently, we also published an editorial guide to the reporting of protocols of randomised pilot and feasibility trials [ 6 ], recommending the use of the CONSORT extension guideline [ 2 , 3 ] alongside the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) checklist [ 7 ].

These publications have focused on guidelines for reporting randomised pilot and feasibility trials, but as the number of manuscript submissions to the journal continues to increase (by 200% from 2015 to 2018), there has arisen a need for some guidance on the reporting of non-randomised pilot and feasibility studies. Many non-randomised studies are undertaken before a randomised pilot or feasibility trial takes place and may comprise a wide spectrum of study designs. In this editorial, we discuss the most common types of non-randomised studies seen in the journal and give guidance on how they should be reported. In most cases, we recommend referring to the CONSORT extension to pilot and feasibility trials [ 2 , 3 ] as many of the items (excluding items that are specific to the randomisation nature of the study) will be relevant for reporting other types of pilot and feasibility studies, and the guideline provides helpful examples and commentary for each of the 26 items. Many other guidelines exist on the Equator (Enhancing the QUAlity and Transparency Of health Research) website ( http://www.equator-network.org/ ), and with minor amendments, some can be adapted for reporting certain types of non-randomised studies.

This editorial is based on our experience of submissions to the journal over the past 4 years, and while it does not provide comprehensive coverage of all types of non-randomised pilot and feasibility studies, it will hopefully provide some useful suggestions and signposts to relevant guidance and examples as an aid to reporting these studies. A point to note here is that our work to date has shown that there is a lack of consensus over the usage of the words ‘pilot’ and ‘feasibility’ [ 4 ], and so as a consequence, both terms are currently being used interchangeably in the journal.

Guidance for reporting non-randomised pilot and feasibility studies for submission to the journal

In the journal Pilot and Feasibility Studies , aided by the CONSORT extension to pilot and feasibility trials, authors are encouraged to report the purpose of a feasibility or pilot study in the context of the planned future study. Many types of non-randomised feasibility studies are at an earlier stage of preparation to that of a randomised pilot or feasibility trial. The proposed methodology and procedures for the main randomised controlled trial (RCT) may still be under development and not yet ready to fully pilot test. These studies usually focus on one or more related but substantive areas of development along the RCT preparatory pathway (e.g. intervention development, development of patient-reported outcome measures (PROMS), piloting of several components of the main trial and piloting the feasibility of implementation).

Moreover, not all pilot and feasibility studies relate to trials or interventional studies; some concern testing out design features of future large-scale cohort studies, such as the feasibility of roll-out across a wide area or being able to obtain buy-in from different stakeholders. Other researchers may want to test the feasibility of preliminary hypotheses of associations between variables that may be important to inform future research before any kind of intervention is developed or future study planned.

Table 1 lists the main types of non-randomised feasibility studies seen in the journal, and we provide guidance and examples for reference.

Intervention development

Studies that describe intervention development typically adopt mainly qualitative methods. The Template for Intervention Description and Replication (TIDieR) guideline exists for reporting intervention description [ 8 ]. Intervention development studies often describe a theoretical model that underpins the reasoning behind the intervention and through literature review or focus group work develop a feasible intervention model. This new intervention is then tried out on a small number of patients and adopted or modified as necessary.

The first thematic series of the journal covered intervention development and drew upon the expertise of guest editor, Professor Pat Hoddinott, to oversee the papers contributing to the series. Around the same time, Professor Alicia O’Cathain and colleagues published a guidance paper on maximising the impact of qualitative research in feasibility studies for RCTs (a highly accessed article) [ 9 ]. Hand in hand with the nine papers in the thematic series, these provide a good set of examples covering issues of complex intervention development [ 10 , 11 , 12 ] and strategic optimisation [ 13 ], a person-based approach to enhancing acceptability [ 14 ], intervention mapping [ 15 ] and obtaining clinical collaboration through a Knowledge to Action framework [ 16 ].

Development of PROMs

PROM development, or development of any questionnaire-based outcome measure, has some methodological similarities to the previous section in terms of selection of the proposed items for the PROM. Items generally stem from an underlying theoretical model and literature review, aided by focus group work with some preliminary testing. The PROM is then assessed for its preliminary reliability and validity in certain patient populations related to its intended use. In our second thematic series of the journal, guest editor, Professor Georgina Jones, presents seven papers that represent the types of pilot work that might take place in PROM development, including issues of translation and back-translation for use in another language [ 17 ], time and cost of administration [ 18 ], technology-based assessment [ 19 ] and the use of e-PROMS [ 20 ]. In another study, the authors follow the RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) framework [ 21 ] to pilot and evaluate use at clinic of an adolescent needs assessment tool for type 1 diabetes [ 22 ].

The CONSORT Patient-Reported Outcomes (PRO) guideline for the reporting of PROMs in main RCTs [ 23 ] may provide some further help but it should be adapted in line with the CONSORT extension to pilot and feasibility trials. The COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) guideline for systematic reviews of PROMs [ 24 ] is a comprehensive document to also be aware of especially when reporting aspects of preliminary reliability and validity and when considering the design of a future large-scale validation study.

Piloting several components of the trial

Quite often enough may be known about the study design (e.g. from conducting previous trials in the same area) to not warrant a fully randomised pilot or feasibility trial. But it may still be necessary to try out certain aspects of the intervention delivery to ensure it will work. Generally, either a before-after study design testing out processes related to the intervention arm only, or processes related to the delivery of both arms without randomisation will suffice. In these cases, we would still recommend using the CONSORT extension to pilot and feasibility trials, as it can usually be readily adapted to these situations. Any items not applicable, for example, items 8a–10 about randomisation, can be ignored in a before-after single-arm study or adapted to non-random allocation for a two-arm non-randomised study.

One example of a before-after study examines the feasibility of the Aging, Community and Health—Community Partnership Program, an inter-professional, nurse-led programme to promote diabetes self-management in older adults with type 2 diabetes and multiple chronic conditions [ 25 ]. A non-randomised study example adopts the RE-AIM framework [ 21 ] to assess the feasibility of implementing a modified weight loss programme, Positive Online Weight Reduction for Royal Navy (POWeR-RN), in overweight and obese navy personnel with a wait-list control group [ 26 ].

Piloting the feasibility of implementation of research findings

Implementation of methods to promote the systematic uptake of research findings, including interventions and other evidence-based practices into routine practice, is the topic of our third thematic series—currently an open call. Piloting plans for future implementation and evaluation of research programmes and showing them to be feasible is an important part of implementation research on the continuum of getting research into current practice. While there are journals focussing on implementation research, the preparation that goes into these programmes is not always apparent or well-reported.

Again, we would recommend using the CONSORT extension to pilot and feasibility trials as the basis for reporting such studies with suitable adaptation of items where necessary. The one published paper from the call to date describes the implementation into nutritional rehabilitation units in Malawi of the Kusamala Program, an interactive counselling programme for primary caregivers of children with severe acute malnutrition [ 27 ]. In the GLA:D® Back (Good Life with osteoArthritis in Denmark) before-after study, physiotherapists and chiropractors were trained in intervention delivery of standardised care following national guidelines for low back pain to plan a future implementation-effectiveness study [ 28 ]. Another example seeks to improve the implementation of evidence-based practices by teaching the Generation Parent Management Training Oregon (GenerationPMTO®) model, a parenting intervention, in a university graduate curriculum [ 29 ]. The RE-AIM framework has also been used in this context [ 21 , 22 ].

Feasibility studies in preparation for a cohort or other large scale study

While the majority of studies submitted to the journal are in preparation for a main future RCT, the journal is also open to submission of articles related to pilot and feasibility work for cohort studies or other large-scale observational studies. The STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guideline [ 30 ] provides a checklist of items that should be included in these types of reports, and most items are applicable to pilot and feasibility studies. However, care should be taken to state clearly the aims and feasibility objectives for the pilot work which should differ from those of the main future study. For this reason, it is recommended that the STROBE checklist is used alongside the CONSORT extension to pilot and feasibility trials to ensure that all items relate or are adapted to issues of feasibility.

Examples of feasibility cohort studies that have been published in the journal to date have concerned the feasibility of recruiting and following up children with respiratory tract infections in the community, including collecting microbiological, symptom severity and duration data [ 31 ], and determining the feasibility of recruiting practices and patients with apparent treatment-resistant hypertension for data collection and follow-up of outcomes [ 32 ].

Feasibility studies that test preliminary hypotheses of association

Sometimes, it is necessary to test preliminary hypotheses of associations between variables which if found to be promising may lead to intervention development or other preliminary work. In other cases, the associations may be in preparation for a trial. These studies are in the minority in the journal, but there are several examples to draw upon. Some adopt observational study designs and some are non-randomised experiments. We again recommend the use of the STROBE guideline alongside the CONSORT extension to pilot and feasibility trials with suitable adaptation of items as necessary.

The two examples in Table 1 look at associations between delirium and electroencephalography (EEG) frequency band connectivity readings as potential future therapeutic and diagnostic biomarkers [ 33 ] and whether sound stimulation in the womb is associated with mouth movements in the foetus [ 34 ]. If these associations are observed, then further future research can be planned.

We have provided guidance for reporting non-randomised pilot and feasibility studies. In most cases, existing guidelines can be adapted and utilised for this purpose, and we have taken some sample guidelines from the Equator website. While we have categorised studies into several common types, as can be seen from the published examples, there is overlap in the types of studies discussed with some examples fitting under more than one sub-heading.

In this editorial, we have taken all examples from the journal Pilot and Feasibility Studies . Many journals still do not have a policy of publishing pilot and feasibility studies. In a previous review of four subject-specific journals and three general mainstream medical journals, Lancaster et al. [ 35 ] identified only 90/4449 (2%) research studies published between 2000 and 2001 that called themselves ‘pilot’ or ‘feasibility’ studies. The majority were studies piloting a new treatment or technique (70%), piloting guidelines (11%), or screening programmes (5%). Surprisingly at the time, only 4 out of the 90 pilot/feasibility studies across all 7 journals were identified as being in preparation for a future RCT. Today, with the publication of the CONSORT extension to pilot and feasibility trials in 2016 [ 2 , 3 ] and aided by other influential papers [ 36 , 37 ], this number has improved, and we are starting to see phases of pilot and feasibility work published along the RCT preparatory pathway.

Most research submitted to the journal reports on one substantive phase of work at a time, addressing intervention development work or uncertainties in the study design. Research protocol submissions may describe the substantive preparatory phases altogether in one publication as a set of planned sub-studies, for example, theoretical review, intervention development and testing (in a few patients), feasibility testing in a larger patient sample, feasibility of implementation into practice and acceptability to key stakeholders. Problems can arise when researchers attempt to report multiple results from each phase within one paper, and this poses a risk of underreporting all of the pertinent findings.

The publication and sharing of detailed feasibility work has many benefits for researchers across disciplines in learning from each other, in reusing techniques that have proved successful and in avoiding similar pitfalls. Much preparatory and exploratory work is linked to the development and evaluation of complex interventions and as such should comply with the UK Medical Research Council (MRC) guidance [ 38 ]. This guidance is currently being updated, and we welcome mention of the progress that has been made to date in providing a more comprehensive framework for reporting pilot and feasibility studies [ 2 , 3 , 4 ].

We hope that this editorial will be helpful to researchers when reporting non-randomised feasibility and pilot studies. We recommend that authors use the current guidance available and ensure items are included to emphasise the goal of feasibility, such as specific feasibility objectives, feasibility outcomes and progression criteria. In writing this guidance, we have tried to identify and clarify the main kinds of issues we repeatedly see in our roles as Editors-in-Chief in an attempt to help researchers in reporting their work. We would like to end by re-iterating the message that reporting guideline publications that contain explanation and elaboration commentary on each item are very useful reference documents to consult not only at the end of a study when writing up the results, but also at the planning stage of a study when constructing an appropriate study design.

Availability of data and materials

Not applicable

Abbreviations

Consolidated Standards of Reporting Trials

COnsensus-based Standards for the selection of health Measurement INstruments

Enhancing the QUAlity and Transparency Of health Research

Generation Parent Management Training Oregon Model

Good Life with osteoArthritis in Denmark

Medical Research Council

Pilot and Feasibility Studies

Patient-Reported Outcomes

Patient-reported outcome measure

Positive Online Weight Reduction for Royal Navy

Randomised controlled trial

Reach, Effectiveness, Adoption, Implementation, Maintenance

Standard Protocol Items: Recommendations for Interventional Trials

Strengthening the Reporting of Observational Studies in Epidemiology

Template for Intervention Description and Replication

Lancaster GA. Pilot and feasibility studies come of age ! Pilot and Feasibility Studies 2015;1,1. Available at: https://pilotfeasibilitystudies.biomedcentral.com/ (last accessed 2/7/19).

Eldridge S, Chan C, Campbell M, Bond C, Hopewell S, Thabane L, Lancaster GA. CONSORT Statement: extension to randomised pilot and feasibility trials. BMJ . 2016;355:i5239.

Article   Google Scholar  

Eldridge S, Chan C, Campbell M, Bond C, Hopewell S, Thabane L, Lancaster GA. CONSORT Statement: extension to randomised pilot and feasibility trials. Pilot and Feasibility Studies . 2016;2:64.

Eldridge S, Lancaster GA, Campbell M, Thabane L, Hopewell S, Coleman C, Bond C. Defining feasibility and pilot studies in preparation for randomised controlled trials: using consensus methods and validation to develop a conceptual framework. PloS One . 2016;11(3):e0150205.

Thabane L, Hopewell S, Lancaster GA, Bond C, Coleman C, Campbell M, Eldridge S. Methods and processes for development of a CONSORT extension for reporting pilot randomized controlled trials. Pilot and Feasibility Studies . 2016;2:25.

Thabane L, Lancaster GA. A guide to the reporting of protocols of pilot and feasibility trials. Pilot and Feasibility Studies . 2019;5:37.

Chan A-W, Tetzlaff JM, Altman DG, Laupacis A, Gøtzsche PC, Krleža-Jerić K, Hróbjartsson A, Mann H, Dickersin K, Berlin J, Doré C, Parulekar W, Summerskill W, Groves T, Schulz K, Sox H, Rockhold FW, Rennie D, Moher D. SPIRIT 2013 Statement: defining standard protocol items for clinical trials. Ann Intern Med. 2013;158(3):200–7.

Hoffman T, Glasziou P, Boutron I, Milne R, Perera R, Moher D, Altman D, Barbour V, Macdonald H, Johnston M, Lamb S, Dixon-Woods M, McCulloch P, Wyatt J, Chan A, Michie S. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348:g1687.

O’Cathain A, Hoddinott P, Lewin S, Thomas KJ, Young B, Adamson J, Jansen YJFM, Mills N, Moore G, Donovan JL. Maximising the impact of qualitative research in feasibility studies for randomised controlled trials: guidance for researchers. Pilot and Feasibility Studies . 2015;1:32.

Sugg HVR, Richards DA, Frost J. Optimising the acceptability and feasibility of novel complex interventions: an iterative, person-based approach to developing the UK Morita therapy outpatient protocol. Pilot and Feasibility Studies . 2017;3:37.

Winder R, Richards SH, Campbell JL, Richards DA, Dickens C, Gandhi M, Wright C, Turner K. Development and refinement of a complex intervention within cardiac rehabilitation services: experiences from the CADENCE feasibility study. Pilot and Feasibility Studies . 2017;3:9.

Murray J, Williams B, Hoskins G, Skar S, McGhee J, Treweek S, Sniehotta FF, Sheikh A, Brown G, Hagen S, Cameron L, Jones C, Gauld D. A theory-informed approach to developing visually mediated interventions to change behaviour using an asthma and physical activity intervention exemplar. Pilot and Feasibility Studies . 2016;2:46.

Levati S, Campbell P, Frost R, Dougall N, Wells M, Donaldson C, Hagen S. Optimisation of complex health interventions prior to a randomised controlled trial: a scoping review of strategies used. Pilot and Feasibility Studies . 2016;2:17.

Yardley L, Ainsworth B, Arden-Close E, Muller I. The person-based approach to enhancing the acceptability and feasibility of interventions. Pilot and Feasibility Studies . 2015;1:37.

Greaves CJ, Wingham J, Deighan C, Doherty P, Elliott J, Armitage W, Clark M, Austin J, Abraham J, Frost J, Singh S, Jolly K, Paul K, Taylor L, Buckingham S, Davis R, Dalal H, Taylor RS. Optimising self-care support for people with heart failure and their caregivers: development of the Rehabilitation Enablement in Chronic Heart Failure (REACH-HF) intervention using intervention mapping. Pilot and Feasibility Studies . 2016;2:37.

Sturgiss EA, Douglas K. A collaborative process for developing a weight management toolkit for general practitioners in Australia—an intervention development study using the Knowledge To Action framework. Pilot and Feasibility Studies . 2016;2:20.

Schnohr CW, Rasmussen CL, Langberg H, Bjørner JB. Danish translation of a physical function item bank from the Patient-Reported Outcome Measurement Information System (PROMIS). Pilot and Feasibility Studies . 2017;3:29.

Skinner MW, Chai-Adisaksopha C, Curtis R, Frick N, Nichol M, Noone D, O’Mahony B, Page D, Stonebraker JS, Iorio A. The Patient Reported Outcomes, Burdens and Experiences (PROBE) Project: development and evaluation of a questionnaire assessing patient reported outcomes in people with haemophilia. Pilot and Feasibility Studies . 2018;4:58.

Article   CAS   Google Scholar  

Khetani MA, McManus BM, Arestad K, Richardson Z, Charlifue-Smith R, Rosenberg C, Rigau B. Technology-based functional assessment in early childhood intervention: a pilot study. Pilot and Feasibility Studies . 2018;4:65.

O’Connell S, Palmer R, Withers K, Saha N, Puntoni S, Carolan-Rees G. Requirements for the collection of electronic PROMS either “in clinic” or “at home” as part of the PROMs, PREMs and Effectiveness Programme (PPEP) in Wales: a feasibility study using a generic PROM tool. Pilot and Feasibility Studies . 2018;4:90.

Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322–7. www.RE-AIM.org .

Cooper H, Lancaster GA, Gichuru P, Peak M. A mixed methods study to evaluate the feasibility of using the Adolescent Diabetes Needs Assessment Tool App in paediatric diabetes care in preparation for a longitudinal cohort study. Pilot and Feasibility Studies . 2018;4:13.

Calvert M, Blazeby J, Altman DG, Revicki DA, Moher D, Brundage MD, CONSORT PRO Group. Reporting of patient-reported outcomes in randomized trials: the CONSORT PRO extension. JAMA. 2013;309(8):814–22.

Mokkink LB, Prinsen CAC, Patrick DL, Alonso J, Bouter LM, De Vet HCW, Terwee CB. COSMIN methodology for systematic reviews of Patient-Reported Outcome Measures (PROMs) user manual. Amsterdam: Department of Epidemiology and Biostatistics, Amsterdam Public Health research institute; 2018. www.cosmin.nl .

Google Scholar  

Markle-Reid M, Ploeg J, Fisher K, Reimer H, Kaasalainen S, Gafni A, Gruneir A, Kirkconnell R, Marzouk S, Akhtar-Danesh N, Thabane L, Rojas-Fernandez C, Upshur R. The Aging, Community and Health Research Unit—Community Partnership Program for older adults with type 2 diabetes and multiple chronic conditions: a feasibility study. Pilot and Feasibility Studies . 2016;2:24.

Garip G, Morton K, Bridger R, Yardley L. Evaluating the feasibility of a web-based weight loss programme for naval service personnel with excess body weight. Pilot and Feasibility Studies . 2017; 3 :6.

Daniel AI, van den Heuvel M, Gladstone M, Bwanali M, Voskuijl W, Bourdon C, Potani I, Fernandes S, Njirammadzi J, Bandsma RHJ. A mixed methods feasibility study of the Kusamala Program at a nutritional rehabilitation unit in Malawi. Pilot and Feasibility Studies . 2018;4:151.

Kongsted A, Hartvigsen J, Boyle E, Ris I, Kjaer P, Thomassen L, Vach W. GLA:D® Back: group-based patient education integrated with exercises to support self-management of persistent back pain — feasibility of implementing standardised care by a course for clinicians. Pilot and Feasibility Studies . 2019;5:65.

Baumann AA, Domenech Rodríguez MM, Wieling E, Parra-Cardona JR, Rains L, Forgatch M. Teaching GenerationPMTO, an evidence-based parent intervention, in a university setting using a blended learning strategy. Pilot and Feasibility Studies . 2019;5:91.

von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: guidelines for reporting observational studies. BMJ. 2007;335(7624):806–8.

Anderson EC, Ingle S, Muir M, Beck CR, Leeming JP, Kesten J, Cabral C, Hay AD. Population-based paediatric respiratory infection surveillance: a prospective inception feasibility cohort study. Pilot and Feasibility Studies . 2018;4:182.

Hayes P, Kielty H, Casey M, Glynn LG, Molloy GJ, Durand H, Newell J, Murphy AW. Prognosis of patients with apparent treatment-resistant hypertension—a feasibility study. Pilot and Feasibility Studies . 2018;4:43.

Fleischmann R, Traenkner S, Kraft A, Schmidt S, Schreiber SJ, Brandt SA. Delirium is associated with frequency band specific dysconnectivity in intrinsic connectivity networks: preliminary evidence from a large retrospective pilot case-control study. Pilot and Feasibility Studies . 2019;5:2.

Reissland N, Francis B, Buttanshaw L, Austen JM, Reid V. Do fetuses move their lips to the sound that they hear? An observational feasibility study on auditory stimulation in the womb. Pilot and Feasibility Studies . 2016;2:14.

Lancaster GA, Dodd SR, Williamson PR. Design and analysis of pilot studies: recommendations for good practice. Journal of Evaluation in Clinical Practice . 2004; 10 (2):307–12.

Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, Robson R, Thabane M, Goldsmith CH. A tutorial on pilot studies: the what, why and how. BMC Medical Research Methodology . 2010; 10 (1).

Arain M, Campbell MC, Cooper CL, Lancaster GA. What is a pilot or feasibility study? A review of current practice and editorial policy. BMC Medical Research Methodology . 2010;10:67.

Craig P, Dieppe P, Macintyre S, Mitchie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ . 2008; 337 :a1655. www.mrc.ac.uk/complexinterventionsguidance .

Download references

Acknowledgements

Pilot And Feasibility Studies (PAFS) working group: Professor Sandra Eldridge and Claire Chan, Queen Mary University of London; Professor Gillian Lancaster, Keele University; Professor Lehana Thabane, McMaster University, Canada; Dr Sally Hopewell, Oxford University; Emeritus Professor Christine Bond, Aberdeen University; Emeritus Professor Mike Campbell, Sheffield University.

Author information

Authors and affiliations.

School of Primary, Social and Community Care, Keele University, Newcastle-under-Lyme, UK

Gillian A. Lancaster & Lehana Thabane

You can also search for this author in PubMed   Google Scholar

Contributions

GL wrote this editorial. LT commented on the draft. Both authors approved the final version.

Corresponding author

Correspondence to Gillian A. Lancaster .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

Gillian Lancaster and Lehana Thabane are co-Editors-in-Chief of Pilot and Feasibility Studies .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Lancaster, G.A., Thabane, L. Guidelines for reporting non-randomised pilot and feasibility studies. Pilot Feasibility Stud 5 , 114 (2019). https://doi.org/10.1186/s40814-019-0499-1

Download citation

Received : 22 July 2019

Accepted : 10 September 2019

Published : 06 October 2019

DOI : https://doi.org/10.1186/s40814-019-0499-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

ISSN: 2055-5784

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

What is the difference between feasibility studies and pilot testing?

Feasibility study.

"Feasibility studies are pieces of research done before a main study to answer the question ‘Can this study be done?’ They are used to estimate important parameters that are needed to design the main study” [1] . Data collected would not be analyzed or included in publications.

• Going to a potential site to see if the research is possible
• Checking to see what is the best approach to the research
• Going through a consent process with friends to see if the information is comprehensible
• Sending your survey instrument to a few experts in the field for their feedback as to whether or not the questions are appropriate for the topic and/or cohort of the research
• Feedback from colleagues and peers about research design
• Student researcher designs questionnaire for their study’s target population and asks someone from a different population to test the questionnaire

A researcher planning to conduct interviews regarding landowner perspectives of land use regulations contacts the US Forest Service to ask how they have typically approached land owners in the past and asks for feedback on their planned questions.

Pilot testing:

“A small scale-study conducted prior to conducting an actual experiment; designed to test and refine procedures.”

 Examples:

• Checking to see if the designed tool works
• Asking people to complete a survey to find out whether a question results in the requested information
• Testing the intervention with four people before trying it with 60 people
• Asking people to complete your survey and then revising the questions based on their responses
• Revising the study after analyzing preliminary data and determining that the data do not address their research question
• Student researcher designs questionnaire for their study’s target population, asks the population to try out the questionnaire, and the questions are revised based on the responses

A researcher planning to conduct interviews regarding landowner perspectives of land use regulations conducts interviews with 5 people to test the questions and see if they get answers that make sense. The researchers may revise their interview guides based on the initial data collected.

Q: Does my feasibility study/pilot testing require IRB Review?

A: The federal regulations indicate that pilot testing meets the definition of research involving human subjects and requires IRB review. However, feasibility studies typically do not meet the definition of research involving human subjects and therefore would not require IRB review.

In order for the IRB to determine whether your activities constitute a feasibility study or pilot testing, and subsequently, whether they require IRB review, please complete and submit the initial sections of the HRPP application and protocol form in Cayuse. Instructions can be found on the " Preparing an Initial Submission " page. In this form, it is helpful to note with whom the study or testing is going to be done and how the data will be used.

[1] National Institute for Health Research; https://www.nihr.ac.uk/glossary/

Watch CBS News

Experimental pill can ease hot flashes for women in menopause, new data suggests

By Sara Moniuszko

Edited By Sarah Lynch Baldwin

August 23, 2024 / 10:39 AM EDT / CBS News

More than 75% of women experience menopause-related hot flashes during their lives — and an experimental pill could help ease them, a new study suggests. 

In a study published in JAMA Thursday, researchers found the drug, elinzanetant, demonstrated "statistically significant reductions" in the frequency and severity of hot flashes for women in menopause — without the use of hormones. 

Not using hormones in the drug is important because it's not a safe option for everyone, Dr. Céline Gounder, CBS News medical contributor and editor-at-large for public health at KFF Health News, said on "CBS Mornings" Friday .

"Some women cannot take hormones, so maybe they've had breast cancer, heart attacks, stroke, blood clots," she said, adding some women are also afraid to take hormones. "This dates back to a study that was done over 20 years ago on hormone replacement therapy. There were a lot of problems with that study, and now, in retrospect, we understand for the vast majority of women, hormone replacement therapy is safe, and I think that's really important to emphasize that, because a lot of women are not using it, who could be using it, but now we have some alternatives for women who can't or don't want to take hormone replacement."

The two main side effects seen in the clinical trials were headache and fatigue, but longer-term effects are still unknown.

"This is on data that's been reported out to 26 weeks of use. So obviously, women would be on these medications for much longer. Are there longer term side effects? We don't know," Gounder said.

Exactly how soon the medication could be available is also unknown. But Bayer, the company that makes it, submitted an application over the summer to the U.S. Food and Drug Administration and expects to hear back in September.

Sara Moniuszko is a health and lifestyle reporter at CBSNews.com. Previously, she wrote for USA Today, where she was selected to help launch the newspaper's wellness vertical. She now covers breaking and trending news for CBS News' HealthWatch.

More from CBS News

60% of baby foods in study don't meet nutrition standards

Baby dies after being left in hot car for over 7 hours, officials say

What are the differences between IVF and IUI, the fertility treatment used by Tim Walz's family?

Multiple people killed in knife attack at festival in German city

Motion planning method and experimental research of medical moxibustion robot of double manipulator arms

• Technical Paper
• Published: 22 August 2024
• Volume 46 , article number  564 , ( 2024 )

Cite this article

• Zhengyao Yi 1 ,
• Haoming Li 1 ,
• Jiasheng Zhu   ORCID: orcid.org/0009-0001-1657-8793 1 ,
• Bingxing Feng 1 ,
• Jie Cao 1 ,
• Xianshu Lu 2 &
• Baocheng Wang 3  

In order to alleviate the contradiction between the increasing demand of seafarers for moxibustion physiotherapy and the shortage of moxibustion doctors, a medical double-arm moxibustion robot was designed by using a six-degree-of-freedom mechanical arm and a four-degree-of-freedom mechanical arm to simulate traditional Chinese medicine moxibustion techniques. The robot coordinate system was established by D-H parameter method, and the forward and inverse kinematics of the robot model were calculated. The robot model was established and simulated by Robotics Toolbox in MATLAB. The angular velocity and angular acceleration curves of each joint and the trajectory and displacement of the robot end were obtained, and the feasibility of robot trajectory planning was verified. Through the preliminary design, the collaborative process of task assignment for double moxibustion robot was established. The simulation test bench was built to further simulate the temperature of human epidermis, and the relationship between the end distance of moxibustion robot and the heating of human epidermis was determined. The simulation and experimental results show that: a) The robot does not appear serious impact or stutter phenomenon in the simulation process, and the kinematics performance is good, which verifies the feasibility of the robot model; and b) during the simulation test, the heating temperature of human epidermis can be maintained at 43 °C, which realizes the expected moxibustion temperature of patients and verifies the effectiveness of the robot model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Research on Stability of Dual-Arm Cooperative Execution Task

Workspace Simulation and Analysis of a Dual-Arm Nursing Robot

A three-DoF upper limb exoskeleton’s design, modeling, and interaction-based control

Explore related subjects.

• Artificial Intelligence

Ministry of Transport of the People's Republic of China. 2020 China Crew Development Report. [2021–06–25]. https://zs.mot.gov.cn/mot/s

Jinlu S (2010) Simulation analysis of seafarers’ seaworthiness in small organization environment. Chinese Journal of Safety Science 20(08):44–48

Google Scholar  

Tao Yu, Xiao W, Xiaopu Z, Aimin Z, Yong J, Jinhan Mo (2018) Distribution characteristics of BTEX in ship cabin environment and health risk assessment of crew exposure[C]//. Academic Conference on Environment and Health-Precise Environmental Health : Challenges for Interdisciplinary CooperationThesis Series 2018:432–433

Li Xiaopeng presided over the meeting and emphasized: optimizing the crew development environment to promote the construction of high-quality crew team.Transportation enterprise management, 2021, 36(02): 27.

Yan Caiju, Gu Yan. Clinical observation of moxibustion therapy combined with Gua Sha in the treatment of chronic pelvic inflammatory disease of cold-damp stagnation type. Frontier of international medical research, 2022, 6(2).

Kim JI, Choi JY, Lee H, Lee MS, Ernst E (2010) Moxibustion for hypertension: A systematic review. BMC Cardiovasc Disord 10(1):33

Article   Google Scholar  

Yin Shao, Zhu Fengya, Li Zhao, Che Deya, Li Liuying,Feng Jie, Zhang Lu, Huo Zhenyi. An Overview of Systematic Reviews of Moxibustion for Knee Osteoarthritis. Frontiers in Physiology, 2022, 13.

Ziqiang Ni, Tianmiao W, Da L (2015) An overview of the development of medical robotics. Journal of Mechanical Engineering 51(13):45–52

Zhang Jingxin Lu, Dongdong LQ, Wang Xuejun Lu, Mengye ZX, Yang Xiaoyuan Gu, Jiyu SY, Tiancheng Xu (2018) Research progress and key technology analysis of intelligent acupuncture robot. China Digital Medicine 13(10):2–4

Deng J, Yin C, Chen M, Tingbiao Wu, Zhang R, Zhang J (2019) Design and use of a moxibustion glasses holder. Chinese Acupuncture 39(10):1137–1140

Gao Ling, Wang Dongbin, Liu Huirong.Development and characteristics of multi-acupoint moxibustion apparatus.Chinese Acupuncture, 2018, 38(9): 1013–1015.

Li Min, Sun Zhiling. Development and characteristics of a fully automatic temperature-controlled moxibustion box.Chinese Acupuncture, 2019, 39(6): 649–650.

Wei W, Mei L, Ligong W (2012) Application and Significance of Multifunctional Moxibustion Treatment Bed. Chinese Acupuncture 32(7):665–667

Dai Yaonan, Chen Xubing. Design and implementation of thermostatic moxibustion robot for human back spine.Computer engineering and application, 2019, 55(9): 216–222.

Zhao Guoyou, Liu Yicheng, Tu Haiyan, Zhang Hanrui, Xia Shilin, Li Yingkun.Design and application of moxibustion manipulator.Acupuncture research, 2020, 45(11): 5.

Xia S, Tian S, Zhang H, Li Y, Haiyan Tu, Zhao G (2021) A design and application of moxibustion apparatus for multi-joint moxibustion manipulator. Chinese Acupuncture and Moxibustion 41(2):4

Yong Y, Lei M, Jiaqi X, Xiangyu Ye, Yang Z, Huayuan Y (2020) Design and implementation of an intelligent moxibustion manipulation simulation system. Beijing Biomed Eng 39(6):7

Yuhao Jin, Rong Yi, Jiangqiong Meng, Qiming Yang, Taipin Guo, Zeyi Li, Ruonan Li, Xiaoling Bai. Design and application of an automatic temperature-controlled moxibustion instrument.Acupuncture Clinical, 2018,34 (11) : 70–72.

Denavit J, Hartenberg RS (1995) A kinematic notation for lowerpair mechanisms based on matrices. J Appl Mech 22:215–221

Qazani M, Pedrammehr S, Rahmani A, Danaei B, Ettefagh MM, Rajab AKS, Abdi H (2015) Kinematic analysis and workspace determination of hexarot-a novel 6-DOF parallel manipulator with a rotation-symmetric arm system. Robotica 33(8):1686–1703

Zixing C (2009) Robotics, 2nd edn. Tsinghua University Press, Beijing

LIAO S, LI J. Kinematic simulation and analysis of robot based on MATLAB[C]/ /American Institute of Physics Conference Series, 2018: 020066–1–020066–5.

Cheng Kai, Liu Bo. Experimental observation on the effect of moxibustion at different temperatures on local skin burns.Clinical Medical Literature Electronic Journal, 2020,7 (30) : 57.

Xin Xu, Bin D, Qi W, Peiheng He, Ningbo L (2020) The skin temperature control system of moxibustion point based on self-tuning fuzzy PID. Electronic measurement technology 43(22):39–44

Download references

This work was supported by the Dalian Science and Technology Innovation Fund (Grant No. 2021JJ13SN50) and Dalian Shield Safe Technology Ltd. (Grant No. 2020067).

Author information

Authors and affiliations.

School of Navigation and Ship Engineering, Dalian Ocean University, Dalian, 116023, China

Zhengyao Yi, Haoming Li, Jiasheng Zhu, Bingxing Feng & Jie Cao

Noh Hyun-Sook Hospital of Korean Medicine in Ansan, Gyeonggi Province, Seoul, 999007, Korea

Maternal and Child Health Care Hospital of Dalian Women and Children Medical Center Group, Dalian, 116021, China

Baocheng Wang

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Jiasheng Zhu .

Ethics declarations

Conflict of interest.

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. The funders had no role in the design of the study, in the collection, analyses, or interpretation of data, in the writing of the manuscript, or in the decision to publish the results.

Additional information

Technical Editor: Rogério Sales Gonçalves.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Yi, Z., Li, H., Zhu, J. et al. Motion planning method and experimental research of medical moxibustion robot of double manipulator arms. J Braz. Soc. Mech. Sci. Eng. 46 , 564 (2024). https://doi.org/10.1007/s40430-024-05139-8

Download citation

Received : 04 January 2024

Accepted : 05 August 2024

Published : 22 August 2024

DOI : https://doi.org/10.1007/s40430-024-05139-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Moxibustion robot
• Kinematics analysis
• D-H parameter method
• Computer simulation
• Find a journal
• Publish with us
• Track your research

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Experimental study of laser scattering protection system for low-speed aircraft

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Validation, Writing – original draft

Affiliation Department of Aerospace Engineering, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea

Roles Funding acquisition, Supervision, Writing – review & editing

* E-mail: [email protected]

• Elliott Donghyun Kim, 

• Published: August 22, 2024
• https://doi.org/10.1371/journal.pone.0308979
• Reader Comments

This study introduces a laser scattering system to protect a low-speed aircraft. Scattering was selected to reduce the laser’s intensity targeting the sensor of an aircraft and simultaneously maintaining the functionality of aircraft optics. Mie scattering, known for effectively decreasing short-wave infrared light, was employed by utilizing water aerosols having a diameter of 1 to 5 μm. Experimental results regarding the decrease of the laser intensity via scattering confirmed that the theoretical and experimental values resulted in a similar decrease rate under static conditions. To validate the theoretical values, the path length, which the laser passing through water aerosols, was changed. To assess the system’s feasibility in flow conditions, a low-speed wind tunnel was employed to generate two flow speeds: 5.5 m/s and 17.6 m/s. Remarkably, the reduction of laser intensity was only affected by the path length, and was somewhat unaffected regardless of flow speed and the uniformity of the flow, only to the path length. In all cases, the initial laser intensity was set to 10 mW. Under static conditions, the intensity dropped to 8.21 mW, showing a decrease of 17.9%. In flow conditions of 5.5 m/s, 17.6 m/s, and in distorted flow, the laser intensity decreased by 18.3%, 18.1%, and 18% respectively. As a preliminary study, these results demonstrate the system’s capability to protect a low-speed aircraft targeted by lasers even under dynamic flow conditions, may suggest a possibility of providing a practical defence solution.

Citation: Kim ED, Park G (2024) Experimental study of laser scattering protection system for low-speed aircraft. PLoS ONE 19(8): e0308979. https://doi.org/10.1371/journal.pone.0308979

Editor: Sushank Chaudhary, Guangdong University of Petrochemical Technology, CHINA

Received: January 9, 2024; Accepted: August 1, 2024; Published: August 22, 2024

Copyright: © 2024 Kim, Park. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper.

Funding: National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No:NRF-2021M3F6A108598013). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

1 Introduction

A low-speed aircraft neutralization system using a high-powered laser was developed owing to its precision, rapid operation, and cost-effectiveness [ 1 , 2 ]. Recent advancements in the aircraft neutralization system have introduced the “soft kill” technique, wherein a high-powered laser focuses on the aircraft’s electronic circuits to neutralize the optical seeker instead of destructively impacting the aircraft [ 3 ]. By aiming the electronic circuits, the laser for the soft kill method requires less power to neutralize a targeted aircraft [ 4 ], compared to the “hard kill” technique, which applies physical damage to a targeted aircraft while airborne using chemical lasers capable of generating megawatts of intensity [ 3 – 5 ]. Furthermore, the operation time is significantly lower than that of the previous hard kill system. Based on these advantages, high-powered lasers for soft kill methods have mainly been developed for surface-to-air defences targeting a low-speed aircraft [ 6 ].

Following the rapid advancement of soft kill techniques, laser defence systems have also been developed by reducing the intensity of the laser. The laser intensity reduction has been achieved through absorption, reflection, and scattering [ 7 – 9 ]. The laser absorption technique incorporates a unique incident laser emitted from an aircraft that generates heat along the laser trajectory, forming an aerodynamic lens capable of absorbing energy from a high-powered laser directed towards the aircraft [ 7 ]. The laser-reflection technique use reflection by coating an ablative material on to the surface of an aircraft, capable of absorbing the energy of a high-powered laser before it reaches the target [ 9 ].

Nonetheless, the light-absorption-based protection system, which predominantly converts incident light into heat energy [ 10 ], can potentially cause aircraft malfunctions, as the system itself could heat up and damage circuits due to the kilowatt power of the laser. Additionally, experiments have revealed that, in the case of reflection systems, coated aircrafts are more vulnerable to damage. This is due to the presence of impurities such as dust particles on the surface of the aircraft, which cause high-powered lasers to initially melt these impurities. This leads to a greater liability for aircraft destruction in coated aircrafts using the reflection system compared to situations without coating materials [ 11 ].

Given the difficulty of using absorption and reflection methods for safeguarding aircraft against lasers, this study aims to introduce a novel concept for a defence system utilizing scattering, which can effectively protect a low-speed aircraft targeted by such lasers. In contrast to the absorption method, the scattering method introduced in this study can be achieved simply by adapting particles along the path of the laser, offering an effective approach, as it eliminates the need for additional emitting devices. Furthermore, compared to the reflection method, the scattering method exclusively safeguards the internal sensors, thereby avoiding any concerns of melting the surface particles. Additionally, because the scattering amount of the laser varies with wavelength, even under identical flow conditions and particle sizes, a laser defence system relying on scattering can effectively reduce the intensity of lasers while simultaneously ensuring the functionality of internal circuits, such as seekers [ 12 ]. Therefore, by leveraging the properties of scattering, such as particle size, this study presents a methodology that effectively obstructs lasers while allowing the seeker’s laser, operating at a different wavelengths, to penetrate through, thereby preserving its functionality. Hirst et al. [ 13 ] conducted a comprehensive investigation into the spatial intensity distribution of light scattered by individual airborne particles, both spherical and nonspherical. Their work demonstrated how laser light scattering can classify particles based on shape and size, offering valuable insights into the scattering behavior of various particle types. Additionally, the research by Miles et al. [ 14 ] on Filtered Rayleigh Scattering (FRS) provides a valuable reference for understanding particle scattering in low-speed and high-speed flows. This demonstrates that a scattering system may operate effectively in low-speed conditions.

In line with this concept, this study presents a novel technology for anti-aircraft protection against lasers that target low-speed aircraft. To effectively defend against lasers, Mie scattering was induced to reduce the laser intensity by employing water aerosols to scatter the incident laser and decrease its intensity. The feasibility of this system was evaluated under various flow conditions. The experiment was conducted in a static state with no flow conditions to establish the fundamental principles of the protection system. Additionally in this condition, the path length was changed to validate the calculations of the theoretical values.

To generate flow, a wind tunnel was operated under two different conditions: a low and high rotation conditions, resulting in average speeds of 5.5 m/s and 17.6 m/s, respectively, simulating the flow conditions experienced by a low-speed aircraft. Moreover, to further assess the scattering system, a distorted flow generator was inserted into the test section of the wind tunnel to induce a non-uniform flow. Under non-uniform flow conditions, such as those influenced by atmospheric turbulences, density changes, occurring aero-optical effects which is possible to affect the scattering amount compared to uniform flow conditions [ 15 , 16 ]. For the test case with a flow speed of 17.6 m/s, the amount of scattering depending on the flow uniformity was compared and discussed.

2 Theoretical background of scattering for a protection system

2.1 scattering theory.

2.2 Selection of wavelength

When developing a protective system for low-speed aircraft, it is essential to specify the laser wavelength. This is because the scattering varies according to the wavelength of the emitted laser. Accordingly, this study presents a protective mechanism to safeguard against lasers within the Short-Wave Infrared (SWIR) range, notably at 1.064 μ m, a wavelength commonly adopted for high-powered laser systems [ 23 ].

Fig 1 shows the relative transmission rates for various wavelengths under atmospheric conditions. The 1.064 μ m wavelength light exhibits a notably superior transmission rate compared to the visible light region [ 24 ]. Furthermore, in comparison to higher wavelengths exceeding 10 μ m, light with a wavelength of 1.064 μ m displays lower scattering losses. As a result, the selection of the 1.064 μ m wavelength light for laser systems is driven by its superior transmission rate and reduced scattering losses, setting it apart from other wavelength options. Thus, the scattering system designed for the 1.064 μ m wavelength light should employ particles capable of effectively scattering this specific wavelength due to its exceptional transmission properties and minimal scattering losses.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

Reproduced with permission from [ 25 ].

https://doi.org/10.1371/journal.pone.0308979.g001

Fig 2 shows the primary scattering observed at various wavelengths. Specifically, for the 1.064 μ m wavelength light, Mie scattering emerges as the dominant mechanism, while Rayleigh scattering exhibits a minimal scattering effect. Furthermore, geometric scattering does not occur within the 1.064 μ m wavelength range. As a result, this study primarily focuses on Mie scattering, which is the dominant scattering mechanism within the 1.064 μ m wavelength range and serves as the main mechanism for reducing laser intensity.

Data adapted from [ 26 ].

https://doi.org/10.1371/journal.pone.0308979.g002

2.3 Selection of scattering material

The effectiveness of Mie scattering is widely recognized when scattering particles exhibit spherical shapes [ 27 ]. Therefore, in this study, water aerosols were chosen as the scattering material for the irradiated laser because of the comparatively high surface tension of water. This characteristic enables the possibility of creating a more spherical particles than in other liquids [ 28 ].

Fig 3 illustrates the selection of the water aerosol diameter based on the calculation of scattering efficiency in relation to particle size. The MiePlot program [ 29 ] released open source by Laven, was employed to analyze water particles scattering 1.064 μ m wavelength light across a particle diameter range of 0.1 to 100 μ m. The scattering efficiency is defined as the ratio of the power scattered to the incident power on the cross-sectional area of the particles, which is a non-dimensional parameter [ 30 ]. The calculation outcomes revealed that particles with diameters ranging from 1 to 5 μ m exhibited the highest scattering efficiency, aligning with the Mie scattering regime. Consequently, water aerosol particles with a diameter of 1 to 5 μ m, which yield efficient Mie scattering, were chosen as the optimal particle size for the experiment.

https://doi.org/10.1371/journal.pone.0308979.g003

2.4 Estimated laser intensity

2.5 Feasibility of the protection system

Fig 4 illustrates the application of the protection system. The scattering-based system was designed to be capable of effectively blocking a laser aiming at a low-speed aircraft while simultaneously blocking or affecting other internal sensor components of the aircraft. This concept is possible, due to the fact that the sensors within the aircraft, such as seekers, operating within the mid-wave infrared (MWIR) range and, approximately having a wavelength of 4 to 6 μ m [ 38 , 39 ], exhibit lower scattering efficiency compared to other wavelengths. In addition, scattering was caused by water aerosols in this study, thereby eliminating the occurrence of geometric scattering in the MWIR range. This observation indicates that the utilization of Mie scattering may have the potential to block high-powered lasers with a 1.064 μ m wavelength while maintaining the functionality of aircraft seekers. In hypothetical real-world scenarios, when a high-powered laser targets the circuits of a low-speed aircraft, the scattering system reduces the laser’s intensity. Meanwhile, simultaneously, the aircraft’s sensors remain somewhat unaffected, due to their use of different light wavelengths, ensuring the aircraft’s proper functioning.

https://doi.org/10.1371/journal.pone.0308979.g004

Fig 5 shows the decrease in laser intensity across various wavelengths due to water aerosol scattering with diameters ranging from 1 to 5 μ m. The calculations assumed a laser path passing through 100 mm water aerosols. The results indicate that within the 1.064 μ m wavelength range, practically employed in high-powered lasers, a reduction of 79.3% is observed. However, within the 4 μ m wavelength range, which the aircraft’s internal sensor uses, the decrease is only 17.9%. Thus, an effective low-speed aircraft protection system is introduced in this study.

https://doi.org/10.1371/journal.pone.0308979.g005

3 Experimental details

3.1 facility and instrumentation.

In pursuit of developing a laser protection system to ensure the safety of aircraft, a series of experiments were conducted under static conditions to validate the theoretical values. Furthermore, to simulate the dynamic flow conditions experienced by an aircraft, the experiment involved modifying the flow environment using a wind tunnel, allowing for the practical assessment of laser intensity reduction through water aerosols. Based on the obtained results, a novel protection system was introduced to decrease the intensity of a laser targeting a low-speed aircraft by utilizing water aerosols under flow conditions.

Fig 6 shows the schematic of the scattering protection system installed in the wind tunnel. The protection system setup consists of two major components: a scattering setup and an optical setup. The scattering setup demonstrates the use of water aerosols to scatter the incident laser, while the optical setup shows the laser and detector employed in the experiment to calculate the reduced laser intensity.

https://doi.org/10.1371/journal.pone.0308979.g006

Water aerosols were employed to scatter the laser and reduce its intensity. To induce Mie scattering using water aerosols, a 3-jet type Collison nebulizer (CH Technologies, Collison nebulizer) was utilized in the experiment, as shown in the scattering setup in Fig 6 . The Collison nebulizer generates water aerosols with a Mass Median Aerodynamic Diameter (MMAD) of 2.5 μ m and a Geometric Standard Deviation (GSD) of 1.8. These values ensured that the diameter of the aerosols fell within the range of 1 μ m to 5 μ m. The generation of water aerosols using a nebulizer relies on the Bernoulli principle, in which compressed air is introduced at a high velocity through the small orifice of the nebulizer, leading to the fragmentation of water into small droplets. Water aerosol atomization and collision with the wall of the nebulizer result in a further reduction in the droplet size. In addition, gravity causes larger particles to settle on the water surface, whereas specialized traps integrated into a curved outlet tube eliminate additional heavy water particles. In this experiment, an air compressor was used to deliver 20 psi (1.4 bar) to the Collision nebulizer, resulting in a uniform horizontal aerosol flow of 15 mm, ensuring that the 1.064 μ m wavelength laser maintained an path length ( L ) of 15 mm in all experimental cases. In addition, a rotational plate was used to support the Collision nebulizer and allowed it to rotate in order to change the path length.

The optical setup in Fig 6 shows the laser and detector used in the experiment. High-powered lasers commonly utilize a wavelength of 1.064 μ m. To replicate this particular laser, an Nd: YVO4 laser (CNIL laser, MIL-III-1064 10mW) with a wavelength matching the desired specifications was employed. Although practical lasers that target low-speed aircrafts employ high-powered beams, this experiment utilized a lower-powered laser with the same wavelength, which is suitable for laboratory-scale applications. Furthermore, in the context of scattering, the laser power becomes inconsequential and only the wavelength is significant. Hence, a 1.064 μ laser with low power was chosen for this purpose. To detect the 1.064 μ m wavelength laser, which is within the infrared range, an optical power detector (Newport, 918D-ST-SL) was employed to capture the laser signal, and a power meter (Newport, 843-R-USB) was utilized to measure the laser intensity.

3.2 Flow condition

Fig 7 shows the wind tunnel at Korea Advanced Institute of Science and Technology (KAIST). The wind tunnel [ 40 ] contains several key components, including a blower, diffuser, settling chamber, contraction, and a constant-area test section measuring 300 × 300 mm 2 . The blower introduces ambient air into the wind tunnel, whereas the diffuser helps recover the static pressure. The settling chamber straightens the airflow using screens, while the contraction section accelerates the flow from the settling chamber into the test section. The generated flow from the wind tunnel is controlled by adjusting the revolutions per minute (RPM) of the blower’s motor. The wind tunnel utilized in the experiment is a continuous type. The flow is generated by a motor inside the wind tunnel, capable of sustaining continuous testing, thereby allowing the test duration to be set as desired. Similar to the wind tunnel experiments conducted by Mohebbi et al. [ 41 ], where a suction open circuit wind tunnel with a test section measuring 100 × 100 cm 2 was used, the setup used in the experiment also involved detailed aerodynamic testing.

https://doi.org/10.1371/journal.pone.0308979.g007

In this experiment, two different flow conditions were used: a low-rotation condition to generate a flow speed of 5.5 m/s and a high-rotation condition for a flow speed of 17.6 m/s. The low-rotation condition of the wind tunnel was obtained by setting the blower’s motor to 10 RPM, generating a flow speed of 5.5 m/s. In contrast, the high-rotation condition was obtained by adjusting the motor to 30 RPM, resulting in a flow speed of 17.6 m/s.

Fig 8 shows the experimental setup to simulate the distorted flow conditions that an aircraft may encounter and a 5-hole probe (AeroProbe, Fast Response Probe) used to measure the distortion rate of the flow. This study examines how the laser scattering system is influenced by the non-uniformity of the flow by applying a distorted flow condition for a test case with a flow speed of 17.6 m/s. The distorted flow generator was a rubber coil placed in the test section of the wind tunnel to create a non-uniform flow pattern. The flow deviated from its original direction as it passes through the generator, resulting in a distorted flow. According to Eq (3) , the path length is the main factor that reduces the laser intensity under static state conditions.

https://doi.org/10.1371/journal.pone.0308979.g008

However, when the flow is distorted, it can alter the interactions among particles in the fluid. This interactions may include increased collisions among particles, impacting the scattering behavior and potentially causing a change in the scattering coefficient ( β A ). To ascertain the uniformity of the flow, a 5-hole probe was used to measure the distorted flow velocity in the test section. The performance of the distorted flow generator was evaluated by measuring the flow speed at three locations along the test section of the wind tunnel: 75 mm, 150 mm, and 225 mm.

Fig 9 shows the flow velocities under the experimental conditions. The flow cases were classified into three groups: case (a) representing a flow condition of 5.5 m/s; case (b) representing a flow condition of 17.6 m/s; and case (c) representing the distorted flow condition. The results indicate average flow speeds of 5.5 m/s and 17.6 m/s of the wind tunnel conditions, respectively, and are shown with error bars. Moreover, the average and standard deviation of the 17.6 m/s flow speed condition are depicted both with and without the distorted flow generator.

https://doi.org/10.1371/journal.pone.0308979.g009

The flow speeds along the u , v , and w axes were determined by employing the 5-hole probe within the test section of the wind tunnel. This enabled a direct comparison of the steady and distorted rates of the generated flow. The comparison between the uniform and distorted flows revealed higher speed values along the v and w axes, in which the identical values of the speed are 0, indicating the presence of flow distortion. This is because, in an ideal uniform flow along the wind tunnel, the flow speed is only along the u-axis. However, a distorted flow has speed components along the v and w axes, indicating that the flow is not uniformly horizontal, resulting in a distorted flow.

The results reveal that the distorted flow displays a notably higher standard deviation rate of 0.2980 compared to the flow without the distorted flow generator, which registers a standard deviation rate of 0.0342. This emphasizes the increased non-uniformity in the flow field.

4 Results and discussion

4.1 theory verification experiment, 4.1.1 static state experiment..

To provide validation of the laser scattering system, measurements of the reduced laser intensity were conducted under static conditions. Across all experimental conditions, the total test time was 45 s, and the aerosol spray time was set to 30 s. The stabilization of the laser was maintained for 5 s before the water aerosols were sprayed. The path length for the 1.064 μ m wavelength laser passing through the water aerosol was fixed at 15 mm, and the initial laser intensity was set at 10 mW.

Fig 10 shows the decreased laser intensity under static state conditions. When the 1.064 μ m laser encountered the aerosol flow with a path width of 15 mm in the static state, the intensity decreased from 10 mW to 8.21 mW, indicating a 17.9% reduction. The theoretical calculations based on Eq (3) indicate that the laser intensity decreases to 7.89 mW, indicating an error rate of 4.05%. The results demonstrated that under static conditions, the scattering protection system effectively reduced the intensity of the incident laser. This outcome also confirms the accuracy of the calculations used to derive Eq (3) .

https://doi.org/10.1371/journal.pone.0308979.g010

4.1.2 Static state experiment with rotation.

The static state experiment demonstrated the accuracy of the theoretical and experimental decrease in laser intensity, as shown in Eq (3) . Nevertheless, Eq (3) indicates that the reduction in laser intensity is solely influenced by the path length. To further validate Eq (3) , a rotation experiment was conducted to measure the decrease in laser intensity by water aerosol scattering at different path lengths. Experiments were conducted by rotating the Collison nebulizer to change the path length and observe the resulting difference in laser intensity. The angles were set at 0, 15, 30, 45, 60, and 75 degrees, corresponding to cases 1 through 6, respectively. The rotational experiments were conducted in a static state, where no flow exists, eliminating all variables except the path length. The initial laser intensity was set to 10 mW with a test time of 45 s, and the aerosols were sprayed for 35 s while the Collison nebulizer was rotated. Additionally, in order to mitigate any errors occurring by the change of the path length ( L ), which is the parameter L in Eq (3) , the laser alignment was precisely maintained.

Fig 11 shows the schematic and results of the rotational case experiment. The rotation angle was converted into the path length. In case 6, with a rotation angle of 75 degrees, the laser encountered turbulent flow from the sprayed aerosol, resulting in a non-ideal path length and a notably high error rate. However, the results of the remaining tests indicated that the theoretical calculations based on Eq (3) closely align with the experimental values, with an error rate below 6% across all cases.

https://doi.org/10.1371/journal.pone.0308979.g011

Table 1 shows the values of the rotation angle of the Collison nebulizer converted into the path length and compares the theoretical and experimental values. The experimental results demonstrated that the calculated values for reduced laser intensity correlated with the experimental values, confirming the feasibility of the protection system. Moreover, this result validates the accuracy of the calculations used to obtain Eq (3) .

https://doi.org/10.1371/journal.pone.0308979.t001

4.2 Experiment with flow

To demonstrate the system’s capability to operate effectively in flow conditions, experiments involved in flows at different speed, as well as uniform and distorted flow conditions were conducted. In the wind tunnel experiment, the variables remained consistent with those in the static state and the rotation experiment.

4.2.1 Flow speed 5.5 m/s-case (a).

Fig 12 shows the decreased laser intensity caused by water aerosol in the 5.5 m/s flow condition, which is case (a). The experimental setup and procedures were identical to those of the static state experiment, with the wind tunnel running continuously at a flow speed of 5.5 m/s throughout the experiment. The results showed a reduction in the laser intensity to 8.17 mW, indicating an 18.3% decrease, corresponding to an error rate of 3.42% compared with the theoretical value. The results indicate that even under 5.5 m/s flow condition, the scattering system reduces the laser intensity, which is consistent with the results observed in the static state condition experiment.

https://doi.org/10.1371/journal.pone.0308979.g012

4.2.2 Flow speed 17.6 m/s-case (b).

Fig 13 shows the results of the decreased laser intensity in the 17.6 m/s flow condition. The laser intensity decreased to 8.19 mW, exhibiting an error rate of 3.67% compared with the theoretical value. These results are consistent with the findings obtained in the static state and 5.5 m/s flow speed conditions.

https://doi.org/10.1371/journal.pone.0308979.g013

Based on the values calculated using Eq (3) , the decrease in the laser intensity due to scattering was determined by the path length ( L ). Consequently, it was concluded that the reduction in the laser intensity caused by water aerosols remained unaffected by the flow speed.

4.2.3 Distorted flow-case (c).

Fig 14 shows the change in laser intensity by scattering under distorted flow conditions achieved by inserting a distorted flow generator in the test section of the wind tunnel for the 17.6 m/s condition, which corresponds to case (c). The results indicated that a decrease in the laser intensity to 8.20 mW, resulting in an 18% reduction. The error rate was 3.80% compared to the theoretical value. Notably, the outcomes closely resemble those obtained in Fig 13 , which is case (b), where the flow speed remains the same, but the flow is steady.

https://doi.org/10.1371/journal.pone.0308979.g014

Table 2 summarizes the experimental results. The results demonstrate that under distorted flow conditions, the reduced laser intensity due to water aerosols remained nearly consistent with the outcomes of previous experiments. This emphasizes that the non-uniformity of the distorted flow does not significantly impact the scattering coefficient calculation, as described in Eq (3) , which shows that the sole variable affecting the change in laser intensity is the path length. Furthermore, as shown in Fig 1 , the high transmittance rate in the atmosphere of the 1.064 μ m wavelength laser shows that the uniformity of the flow does not influence the laser’s intensity change.

https://doi.org/10.1371/journal.pone.0308979.t002

5 Conclusions

In this study, a laser protection system concept was introduced. Scattering was selected as a new defence method because of its ability to protect an aircraft from lasers while ensuring the functionality of aircraft optics when employed under real-flight conditions. Specifically, Mie scattering was used to reduce the intensity of the laser, known to effectively decrease the power of a laser having a wavelength of 1.064 μ m used in actual high-powered lasers. Following the principles of small particle theory, water aerosols with a diameter ranging from 1 to 5 μ m were selected as the scattering particles to efficiently induce Mie scattering and thereby reduce the intensity of the incident laser.

Confirming the feasibility of the scattering protection system involved conducting a laser intensity decrease experiment in a static state. The experiment showed a decrease in laser intensity from 10 mW to 8.21 mW, indicating the potential for decreasing the 1.064 μ m laser intensity through scattering via water aerosols. In addition, the experimental results revealed a notable resemblance to the calculated theoretical values. To validate the accuracy of the theoretical calculations, rotational experiments were conducted, involving variations in the path length, indicating the trajectory of the laser crossing through the water aerosols. As the path length increased, the scattered laser amount also increased, leading to a greater reduction in laser intensity. This outcome affirmed the accuracy of calculations based on the modified Beer-Lambert law, which asserts that the path length is the sole variable of the laser intensity decrease. Moreover, an error rate of less than 6% was shown in each case when the path lengths were adjusted, leading to the conclusion that the calculation results were accurate.

Additionally, a wind tunnel was used to validate the capability of the system to operate under flow conditions. This was based on the consideration that in real-state conditions, an aircraft targeted by a high-powered laser experiences flow. Furthermore, a distorted flow generator was introduced into the tunnel to intentionally create a distorted flow and assess the resulting reduction in laser intensity. The experiments conducted at flow speeds of 5.5 m/s and 17.6 m/s showed values of 8.17 mW and 8.19 mW, respectively, indicating minimal variation. Furthermore, under distorted flow conditions, the degree of flow uniformity did not significantly impact the reduction in laser intensity. In the case of distorted flow in the 17.6 m/s flow case, the laser intensity decreased from 10 mW to 8.20 mW, showing results closely aligned with the previous experiments. Additionally, under both static state and flow conditions, the experimental values exhibited an error rate of less than 5% compared to the calculated values across all cases.

The static state and flow speeds of 5.5 m/s, 17.6 m/s, as well as distorted flow conditions, all exhibit a similar decrease in laser intensity. These results align with the calculations, which indicates that the parameter influencing the change in laser intensity is the path length ( L ). This demonstrates that even in different flow conditions the path length remains constant, leading to nearly identical reductions in laser intensity in each case.

These results indicate that the reduction in laser intensity through water aerosol spraying can be employed in diverse environments, at various air speeds, as well as under distorted flow conditions.

This study is a fundamental research focused on the novel laser scattering protection system concept that concerns the interaction between water aerosol and the laser. Several factors are good to be addressed in the future, such as the effects of vibrations, the challenges of miniaturizing the system for practical implementation and the effect from high-powered laser that is often the case in real scenarios. These aspects seem critical for developing a robust, realistic, and reliable protection system. Future research could address these critical areas to contribute to the design and development of a more effective and practical laser protection system.

Acknowledgments

The first author would like to thank Dr. Keunyeong Kim for providing valuable advice during the preparation of this paper.

• View Article
• Google Scholar
• 6. Weise TH, Jung M, Langhans D, Gowin M. Overview of directed energy weapon developments. In: 2004 12th Symposium on Electromagnetic Launch Technology. IEEE; 2004. p. 483–489.
• 8. Lasbury ME. The Realization of Star Trek Technologies: The Science, Not Fiction, Behind Brain Implants, Plasma Shields, Quantum Computing, and More. Springer; 2017.
• 10. West W. Absorption of electromagnetic radiation. AccessScience,© McGraw-Hill Companies; 2008.
• PubMed/NCBI
• 14. MILES R, FORKEY J, LEMPERT W. Filtered Rayleigh scattering measurements in supersonic/hypersonic facilities. In: 28th Joint Propulsion Conference and Exhibit; 1992. p. 3894.
• 17. Gonis A, Butler WH. Multiple scattering in solids. Springer Science & Business Media; 1999.
• 20. Stuke S. Characterizing thin clouds using aerosol optical depth information. PhD thesis, University of Innsbruck. 2016; p. 1–75.
• 21. Allegrini M, García N, Marti O. Nanometer scale science and technology. IOS press; 2001.
• 24. Weichel H. Laser beam propagation in the atmosphere. SPIE press; 1990.
• 25. Fingas M. Oil spill science and technology. Gulf professional publishing; 2016.
• 28. Myers D. Interfaces, and Colloids: Principles and Applications. VCH Publishers; 1991.
• 35. Prahl S. Mie Scattering Calculator; 2007 (accessed 17 January 2023). Available from: http://omlc.org/calc/mie_calc.html .
• 36. Bohren CF, Huffman DR. Absorption and scattering of light by small particles. John Wiley & Sons; 1983.
• 37. Born M, Wolf E. Principles of optics: electromagnetic theory of propagation, interference and diffraction of light. Elsevier; 2013.
• 39. Pastor A. Infrared guidance systems. A review of two man-portable defense applications. Universitat Oberta de Catalunya, Barcelona, Spain,. 2019. https://doi.org/10.31219/osf.io/c6gxf