complex problem solving unir

CONCEPTUAL ANALYSIS article

Complex problem solving: what it is and what it is not.

$\r\nDietrich Drner$

1 Department of Psychology, University of Bamberg, Bamberg, Germany
2 Department of Psychology, Heidelberg University, Heidelberg, Germany

Computer-simulated scenarios have been part of psychological research on problem solving for more than 40 years. The shift in emphasis from simple toy problems to complex, more real-life oriented problems has been accompanied by discussions about the best ways to assess the process of solving complex problems. Psychometric issues such as reliable assessments and addressing correlations with other instruments have been in the foreground of these discussions and have left the content validity of complex problem solving in the background. In this paper, we return the focus to content issues and address the important features that define complex problems.

Succeeding in the 21st century requires many competencies, including creativity, life-long learning, and collaboration skills (e.g., National Research Council, 2011 ; Griffin and Care, 2015 ), to name only a few. One competence that seems to be of central importance is the ability to solve complex problems ( Mainzer, 2009 ). Mainzer quotes the Nobel prize winner Simon (1957) who wrote as early as 1957:

The capacity of the human mind for formulating and solving complex problems is very small compared with the size of the problem whose solution is required for objectively rational behavior in the real world or even for a reasonable approximation to such objective rationality. (p. 198)

The shift from well-defined to ill-defined problems came about as a result of a disillusion with the “general problem solver” ( Newell et al., 1959 ): The general problem solver was a computer software intended to solve all kind of problems that can be expressed through well-formed formulas. However, it soon became clear that this procedure was in fact a “special problem solver” that could only solve well-defined problems in a closed space. But real-world problems feature open boundaries and have no well-determined solution. In fact, the world is full of wicked problems and clumsy solutions ( Verweij and Thompson, 2006 ). As a result, solving well-defined problems and solving ill-defined problems requires different cognitive processes ( Schraw et al., 1995 ; but see Funke, 2010 ).

Well-defined problems have a clear set of means for reaching a precisely described goal state. For example: in a match-stick arithmetic problem, a person receives a false arithmetic expression constructed out of matchsticks (e.g., IV = III + III). According to the instructions, moving one of the matchsticks will make the equations true. Here, both the problem (find the appropriate stick to move) and the goal state (true arithmetic expression; solution is: VI = III + III) are defined clearly.

Ill-defined problems have no clear problem definition, their goal state is not defined clearly, and the means of moving towards the (diffusely described) goal state are not clear. For example: The goal state for solving the political conflict in the near-east conflict between Israel and Palestine is not clearly defined (living in peaceful harmony with each other?) and even if the conflict parties would agree on a two-state solution, this goal again leaves many issues unresolved. This type of problem is called a “complex problem” and is of central importance to this paper. All psychological processes that occur within individual persons and deal with the handling of such ill-defined complex problems will be subsumed under the umbrella term “complex problem solving” (CPS).

Systematic research on CPS started in the 1970s with observations of the behavior of participants who were confronted with computer simulated microworlds. For example, in one of those microworlds participants assumed the role of executives who were tasked to manage a company over a certain period of time (see Brehmer and Dörner, 1993 , for a discussion of this methodology). Today, CPS is an established concept and has even influenced large-scale assessments such as PISA (“Programme for International Student Assessment”), organized by the Organization for Economic Cooperation and Development ( OECD, 2014 ). According to the World Economic Forum, CPS is one of the most important competencies required in the future ( World Economic Forum, 2015 ). Numerous articles on the subject have been published in recent years, documenting the increasing research activity relating to this field. In the following collection of papers we list only those published in 2010 and later: theoretical papers ( Blech and Funke, 2010 ; Funke, 2010 ; Knauff and Wolf, 2010 ; Leutner et al., 2012 ; Selten et al., 2012 ; Wüstenberg et al., 2012 ; Greiff et al., 2013b ; Fischer and Neubert, 2015 ; Schoppek and Fischer, 2015 ), papers about measurement issues ( Danner et al., 2011a ; Greiff et al., 2012 , 2015a ; Alison et al., 2013 ; Gobert et al., 2015 ; Greiff and Fischer, 2013 ; Herde et al., 2016 ; Stadler et al., 2016 ), papers about applications ( Fischer and Neubert, 2015 ; Ederer et al., 2016 ; Tremblay et al., 2017 ), papers about differential effects ( Barth and Funke, 2010 ; Danner et al., 2011b ; Beckmann and Goode, 2014 ; Greiff and Neubert, 2014 ; Scherer et al., 2015 ; Meißner et al., 2016 ; Wüstenberg et al., 2016 ), one paper about developmental effects ( Frischkorn et al., 2014 ), one paper with a neuroscience background ( Osman, 2012 ) 1 , papers about cultural differences ( Güss and Dörner, 2011 ; Sonnleitner et al., 2014 ; Güss et al., 2015 ), papers about validity issues ( Goode and Beckmann, 2010 ; Greiff et al., 2013c ; Schweizer et al., 2013 ; Mainert et al., 2015 ; Funke et al., 2017 ; Greiff et al., 2017 , 2015b ; Kretzschmar et al., 2016 ; Kretzschmar, 2017 ), review papers and meta-analyses ( Osman, 2010 ; Stadler et al., 2015 ), and finally books ( Qudrat-Ullah, 2015 ; Csapó and Funke, 2017b ) and book chapters ( Funke, 2012 ; Hotaling et al., 2015 ; Funke and Greiff, 2017 ; Greiff and Funke, 2017 ; Csapó and Funke, 2017a ; Fischer et al., 2017 ; Molnàr et al., 2017 ; Tobinski and Fritz, 2017 ; Viehrig et al., 2017 ). In addition, a new “Journal of Dynamic Decision Making” (JDDM) has been launched ( Fischer et al., 2015 , 2016 ) to give the field an open-access outlet for research and discussion.

This paper aims to clarify aspects of validity: what should be meant by the term CPS and what not? This clarification seems necessary because misunderstandings in recent publications provide – from our point of view – a potentially misleading picture of the construct. We start this article with a historical review before attempting to systematize different positions. We conclude with a working definition.

Historical Review

The concept behind CPS goes back to the German phrase “komplexes Problemlösen” (CPS; the term “komplexes Problemlösen” was used as a book title by Funke, 1986 ). The concept was introduced in Germany by Dörner and colleagues in the mid-1970s (see Dörner et al., 1975 ; Dörner, 1975 ) for the first time. The German phrase was later translated to CPS in the titles of two edited volumes by Sternberg and Frensch (1991) and Frensch and Funke (1995a) that collected papers from different research traditions. Even though it looks as though the term was coined in the 1970s, Edwards (1962) used the term “dynamic decision making” to describe decisions that come in a sequence. He compared static with dynamic decision making, writing:

In dynamic situations, a new complication not found in the static situations arises. The environment in which the decision is set may be changing, either as a function of the sequence of decisions, or independently of them, or both. It is this possibility of an environment which changes while you collect information about it which makes the task of dynamic decision theory so difficult and so much fun. (p. 60)

The ability to solve complex problems is typically measured via dynamic systems that contain several interrelated variables that participants need to alter. Early work (see, e.g., Dörner, 1980 ) used a simulation scenario called “Lohhausen” that contained more than 2000 variables that represented the activities of a small town: Participants had to take over the role of a mayor for a simulated period of 10 years. The simulation condensed these ten years to ten hours in real time. Later, researchers used smaller dynamic systems as scenarios either based on linear equations (see, e.g., Funke, 1993 ) or on finite state automata (see, e.g., Buchner and Funke, 1993 ). In these contexts, CPS consisted of the identification and control of dynamic task environments that were previously unknown to the participants. Different task environments came along with different degrees of fidelity ( Gray, 2002 ).

According to Funke (2012) , the typical attributes of complex systems are (a) complexity of the problem situation which is usually represented by the sheer number of involved variables; (b) connectivity and mutual dependencies between involved variables; (c) dynamics of the situation, which reflects the role of time and developments within a system; (d) intransparency (in part or full) about the involved variables and their current values; and (e) polytely (greek term for “many goals”), representing goal conflicts on different levels of analysis. This mixture of features is similar to what is called VUCA (volatility, uncertainty, complexity, ambiguity) in modern approaches to management (e.g., Mack et al., 2016 ).

In his evaluation of the CPS movement, Sternberg (1995) compared (young) European approaches to CPS with (older) American research on expertise. His analysis of the differences between the European and American traditions shows advantages but also potential drawbacks for each side. He states (p. 301): “I believe that although there are problems with the European approach, it deals with some fundamental questions that American research scarcely addresses.” So, even though the echo of the European approach did not enjoy strong resonance in the US at that time, it was valued by scholars like Sternberg and others. Before attending to validity issues, we will first present a short review of different streams.

Different Approaches to CPS

In the short history of CPS research, different approaches can be identified ( Buchner, 1995 ; Fischer et al., 2017 ). To systematize, we differentiate between the following five lines of research:

(a) The search for individual differences comprises studies identifying interindividual differences that affect the ability to solve complex problems. This line of research is reflected, for example, in the early work by Dörner et al. (1983) and their “Lohhausen” study. Here, naïve student participants took over the role of the mayor of a small simulated town named Lohhausen for a simulation period of ten years. According to the results of the authors, it is not intelligence (as measured by conventional IQ tests) that predicts performance, but it is the ability to stay calm in the face of a challenging situation and the ability to switch easily between an analytic mode of processing and a more holistic one.

(b) The search for cognitive processes deals with the processes behind understanding complex dynamic systems. Representative of this line of research is, for example, Berry and Broadbent’s (1984) work on implicit and explicit learning processes when people interact with a dynamic system called “Sugar Production”. They found that those who perform best in controlling a dynamic system can do so implicitly, without explicit knowledge of details regarding the systems’ relations.

(c) The search for system factors seeks to identify the aspects of dynamic systems that determine the difficulty of complex problems and make some problems harder than others. Representative of this line of research is, for example, work by Funke (1985) , who systematically varied the number of causal effects within a dynamic system or the presence/absence of eigendynamics. He found, for example, that solution quality decreases as the number of systems relations increases.

(d) The psychometric approach develops measurement instruments that can be used as an alternative to classical IQ tests, as something that goes “beyond IQ”. The MicroDYN approach ( Wüstenberg et al., 2012 ) is representative for this line of research that presents an alternative to reasoning tests (like Raven matrices). These authors demonstrated that a small improvement in predicting school grade point average beyond reasoning is possible with MicroDYN tests.

(e) The experimental approach explores CPS under different experimental conditions. This approach uses CPS assessment instruments to test hypotheses derived from psychological theories and is sometimes used in research about cognitive processes (see above). Exemplary for this line of research is the work by Rohe et al. (2016) , who test the usefulness of “motto goals” in the context of complex problems compared to more traditional learning and performance goals. Motto goals differ from pure performance goals by activating positive affect and should lead to better goal attainment especially in complex situations (the mentioned study found no effect).

To be clear: these five approaches are not mutually exclusive and do overlap. But the differentiation helps to identify different research communities and different traditions. These communities had different opinions about scaling complexity.

The Race for Complexity: Use of More and More Complex Systems

In the early years of CPS research, microworlds started with systems containing about 20 variables (“Tailorshop”), soon reached 60 variables (“Moro”), and culminated in systems with about 2000 variables (“Lohhausen”). This race for complexity ended with the introduction of the concept of “minimal complex systems” (MCS; Greiff and Funke, 2009 ; Funke and Greiff, 2017 ), which ushered in a search for the lower bound of complexity instead of the higher bound, which could not be defined as easily. The idea behind this concept was that whereas the upper limits of complexity are unbound, the lower limits might be identifiable. Imagine starting with a simple system containing two variables with a simple linear connection between them; then, step by step, increase the number of variables and/or the type of connections. One soon reaches a point where the system can no longer be considered simple and has become a “complex system”. This point represents a minimal complex system. Despite some research having been conducted in this direction, the point of transition from simple to complex has not been identified clearly as of yet.

Some years later, the original “minimal complex systems” approach ( Greiff and Funke, 2009 ) shifted to the “multiple complex systems” approach ( Greiff et al., 2013a ). This shift is more than a slight change in wording: it is important because it taps into the issue of validity directly. Minimal complex systems have been introduced in the context of challenges from large-scale assessments like PISA 2012 that measure new aspects of problem solving, namely interactive problems besides static problem solving ( Greiff and Funke, 2017 ). PISA 2012 required test developers to remain within testing time constraints (given by the school class schedule). Also, test developers needed a large item pool for the construction of a broad class of problem solving items. It was clear from the beginning that MCS deal with simple dynamic situations that require controlled interaction: the exploration and control of simple ticket machines, simple mobile phones, or simple MP3 players (all of these example domains were developed within PISA 2012) – rather than really complex situations like managerial or political decision making.

As a consequence of this subtle but important shift in interpreting the letters MCS, the definition of CPS became a subject of debate recently ( Funke, 2014a ; Greiff and Martin, 2014 ; Funke et al., 2017 ). In the words of Funke (2014b , p. 495):

It is funny that problems that nowadays come under the term ‘CPS’, are less complex (in terms of the previously described attributes of complex situations) than at the beginning of this new research tradition. The emphasis on psychometric qualities has led to a loss of variety. Systems thinking requires more than analyzing models with two or three linear equations – nonlinearity, cyclicity, rebound effects, etc. are inherent features of complex problems and should show up at least in some of the problems used for research and assessment purposes. Minimal complex systems run the danger of becoming minimal valid systems.

Searching for minimal complex systems is not the same as gaining insight into the way how humans deal with complexity and uncertainty. For psychometric purposes, it is appropriate to reduce complexity to a minimum; for understanding problem solving under conditions of overload, intransparency, and dynamics, it is necessary to realize those attributes with reasonable strength. This aspect is illustrated in the next section.

Importance of the Validity Issue

The most important reason for discussing the question of what complex problem solving is and what it is not stems from its phenomenology: if we lose sight of our phenomena, we are no longer doing good psychology. The relevant phenomena in the context of complex problems encompass many important aspects. In this section, we discuss four phenomena that are specific to complex problems. We consider these phenomena as critical for theory development and for the construction of assessment instruments (i.e., microworlds). These phenomena require theories for explaining them and they require assessment instruments eliciting them in a reliable way.

The first phenomenon is the emergency reaction of the intellectual system ( Dörner, 1980 ): When dealing with complex systems, actors tend to (a) reduce their intellectual level by decreasing self-reflections, by decreasing their intentions, by stereotyping, and by reducing their realization of intentions, (b) they show a tendency for fast action with increased readiness for risk, with increased violations of rules, and with increased tendency to escape the situation, and (c) they degenerate their hypotheses formation by construction of more global hypotheses and reduced tests of hypotheses, by increasing entrenchment, and by decontextualizing their goals. This phenomenon illustrates the strong connection between cognition, emotion, and motivation that has been emphasized by Dörner (see, e.g., Dörner and Güss, 2013 ) from the beginning of his research tradition; the emergency reaction reveals a shift in the mode of information processing under the pressure of complexity.

The second phenomenon comprises cross-cultural differences with respect to strategy use ( Strohschneider and Güss, 1999 ; Güss and Wiley, 2007 ; Güss et al., 2015 ). Results from complex task environments illustrate the strong influence of context and background knowledge to an extent that cannot be found for knowledge-poor problems. For example, in a comparison between Brazilian and German participants, it turned out that Brazilians accept the given problem descriptions and are more optimistic about the results of their efforts, whereas Germans tend to inquire more about the background of the problems and take a more active approach but are less optimistic (according to Strohschneider and Güss, 1998 , p. 695).

The third phenomenon relates to failures that occur during the planning and acting stages ( Jansson, 1994 ; Ramnarayan et al., 1997 ), illustrating that rational procedures seem to be unlikely to be used in complex situations. The potential for failures ( Dörner, 1996 ) rises with the complexity of the problem. Jansson (1994) presents seven major areas for failures with complex situations: acting directly on current feedback; insufficient systematization; insufficient control of hypotheses and strategies; lack of self-reflection; selective information gathering; selective decision making; and thematic vagabonding.

The fourth phenomenon describes (a lack of) training and transfer effects ( Kretzschmar and Süß, 2015 ), which again illustrates the context dependency of strategies and knowledge (i.e., there is no strategy that is so universal that it can be used in many different problem situations). In their own experiment, the authors could show training effects only for knowledge acquisition, not for knowledge application. Only with specific feedback, performance in complex environments can be increased ( Engelhart et al., 2017 ).

These four phenomena illustrate why the type of complexity (or degree of simplicity) used in research really matters. Furthermore, they demonstrate effects that are specific for complex problems, but not for toy problems. These phenomena direct the attention to the important question: does the stimulus material used (i.e., the computer-simulated microworld) tap and elicit the manifold of phenomena described above?

Dealing with partly unknown complex systems requires courage, wisdom, knowledge, grit, and creativity. In creativity research, “little c” and “BIG C” are used to differentiate between everyday creativity and eminent creativity ( Beghetto and Kaufman, 2007 ; Kaufman and Beghetto, 2009 ). Everyday creativity is important for solving everyday problems (e.g., finding a clever fix for a broken spoke on my bicycle), eminent creativity changes the world (e.g., inventing solar cells for energy production). Maybe problem solving research should use a similar differentiation between “little p” and “BIG P” to mark toy problems on the one side and big societal challenges on the other. The question then remains: what can we learn about BIG P by studying little p? What phenomena are present in both types, and what phenomena are unique to each of the two extremes?

Discussing research on CPS requires reflecting on the field’s research methods. Even if the experimental approach has been successful for testing hypotheses (for an overview of older work, see Funke, 1995 ), other methods might provide additional and novel insights. Complex phenomena require complex approaches to understand them. The complex nature of complex systems imposes limitations on psychological experiments: The more complex the environments, the more difficult is it to keep conditions under experimental control. And if experiments have to be run in labs one should bring enough complexity into the lab to establish the phenomena mentioned, at least in part.

There are interesting options to be explored (again): think-aloud protocols , which have been discredited for many years ( Nisbett and Wilson, 1977 ) and yet are a valuable source for theory testing ( Ericsson and Simon, 1983 ); introspection ( Jäkel and Schreiber, 2013 ), which seems to be banned from psychological methods but nevertheless offers insights into thought processes; the use of life-streaming ( Wendt, 2017 ), a medium in which streamers generate a video stream of think-aloud data in computer-gaming; political decision-making ( Dhami et al., 2015 ) that demonstrates error-proneness in groups; historical case studies ( Dörner and Güss, 2011 ) that give insights into the thinking styles of political leaders; the use of the critical incident technique ( Reuschenbach, 2008 ) to construct complex scenarios; and simulations with different degrees of fidelity ( Gray, 2002 ).

The methods tool box is full of instruments that have to be explored more carefully before any individual instrument receives a ban or research narrows its focus to only one paradigm for data collection. Brehmer and Dörner (1993) discussed the tensions between “research in the laboratory and research in the field”, optimistically concluding “that the new methodology of computer-simulated microworlds will provide us with the means to bridge the gap between the laboratory and the field” (p. 183). The idea behind this optimism was that computer-simulated scenarios would bring more complexity from the outside world into the controlled lab environment. But this is not true for all simulated scenarios. In his paper on simulated environments, Gray (2002) differentiated computer-simulated environments with respect to three dimensions: (1) tractability (“the more training subjects require before they can use a simulated task environment, the less tractable it is”, p. 211), correspondence (“High correspondence simulated task environments simulate many aspects of one task environment. Low correspondence simulated task environments simulate one aspect of many task environments”, p. 214), and engagement (“A simulated task environment is engaging to the degree to which it involves and occupies the participants; that is, the degree to which they agree to take it seriously”, p. 217). But the mere fact that a task is called a “computer-simulated task environment” does not mean anything specific in terms of these three dimensions. This is one of several reasons why we should differentiate between those studies that do not address the core features of CPS and those that do.

What is not CPS?

Even though a growing number of references claiming to deal with complex problems exist (e.g., Greiff and Wüstenberg, 2015 ; Greiff et al., 2016 ), it would be better to label the requirements within these tasks “dynamic problem solving,” as it has been done adequately in earlier work ( Greiff et al., 2012 ). The dynamics behind on-off-switches ( Thimbleby, 2007 ) are remarkable but not really complex. Small nonlinear systems that exhibit stunningly complex and unstable behavior do exist – but they are not used in psychometric assessments of so-called CPS. There are other small systems (like MicroDYN scenarios: Greiff and Wüstenberg, 2014 ) that exhibit simple forms of system behavior that are completely predictable and stable. This type of simple systems is used frequently. It is even offered commercially as a complex problem-solving test called COMPRO ( Greiff and Wüstenberg, 2015 ) for business applications. But a closer look reveals that the label is not used correctly; within COMPRO, the used linear equations are far from being complex and the system can be handled properly by using only one strategy (see for more details Funke et al., 2017 ).

Why do simple linear systems not fall within CPS? At the surface, nonlinear and linear systems might appear similar because both only include 3–5 variables. But the difference is in terms of systems behavior as well as strategies and learning. If the behavior is simple (as in linear systems where more input is related to more output and vice versa), the system can be easily understood (participants in the MicroDYN world have 3 minutes to explore a complex system). If the behavior is complex (as in systems that contain strange attractors or negative feedback loops), things become more complicated and much more observation is needed to identify the hidden structure of the unknown system ( Berry and Broadbent, 1984 ; Hundertmark et al., 2015 ).

Another issue is learning. If tasks can be solved using a single (and not so complicated) strategy, steep learning curves are to be expected. The shift from problem solving to learned routine behavior occurs rapidly, as was demonstrated by Luchins (1942) . In his water jar experiments, participants quickly acquired a specific strategy (a mental set) for solving certain measurement problems that they later continued applying to problems that would have allowed for easier approaches. In the case of complex systems, learning can occur only on very general, abstract levels because it is difficult for human observers to make specific predictions. Routines dealing with complex systems are quite different from routines relating to linear systems.

What should not be studied under the label of CPS are pure learning effects, multiple-cue probability learning, or tasks that can be solved using a single strategy. This last issue is a problem for MicroDYN tasks that rely strongly on the VOTAT strategy (“vary one thing at a time”; see Tschirgi, 1980 ). In real-life, it is hard to imagine a business manager trying to solve her or his problems by means of VOTAT.

What is CPS?

In the early days of CPS research, planet Earth’s dynamics and complexities gained attention through such books as “The limits to growth” ( Meadows et al., 1972 ) and “Beyond the limits” ( Meadows et al., 1992 ). In the current decade, for example, the World Economic Forum (2016) attempts to identify the complexities and risks of our modern world. In order to understand the meaning of complexity and uncertainty, taking a look at the worlds’ most pressing issues is helpful. Searching for strategies to cope with these problems is a difficult task: surely there is no place for the simple principle of “vary-one-thing-at-a-time” (VOTAT) when it comes to global problems. The VOTAT strategy is helpful in the context of simple problems ( Wüstenberg et al., 2014 ); therefore, whether or not VOTAT is helpful in a given problem situation helps us distinguish simple from complex problems.

Because there exist no clear-cut strategies for complex problems, typical failures occur when dealing with uncertainty ( Dörner, 1996 ; Güss et al., 2015 ). Ramnarayan et al. (1997) put together a list of generic errors (e.g., not developing adequate action plans; lack of background control; learning from experience blocked by stereotype knowledge; reactive instead of proactive action) that are typical of knowledge-rich complex systems but cannot be found in simple problems.

Complex problem solving is not a one-dimensional, low-level construct. On the contrary, CPS is a multi-dimensional bundle of competencies existing at a high level of abstraction, similar to intelligence (but going beyond IQ). As Funke et al. (2018) state: “Assessment of transversal (in educational contexts: cross-curricular) competencies cannot be done with one or two types of assessment. The plurality of skills and competencies requires a plurality of assessment instruments.”

There are at least three different aspects of complex systems that are part of our understanding of a complex system: (1) a complex system can be described at different levels of abstraction; (2) a complex system develops over time, has a history, a current state, and a (potentially unpredictable) future; (3) a complex system is knowledge-rich and activates a large semantic network, together with a broad list of potential strategies (domain-specific as well as domain-general).

Complex problem solving is not only a cognitive process but is also an emotional one ( Spering et al., 2005 ; Barth and Funke, 2010 ) and strongly dependent on motivation (low-stakes versus high-stakes testing; see Hermes and Stelling, 2016 ).

Furthermore, CPS is a dynamic process unfolding over time, with different phases and with more differentiation than simply knowledge acquisition and knowledge application. Ideally, the process should entail identifying problems (see Dillon, 1982 ; Lee and Cho, 2007 ), even if in experimental settings, problems are provided to participants a priori . The more complex and open a given situation, the more options can be generated (T. S. Schweizer et al., 2016 ). In closed problems, these processes do not occur in the same way.

In analogy to the difference between formative (process-oriented) and summative (result-oriented) assessment ( Wiliam and Black, 1996 ; Bennett, 2011 ), CPS should not be reduced to the mere outcome of a solution process. The process leading up to the solution, including detours and errors made along the way, might provide a more differentiated impression of a person’s problem-solving abilities and competencies than the final result of such a process. This is one of the reasons why CPS environments are not, in fact, complex intelligence tests: research on CPS is not only about the outcome of the decision process, but it is also about the problem-solving process itself.

Complex problem solving is part of our daily life: finding the right person to share one’s life with, choosing a career that not only makes money, but that also makes us happy. Of course, CPS is not restricted to personal problems – life on Earth gives us many hard nuts to crack: climate change, population growth, the threat of war, the use and distribution of natural resources. In sum, many societal challenges can be seen as complex problems. To reduce that complexity to a one-hour lab activity on a random Friday afternoon puts it out of context and does not address CPS issues.

Theories about CPS should specify which populations they apply to. Across populations, one thing to consider is prior knowledge. CPS research with experts (e.g., Dew et al., 2009 ) is quite different from problem solving research using tasks that intentionally do not require any specific prior knowledge (see, e.g., Beckmann and Goode, 2014 ).

More than 20 years ago, Frensch and Funke (1995b) defined CPS as follows:

CPS occurs to overcome barriers between a given state and a desired goal state by means of behavioral and/or cognitive, multi-step activities. The given state, goal state, and barriers between given state and goal state are complex, change dynamically during problem solving, and are intransparent. The exact properties of the given state, goal state, and barriers are unknown to the solver at the outset. CPS implies the efficient interaction between a solver and the situational requirements of the task, and involves a solver’s cognitive, emotional, personal, and social abilities and knowledge. (p. 18)

The above definition is rather formal and does not account for content or relations between the simulation and the real world. In a sense, we need a new definition of CPS that addresses these issues. Based on our previous arguments, we propose the following working definition:

Complex problem solving is a collection of self-regulated psychological processes and activities necessary in dynamic environments to achieve ill-defined goals that cannot be reached by routine actions. Creative combinations of knowledge and a broad set of strategies are needed. Solutions are often more bricolage than perfect or optimal. The problem-solving process combines cognitive, emotional, and motivational aspects, particularly in high-stakes situations. Complex problems usually involve knowledge-rich requirements and collaboration among different persons.

The main differences to the older definition lie in the emphasis on (a) the self-regulation of processes, (b) creativity (as opposed to routine behavior), (c) the bricolage type of solution, and (d) the role of high-stakes challenges. Our new definition incorporates some aspects that have been discussed in this review but were not reflected in the 1995 definition, which focused on attributes of complex problems like dynamics or intransparency.

This leads us to the final reflection about the role of CPS for dealing with uncertainty and complexity in real life. We will distinguish thinking from reasoning and introduce the sense of possibility as an important aspect of validity.

CPS as Combining Reasoning and Thinking in an Uncertain Reality

Leading up to the Battle of Borodino in Leo Tolstoy’s novel “War and Peace”, Prince Andrei Bolkonsky explains the concept of war to his friend Pierre. Pierre expects war to resemble a game of chess: You position the troops and attempt to defeat your opponent by moving them in different directions.

“Far from it!”, Andrei responds. “In chess, you know the knight and his moves, you know the pawn and his combat strength. While in war, a battalion is sometimes stronger than a division and sometimes weaker than a company; it all depends on circumstances that can never be known. In war, you do not know the position of your enemy; some things you might be able to observe, some things you have to divine (but that depends on your ability to do so!) and many things cannot even be guessed at. In chess, you can see all of your opponent’s possible moves. In war, that is impossible. If you decide to attack, you cannot know whether the necessary conditions are met for you to succeed. Many a time, you cannot even know whether your troops will follow your orders…”

In essence, war is characterized by a high degree of uncertainty. A good commander (or politician) can add to that what he or she sees, tentatively fill in the blanks – and not just by means of logical deduction but also by intelligently bridging missing links. A bad commander extrapolates from what he sees and thus arrives at improper conclusions.

Many languages differentiate between two modes of mentalizing; for instance, the English language distinguishes between ‘thinking’ and ‘reasoning’. Reasoning denotes acute and exact mentalizing involving logical deductions. Such deductions are usually based on evidence and counterevidence. Thinking, however, is what is required to write novels. It is the construction of an initially unknown reality. But it is not a pipe dream, an unfounded process of fabrication. Rather, thinking asks us to imagine reality (“Wirklichkeitsfantasie”). In other words, a novelist has to possess a “sense of possibility” (“Möglichkeitssinn”, Robert Musil; in German, sense of possibility is often used synonymously with imagination even though imagination is not the same as sense of possibility, for imagination also encapsulates the impossible). This sense of possibility entails knowing the whole (or several wholes) or being able to construe an unknown whole that could accommodate a known part. The whole has to align with sociological and geographical givens, with the mentality of certain peoples or groups, and with the laws of physics and chemistry. Otherwise, the entire venture is ill-founded. A sense of possibility does not aim for the moon but imagines something that might be possible but has not been considered possible or even potentially possible so far.

Thinking is a means to eliminate uncertainty. This process requires both of the modes of thinking we have discussed thus far. Economic, political, or ecological decisions require us to first consider the situation at hand. Though certain situational aspects can be known, but many cannot. In fact, von Clausewitz (1832) posits that only about 25% of the necessary information is available when a military decision needs to be made. Even then, there is no way to guarantee that whatever information is available is also correct: Even if a piece of information was completely accurate yesterday, it might no longer apply today.

Once our sense of possibility has helped grasping a situation, problem solvers need to call on their reasoning skills. Not every situation requires the same action, and we may want to act this way or another to reach this or that goal. This appears logical, but it is a logic based on constantly shifting grounds: We cannot know whether necessary conditions are met, sometimes the assumptions we have made later turn out to be incorrect, and sometimes we have to revise our assumptions or make completely new ones. It is necessary to constantly switch between our sense of possibility and our sense of reality, that is, to switch between thinking and reasoning. It is an arduous process, and some people handle it well, while others do not.

If we are to believe Tuchman’s (1984) book, “The March of Folly”, most politicians and commanders are fools. According to Tuchman, not much has changed in the 3300 years that have elapsed since the misguided Trojans decided to welcome the left-behind wooden horse into their city that would end up dismantling Troy’s defensive walls. The Trojans, too, had been warned, but decided not to heed the warning. Although Laocoön had revealed the horse’s true nature to them by attacking it with a spear, making the weapons inside the horse ring, the Trojans refused to see the forest for the trees. They did not want to listen, they wanted the war to be over, and this desire ended up shaping their perception.

The objective of psychology is to predict and explain human actions and behavior as accurately as possible. However, thinking cannot be investigated by limiting its study to neatly confined fractions of reality such as the realms of propositional logic, chess, Go tasks, the Tower of Hanoi, and so forth. Within these systems, there is little need for a sense of possibility. But a sense of possibility – the ability to divine and construe an unknown reality – is at least as important as logical reasoning skills. Not researching the sense of possibility limits the validity of psychological research. All economic and political decision making draws upon this sense of possibility. By not exploring it, psychological research dedicated to the study of thinking cannot further the understanding of politicians’ competence and the reasons that underlie political mistakes. Christopher Clark identifies European diplomats’, politicians’, and commanders’ inability to form an accurate representation of reality as a reason for the outbreak of World War I. According to Clark’s (2012) book, “The Sleepwalkers”, the politicians of the time lived in their own make-believe world, wrongfully assuming that it was the same world everyone else inhabited. If CPS research wants to make significant contributions to the world, it has to acknowledge complexity and uncertainty as important aspects of it.

For more than 40 years, CPS has been a new subject of psychological research. During this time period, the initial emphasis on analyzing how humans deal with complex, dynamic, and uncertain situations has been lost. What is subsumed under the heading of CPS in modern research has lost the original complexities of real-life problems. From our point of view, the challenges of the 21st century require a return to the origins of this research tradition. We would encourage researchers in the field of problem solving to come back to the original ideas. There is enough complexity and uncertainty in the world to be studied. Improving our understanding of how humans deal with these global and pressing problems would be a worthwhile enterprise.

Author Contributions

JF drafted a first version of the manuscript, DD added further text and commented on the draft. JF finalized the manuscript.

Authors Note

After more than 40 years of controversial discussions between both authors, this is the first joint paper. We are happy to have done this now! We have found common ground!

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors thank the Deutsche Forschungsgemeinschaft (DFG) for the continuous support of their research over many years. Thanks to Daniel Holt for his comments on validity issues, thanks to Julia Nolte who helped us by translating German text excerpts into readable English and helped us, together with Keri Hartman, to improve our style and grammar – thanks for that! We also thank the two reviewers for their helpful critical comments on earlier versions of this manuscript. Finally, we acknowledge financial support by Deutsche Forschungsgemeinschaft and Ruprecht-Karls-Universität Heidelberg within their funding programme Open Access Publishing .

^ The fMRI-paper from Anderson (2012) uses the term “complex problem solving” for tasks that do not fall in our understanding of CPS and is therefore excluded from this list.

Alison, L., van den Heuvel, C., Waring, S., Power, N., Long, A., O’Hara, T., et al. (2013). Immersive simulated learning environments for researching critical incidents: a knowledge synthesis of the literature and experiences of studying high-risk strategic decision making. J. Cogn. Eng. Deci. Mak. 7, 255–272. doi: 10.1177/1555343412468113

CrossRef Full Text | Google Scholar

Anderson, J. R. (2012). Tracking problem solving by multivariate pattern analysis and hidden markov model algorithms. Neuropsychologia 50, 487–498. doi: 10.1016/j.neuropsychologia.2011.07.025

PubMed Abstract | CrossRef Full Text | Google Scholar

Barth, C. M., and Funke, J. (2010). Negative affective environments improve complex solving performance. Cogn. Emot. 24, 1259–1268. doi: 10.1080/02699930903223766

Beckmann, J. F., and Goode, N. (2014). The benefit of being naïve and knowing it: the unfavourable impact of perceived context familiarity on learning in complex problem solving tasks. Instruct. Sci. 42, 271–290. doi: 10.1007/s11251-013-9280-7

Beghetto, R. A., and Kaufman, J. C. (2007). Toward a broader conception of creativity: a case for “mini-c” creativity. Psychol. Aesthetics Creat. Arts 1, 73–79. doi: 10.1037/1931-3896.1.2.73

Bennett, R. E. (2011). Formative assessment: a critical review. Assess. Educ. Princ. Policy Pract. 18, 5–25. doi: 10.1080/0969594X.2010.513678

Berry, D. C., and Broadbent, D. E. (1984). On the relationship between task performance and associated verbalizable knowledge. Q. J. Exp. Psychol. 36, 209–231. doi: 10.1080/14640748408402156

Blech, C., and Funke, J. (2010). You cannot have your cake and eat it, too: how induced goal conflicts affect complex problem solving. Open Psychol. J. 3, 42–53. doi: 10.2174/1874350101003010042

Brehmer, B., and Dörner, D. (1993). Experiments with computer-simulated microworlds: escaping both the narrow straits of the laboratory and the deep blue sea of the field study. Comput. Hum. Behav. 9, 171–184. doi: 10.1016/0747-5632(93)90005-D

Buchner, A. (1995). “Basic topics and approaches to the study of complex problem solving,” in Complex Problem Solving: The European Perspective , eds P. A. Frensch and J. Funke (Hillsdale, NJ: Erlbaum), 27–63.

Google Scholar

Buchner, A., and Funke, J. (1993). Finite state automata: dynamic task environments in problem solving research. Q. J. Exp. Psychol. 46A, 83–118. doi: 10.1080/14640749308401068

Clark, C. (2012). The Sleepwalkers: How Europe Went to War in 1914 . London: Allen Lane.

Csapó, B., and Funke, J. (2017a). “The development and assessment of problem solving in 21st-century schools,” in The Nature of Problem Solving: Using Research to Inspire 21st Century Learning , eds B. Csapó and J. Funke (Paris: OECD Publishing), 19–31.

Csapó, B., and Funke, J. (eds) (2017b). The Nature of Problem Solving. Using Research to Inspire 21st Century Learning. Paris: OECD Publishing.

Danner, D., Hagemann, D., Holt, D. V., Hager, M., Schankin, A., Wüstenberg, S., et al. (2011a). Measuring performance in dynamic decision making. Reliability and validity of the Tailorshop simulation. J. Ind. Differ. 32, 225–233. doi: 10.1027/1614-0001/a000055

CrossRef Full Text

Danner, D., Hagemann, D., Schankin, A., Hager, M., and Funke, J. (2011b). Beyond IQ: a latent state-trait analysis of general intelligence, dynamic decision making, and implicit learning. Intelligence 39, 323–334. doi: 10.1016/j.intell.2011.06.004

Dew, N., Read, S., Sarasvathy, S. D., and Wiltbank, R. (2009). Effectual versus predictive logics in entrepreneurial decision-making: differences between experts and novices. J. Bus. Ventur. 24, 287–309. doi: 10.1016/j.jbusvent.2008.02.002

Dhami, M. K., Mandel, D. R., Mellers, B. A., and Tetlock, P. E. (2015). Improving intelligence analysis with decision science. Perspect. Psychol. Sci. 10, 753–757. doi: 10.1177/1745691615598511

Dillon, J. T. (1982). Problem finding and solving. J. Creat. Behav. 16, 97–111. doi: 10.1002/j.2162-6057.1982.tb00326.x

Dörner, D. (1975). Wie Menschen eine Welt verbessern wollten [How people wanted to improve a world]. Bild Der Wissenschaft 12, 48–53.

Dörner, D. (1980). On the difficulties people have in dealing with complexity. Simulat. Gam. 11, 87–106. doi: 10.1177/104687818001100108

Dörner, D. (1996). The Logic of Failure: Recognizing and Avoiding Error in Complex Situations. New York, NY: Basic Books.

Dörner, D., Drewes, U., and Reither, F. (1975). “Über das Problemlösen in sehr komplexen Realitätsbereichen,” in Bericht über den 29. Kongreß der DGfPs in Salzburg 1974, Band 1 , ed. W. H. Tack (Göttingen: Hogrefe), 339–340.

Dörner, D., and Güss, C. D. (2011). A psychological analysis of Adolf Hitler’s decision making as commander in chief: summa confidentia et nimius metus. Rev. Gen. Psychol. 15, 37–49. doi: 10.1037/a0022375

Dörner, D., and Güss, C. D. (2013). PSI: a computational architecture of cognition, motivation, and emotion. Rev. Gen. Psychol. 17, 297–317. doi: 10.1037/a0032947

Dörner, D., Kreuzig, H. W., Reither, F., and Stäudel, T. (1983). Lohhausen. Vom Umgang mit Unbestimmtheit und Komplexität. Bern: Huber.

Ederer, P., Patt, A., and Greiff, S. (2016). Complex problem-solving skills and innovativeness – evidence from occupational testing and regional data. Eur. J. Educ. 51, 244–256. doi: 10.1111/ejed.12176

Edwards, W. (1962). Dynamic decision theory and probabiIistic information processing. Hum. Factors 4, 59–73. doi: 10.1177/001872086200400201

Engelhart, M., Funke, J., and Sager, S. (2017). A web-based feedback study on optimization-based training and analysis of human decision making. J. Dynamic Dec. Mak. 3, 1–23.

Ericsson, K. A., and Simon, H. A. (1983). Protocol Analysis: Verbal Reports As Data. Cambridge, MA: Bradford.

Fischer, A., Greiff, S., and Funke, J. (2017). “The history of complex problem solving,” in The Nature of Problem Solving: Using Research to Inspire 21st Century Learning , eds B. Csapó and J. Funke (Paris: OECD Publishing), 107–121.

Fischer, A., Holt, D. V., and Funke, J. (2015). Promoting the growing field of dynamic decision making. J. Dynamic Decis. Mak. 1, 1–3. doi: 10.11588/jddm.2015.1.23807

Fischer, A., Holt, D. V., and Funke, J. (2016). The first year of the “journal of dynamic decision making.” J. Dynamic Decis. Mak. 2, 1–2. doi: 10.11588/jddm.2016.1.28995

Fischer, A., and Neubert, J. C. (2015). The multiple faces of complex problems: a model of problem solving competency and its implications for training and assessment. J. Dynamic Decis. Mak. 1, 1–14. doi: 10.11588/jddm.2015.1.23945

Frensch, P. A., and Funke, J. (eds) (1995a). Complex Problem Solving: The European Perspective. Hillsdale, NJ: Erlbaum.

Frensch, P. A., and Funke, J. (1995b). “Definitions, traditions, and a general framework for understanding complex problem solving,” in Complex Problem Solving: The European Perspective , eds P. A. Frensch and J. Funke (Hillsdale, NJ: Lawrence Erlbaum), 3–25.

Frischkorn, G. T., Greiff, S., and Wüstenberg, S. (2014). The development of complex problem solving in adolescence: a latent growth curve analysis. J. Educ. Psychol. 106, 1004–1020. doi: 10.1037/a0037114

Funke, J. (1985). Steuerung dynamischer Systeme durch Aufbau und Anwendung subjektiver Kausalmodelle. Z. Psychol. 193, 435–457.

Funke, J. (1986). Komplexes Problemlösen - Bestandsaufnahme und Perspektiven [Complex Problem Solving: Survey and Perspectives]. Heidelberg: Springer.

Funke, J. (1993). “Microworlds based on linear equation systems: a new approach to complex problem solving and experimental results,” in The Cognitive Psychology of Knowledge , eds G. Strube and K.-F. Wender (Amsterdam: Elsevier Science Publishers), 313–330.

Funke, J. (1995). “Experimental research on complex problem solving,” in Complex Problem Solving: The European Perspective , eds P. A. Frensch and J. Funke (Hillsdale, NJ: Erlbaum), 243–268.

Funke, J. (2010). Complex problem solving: a case for complex cognition? Cogn. Process. 11, 133–142. doi: 10.1007/s10339-009-0345-0

Funke, J. (2012). “Complex problem solving,” in Encyclopedia of the Sciences of Learning , Vol. 38, ed. N. M. Seel (Heidelberg: Springer), 682–685.

Funke, J. (2014a). Analysis of minimal complex systems and complex problem solving require different forms of causal cognition. Front. Psychol. 5:739. doi: 10.3389/fpsyg.2014.00739

Funke, J. (2014b). “Problem solving: what are the important questions?,” in Proceedings of the 36th Annual Conference of the Cognitive Science Society , eds P. Bello, M. Guarini, M. McShane, and B. Scassellati (Austin, TX: Cognitive Science Society), 493–498.

Funke, J., Fischer, A., and Holt, D. V. (2017). When less is less: solving multiple simple problems is not complex problem solving—A comment on Greiff et al. (2015). J. Intell. 5:5. doi: 10.3390/jintelligence5010005

Funke, J., Fischer, A., and Holt, D. V. (2018). “Competencies for complexity: problem solving in the 21st century,” in Assessment and Teaching of 21st Century Skills , eds E. Care, P. Griffin, and M. Wilson (Dordrecht: Springer), 3.

Funke, J., and Greiff, S. (2017). “Dynamic problem solving: multiple-item testing based on minimally complex systems,” in Competence Assessment in Education. Research, Models and Instruments , eds D. Leutner, J. Fleischer, J. Grünkorn, and E. Klieme (Heidelberg: Springer), 427–443.

Gobert, J. D., Kim, Y. J., Pedro, M. A. S., Kennedy, M., and Betts, C. G. (2015). Using educational data mining to assess students’ skills at designing and conducting experiments within a complex systems microworld. Think. Skills Creat. 18, 81–90. doi: 10.1016/j.tsc.2015.04.008

Goode, N., and Beckmann, J. F. (2010). You need to know: there is a causal relationship between structural knowledge and control performance in complex problem solving tasks. Intelligence 38, 345–352. doi: 10.1016/j.intell.2010.01.001

Gray, W. D. (2002). Simulated task environments: the role of high-fidelity simulations, scaled worlds, synthetic environments, and laboratory tasks in basic and applied cognitive research. Cogn. Sci. Q. 2, 205–227.

Greiff, S., and Fischer, A. (2013). Measuring complex problem solving: an educational application of psychological theories. J. Educ. Res. 5, 38–58.

Greiff, S., Fischer, A., Stadler, M., and Wüstenberg, S. (2015a). Assessing complex problem-solving skills with multiple complex systems. Think. Reason. 21, 356–382. doi: 10.1080/13546783.2014.989263

Greiff, S., Stadler, M., Sonnleitner, P., Wolff, C., and Martin, R. (2015b). Sometimes less is more: comparing the validity of complex problem solving measures. Intelligence 50, 100–113. doi: 10.1016/j.intell.2015.02.007

Greiff, S., Fischer, A., Wüstenberg, S., Sonnleitner, P., Brunner, M., and Martin, R. (2013a). A multitrait–multimethod study of assessment instruments for complex problem solving. Intelligence 41, 579–596. doi: 10.1016/j.intell.2013.07.012

Greiff, S., Holt, D. V., and Funke, J. (2013b). Perspectives on problem solving in educational assessment: analytical, interactive, and collaborative problem solving. J. Problem Solv. 5, 71–91. doi: 10.7771/1932-6246.1153

Greiff, S., Wüstenberg, S., Molnár, G., Fischer, A., Funke, J., and Csapó, B. (2013c). Complex problem solving in educational contexts—something beyond g: concept, assessment, measurement invariance, and construct validity. J. Educ. Psychol. 105, 364–379. doi: 10.1037/a0031856

Greiff, S., and Funke, J. (2009). “Measuring complex problem solving: the MicroDYN approach,” in The Transition to Computer-Based Assessment. New Approaches to Skills Assessment and Implications for Large-Scale Testing , eds F. Scheuermann and J. Björnsson (Luxembourg: Office for Official Publications of the European Communities), 157–163.

Greiff, S., and Funke, J. (2017). “Interactive problem solving: exploring the potential of minimal complex systems,” in The Nature of Problem Solving: Using Research to Inspire 21st Century Learning , eds B. Csapó and J. Funke (Paris: OECD Publishing), 93–105.

Greiff, S., and Martin, R. (2014). What you see is what you (don’t) get: a comment on Funke’s (2014) opinion paper. Front. Psychol. 5:1120. doi: 10.3389/fpsyg.2014.01120

Greiff, S., and Neubert, J. C. (2014). On the relation of complex problem solving, personality, fluid intelligence, and academic achievement. Learn. Ind. Diff. 36, 37–48. doi: 10.1016/j.lindif.2014.08.003

Greiff, S., Niepel, C., Scherer, R., and Martin, R. (2016). Understanding students’ performance in a computer-based assessment of complex problem solving: an analysis of behavioral data from computer-generated log files. Comput. Hum. Behav. 61, 36–46. doi: 10.1016/j.chb.2016.02.095

Greiff, S., Stadler, M., Sonnleitner, P., Wolff, C., and Martin, R. (2017). Sometimes more is too much: a rejoinder to the commentaries on Greiff et al. (2015). J. Intell. 5:6. doi: 10.3390/jintelligence5010006

Greiff, S., and Wüstenberg, S. (2014). Assessment with microworlds using MicroDYN: measurement invariance and latent mean comparisons. Eur. J. Psychol. Assess. 1, 1–11. doi: 10.1027/1015-5759/a000194

Greiff, S., and Wüstenberg, S. (2015). Komplexer Problemlösetest COMPRO [Complex Problem-Solving Test COMPRO]. Mödling: Schuhfried.

Greiff, S., Wüstenberg, S., and Funke, J. (2012). Dynamic problem solving: a new assessment perspective. Appl. Psychol. Measure. 36, 189–213. doi: 10.1177/0146621612439620

Griffin, P., and Care, E. (2015). “The ATC21S method,” in Assessment and Taching of 21st Century Skills , eds P. Griffin and E. Care (Dordrecht, NL: Springer), 3–33.

Güss, C. D., and Dörner, D. (2011). Cultural differences in dynamic decision-making strategies in a non-linear, time-delayed task. Cogn. Syst. Res. 12, 365–376. doi: 10.1016/j.cogsys.2010.12.003

Güss, C. D., Tuason, M. T., and Orduña, L. V. (2015). Strategies, tactics, and errors in dynamic decision making in an Asian sample. J. Dynamic Deci. Mak. 1, 1–14. doi: 10.11588/jddm.2015.1.13131

Güss, C. D., and Wiley, B. (2007). Metacognition of problem-solving strategies in Brazil, India, and the United States. J. Cogn. Cult. 7, 1–25. doi: 10.1163/156853707X171793

Herde, C. N., Wüstenberg, S., and Greiff, S. (2016). Assessment of complex problem solving: what we know and what we don’t know. Appl. Meas. Educ. 29, 265–277. doi: 10.1080/08957347.2016.1209208

Hermes, M., and Stelling, D. (2016). Context matters, but how much? Latent state – trait analysis of cognitive ability assessments. Int. J. Sel. Assess. 24, 285–295. doi: 10.1111/ijsa.12147

Hotaling, J. M., Fakhari, P., and Busemeyer, J. R. (2015). “Dynamic decision making,” in International Encyclopedia of the Social & Behavioral Sciences , 2nd Edn, eds N. J. Smelser and P. B. Batles (New York, NY: Elsevier), 709–714.

Hundertmark, J., Holt, D. V., Fischer, A., Said, N., and Fischer, H. (2015). System structure and cognitive ability as predictors of performance in dynamic system control tasks. J. Dynamic Deci. Mak. 1, 1–10. doi: 10.11588/jddm.2015.1.26416

Jäkel, F., and Schreiber, C. (2013). Introspection in problem solving. J. Problem Solv. 6, 20–33. doi: 10.7771/1932-6246.1131

Jansson, A. (1994). Pathologies in dynamic decision making: consequences or precursors of failure? Sprache Kogn. 13, 160–173.

Kaufman, J. C., and Beghetto, R. A. (2009). Beyond big and little: the four c model of creativity. Rev. Gen. Psychol. 13, 1–12. doi: 10.1037/a0013688

Knauff, M., and Wolf, A. G. (2010). Complex cognition: the science of human reasoning, problem-solving, and decision-making. Cogn. Process. 11, 99–102. doi: 10.1007/s10339-010-0362-z

Kretzschmar, A. (2017). Sometimes less is not enough: a commentary on Greiff et al. (2015). J. Intell. 5:4. doi: 10.3390/jintelligence5010004

Kretzschmar, A., Neubert, J. C., Wüstenberg, S., and Greiff, S. (2016). Construct validity of complex problem solving: a comprehensive view on different facets of intelligence and school grades. Intelligence 54, 55–69. doi: 10.1016/j.intell.2015.11.004

Kretzschmar, A., and Süß, H.-M. (2015). A study on the training of complex problem solving competence. J. Dynamic Deci. Mak. 1, 1–14. doi: 10.11588/jddm.2015.1.15455

Lee, H., and Cho, Y. (2007). Factors affecting problem finding depending on degree of structure of problem situation. J. Educ. Res. 101, 113–123. doi: 10.3200/JOER.101.2.113-125

Leutner, D., Fleischer, J., Wirth, J., Greiff, S., and Funke, J. (2012). Analytische und dynamische Problemlösekompetenz im Lichte internationaler Schulleistungsvergleichsstudien: Untersuchungen zur Dimensionalität. Psychol. Rundschau 63, 34–42. doi: 10.1026/0033-3042/a000108

Luchins, A. S. (1942). Mechanization in problem solving: the effect of einstellung. Psychol. Monogr. 54, 1–95. doi: 10.1037/h0093502

Mack, O., Khare, A., Krämer, A., and Burgartz, T. (eds) (2016). Managing in a VUCA world. Heidelberg: Springer.

Mainert, J., Kretzschmar, A., Neubert, J. C., and Greiff, S. (2015). Linking complex problem solving and general mental ability to career advancement: does a transversal skill reveal incremental predictive validity? Int. J. Lifelong Educ. 34, 393–411. doi: 10.1080/02601370.2015.1060024

Mainzer, K. (2009). Challenges of complexity in the 21st century. An interdisciplinary introduction. Eur. Rev. 17, 219–236. doi: 10.1017/S1062798709000714

Meadows, D. H., Meadows, D. L., and Randers, J. (1992). Beyond the Limits. Vermont, VA: Chelsea Green Publishing.

Meadows, D. H., Meadows, D. L., Randers, J., and Behrens, W. W. (1972). The Limits to Growth. New York, NY: Universe Books.

Meißner, A., Greiff, S., Frischkorn, G. T., and Steinmayr, R. (2016). Predicting complex problem solving and school grades with working memory and ability self-concept. Learn. Ind. Differ. 49, 323–331. doi: 10.1016/j.lindif.2016.04.006

Molnàr, G., Greiff, S., Wüstenberg, S., and Fischer, A. (2017). “Empirical study of computer-based assessment of domain-general complex problem-solving skills,” in The Nature of Problem Solving: Using research to Inspire 21st Century Learning , eds B. Csapó and J. Funke (Paris: OECD Publishing), 125–141.

National Research Council (2011). Assessing 21st Century Skills: Summary of a Workshop. Washington, DC: The National Academies Press.

Newell, A., Shaw, J. C., and Simon, H. A. (1959). A general problem-solving program for a computer. Comput. Automat. 8, 10–16.

Nisbett, R. E., and Wilson, T. D. (1977). Telling more than we can know: verbal reports on mental processes. Psychol. Rev. 84, 231–259. doi: 10.1037/0033-295X.84.3.231

OECD (2014). “PISA 2012 results,” in Creative Problem Solving: Students’ Skills in Tackling Real-Life problems , Vol. 5 (Paris: OECD Publishing).

Osman, M. (2010). Controlling uncertainty: a review of human behavior in complex dynamic environments. Psychol. Bull. 136, 65–86. doi: 10.1037/a0017815

Osman, M. (2012). The role of reward in dynamic decision making. Front. Neurosci. 6:35. doi: 10.3389/fnins.2012.00035

Qudrat-Ullah, H. (2015). Better Decision Making in Complex, Dynamic Tasks. Training with Human-Facilitated Interactive Learning Environments. Heidelberg: Springer.

Ramnarayan, S., Strohschneider, S., and Schaub, H. (1997). Trappings of expertise and the pursuit of failure. Simulat. Gam. 28, 28–43. doi: 10.1177/1046878197281004

Reuschenbach, B. (2008). Planen und Problemlösen im Komplexen Handlungsfeld Pflege. Berlin: Logos.

Rohe, M., Funke, J., Storch, M., and Weber, J. (2016). Can motto goals outperform learning and performance goals? Influence of goal setting on performance, intrinsic motivation, processing style, and affect in a complex problem solving task. J. Dynamic Deci. Mak. 2, 1–15. doi: 10.11588/jddm.2016.1.28510

Scherer, R., Greiff, S., and Hautamäki, J. (2015). Exploring the relation between time on task and ability in complex problem solving. Intelligence 48, 37–50. doi: 10.1016/j.intell.2014.10.003

Schoppek, W., and Fischer, A. (2015). Complex problem solving – single ability or complex phenomenon? Front. Psychol. 6:1669. doi: 10.3389/fpsyg.2015.01669

Schraw, G., Dunkle, M., and Bendixen, L. D. (1995). Cognitive processes in well-defined and ill-defined problem solving. Appl. Cogn. Psychol. 9, 523–538. doi: 10.1002/acp.2350090605

Schweizer, F., Wüstenberg, S., and Greiff, S. (2013). Validity of the MicroDYN approach: complex problem solving predicts school grades beyond working memory capacity. Learn. Ind. Differ. 24, 42–52. doi: 10.1016/j.lindif.2012.12.011

Schweizer, T. S., Schmalenberger, K. M., Eisenlohr-Moul, T. A., Mojzisch, A., Kaiser, S., and Funke, J. (2016). Cognitive and affective aspects of creative option generation in everyday life situations. Front. Psychol. 7:1132. doi: 10.3389/fpsyg.2016.01132

Selten, R., Pittnauer, S., and Hohnisch, M. (2012). Dealing with dynamic decision problems when knowledge of the environment is limited: an approach based on goal systems. J. Behav. Deci. Mak. 25, 443–457. doi: 10.1002/bdm.738

Simon, H. A. (1957). Administrative Behavior: A Study of Decision-Making Processes in Administrative Organizations , 2nd Edn. New York, NY: Macmillan.

Sonnleitner, P., Brunner, M., Keller, U., and Martin, R. (2014). Differential relations between facets of complex problem solving and students’ immigration background. J. Educ. Psychol. 106, 681–695. doi: 10.1037/a0035506

Spering, M., Wagener, D., and Funke, J. (2005). The role of emotions in complex problem solving. Cogn. Emot. 19, 1252–1261. doi: 10.1080/02699930500304886

Stadler, M., Becker, N., Gödker, M., Leutner, D., and Greiff, S. (2015). Complex problem solving and intelligence: a meta-analysis. Intelligence 53, 92–101. doi: 10.1016/j.intell.2015.09.005

Stadler, M., Niepel, C., and Greiff, S. (2016). Easily too difficult: estimating item difficulty in computer simulated microworlds. Comput. Hum. Behav. 65, 100–106. doi: 10.1016/j.chb.2016.08.025

Sternberg, R. J. (1995). “Expertise in complex problem solving: a comparison of alternative conceptions,” in Complex Problem Solving: The European Perspective , eds P. A. Frensch and J. Funke (Hillsdale, NJ: Erlbaum), 295–321.

Sternberg, R. J., and Frensch, P. A. (1991). Complex Problem Solving: Principles and Mechanisms. (eds) R. J. Sternberg and P. A. Frensch. Hillsdale, NJ: Erlbaum.

Strohschneider, S., and Güss, C. D. (1998). Planning and problem solving: differences between brazilian and german students. J. Cross-Cult. Psychol. 29, 695–716. doi: 10.1177/0022022198296002

Strohschneider, S., and Güss, C. D. (1999). The fate of the Moros: a cross-cultural exploration of strategies in complex and dynamic decision making. Int. J. Psychol. 34, 235–252. doi: 10.1080/002075999399873

Thimbleby, H. (2007). Press On. Principles of Interaction. Cambridge, MA: MIT Press.

Tobinski, D. A., and Fritz, A. (2017). “EcoSphere: a new paradigm for problem solving in complex systems,” in The Nature of Problem Solving: Using Research to Inspire 21st Century Learning , eds B. Csapó and J. Funke (Paris: OECD Publishing), 211–222.

Tremblay, S., Gagnon, J.-F., Lafond, D., Hodgetts, H. M., Doiron, M., and Jeuniaux, P. P. J. M. H. (2017). A cognitive prosthesis for complex decision-making. Appl. Ergon. 58, 349–360. doi: 10.1016/j.apergo.2016.07.009

Tschirgi, J. E. (1980). Sensible reasoning: a hypothesis about hypotheses. Child Dev. 51, 1–10. doi: 10.2307/1129583

Tuchman, B. W. (1984). The March of Folly. From Troy to Vietnam. New York, NY: Ballantine Books.

Verweij, M., and Thompson, M. (eds) (2006). Clumsy Solutions for A Complex World. Governance, Politics and Plural Perceptions. New York, NY: Palgrave Macmillan. doi: 10.1057/9780230624887

Viehrig, K., Siegmund, A., Funke, J., Wüstenberg, S., and Greiff, S. (2017). “The heidelberg inventory of geographic system competency model,” in Competence Assessment in Education. Research, Models and Instruments , eds D. Leutner, J. Fleischer, J. Grünkorn, and E. Klieme (Heidelberg: Springer), 31–53.

von Clausewitz, C. (1832). Vom Kriege [On war]. Berlin: Dämmler.

Wendt, A. N. (2017). The empirical potential of live streaming beyond cognitive psychology. J. Dynamic Deci. Mak. 3, 1–9. doi: 10.11588/jddm.2017.1.33724

Wiliam, D., and Black, P. (1996). Meanings and consequences: a basis for distinguishing formative and summative functions of assessment? Br. Educ. Res. J. 22, 537–548. doi: 10.1080/0141192960220502

World Economic Forum (2015). New Vsion for Education Unlocking the Potential of Technology. Geneva: World Economic Forum.

World Economic Forum (2016). Global Risks 2016: Insight Report , 11th Edn. Geneva: World Economic Forum.

Wüstenberg, S., Greiff, S., and Funke, J. (2012). Complex problem solving — more than reasoning? Intelligence 40, 1–14. doi: 10.1016/j.intell.2011.11.003

Wüstenberg, S., Greiff, S., Vainikainen, M.-P., and Murphy, K. (2016). Individual differences in students’ complex problem solving skills: how they evolve and what they imply. J. Educ. Psychol. 108, 1028–1044. doi: 10.1037/edu0000101

Wüstenberg, S., Stadler, M., Hautamäki, J., and Greiff, S. (2014). The role of strategy knowledge for the application of strategies in complex problem solving tasks. Technol. Knowl. Learn. 19, 127–146. doi: 10.1007/s10758-014-9222-8

Keywords : complex problem solving, validity, assessment, definition, MicroDYN

Citation: Dörner D and Funke J (2017) Complex Problem Solving: What It Is and What It Is Not. Front. Psychol. 8:1153. doi: 10.3389/fpsyg.2017.01153

Received: 14 March 2017; Accepted: 23 June 2017; Published: 11 July 2017.

Reviewed by:

Copyright © 2017 Dörner and Funke. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Joachim Funke, [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

SUGGESTED TOPICS
The Magazine
Newsletters
Managing Yourself
Managing Teams
Work-life Balance
The Big Idea
Data & Visuals
Reading Lists
Case Selections
HBR Learning
Topic Feeds
Account Settings
Email Preferences

What It Takes to Think Deeply About Complex Problems

Tony Schwartz

Three ways to embrace a more nuanced, spacious perspective.

The problems we’re facing often seem as intractable as they do complex. But as Albert Einstein famously observed, “We cannot solve our problems with the same level of thinking that created them.” So what does it take to increase the complexity of our thinking? To cultivate a more nuanced, spacious perspective, start by challenging your convictions. Ask yourself, “What am I not seeing here?” and “What else might be true?” Second, do your most challenging task first every day, when your mind is fresh and before distractions arise. And third, pay attention to how you’re feeling. Embracing complexity means learning to better manage tough emotions like fear and anger.

The problems we’re facing often seem as complex as they do intractable. And as Albert Einstein is often quoted as saying, “We cannot solve our problems with the same level of thinking that created them.” So what does it take to increase the complexity of our thinking?

Tony Schwartz is the CEO of The Energy Project and the author of The Way We’re Working Isn’t Working . Become a fan of The Energy Project on Facebook .

Partner Center

Courses for Individuals

Understanding and Solving Complex Business Problems

Systems represented by buildings connecting as data points. image number null

Course Dates	Format	Location	Duration	Time Commitment	Price
In Person	Cambridge, MA	2 days	8 hours/day	$4,500
Dec 12-13, 2024	Live Online	N/A	2 days	8 hours/day	$4,500

Management and Leadership

Certificate Credits

- Operations

- Systems Thinking

Participants

Course Highlights

Discover MIT's unique, powerful, and integrative System Dynamics approach to assess problems that will not go away
Experience the Beer Game, which simulates the supply chain of the beer industry
Learn a new way of thinking about and resolving complex, persistent problems that emerge from change
Earn a certificate of course completion from the MIT Sloan School of Management

Why attend Understanding and Solving Complex Business Problems?

Systems thinking was designed to improve people's ability to manage organizations comprehensively in a volatile global environment. It offers managers a framework for understanding complex situations and the dynamics those situations produce. Systems thinking is a response to the rapid changes in technology, population, and economic activity that are transforming the world, and as a way to deal with the ever-increasing complexity of today's business.

Senior managers can use systems thinking to design policies that lead their organizations to high performance. The program is intended to give participants the tools and confidence to manage organizations with full understanding and solid strategy.

Course experience

This complex problem-solving course introduces participants to MIT's unique, powerful, and integrative System Dynamics approach to assess problems that will not go away and to produce the results they want. Through exercises and simulation models, participants experience the long-term side effects and impacts of decisions and understand the ways in which performance is tied to structures and policies.

People playing the ‘Beer Game’ while sitting at a table.

Sample Schedule—Subject to Change

This program is designed for executives with decision-making responsibility who are looking for fresh ideas to resolve organizational problems.

Past participants have included

VPs and EVPs
Corporate planners and strategists
Senior Project Managers
Product Development Managers

Request Course Information

Receive email updates related to this course, including faculty news and additional offering dates.

Review our Privacy Policy.

This program is designed to empower you to analyze complex problems in any area by using powerful yet very simple tools which are also very easy to use in real world, I enjoyed it a lot.

—Jia X.

Enroll Now!

What mit’s beer game teaches about panic hoarding, mit’s ‘beer game’ shows humans are weakest link in supply chains, the beer game, john sterman on system dynamics, how the ‘beer game’ helps retailers solve toilet paper crisis, course offerings.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings
My Bibliography
Collections
Citation manager

Save citation to file

Email citation, add to collections.

Create a new collection
Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

Search in PubMed
Search in NLM Catalog
Add to Search

Complex Problem Solving: What It Is and What It Is Not

Affiliations.

1 Department of Psychology, University of BambergBamberg, Germany.
2 Department of Psychology, Heidelberg UniversityHeidelberg, Germany.
PMID: 28744242
PMCID: PMC5504467
DOI: 10.3389/fpsyg.2017.01153

Keywords: MicroDYN; assessment; complex problem solving; definition; validity.

PubMed Disclaimer

Related information

Cited in Books

LinkOut - more resources

Full text sources.

Europe PubMed Central
Frontiers Media SA
PubMed Central

Other Literature Sources

scite Smart Citations

Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Accreditation
Value of Accreditation
Standards and Process
Search Accredited Schools

Bright green megaphone icon against dark teal weave pattern

Educational Membership
Business Membership
Find a Member

Learning and Events
Conferences
Webinars and Online Courses

All Insights
B-School Leadership
Future of Work
Societal Impact

Four colored boxes with different icons: notebook, camera lens, compass, and binoculars

Leadership and Governance
Media Center

Accredited School Search
Advertise, Sponsor, Exhibit
Tips and Advice
Is Business School Right for Me?

Solving Problems Through Systems Thinking

Systems thinking in business education encourages decision-making within a broader context, exclusive of a single disciplinary approach.
Employers need agile graduates capable of solving problems that exist in confusing systems and often alongside concurrent challenges.
Some traditional educational methods can hinder a more integrated approach to addressing complex issues.

Peter Møllgaard [00:15]: OK, so systems thinking is very important for business education today because we have all these polycrises, we have complex settings. So businesses need to take decisions acknowledging that there is a wider context. And if you ignore that, then you will suboptimize. The decisions that businesses will take would not be the right decisions.

And we need, of course, to take that into the classroom to make sure that our students, our graduates, will not suboptimize, will actually understand that they are part of a, that what they do is part of a bigger system.

Whatever that system might be depends on the concrete situation. So this way of looking at an issue at hand with a number of different, from a number of different perspectives is exactly what we need our students to learn.

We need to make sure that our students, our graduates, will actually understand that what they do is part of a bigger system.

[01:13]: When I talk to employers, what they are facing are a number of different crises that happen simultaneously: geopolitical crisis, climate change, whatnot. So there are lots of different things going on. And employers need to be agile. They need to operate in that very confusing system, really.

And in that confusing system, they need to have graduates out of business schools that can actually deal with that and can also engage in multidisciplinary, multigenerational teams that will solve these things.

So I think if you look at it from a very abstract point of view, this agility and the systems thinking are very well connected and would solve the issues that employers need to have solved these days.

So one example could be if you want to change the waterways in Ghana.

[02:10]: So we have a development problem, a Danish development project in Ghana, and you could think that you can just take solutions from Denmark and plug them in Ghana.

Of course, that would ignore the very different society that you are. The system is different. Simply, it’s a different legal system. There’s a lot less legal control. It’s a different behavior.

So, for example, in Ghana, people regularly just plug into the water pipes and say, well, I need water. So that would be illegal, but nobody cares, and they do that.

So if we want to take our solutions from Denmark, we need to understand that the behavior and the legal system is different, and only then can you become efficient in providing solutions to the Ghanaian society.

So in the classroom, we incorporate anthropology. So you need to understand behavior and actually observe behavior.

You need to be able to integrate the different disciplines in one solution. And so, if you get too hardcore into one discipline, then there’s a chance that you don’t open up.

[03:07]: What are they actually doing? Not what you think they should do or could do. We have legal aspects, we have globalization aspects, cultural aspects—a lot of different aspects to cover the system.

And of course, the ultimate aim is that you integrate all those different aspects, when you look at the problem at hand, for example, improving water pipes in Ghana, which is a hugely valuable thing to do in Ghana.

Traditional ways of teaching can get in the way because what we need is that there is an interface with other disciplines, right?

So that’s what I call integrative thinking—that you need to be able to integrate the different disciplines in one solution. And so, if you get too hardcore into one discipline, then there’s a chance that you don’t open up.

[03:57]: You get religious with that particular methodology. And that’s not good when you need to be able to work in multidisciplinary teams or just apply a systems thinking in your own head.

I’m not sure we have challenges that we can’t solve ourselves, but there are often … a certain conservatism when it comes to changing curricula. And I think that’s something we need to work with, but that’s our own system. We should be able to work with that.

cross-disciplinary
future of work

Fostering complex problem solving for diverse learners: engaging an ethos of intentionality toward equitable access

Published: 14 April 2020
Volume 68 , pages 679–702, ( 2020 )

Cite this article

Krista D. Glazewski 1 &
Peggy A. Ertmer 2 , 3

1936 Accesses

8 Citations

2 Altmetric

Explore all metrics

Complex problem solving is an effective means to engage students in disciplinary content while also furnishing critical non-cognitive and life skills. Despite increased adoption of complex problem-solving methods in K-12 classrooms today (e.g., case-, project-, or problem-based learning), we know little about how to make these approaches accessible to linguistically and culturally diverse (LCD) students. In this paper, we promote a conceptual framework, based on an ethos of intentionality , that supports culturally responsive teaching (CRT). We provide specific questions to guide teachers’ implementation of an ethos of intentionality, through critical reflection and meaningful action, and discuss a framework for culturally relevant practice that operationalizes key central tenets (e.g., high expectations, cultural competence, and critical consciousness). Finally, we include strategies that can help teachers and designers translate the principles of the CRT framework into action with a specific focus on complex problem solving in classrooms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

Get 10 units per month
Download Article/Chapter or eBook
1 Unit = 1 Article or 1 Chapter
Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Using Culturally Embedded Problem-Solving Tasks to Promote Equity Within Mathematical Inquiry Communities

Designing Problem-Solving for Meaningful Learning: A Discussion of Asia-Pacific Research

Addressing cultural diversity: effects of a problem-based intercultural learning unit

Explore related subjects.

Artificial Intelligence
Digital Education and Educational Technology

Aronson, B., & Laughter, J. (2015). The theory and practice of culturally relevant education: A synthesis of research across content areas. Review of Educational Research, 86 (1), 163–206. https://doi.org/10.3102/0034654315582066 .

Article Google Scholar

Amaral, O. M., Garrison, L., & Klentschy, M. (2002). Helping English learners increase achievement through inquiry-based science instruction. Bilingual Research Journal, 26 (2), 213–219. https://doi.org/10.1080/15235882.2002.10668709 .

Baker, B. D., & Corcoran, S. P. (2012). The stealth inequalities of school funding: How local tax systems and state aid formulas undermine equality . Washington, DC: Center for American Progress. Retrieved March 14, 2020 from http://www.americanprogress.org/wpcontent/uploads/2012/09/StealthInequities.pdf .

Barton, K. C., & McCully, A. W. (2012). Trying to see things differently: Northern Ireland students struggle to understand alternative historical perspectives. Theory and Research in Social Education, 40 (4), 371–408. https://doi.org/10.1080/00933104.2012.710928 .

Bell, P., Van Horne, K., & Cheng, B. H. (2017). Special issue: Designing learning environments for equitable disciplinary identification. Journal of the Learning Sciences, 26 , 367–375. https://doi.org/10.1080/10508406.2017.1336021 .

Belland, B. R. (2010). Portraits of middle school students constructing evidence-based arguments during problem-based learning: The impact of computer-based scaffolds. Educational Technology Research and Development, 58 (3), 285–309. https://doi.org/10.1007/s11423-009-9139-4 .

Belland, B. R., Ertmer, P. A., & Simons, K. D. (2006). Perceptions of the value of problem-based learning among students with special needs and their teachers. Interdisciplinary Journal of Problem-Based Learning, 1 (2), 1–18. https://doi.org/10.7771/1541-5015.1024 .

Belland, B. R., Glazewski, K. D., & Ertmer, P. A. (2009). Inclusion and problem-based learning: Roles of students in mixed-ability groups. Research on Middle Level Education , 32 (9), 1–19. https://doi.org/10.1080/19404476.2009.11462062 .

Bilgin, I., Karakuyu, Y., & Ay, Y. (2015). The effects of project-based learning on undergraduate students’ achievement and self-efficacy beliefs towards science teaching. Eurasia Journal of Mathematics, Science and Technology Education, 11 (3), 469–477. https://doi.org/10.12973/eurasia.2014.1015a .

Birr Moje, E., McIntosh Ciechanowski, K., Kramer, K., Ellis, L., Carrillo, R., & Collazo, T. (2004). Working toward third space in content area literacy: An examination of everyday funds of knowledge and discourse. Reading Research Quarterly, 39 (1), 38–70. https://doi.org/10.1598/rrq.39.1.4 .

Blasco, M. (2015). Making the tacit explicit: Rethinking culturally inclusive pedagogy in international student academic adaptation. Pedagogy, Culture and Society, 23 (1), 85–106.

Bogard, T., Liu, M., & Chiang, Y.-H. V. (2018). Thresholds of knowledge development in complex problem solving: A multiple case-study of advanced learners' cognitive processes. Educational Technology Research and Development, 61 , 465–503.

Bonner, E. P. (2014). Investigating practices of highly successful mathematics teachers of traditionally underserved students. Educational Studies in Mathematics, 86 (3), 377–399. https://doi.org/10.1007/s10649-014-9533-7 .

Brown, B. A. (2006). “It isn’t no slang that can be said about this stuff”: Language, identity, and appropriating science discourse. Journal of Research in Science Teaching, 43 (1), 96–126. https://doi.org/10.1002/tea.20096 .

Brown, B. A., Reveles, J. M., & Kelly, G. J. (2005). Scientific literacy and discursive identity: A theoretical framework for understanding science learning. Science Education, 89 (5), 779–802. https://doi.org/10.1002/sce.20069 .

Brown, J. C., Ring-Whalen, E. A., Roehrig, G. H., & Ellis, J. (2018). Advancing culturally responsive science education in secondary classrooms through an induction course. International Journal of Designs for Learning, 9 (1), 14–33.

Calabrese Barton, A., & Tan, E. (2010). We be Burnin’! Agency, identity, and science learning. Journal of the Learning Sciences, 19 (2), 187–229. https://doi.org/10.1080/10508400903530044 .

Danowitz, M. A., & Tuitt, F. (2011). Enacting inclusivity through engaged pedagogy: A higher education perspective. Equity and Excellence in Education, 44 (1), 40–56. https://doi.org/10.1080/10665684.2011.539474 .

Derry, S. J., Hmelo-Silver, C. E., Nagarajan, A., Chernobilsky, E., & Beitzel, B. (2006). Cognitive transfer revisited: Can we exploit new media to solve old problems on a large scale? Journal of Educational Computing Research , 35 , 145–162. https://doi.org/10.2190/0576-R724-T149-5432 .

Deslauriers, L., Schelew, E., & Wieman, C. (2011). Improved learning in a large-enrollment physics class. Science, 332 (6031), 862–864. https://doi.org/10.1126/science.1201783 .

Dolan, J. E. (2016). Splicing the divide: A review of research on the evolving digital divide among K-12 students. Journal of Research on Technology in Education, 48 (1), 16–37. https://doi.org/10.1080/15391523.2015.1103147 .

Eberlein, T., Kampmeier, J., Minderhout, V., Moog, R. S., Platt, T., Varma-Nelson, P., et al. (2008). Pedagogies of engagement in science: A comparison of PBL, POGIL, and PLTL. Biochemistry and Molecular Biology Education, 36 (4), 262–273. https://doi.org/10.1002/bmb.20204 .

Ertmer, P. A., & Glazewski, K. D. (2018). Problem-based learning: Essential design considerations. In R. A. Reiser & J. V. Dempsey (Eds.), Trends and issues in instructional design and technology (Vol. 4, pp. 286–295). New York: Pearson.

Google Scholar

Ertmer, P. A., & Glazewski, K. (2019). Scaffolding in PBL environments: Structuring and problematizing relevant task features. In N. Dabbah, W. Hung, & M. Moallem (Eds.), Handbook of problem-based learning (pp. 321–342). Hoboken, NY: Wiley Publishing.

Chapter Google Scholar

Ertmer, P. A., & Simons, K. D. (2006). Jumping the PBL implementation hurdle: Supporting the efforts of K-12 teachers. Interdisciplinary Journal of Problem-Based Learning . https://doi.org/10.7771/1541-5015.1005 .

Esteban-Guitart, M., & Moll, L. C. (2014). Funds of identity: A new concept based on the Funds of Knowledge approach. Culture & Psychology, 20 (1), 31–48.

Gallagher, S. A. (2005). Adapting problem-based learning for gifted students. In F. A. Karnes & S. M. Bean (Eds.), Methods and materials for teaching the gifted (pp. 285–311). Waco, TX: Prufrock Press.

Gallagher, S. A., & Gallagher, J. J. (2013). Using problem-based learning to explore unseen academic potential. Interdisciplinary Journal of Problem-Based Learning, 7 (1), 3–15. https://doi.org/10.7771/1541-5015.1322 .

Gay, G. (2001). Preparing for culturally responsive teaching. Journal of Teacher Education, 53 (2), 106–116. https://doi.org/10.1177/0022487104267587 .

Gay, G. (2010). Culturally responsive teaching: Theory, research, and practice . New York: Teachers College Press.

Gay, G. (2013). Teaching to and through cultural diversity. Curriculum Inquiry, 43 (1), 48–70. https://doi.org/10.1111/curi.12002 .

Glazewski, K. D., & Hmelo-Silver, C. E. (2019). Scaffolding and supporting use of information for ambitious learning practices. Information and Learning Sciences , 120 (1/2), 39–58. https://doi.org/10.1108/ILS-08-2018-0087 .

Goldberg, T. (2013). It’s in my veins: Identity and disciplinary practice in students’ discussions of a historical issue. Theory and Research in Social Education, 41 (1), 33–64. https://doi.org/10.1080/00933104.2012.757265 .

Goldman, S. R. (2003). Learning in complex domains: When and why do multiple representations help? Learning and Instruction, 13 (2), 239–244. https://doi.org/10.1016/s0959-4752(02)00023-3 .

González, N., Moll, L. C., & Amanti, C. (Eds.). (2005). Funds of knowledge: Theorizing practices in households, communities, and classrooms . Mahwah: Lawrence Erlbaum.

Gorski, P. C. (2008). Insisting on digital equity: Reframing the dominant discourse on multicultural education and technology. Urban Education, 44 , 348–364. https://doi.org/10.1177/0042085908318712 .

Gray, C. M., & Boling, E. (2016). Inscribing ethics and values in designs for learning: A problematic. Educational Technology Research and Development, 64 , 969–1001. https://doi.org/10.1007/s11423-016-9478-x .

Gutiérrez, K. D. (2008). Developing a sociocritical literacy in the third space. Reading Research Quarterly, 43 (2), 148–164. https://doi.org/10.1598/rrq.43.2.3 .

Gutierrez, K. D., Baquedano-Lopez, P., & Tegeda, C. (1999). Rethinking diversity: Hybridity and hybrid language practices in the third space. Mind, Culture, and Activity, 6 , 286–303. https://doi.org/10.1080/10749039909524733 .

Hanney, R., & Savin-Baden, M. (2013). The problem of projects: Understanding the theoretical underpinnings of project-led PBL. London Review of Education, 11 (1), 7–19. https://doi.org/10.1080/14748460.2012.761816 .

Hartman, P., Renguette, C., & Seig, M. T. (2018). Problem-based teacher-mentor education: Fostering literacy acquisition in multicultural classrooms. Interdisciplinary Journal of Problem-Based Learning . https://doi.org/10.7771/1541-5015.1659 .

Haruehansawasin, S., & Kiattikomol, P. (2018). Scaffolding in problem-based learning for low-achieving learners. The Journal of Educational Research, 111 (3), 363–370. https://doi.org/10.1080/00220671.2017.1287045 .

Hasni, A., Bousadra, F., Belletête, V., Benabdallah, A., Nicole, M. C., & Dumais, N. (2016). Trends in research on project-based science and technology teaching and learning at K–12 levels: A systematic review. Studies in Science Education, 52 (2), 199–231. https://doi.org/10.1080/03057267.2016.1226573 .

Hmelo-Silver, C. E. (2004). Problem-based learning: What and how do students learn? Educational Psychology Review, 16 (3), 235–266. https://doi.org/10.1023/B:EDPR.0000034022.16470.f3 .

Hmelo-Silver, C. E. (2013). Creating a learning space in problem-based learning. Interdisciplinary Journal of Problem-Based Learning . https://doi.org/10.7771/1541-5015.1334 .

Hmelo-Silver, C. E., & DeSimone, C. (2013). Problem-based learning: An instructional model of collaborative learning. In C. E. Hmelo-Silver, C. A. Chinn, C. K. K. Chan, & A. O’Donnell (Eds.), The international handbook of collaborative learning (pp. 382–398). New York: Routledge.

Hmelo-Silver, C. E., Duncan, R. G., & Chinn, C. A. (2007). Scaffolding and achievement in problem-based and inquiry learning: A response to Kirschner, Sweller, and Clark. Educational Psychologist, 42 (2), 99–107. https://doi.org/10.1080/00461520701263368 .

Hooks, B. (1994). Teaching to transgress: Education as the practice of freedom . New York: Routledge.

Howard, T. C. (2003). Culturally relevant pedagogy: Ingredients for critical teacher reflection. Theory into Practice, 42 (3), 195–202. https://doi.org/10.1207/s15430421tip4203_5 .

Hung, W. (2013). Problem-based learning: A learning environment for enhancing learning transfer. New Directions for Adult and Continuing Education, 137 , 28–38. https://doi.org/10.1002/ace .

Hwang, G. J., Chiu, L. Y., & Chen, C. H. (2015). A contextual game-based learning approach to improving students’ inquiry-based learning performance in social studies courses. Computers and Education, 81 , 13–25. https://doi.org/10.1016/j.compedu.2014.09.006 .

Jonassen, D. H. (2000). Toward a design theory of problem solving. Educational Technology Research and Development, 86 (2), 75–85. https://doi.org/10.1007/BF02300500 .

Jonassen, D., Strobel, J., & Lee, C. B. (2006). Everyday problem solving in engineering: Lessons for engineering educators. Journal of engineering education, 95 (2), 139–151. https://doi.org/10.1002/j.2168-9830.2006.tb00885.x .

Johnson, C. C. (2011). The road to culturally relevant science: Exploring how teachers navigate change in pedagogy. Journal of Research in Science Teaching, 48 (2), 170–198. https://doi.org/10.1002/tea.20405 .

Kim, H. (2016). Inquiry-based science and technology enrichment program for middle school-aged female students. Journal of Science Education and Technology, 25 , 174–186. https://doi.org/10.1007/s10956-015-9584-2 .

Kim, N. J., Belland, B. R., Lefler, M., Andreasen, L., Walker, A., & Axelrod, D. (2019). Computer-based scaffolding targeting individual versus groups in problem-centered instruction for STEM education: Meta-analysis. Educational Psychology Review . https://doi.org/10.1007/s10648-019-09502-3 .

Kolodner, J. L., Camp, P. J., Crismond, D., Fasse, J. G., Holbrook, J., Puntambekar, S., et al. (2003). Problem-based learning meets case-based reasoning in the middle school science classroom: Putting learning by design into practice. Journal of the Learning Sciences, 12 , 495–547. https://doi.org/10.1207/S15327809JLS1204_2 .

Komis, V., Romero, M., & Misirli, A. (2016). A scenario-based approach for designing educational robotics activities for co-creative problem solving. In International Conference EduRobotics 2016 (pp. 158–169). Cham: Springer.

Krapp, A., & Prenzel, M. (2011). Research on interest in science: Theories, methods, and findings. International Journal of Science Education, 33 (1), 27–50. https://doi.org/10.1080/09500693.2010.518645 .

Ladson-Billings, G. (1995). Toward a theory of culturally relevant pedagogy. American Educational Research Journal, 32 (3), 465–491.

Ladson-Billings, G. (2006). Liberatory consequences of literacy: A case of culturally relevant instruction for African American students. The Journal of Negro Education, 61 (3), 378–391. https://www.jstor.org/stable/2295255 .

Ladson-Billings, G. (2007). Pushing past the achievement gap: An essay on the language of deficit. Journal of Negro Education, 76 (3), 316–323.

Ladson-Billings, G. (2015). Culturally relevant pedagogy 2.0: a.k.a the Remix. Harvard Educational Review, 84 (1), 74–84. https://doi.org/10.17763/haer.84.1.p2rj131485484751 .

Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation . Cambridge: Cambridge University Press.

Book Google Scholar

Lazonder, A. W., & Harmsen, R. (2016). Meta-analysis of inquiry-based learning: Effects of guidance. Review of Educational Research, 86 (3), 681–718. https://doi.org/10.3102/0034654315627366 .

Lee, O. (2001). Culture and language in science education: What do we know and what do we need to know? Journal of Research in Science Teaching, 38 (5), 499–501. https://doi.org/10.1002/tea.1015 .

Lee, O. (2004). Teacher change in beliefs and practices in science and literacy instruction with English language learners. Journal of Research in Science Teaching, 41 (1), 65–93. https://doi.org/10.1002/tea.10125 .

Lee, O., & Buxton, C. A. (2013). Integrating science and English proficiency for English Language Learners. Theory into Practice, 52 , 36–42. https://doi.org/10.1007/s11423-016-9478-x .

Lee, O., Buxton, C., Lewis, S., & LeRoy, K. (2006). Science inquiry and student diversity: Enhanced abilities and continuing difficulties after an instructional intervention. Journal of Research in Science Teaching, 43 (7), 607–636. https://doi.org/10.1002/tea.20141 .

Lee, O., & Fradd, S. H. (1998). Science for all, including students from non-English language backgrounds. Educational Researcher, 27 (4), 12–21.

Lee, O., Quinn, H., & Valdés, G. (2013). Science and language for English language learners in relation to Next Generation Science Standards and with implications for Common Core State Standards for English language arts and mathematics. Educational Researcher, 42 (4), 223–233. https://doi.org/10.3102/0013189X13480524 .

Liu, M., Hsieh, P., Cho, Y., & Schallert, D. L. (2006). Middle school students’ self-efficacy, attitudes, and achievement in a computer-enhanced problem-based learning environment. Journal of Interactive Learning Research, 17 (3), 225–242.

Liu, M., Liu, S., Pan, Z., Zou, W., & Li, C. (2019). Examining science learning and attitude by at-risk students after they used a multimedia-enriched problem-based learning environment. Interdisciplinary Journal of Problem-Based Learning, 13 (1), 1–11. https://doi.org/10.7771/1541-5015.1752 .

Llosa, L., Lee, O., Jiang, F., Haas, A., O’Connor, C., Van Booven, C. D., et al. (2016). Impact of a large-scale science intervention focused on English Language Learners. American Educational Research Journal, 53 (2), 395–424. https://doi.org/10.3102/0002831216637348 .

Lopez, A. E. (2011). Culturally relevant pedagogy and critical literacy in diverse English classrooms: A case study of a secondary English teacher’s activism and agency. English Teaching: Practice and Critique, 10 (4), 75–93.

Lynch, S., Kuipers, J., Pyke, C., & Szesze, M. (2005). Examining the effects of a highly rated science curriculum unit on diverse students: Results from a planning grant. Journal of Research in Science Teaching, 42 (8), 912–946. https://doi.org/10.1002/tea.20080 .

Maker, C. J., & Zimmerman, R. (2008). Problem solving in a complex world: Integrating DISCOVER, TASC, and PBL in a teacher education project. Gifted Education International, 24 , 160–178. https://doi.org/10.1177/026142940802400305 .

Milner, H. R., IV. (2010). What does teacher education have to do with teaching? Implications for diversity studies. Journal of Teacher Education, 61 (1–2), 118–131. https://doi.org/10.1177/0022487109347670 .

Moje, E. B., Collazo, T., Carrillo, R., & Marx, R. W. (2001). “Maestro, what is ‘quality’?”: Language, literacy, and discourse in project-based science. Journal of Research in Science Teaching, 38 (4), 469–498. https://doi.org/10.1002/tea.1014 .

Morgan, K., & Brooks, D. W. (2012). Investigating a method of scaffolding student-designed experiments. Journal of Science Education and Technology, 21 (4), 513–522. https://doi.org/10.1007/s10956-011-9343-y .

Morrison, K. A., Robbins, H. H., & Rose, D. G. (2008). Operationalizing culturally relevant pedagogy: A synthesis of classroom-based research. Equity & Excellence in Education, 41 , 433–452. https://doi.org/10.1080/10665680802400006 .

National Center for Education Statistics. (2019a). Fast facts: Back to school statistics . https://nces.ed.gov/fastfacts/display.asp?id=372 .

National Center for Education Statistics. (2019b). Fast facts: English language learners . https://nces.ed.gov/FastFacts/display.asp?id=96 .

National Commission on Teaching and America’s Future. (2004). Fifty years after Brown v. Board of Education: A two-tiered education system . Washington, DC: Author.

National Council for the Social Studies (NCSS). (2013). The college, career, and civic life (C3) framework for Social Studies state standards: Guidance for enhancing the rigor of K- 12 civics, economics, geography, and history . https://www.socialstudies.org/system/files/c3/C3-Framework-for-Social-Studies.pdf .

NGSS Lead States. (2013). Next generation science standards: For states, by states . Washington, DC: National Academies Press. www.nextgenscience.org .

Paris, D. (2012). Culturally sustaining pedagogy: A needed change in stance, terminology, and practice. Educational Researcher, 41 (3), 93–97.

Pedersen, S., & Liu, M. (2002). The transfer of problem-solving skills from a problem-based learning environment: The effect of modeling an expert’s cognitive processes. Journal of Research on Technology in Education, 35 (2), 303–320. https://doi.org/10.1080/15391523.2002.10782388 .

Piaget, J. (1969). The psychology of the child . New York: Basic Books.

Ritzhaupt, A. D., Liu, F., Dawson, K., & Barron, A. E. (2014). Differences in student information and communication technology literacy based on socio-economic status, ethnicity, and gender. Journal of Research on Technology in Education, 45 , 291–307. https://doi.org/10.1080/15391523.2013.10782607 .

Saleh, A., Hmelo-Silver, C., Glazewski, K., Chen, Y., Mott, B., Rowe, J. P., & Lester, J. C. (2019). Collaborative inquiry play: A design case to frame integration of collaborative problem solving with story-centric games. Information and Learning Sciences , 120 (7/8), 547–566. https://doi.org/10.1108/ILS-03-2019-0024 .

Savin-Baden, M. (2016). The impact of transdisciplinary threshold concepts on student engagement in Problem-Based Learning: A conceptual synthesis. Interdisciplinary Journal of Problem-Based Learning, 10 (2), 9–13. https://doi.org/10.7771/1541-5015.1588 .

Schmidt, H. G., van der Molen, H. T., te Winkel, W. W. R., & Wijnen, W. H. F. W. (2009). Constructivist, problem-based learning does work: A meta-analysis of curricular comparisons involving a single medical school. Educational Psychologist, 44 (4), 227–249. https://doi.org/10.1080/00461520903213592 .

Simons, K. D., & Klein, J. D. (2007). The impact of scaffolding and student achievement levels in a problem-based learning environment. Instructional Science, 35 (1), 41–72. https://doi.org/10.1007/s11251-006-9002-5 .

Singh, K., Granville, M., & Dika, S. (2002). Mathematics and science achievement: Effects of motivation, interest, and academic engagement. Journal of Educational Research, 95 (6), 323–332. https://doi.org/10.1080/00220670209596607 .

Slavin, R. E. (1987). Ability grouping and student achievement in elementary schools: A best-evidence synthesis. Review of Educational Research, 57 (3), 293–336. https://doi.org/10.3102/00346543057003293 .

Sorge, C. (2007). What happens? Relationship of age and gender with science attitudes from elementary to middle school. Science Educator, 16 (2), 33–37.

Souto-Manning, M. (2009). Negotiating culturally responsive pedagogy through multicultural children’s literature: Towards critical democratic literacy practices in a first grade classroom. Journal of Early Childhood Literacy, 9 (1), 50–74. https://doi.org/10.1177/1468798408101105 .

Spector, J. M. (2008). A modeling methodology for assessing learning in complex domains. In P. Blumschein, W. Hung, & D. Jonassen (Eds.), Model-based approaches to learning: Using systems, models, and simulations to improve understanding and problem solving in complex domains (pp. 163–177). Rotterdam, NL: Sense Publishers.

Spratt, J., & Florian, L. (2015). Inclusive pedagogy: From learning to action. Supporting each individual in the context of “everybody”. Teaching and Teacher Education, 49 , 89–96. https://doi.org/10.1016/j.tate.2015.03.006 .

Tawfik, A., & Kolodner, J. L. (2016). Systematizing scaffolding for problem-based learning: A view from case-based reasoning. Interdisciplinary Journal of Problem-Based Learning, 10 (1), 5–6. https://doi.org/10.7771/1541-5015.1608 .

Ushomirsky, N., & Williams, D. (2015). Funding gaps 2015: Too many states still spend less on educating students who need the most . Washington, DC.

Van Horne, K., & Bell, P. (2017). Youth disciplinary identification during participation in contemporary project-based science investigations in school. Journal of the Learning Sciences, 26 (3), 437–476. https://doi.org/10.1080/10508406.2017.1330689 .

Wallace, S., Banks, T., Sedas, M., Glazewski, K., Brush, T. A., & McKay, C. (2017). What will keep the fish alive? Exploring the intersections of designing, making, and inquiry among middle school learners. International Journal of Designs for Learning, 8 (1), 11–21. https://doi.org/10.14434/ijdl.v8i1.22668 .

Warschauer, M. (2011). A literacy approach to the digital divide. Cadernos de Letras, 28 , 5–19.

Watson, W. E., Johnson, L., & Zgourides, G. D. (2002). The influence of ethnic diversity on leadership, group process, and performance: An examination of learning teams. International Journal of Intercultural Relations, 26 (1), 1–16. https://doi.org/10.1016/S0147-1767(01)00032-3 .

Wirkala, C., & Kuhn, D. (2011). Problem-based learning in K–12 education: Is it effective and how does it achieve its effects? American Educational Research Journal, 48 (5), 1157–1186. https://doi.org/10.3102/0002831211419491 .

Woods, D. R. (2014). Problem-oriented learning, problem-based learning, problem-based synthesis, process oriented guided inquiry learning, peer-led team learning, model-eliciting activities, and project-based learning: What is best for you? Industrial and Engineering Chemistry Research, 53 (13), 5337–5354. https://doi.org/10.1021/ie401202k .

Zwiep, S. G., & Straits, W. J. (2013). Inquiry science: The gateway to English language proficiency. Journal of Science Teacher Education, 24 (8), 1315–1331. https://doi.org/10.1007/s10972-013-9357-9 .

Download references

There was no funding for this work.

Author information

Authors and affiliations.

Instructional Systems Technology, Indiana University, Bloomington, IN, 47405, USA

Krista D. Glazewski

Purdue University, West Lafayette, IN, 47907, USA

Peggy A. Ertmer

Littleton, CO, 80120, USA

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Krista D. Glazewski .

Ethics declarations

Conflict of interests.

The authors declare that they have no conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Glazewski, K.D., Ertmer, P.A. Fostering complex problem solving for diverse learners: engaging an ethos of intentionality toward equitable access. Education Tech Research Dev 68 , 679–702 (2020). https://doi.org/10.1007/s11423-020-09762-9

Download citation

Published : 14 April 2020

Issue Date : April 2020

DOI : https://doi.org/10.1007/s11423-020-09762-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Complex problem solving
Linguistic and culturally diverse learners
Inquiry learning
Culturally responsive teaching
Problem-based learning
Project-based learning
Find a journal
Publish with us
Track your research

Newsletters
Best Industries
Business Plans
Home-Based Business
The UPS Store
Customer Service
Black in Business
Your Next Move
Female Founders
Best Workplaces
Company Culture
Public Speaking
HR/Benefits
Productivity
All the Hats
Digital Transformation
Artificial Intelligence
Bringing Innovation to Market
Cloud Computing
Social Media
Data Detectives
Exit Interview
Bootstrapping
Crowdfunding
Venture Capital
Business Models
Personal Finance
Founder-Friendly Investors
Upcoming Events
Inc. 5000 Vision Conference
Become a Sponsor
Cox Business
Verizon Business
Branded Content
Apply Inc. 5000 US

Inc. Premium

OpenAI to Reportedly Release the New Strawberry AI Model in the Next 2 Weeks

Don't let the simple name fool you--the model could be capable of solving complex problems by thinking in multiple steps..

Artificial intelligence market leader OpenAI is reportedly preparing to launch its latest AI model , codenamed Strawberry , on ChatGPT in the next two weeks.

According to new reporting from The Information , OpenAI is planning to imminently launch Strawberry, a model capable of solving complex problems that have beguiled earlier models by thinking in multiple steps.

By all accounts, Strawberry will be considered a reasoning model, meaning that it will be capable of taking on more complicated requests that require multiple steps to complete, from solving tricky algebra problems to developing a comprehensive monthslong marketing campaign. The Information also reported that at launch, Strawberry will only be able to process text, unlike GPT-4o, OpenAI's flagship model that can process images and audio.

While this means Strawberry will reportedly be less prone to errors and hallucinations than GPT-4o, the trade-off is that the model is said to be slower. According to The Information, it can take anywhere from 10 to 20 seconds to respond to a query, and while the model is supposed to avoid deep thinking for simple requests, it doesn't always work that way in practice. Some people who had tested early versions of the model told The Information that these slightly better answers weren't worth the wait.

It's likely that due to its advanced capabilities, Strawberry will be more compute-heavy than its siblings and cost more to use. The Information reported that Strawberry "will likely have rate limits restricting users to some maximum number of messages per hour, with the potential for a higher-priced tier that's faster to respond."

Complex number

$x^2 = -1$

1 Derivation
2 Formal Definition
4.1 Examples
5 Alternate Forms
7.1 Introductory
7.2 Intermediate
7.3 Olympiad

$i$

Formal Definition

$a + bi$

Addition and subtraction of complex numbers are similar to doing the same operations to polynomials -- add the real parts then add the imaginary parts.

$i^2 = -1$

Alternate Forms

$a+bi$

Complex plane
De Moivre's Theorem
Exponential form
Roots of unity

Introductory

2007 AMC 12A Problems/Problem 18

Intermediate

1984 AIME Problem 8
1985 AIME Problem 3
1988 AIME Problem 11
1989 AIME Problem 14
1990 AIME Problem 10
1992 AIME Problem 10
1994 AIME Problem 8
1994 AIME Problem 13
1995 AIME Problem 5
1996 AIME Problem 11
1997 AIME Problem 11
1997 AIME Problem 14
1998 AIME Problem 13
1999 AIME Problem 9
2000 AIME II Problem 9
2002 AIME I Problem 12
2004 AIME I Problem 13
2005 AIME II Problem 9
2009 AIME I Problem 2
2011 AIME II Problem 8
Fundamental Theorem of Algebra
Trigonometry
Real numbers
Imaginary unit
Complex numbers

Something appears to not have loaded correctly.

Click to refresh .

SoftwareDominos

The 7 Timeless Steps to Guide You Through Complex Problem Solving

I. Introduction

As we go through life, we inevitably encounter problems that require extensive forethought, critical thinking , and creativity . Solving complex problems is a crucial skill for success, whether it’s a business challenge, a personal dilemma, or a societal issue.

This guide will explore the fundamentals of complex problem-solving and provide practical tips and strategies for mastering this critical skill.

II. This Series Also Discusses…

This article is part of a series on complex problem-solving. The list below will guide you through the different subtopics.

Complex Problem-Solving Guide in 7 Steps

The Nature of Complex Problems

What Does the Nature of the Problem Tell Us About Its Solution

Gaussian Distributions vs Power Laws

Your Ultimate Guide to Making Sense of Natural and Social Phenomena

Complex Problem-Solving in Groups

An Exploratory Overview of ProbleSolving Processes in Groups

The Power of Critical Thinking

An Essential Guide for Personal and Professional Development

Group-Decision Making

6 Modes That Tell Us How Teams Decide

III. What Is a Complex Problem?

A. generic definition of complex problems.

Four properties allow us to distinguish complex problems from simple ones.

Complex problems accept alternative solutions
Choices can be weighed in multiple ways
Data supports multiple hypotheses
Breakdown of causal chains.

A complex problem presents no trivial or obvious solution. In other words, it shows the following characteristics:

Now that we have defined the general notion of a complex problem, let’s examine some specific cases related to software development , business management , and complexity theory.

B. Examples of Complex Problems

1. complex problems in software development.

A complex software development problem involves intricate interactions between numerous system components and requires a sophisticated understanding of the business problem, computing , algorithms and data structures.

Source: “Domain-Driven Design: Tackling Complexity in the Heart of Software” by Eric Evans

2. Complex Problems in Business Management

In business management , a complex problem is characterized by interconnected elements, uncertainty, and dynamic interactions, making it challenging to predict outcomes and devise straightforward solutions. This is most obviously seen in formulating effective organisational strategies or leading successful enterprise transformations.

Source: “ Strategic Management and Organisational Dynamics: The Challenge of Complexity ” by Ralph D. Stacey

3. Complex Problems in Complexity Theory

From a complexity theory standpoint, a complex problem involves many interacting agents or components, often exhibiting emergent properties that cannot be easily deduced from the properties of individual agents.

Source: “ The Quark and the Jaguar: Adventures in the Simple and the Complex ” by Murray Gell-Mann

Complex problems are contrasted with complicated problems. Complicated problems have clear causes and effects, can be broken down into smaller parts, and have predictable solutions. Complex problems, however, are dynamic, have interconnected parts, and exhibit emergent properties (unpredictable outcomes from the interaction of parts).

Source: “Cynefin Framework” (2007) by Dave Snowden

C. What are Complex Problem Solving Skills?

Complex problem-solving skills involve identifying , analysing , and solving non-routine problems requiring high cognitive effort.

These problems typically involve a large number of variables and require the application of creative and critical thinking skills to identify potential solutions. Individuals with complex problem-solving skills can work through ambiguity and uncertainty and use logical reasoning to develop effective solutions.

IV. Solving Complex Problems: A Generic Approach

While developing a universal solution that works in any context would be very challenging, we will describe a generic approach consisting of seven steps that will assist you in creating a bespoke method suitable to the specific context you are working in.

At the heart of this approach is logical decomposition , or breaking down a complex problem into smaller, more manageable ones and then developing and implementing effective solutions for each. This skill is essential for success in many areas of life, including business, education , and personal relationships.

Logical decomposition is at the heart of scientific thought, as described in Edsger W. Dijkstra’s paper “ On the Role of Scientific Thought “.

The seven steps to solving complex problems are listed below. We will discuss them in great detail in the following sections.

The 7 steps to creative solutions

V. Complex Problem-Solving Skills

A. why are complex problem solving skills essential.

In today’s rapidly changing world, individuals and organizations must possess complex problem-solving skills to succeed. These skills are essential for several reasons:

Dealing with Uncertainty

In many situations, there is no clear-cut solution to a problem. Complex problem-solving skills enable individuals to work through ambiguity and uncertainty and develop effective solutions.

Identifying Root Causes

Complex problems often have multiple causes that are difficult to identify. Individuals with complex problem-solving skills can identify and address the root causes of problems rather than just treating the symptoms.

Developing Creative Solutions

Complex problems require creative solutions that go beyond traditional approaches. Individuals who possess complex problem-solving skills can think outside the box and develop innovative solutions.

Achieving Business Success

Organizations with complex problem-solving skills are better equipped to overcome challenges, identify opportunities, and succeed in today’s competitive business environment.

B. How to Develop Complex Problem-Solving Skills

While some individuals possess a natural aptitude for complex problem-solving, these skills can be developed and improved over time. Here are some tips to help you develop complex problem-solving skills:

1. Build Your Knowledge Base

Developing complex problem-solving skills requires a strong foundation of knowledge in your area of expertise. Stay updated on your field’s latest trends, research, and developments to enhance your problem-solving abilities.

2. Practice Critical Thinking

Developing critical thinking skills is essential for complex problem-solving. Practice questioning assumptions, analyzing information , and evaluating arguments to build critical thinking skills.

3. Welcome Creativity

Complex problems require creative solutions. Embrace your creativity by exploring new ideas, brainstorming solutions, and seeking diverse perspectives.

4. Collaborate with Others

Collaborating with others can help you develop your complex problem-solving skills. Working in a team environment can expose you to new ideas and approaches, help you identify blind spots, and provide opportunities for feedback and support.

5. Seek Out Challenging Problems

Developing complex problem-solving skills requires practice. Seek out challenging problems and apply your problem-solving skills to real-world situations.

VI. Step 1: Understanding the Nature of Complex vs Complicated

A. the cynefin framework.

Complex and complicated problems are two distinct types of challenges that require different approaches to solve. Dave Snowden, a management consultant and researcher, developed the Cynefin framework , a conceptual model used to understand complex systems and situations. The framework identifies five domains: simple, complicated, complex, chaotic, and disordered, and guides how to approach challenges in each domain.

B. Complicated Problems

Complicated Problems:

are characterized by having many interrelated parts and require specialized knowledge and expertise to solve.
have a clear cause-and-effect relationship , and the solution can be discovered by systematically analysing the components.
are best addressed through a top-down, expert-driven approach , where the experts can identify the best solution through analysis and evaluation.

C. Complex Problems

Complex problems are characterized by uncertainty, ambiguity, and the involvement of multiple interconnected factors. There is no clear cause-and-effect relationship, and the solution cannot be found by simply analysing the components. Complex problems require a bottom-up, participatory approach, where multiple perspectives and ideas are considered to develop a solution. The solution may not be apparent initially, but it involves experimentation, adaptation, and feedback.

The Cynefin framework proposes that complex problems belong to the complex domain, where emergent solutions cannot be predicted or prescribed. The complex domain should explore the issue, generate hypotheses, and test them through experimentation. The emphasis is on learning from the process , adapting to changing circumstances, and using feedback to guide the solution.

D. Practical Tips on Identifying an Appropriate Framework

Objective — Classify the problem as complex, complicated, or disordered. This classification will determine the approach to be used.

How it’s done — You can do that by asking the following questions.

Do we have multiple, internally consistent, competing hypotheses explaining the issue?
Does the available data support both theories?

In this case, the problem lies in the complex domain, and the preferred approach is to identify suitable solutions and conduct safe-to-fail experiments. If it’s a complicated (but not complex) problem, the following questions can be answered in the affirmative:

Do we have a single view that explains the problem?
Do we know the engineering part of the solution?
Is the problem sufficiently familiar to be solved by an expert?

VII. Step 2: Identifying and Defining the Problem

A. problem identification.

The first step in problem-solving is identifying the problem. This step involves recognizing that a problem exists and understanding its nature. Some tips for identifying the issue include:

Once you have identified the problem, the next step is to define it. This step involves breaking down the problem into smaller parts and better understanding its nature. Some tips for defining the issue include:

Writing it down: Write down the problem statement clearly and concisely. This will help you to focus on the specific issue and avoid confusion.
Breaking it down: Break the problem into smaller parts to better understand its nature. This can help you identify the underlying causes and potential solutions. The logical decomposition of the issues is vital, and we have dedicated the next section to this.
Identifying the scope: Identify the scope of the problem and determine its impact. This can help you to prioritize the problem and allocate resources accordingly.

Reliable data and statistical analysis skills are crucial in problem-solving. Data provides information and insights necessary for understanding the root cause of the problem. Statistical analysis allows us to make sense of the data and extract meaningful information. This article will discuss the importance of reliable data and statistical analysis skills in problem identification.

B. Practical Tips on Identifying the Problem

Objective — Paint a complete picture of the problem by laying out the details, preferably on a piece of paper, classifying it, and deciding on an approach to solving it.

How it’s done — Write down a complete description of the problem, including its scope and impact on the various stakeholders or aspects of the business. Use data as evidence to support initial hypotheses. Find out if the problem is localised and can be resolved locally or whether it might need escalation and support from higher levels of management.

VIII. Step 3: Gathering and Analyzing Data

A. gathering reliable data.

In today’s fast-paced business environment, reliable data is more critical than ever. Accurate and objective information is vital to identifying problems and determining their root cause.

Reliable data is the basis of any evidence-based decision-making, without which what we have is opinions and assumptions.

Without reliable data, it isn’t easy to make informed decisions that can lead to effective problem-solving. Here are some of the benefits of using reliable data in problem identification:

Objective information: Reliable data provides an objective perspective of the situation.
Evidence-based decision-making: Using reliable data ensures that decisions are based on evidence rather than assumptions or opinions.
Improved accuracy: Reliable data improves the accuracy of problem identification, leading to better solutions.
Better understanding: Reliable data provides a better understanding of the situation, leading to a more comprehensive and holistic approach to problem-solving.
Improved Risk Management : Reliable helps put problems into perspective by allowing analysts to calculate their occurrence probabilities and impacts. Based on impact and likelihood, risk can then be categorised and prioritized.

B. Statistical Analysis Skills

Statistical analysis skills are necessary for making sense of the data and extracting meaningful information. These skills allow us to identify patterns and trends, understand the relationships between different variables, and (sometimes) predict future outcomes.

How statistical analysis can help with complex problem solving.

Some benefits of using statistical analysis skills in problem identification include the following:

Identifying patterns: Statistical analysis skills enable us to identify patterns and trends in the data, which can help identify the problem accurately.
Understanding relationships: Statistical analysis skills help us understand the relationships between different variables, which can help identify the problem’s root cause.
Predictive capabilities: Statistical analysis skills allow us to predict future outcomes based on the data, which can help develop effective solutions.
Objective analysis: Statistical analysis provides objective data analysis, which can help make evidence-based decisions.

Interpreting data, however, requires technical skills to avoid misinterpretations. The following is a common list of statistical analysis mistakes non-professionals can make.

C. How Software Team Leads Can Gather Reliable Data

Software team leads need reliable data on their performance to make informed decisions and identify areas for improvement. Here are some sources where software team leads can gather reliable data on their team’s performance:

Project management tools: Most project management tools have built-in reporting features, allowing team leads to track performance metrics such as task completion rates, sprint velocity, and burn-down charts. This data can be used to identify areas for improvement and make data-driven decisions.
Team feedback: Gathering feedback from team members through one-on-one meetings or anonymous feedback forms can provide valuable insights into team performance . This data can help team leads identify areas where team members may struggle or where additional training or resources may be needed. Crucially, it also provides insights into the organizational culture .
Code analysis tools like SonarQube or Code Climate can provide insights into code quality , maintainability, and security. This data can help team leads identify needed code improvements and prioritize technical debt reduction.
Customer feedback: Customer feedback, such as ratings, reviews, and support tickets, can provide insights into the usability and functionality of deployed applications. This data can help team leads identify areas for improvement and prioritize feature development.

The software team should gather data from multiple sources, use that data to inform decisions and identify areas for improvement. By using reliable data sources and monitoring team performance metrics regularly, software team leads can drive continuous improvement and ensure project success.

D. Practical Tips on Gathering Data to Support the Proposed Hypotheses

Objective — The availability of data can help put the problem into perspective. For example, a dollar figure for the losses due to process inefficiencies can help identify the potential solutions that management will deem feasible.

How it’s done — All modern project management and tracking tools have sophisticated built-in data capture tools that can be exported, cleaned, and analysed for insights.

For example, when evaluating a team’s productivity , you can export data from JIRA, Jenkins, or BitBucket and measure performance metrics such as team velocity, overruns, and time-to-market.

When evidence is insufficient, you can gather more data, abandon the hypothesis, or temporarily shelve it.

IX. Step 4: Logical Decomposition in Problem Solving

A. logical decomposition.

Logical decomposition is a problem-solving technique that breaks down complex problems into smaller, more manageable pieces. It is a structured approach that enables individuals to examine a problem from multiple angles, identify key issues and sub-problems, and develop a solution that addresses each piece of the problem.

The process of logical decomposition involves breaking down the main problem into smaller sub-problems, which are then broken down into smaller pieces. Each piece is analyzed in detail to determine its underlying cause-and-effect relationships and potential solutions. By breaking down the problem into smaller pieces, the individual can better understand the overall situation, identify possible solutions more quickly, and prioritize which sub-problems to address first.

Logical decomposition is particularly useful for dealing with complex issues. It allows individuals to break down a significant, overwhelming problem into smaller, more manageable pieces. This not only makes the problem easier to understand and solve but also less daunting and more approachable. Additionally, by breaking down the problem into smaller pieces, individuals can identify and focus on the underlying root causes of the problem rather than just treating the symptoms.

Logical decomposition is a vital stage of architecting large systems and solutions.

B. Practical Tips on Logical Decomposition

Objective — Most problems worth tackling are also overwhelming in size and complexity (or complicatedness). Luckily, a logical decomposition into specialized areas or modules will help focus the team’s efforts on a small enough subproblem or bring in the right expertise.

How it’s done — This author prefers mindmaps. A mindmap is a tree that starts with a single node and branches off into different areas, views, or perspectives of the problem. Mindmaps help analysts stay focused on a key area and ensure that all aspects of a problem are covered.

Once a mindmap has been created, potential solutions can be explored.

From Abstract Concepts to Tangible Value: Solution Architecture in Modern IT Systems

X. Step 5: Generating and Evaluating (Several) Potential Solutions

Generating multiple solutions to solve a problem is an effective way to increase creativity and innovation in problem-solving. By exploring different options, individuals can identify the strengths and weaknesses of each solution and determine the most effective approach to solving the problem. This section will discuss the advantages and techniques of generating multiple solutions to solve problems more effectively.

A. Advantages of Generating Multiple Solutions

The advantages of generating multiple solutions during problem-solving are:

B. Techniques for Generating Multiple Solutions

Techniques for generating multiple solutions:

C. Practical Tips on Solution Generation and Selection

Objective — The key principle of solution generation is comprehensively exploring the solution space. This exploration allows teams to avoid local minima or overcommitting to a suboptimal solution.

How it’s done — The most effective approach is to bring in several people from different areas of expertise or seniority and to offer every suggestion the opportunity to be heard and thoroughly explored.

Also, different stakeholders might favour solutions that maximise their (potentially) narrow gains. If not consulted, they might actively block the implementation of the selected solution if it adversely impacts their interests.

The technical aspect of problem-solving is relatively easy to generate and implement without budgetary or scheduling constraints . It’s only when you consider the cost and impact of a solution that complexity arises.

5 Key Concepts You Need to Know From Herbert Simon’s Paper on the Architecture of Complexity

XI. Step 6: Implementing and Assessing Solutions

Implementing solutions to complex problems requires a structured approach that considers the unique challenges and variables involved. Effective problem-solving involves implementing practical, feasible, and sustainable solutions.

This section will first discuss two approaches to implementing solutions to complex problems: small, safe-to-fail solutions and solving easy problems with enormous benefits.

A. Implementing Many Safe-to-Fail Solutions

One practical approach to implementing solutions to complex problems is small, safe-to-fail solutions. This technique involves implementing a small-scale solution that can be tested quickly and easily to gather feedback.

Exploring multiple paths allows analysts to avoid over-commitment to suboptimal solutions.

Starting with small-scale solutions allows individuals to gather feedback and adjust before investing significant resources in a more extensive solution. This approach can save time and resources while ensuring that the final solution meets the needs of stakeholders.

Small safe-to-fail experiments effectively deal with complexity where an engineering solution is unknown priori.

B. Prioritizing High-Yield Solutions

Another effective approach to implementing solutions to complex problems is to first solve easy problems with large benefits. This technique involves identifying and solving simple, straightforward problems that significantly impact the overall problem.

By prioritising easy problems, individuals can progress quickly and gain momentum towards solving the larger problem. This approach can also help build trust and credibility with stakeholders, as progress is visible and measurable.

C. A Systematic Approach to Implementing Solutions

It is important to note that both approaches should be used with a broader problem-solving methodology . Effective problem-solving requires a systematic approach that involves identifying the problem, gathering information, analyzing data, developing and evaluating potential solutions, and implementing the best solution. By implementing small, safe-to-fail solutions and solving easy problems with large benefits, individuals can enhance their problem-solving approach and increase the likelihood of success.

In conclusion, implementing solutions to complex problems requires a structured approach that considers the unique challenges and variables involved. Implementing small, safe-to-fail solutions and solving easy problems with large benefits are two effective techniques for enhancing problem-solving. These techniques should be used with a broader problem-solving methodology to ensure the final solution is practical, feasible, and sustainable.

D. Implementing the Solution

Objective — This stage aims to efficiently and effectively implement the (optimal) selected solution(s).

How it’s done — Three principal techniques are required for the implementation of the solution to succeed. The first is conducting safe-to-fail experiments. The second is allocating resources to conduct each experiment. The third is setting up the criteria for success or failure.

XII. Step 7: Evaluating the Solution

Objective — Solutions might work well under laboratory conditions but fail spectacularly in the field. Evaluating solutions after a trial is vital to avoid continuing investment in failed solutions.

How it’s done — The best way to evaluate a solution is to monitor the Key Performance Indicators (KPIs) originally used in the problem diagnosis. When solutions are successful, noticeable and measurable improvements should be observed.

Measuring second-order effects or observing undesirable team or business dynamics changes is key to continuing or aborting initiatives.

Complex problem-solving refers to the ability to solve complex, ambiguous problems that often require creative and innovative solutions. It involves identifying the root cause of a problem, analyzing different variables and factors, developing and evaluating possible solutions, and selecting the best course of action.

Complex problem-solving is essential because it allows individuals and organizations to overcome challenges and obstacles hindering their progress and success. It enables them to identify opportunities, improve processes, and innovate to stay ahead of the competition.

To develop your complex problem-solving skills, you can practice consistently, develop a systematic approach, and leverage the right tools and resources. You can also seek feedback from others, learn from your mistakes, and adopt a growth mindset that values continuous learning and improvement.

Some common obstacles to effective problem-solving include cognitive biases , lack of information, unclear objectives, and groupthink. These obstacles can hinder individuals and teams from developing effective solutions to complex problems.

Various tools and techniques for complex problem-solving include root cause analysis, fishbone diagrams, SWOT analysis, Pareto analysis, decision trees, and scenario planning. These tools can help individuals and teams to analyze complex problems, identify underlying causes, and develop effective solutions.

To improve your decision-making skills, you can develop a structured approach, gather and analyze relevant data, evaluate different options, and consider each alternative’s potential risks and benefits. You can also seek feedback from others and reflect on your past decisions to learn from your mistakes.

Complex problem-solving skills can be applied in various aspects of your personal life, such as improving your relationships, managing your finances, and achieving your goals. You can overcome obstacles and succeed personally by systematically analyzing different variables and factors and developing creative and innovative solutions.

To overcome cognitive biases in problem-solving, you can challenge your assumptions, seek diverse perspectives, and use data and evidence to inform your decisions. You can also use brainstorming and mind-mapping techniques to generate new ideas and avoid tunnel vision.

XIV. Final Words

In conclusion, complex problem-solving is a crucial skill that can significantly impact your professional and personal life. It allows you to navigate complex challenges, identify the root cause of a problem, and develop practical solutions.

By mastering the art of complex problem-solving, you can enhance your critical thinking, analytical skills, and decision-making abilities, which are essential for success in today’s fast-paced and dynamic business environment.

The key to mastering complex problem-solving is to practice consistently, develop a systematic approach, and leverage the right tools and resources. With patience, persistence, and a growth mindset, anyone can become a skilled problem solver and tackle even the most challenging problems.

Blog MHCLG Digital

https://mhclgdigital.blog.gov.uk/2024/09/09/adaptive-funding-8-ways-to-make-funding-effective-in-solving-complex-problems/

Adaptive funding: 8 ways to make funding effective in solving complex problems

a laptop screen showing the 'apply for funding' page on gov.uk

Complex problems

Most of the problems that today’s governments are trying to address are complex. If they had a simple answer, they probably would have been solved by now.

By ‘complex’, I mean that various factors interact in unpredictable ways to produce unpredictable outcomes, and we can therefore only understand why things happen in retrospect. As per Dave Snowden’s Cynefin framework, complex problems differ from ‘complicated’ problems, which also involve a wide range of factors, but once these are analysed, we can make reliable predictions and have confidence in our solutions. In Donald Rumsfeld’s words, complicated problems deal with “known unknowns”, whereas complex problems operate in the realm of “unknown unknowns”.

As government programmes continue to tackle many complex challenges, there is an opportunity to evolve our delivery approaches to ensure they are optimally structured to deal with complexity.

Complexity and the Agile mindset

The more traditional ‘waterfall’ approach to project management, which puts more emphasis on sticking to long-term project plans with clearly defined boundaries and pre-planned timelines, can be an ideal way to manage complicated projects, because with the right expertise and analysis, you can clearly define the problem and build a solution that you are confident will solve it.

But when you are dealing with complexity, this comparatively rigid approach often results in delays, overspend and solutions that you ultimately discover are not fit for purpose. That’s where ‘Agile’ comes in.

In 2001, 17 software engineers met at a ski resort in Utah to discuss their approaches to software development. That meeting ultimately resulted in the publication of the ‘ Manifesto for Agile Software Development ’, which set out some of the values and principles they had adopted to deal with the complex problem of building software that meets user needs.

The Manifesto set out 4 core values:

Individuals and interactions  over processes and tools
Working software  over comprehensive documentation
Customer collaboration  over contract negotiation
Responding to change  over following a plan

Agile and policy development

Since the publication of the Agile Manifesto, this approach has been successfully applied in various other sectors, including government services. In 2009, Henry David Venema and John Drexhage made a case for public policies which embrace the Agile mindset in Creating Adaptive Policies :

"Our world is more complex than ever – highly interconnected, owing to advances in communication and transportation; and highly dynamic, owing to the scale of impact of our collective actions… Policies that cannot perform effectively under dynamic and uncertain conditions run the risk of not achieving their intended purpose, and becoming a hindrance to the ability of individuals, communities and businesses to cope with – and adapt to – change. Far from serving the public good, these policies may actually get in the way."

This sentiment has been echoed in a recent paper, The Radical How , which advocates powerfully for an approach to delivering government programmes “that deliberately and specifically acknowledges complexity and uncertainty, and mitigates for both”.

Adaptive funding

One of the big ‘levers’ government has at its disposal is funding. Whether we are dealing with climate change, housing or healthcare, we can only go so far without fronting up some cash.

But funding programmes tend to be delivered according to the waterfall approach to project management. With the upcoming Spending Review offering an opportunity to reset how government funding is delivered, the time is ripe for a shift towards a more adaptive approach.

The Ministry of Housing, Communities and Local Government (MHCLG), has already started to design funds to account for complexity and uncertainty. But, as far as I can tell, this has happened because different teams could see that the rigid approach previously in place may not be working, rather than because they were consciously trying to create Agile funding programmes.

Adaptive funding is about building flexibility and adaptability into the design and delivery of funding programmes, to account for the complex and uncertain nature of the problems the funding is trying to solve. E mbracing the adaptive policy framework can help policymakers develop a coherent approach to programme design, which should help the government make progress against the complex missions it has set itself.

8 ways to design and deliver adaptive funding

Based loosely on Darren Swanson et al.’s 7 guidelines for crafting adaptive policies, and inspired by policy developments I have seen during my time within MHCLG, I have come up with 8 ways to design and deliver adaptive funding:

1. Decentralise decision-making over funding and promote policy variation.

The idea that central government knows best is rarely true, and usually leads to crude ‘one-size-fits-all’ policies. Different local manifestations of an issue add additional layers of complexity which make already complex problems even more difficult to solve. Local leaders often have a more detailed understanding of the problems in their areas than those in central government. Giving devolved institutions and local authorities greater flexibility to deliver funding according to local priorities and opportunities and allowing different places to come up with different solutions has the potential to increase the chance of success across many policy domains.

2. Test risky assumptions and unknowns with users .

Designing funding programmes based on assumptions that have not been tested with users can lead to huge costs if they turn out to be wrong. To set a programme up for success, policy teams should engage with users (for example, funding recipients or delivery organisations) to test their riskiest assumptions before funding is delivered. This will allow funding teams to refine the design of the programme before huge costs have been incurred.

3. Deliver short, small-scale pilot funds or experiments to test specific hypotheses .

Even if we test assumptions with users before launching a programme, in a complex environment there is always an element of uncertainty about how successful the programme will be. To reduce risk as much as possible, why not start small and scale up as you gain more confidence in each hypothesis? The authors of The Radical How are right, however, in cautioning against simply running lots of pilots. One problem is that pilots often test a whole policy solution rather than a specific hypothesis, which doesn’t always give you the nuanced understanding you need. To rectify this, pilots or experiments should be explicitly designed to test the specific hypotheses upon which the success of the programme depends. It’s also critical that, instead of waiting for a pilot to end before evaluating its success, we seek to learn throughout the pilot.

4. Prioritise continuous learning alongside longer-term evaluations .

Although HM Treasury recommends that government interventions should be evaluated during the intervention as well as after, most funding programmes tend to prioritise the latter. While these evaluations often provide invaluable insights, they usually come to light too late to influence the design of the programme. Conducting user testing will enable teams to iterate based on real-time feedback and correct any design features based on faulty assumptions. Departments should also monitor and evaluate the success of different local initiatives, to identify which solutions are working well, and which are not. By doing this, government can highlight, champion and encourage examples of good practice.

5. Iterate during the course of the programme based on user feedback .

Once a funding team identifies that an assumption is incorrect, or an element of the policy is not working, it’s important that the team is able to make iterations. This will not be possible in all cases (particularly if the fund has already been designed according to a waterfall approach), but where such changes do not cause significant disruption, in-flight course corrections can help to steer the programme in the right direction. For example, if a fund has multiple ‘bidding rounds’, amending the guidance between rounds may help to improve the quality or quantity of future applications.

6. Do not expect funding recipients to set out detailed project plans at the start of a programme .

As it is often difficult (or impossible) to predict what the best solution to a complex problem is, where possible, we should avoid requiring funding recipients to set out highly detailed plans from the outset. This does, of course, involve some risk, as a department would have limited assurance at the outset that the recipient will deliver what it wants (or at least what the department thinks it wants). But there is also significant risk in tying an organisation down to an overly specified plan which has not been tested. This approach might not be appropriate for all organisation types, but local and devolved authorities should be given the space to develop their plans as more becomes known.

7. Give funding recipients flexibility to make changes to their plans.

Linked to the above, government should give local leaders flexibility to make swift changes once it becomes clear that the original plan is no longer fit for purpose. For example, if private sector match funding ceases to be available, a project will need to be re-scoped. Providing trusted funding recipients with more autonomy to adapt their projects and programmes will enable them to respond nimbly to the risks and opportunities of a dynamic and ever-changing world.

8. Simplify funding by adopting a ‘systems thinking’ approach .

The difficulty of tackling a complex problem is often compounded by a complex system of government interventions. Taking a step back and adopting a ‘systems thinking’ approach can help to identify where government has made things unnecessarily difficult for external partners to navigate. Streamlining and simplifying the funding landscape can help to maximise impact by reducing duplicative and unnecessary administrative costs. Even if we cannot make the problem less complex, we can at least try to avoid compounding this complexity with byzantine ‘solutions’.

Considerations and trade-offs

If this adaptive approach is to be given the best chance of success, there are some foundations which should first be in place:

Central government should set specific outcomes that delivery partners are working towards . Those responsible for delivery will then have clarity on what they need to achieve, as well as the flexibility needed to respond effectively.
Delivery partners should have the necessary capacity and capability . Organisations need to be given the time, resources and skills they need if they are expected to solve complex problems.
Funding teams should be multi-disciplinary. By bringing together policy experts, delivery specialists, user researchers, content designers, service designers, analysts and data specialists, funding teams would be able to draw on the diverse perspectives needed to be effective in a complex environment.
Good quality, timely and easily accessible data . To make improvements to funding programmes when things are not working, funding teams need up-to-date information that is consistent, findable and usable. This will allow teams to understand whether the programme is achieving its objectives and change course if needed.

As with any policy approach, there will be trade-offs. For instance, an adaptive approach to funding policy may not provide delivery partners with the certainty they understandably crave. But by giving grant recipients flexibility in delivery, in-flight changes should not create so many issues, particularly if those changes respond to user feedback and are tested before roll-out.

You might also argue that this approach will lead to more unequal outcomes across the country. It is true that giving places more flexibility will inevitably lead to some areas doing better than others. But if recipients are also encouraged to start small, test their hypotheses, and remain vigilant to approaches that are being tested elsewhere, more places should start to move in a positive direction. By embracing an adaptive approach to funding, we have a chance to reset how we work with public, private and third sector organisations, and give ourselves the best chance of achieving our missions.

Cynefin: a tool for situating the problem in a sense-making framework (2017), Annabelle Mark and Dave Snowden. In Applied Systems Thinking for Health Systems Research: a Methodological Handbook , ed, by Don de Savigny, Karl Blanchet and Taghreed Adam, 76-96.
Creating Adaptive Policies: A Guide for Policy-making in an Uncertain World (2009) , Edited by Darren Swanson and Suruchi Bhadwal, International Development Research Centre
The Radical How (2024), Andrew Greenway and Tom Loosemore, UK Options 2040

Subscribe to our blog for the latest updates from the team or visit our careers page if you're interested in working with us.

Sharing and comments

Share this page, leave a comment.

Cancel reply

By submitting a comment you understand it may be published on this public website. Please read our privacy notice to see how the GOV.UK blogging platform handles your information.

Work with us

Find out more about working for MHCLG Digital

Sign up and manage updates

MHCLG on Twitter

Comments and moderation

Read our guidelines

How To Solve Complex Problems

In today’s increasingly complex world, we are constantly faced with ill-defined problems that don’t have a clear solution. From poverty and climate change to crime and addiction, complex situations surround us. Unlike simple problems with a pre-defined or “right” answer, complex problems share several basic characteristics that make them hard to solve. While these problems can be frustrating and overwhelming, they also offer an opportunity for growth and creativity. Complex problem-solving skills are the key to addressing these tough issues.

In this article, I will discuss simple versus complex problems, define complex problem solving, and describe why it is so important in complex dynamic environments. I will also explain how to develop problem-solving skills and share some tips for effectively solving complex problems.

How is simple problem-solving different from complex problem-solving?

Solving problems is about getting from a currently undesirable state to an intended goal state. In other words, about bridging the gap between “what is” and “what ought to be”. However, the challenge of reaching a solution varies based on the kind of problem that is being solved. There are generally three different kinds of problems you should consider.

Simple problems have one problem solution. The goal is to find that answer as quickly and efficiently as possible. Puzzles are classic examples of simple problem solving. The objective is to find the one correct solution out of many possibilities.

Problems are different from puzzles in that they don’t have a known problem solution. As such, many people may agree that there is an issue to be solved, but they may not agree on the intended goal state or how to get there. In this type of problem, people spend a lot of time debating the best solution and the optimal way to achieve it.

Messes are collections of interrelated problems where many stakeholders may not even agree on what the issue is. Unlike problems where there is agreement about what the problem is, in messes, there isn’t agreement amongst stakeholders. In other words, even “what is” can’t be taken for granted. Most complex social problems are messes, made up of interrelated social issues with ill-defined boundaries and goals.

Problems and messes can be complicated or complex

Puzzles are simple, but problems and messes exist on a continuum between complicated and complex. Complicated problems are technical in nature. There may be many involved variables, but the relationships are linear. As a result, complicated problems have step-by-step, systematic solutions. Repairing an engine or building a rocket may be difficult because of the many parts involved, but it is a technical problem we call complicated.

On the other hand, solving a complex problem is entirely different. Unlike complicated problems that may have many variables with linear relationships, a complex problem is characterized by connectivity patterns that are harder to understand and predict.

Characteristics of complex problems and messes

So what else makes a problem complex? Here are seven additional characteristics (from Funke and Hester and Adams ).

Lack of information. There is often a lack of data or information about the problem itself. In some cases, variables are unknown or cannot be measured.
Many goals. A complex problem has a mix of conflicting objectives. In some sense, every stakeholder involved with the problem may have their own goals. However, with limited resources, not all goals can be simultaneously satisfied.
Unpredictable feedback loops. In part due to many variables connected by a range of different relationships, a change in one variable is likely to have effects on other variables in the system. However, because we do not know all of the variables it will affect, small changes can have disproportionate system-wide effects. These unexpected events that have big, unpredictable effects are sometimes called Black Swans.
Dynamic. A complex problem changes over time and there is a significant impact based on when you act. In other words, because the problem and its parts and relationships are constantly changing, an action taken today won’t have the same effects as the same action taken tomorrow.
Time-delayed. It takes a while for cause and effect to be realized. Thus it is very hard to know if any given intervention is working.
Unknown unknowns. Building off the previous point about a lack of information, in a complex problem you may not even know what you don’t know. In other words, there may be very important variables that you are not even aware of.
Affected by (error-prone) humans. Simply put, human behavior tends to be illogical and unpredictable. When humans are involved in a problem, avoiding error may be impossible.

What is complex problem-solving?

“Complex problem solving” is the term for how to address a complex problem or messes that have the characteristics listed above.

Since a complex problem is a different phenomenon than a simple or complicated problem, solving them requires a different approach. Methods designed for simple problems, like systematic organization, deductive logic, and linear thinking don’t work well on their own for a complex problem.

And yet, despite its importance, there isn’t complete agreement about what exactly it is.

How is complex problem solving defined by experts?

Let’s look at what scientists, researchers, and system thinkers have come up with in terms of a definition for solving a complex problem.

As a series of observations and informed decisions

For many employers, the focus is on making smart decisions. These must weigh the future effects to the company of any given solution. According to Indeed.com , it is defined as “a series of observations and informed decisions used to find and implement a solution to a problem. Beyond finding and implementing a solution, complex problem solving also involves considering future changes to circumstance, resources, and capabilities that may affect the trajectory of the process and success of the solution. Complex problem solving also involves considering the impact of the solution on the surrounding environment and individuals.”

As using information to review options and develop solutions

For others, it is more of a systematic way to consider a range of options. According to O*NET , the definition focuses on “identifying complex problems and reviewing related information to develop and evaluate options and implement solutions.”

As a self-regulated psychological process

Others emphasize the broad range of skills and emotions needed for change. In addition, they endorse an inspired kind of pragmatism. For example, Dietrich Dorner and Joachim Funke define it as “a collection of self-regulated psychological processes and activities necessary in dynamic environments to achieve ill-defined goals that cannot be reached by routine actions. Creative combinations of knowledge and a broad set of strategies are needed. Solutions are often more bricolage than perfect or optimal. The problem-solving process combines cognitive, emotional, and motivational aspects, particularly in high-stakes situations. Complex problems usually involve knowledge-rich requirements and collaboration among different persons.”

As a novel way of thinking and reasoning

Finally, some emphasize the multidisciplinary nature of knowledge and processes needed to tackle a complex problem. Patrick Hester and Kevin MacG. Adams have stated that “no single discipline can solve truly complex problems. Problems of real interest, those vexing ones that keep you up at night, require a discipline-agnostic approach…Simply they require us to think systemically about our problem…a novel way of thinking and reasoning about complex problems that encourages increased understanding and deliberate intervention.”

A synthesis definition

By pulling the main themes of these definitions together, we can get a sense of what complex problem-solvers must do:

Gain a better understanding of the phenomena of a complex problem or mess. Use a discipline-agnostic approach in order to develop deliberate interventions. Take into consideration future impacts on the surrounding environment.

Why is complex problem solving important?

Many efforts aimed at complex social problems like reducing homelessness and improving public health – despite good intentions giving more effort than ever before – are destined to fail because their approach is based on simple problem-solving. And some efforts might even unwittingly be contributing to the problems they’re trying to solve.

Einstein said that “We can’t solve problems by using the same kind of thinking we used when we created them.” I think he could have easily been alluding to the need for more complex problem solvers who think differently. So what skills are required to do this?

What are complex problem-solving skills?

The skills required to solve a complex problem aren’t from one domain, nor are they an easily-packaged bundle. Rather, I like to think of them as a balancing act between a series of seemingly opposite approaches but synthesized. This brings a sort of cognitive dissonance into the process, which is itself informative.

It brings F. Scott Fitzgerald’s maxim to mind:

“The test of a first-rate intelligence is the ability to hold two opposing ideas in mind at the same time and still retain the ability to function. One should, for example, be able to see that things are hopeless yet be determined to make them otherwise.”

To see the problem situation clearly, for example, but also with a sense of optimism and possibility.

Here are the top three dialectics to keep in mind:

Thinking and reasoning

Reasoning is the ability to make logical deductions based on evidence and counterevidence. On the other hand, thinking is more about imagining an unknown reality based on thoughts about the whole picture and how the parts could fit together. By thinking clearly, one can have a sense of possibility that prepares the mind to deduce the right action in the unique moment at hand.

As Dorner and Funke explain: “Not every situation requires the same action, and we may want to act this way or another to reach this or that goal. This appears logical, but it is a logic based on constantly shifting grounds: We cannot know whether necessary conditions are met, sometimes the assumptions we have made later turn out to be incorrect, and sometimes we have to revise our assumptions or make completely new ones. It is necessary to constantly switch between our sense of possibility and our sense of reality, that is, to switch between thinking and reasoning. It is an arduous process, and some people handle it well, while others do not.”

Analysis and reductionism combined with synthesis and holism

It’s important to be able to use scientific processes to break down a complex problem into its parts and analyze them. But at the same time, a complex problem is more than the sum of its parts. In most cases, the relationships between the parts are more important than the parts themselves. Therefore, decomposing problems with rigor isn’t enough. What’s needed, once problems are reduced and understood, is a way of understanding the relationships between various components as well as putting the pieces back together. However, synthesis and holism on their own without deductive analysis can often miss details and relationships that matter.

What makes this balancing act more difficult is that certain professions tend to be trained in and prefer one domain over the other. Scientists prefer analysis and reductionism whereas most social scientists and practitioners default to synthesis and holism. Unfortunately, this divide of preferences results in people working in their silos at the expense of multi-disciplinary approaches that together can better “see” complexity.

Situational awareness and self-awareness

Dual awareness is the ability to pay attention to two experiences simultaneously. In the case of complex problems, context really matters. In other words, problem-solving exists in an ecosystem of environmental factors that are not incidental. Personal and cultural preferences play a part as do current events unfolding over time. But as a problem solver, knowing the environment is only part of the equation.

The other crucial part is the internal psychological process unique to every individual who also interacts with the problem and the environment. Problem solvers inevitably come into contact with others who may disagree with them, or be advancing seemingly counterproductive solutions, and these interactions result in emotions and motivations. Without self-awareness, we can become attached to our own subjective opinions, fall in love with “our” solutions, and generally be driven by the desire to be seen as problem solvers at the expense of actually solving the problem.

By balancing these three dialectics, practitioners can better deal with uncertainty as well as stay motivated despite setbacks. Self-regulation among these seemingly opposite approaches also reminds one to stay open-minded.

How do you develop complex problem-solving skills?

There is no one answer to this question, as the best way to develop them will vary depending on your strengths and weaknesses. However, there are a few general things that you can do to improve your ability to solve problems.

Ground yourself in theory and knowledge

First, it is important to learn about systems thinking and complexity theories. These frameworks will help you understand how complex systems work, and how different parts of a system interact with each other. This conceptual understanding will allow you to identify potential solutions to problems more quickly and effectively.

Practice switching between approaches

Second, practice switching between the dialectics mentioned above. For example, in your next meeting try to spend roughly half your time thinking and half your time reasoning. The important part is trying to get habituated to regularly switching lenses. It may seem disjointed at first, but after a while, it becomes second nature to simultaneously see how the parts interact and the big picture.

Focus on the specific problem phenomena

Third, it may sound obvious, but people often don’t spend very much time studying the problem itself and how it functions. In some sense, becoming a good problem-solver involves becoming a problem scientist. Your time should be spent regularly investigating the phenomena of “what is” rather than “what ought to be”. A holistic understanding of the problem is the required prerequisite to coming up with good solutions.

Stay curious

Finally, after we have worked on a problem for a while, we tend to think we know everything about it, including how to solve it. Even if we’re working on a problem, which may change dynamically from day to day, we start treating it more like a puzzle with a definite solution. When that happens, we can lose our motivation to continue learning about the problem. This is very risky because it closes the door to learning from others, regardless of whether we completely agree with them or not.

As Neils Bohr said, “Two different perspectives or models about a system will reveal truths regarding the system that are neither entirely independent nor entirely compatible.”

By staying curious, we can retain our ability to learn on a daily basis.

Tips for how to solve complex problems

Focus on processes over results.

It’s easy to get lost in utopian thinking. Many people spend so much time on “what ought to be” that they forget that problem solving is about the gap between “what is” and “what ought to be”. It is said that “life is a journey, not a destination.” The same is true for complex problem-solving. To do it well, a problem solver must focus on enjoying the process of gaining a holistic understanding of the problem.

Adaptive and iterative methods and tools

A variety of adaptive and iterative methods have been developed to address complexity. They share a laser focus on gaining holistic understanding with tools that best match the phenomena of complexity. They are also non-ideological, trans-disciplinary, and flexible. In most cases, your journey through a set of steps won’t be linear. Rather, as you think and reason, analyze and synthesize, you’ll jump around to get a holistic picture.

In my online course , we generally follow a seven-step method:

Get clear sight with a complex problem-solving frame
Establish a secure base of operation
Gain a deep understanding of the problem
Create an interactive model of the problem
Develop an impact strategy
Create an action plan and implement
Embed systemic solutions

Of course, each of these steps involves testing to see what works and consistently evaluating our process and progress.

Resolution is about systematically managing a problem over time

One last thing to keep in mind. Most social problems are not just solved one day, never to return. In reality, most complex problems are managed, not solved. For all practical purposes, what this means is that “the solution” is a way of systematically dealing with the problem over time. Some find this disappointing, but it’s actually a pragmatic pointer to think about resolution – a way move problems in the right direction – rather than final solutions.

Problem solvers regularly train and practice

If you need help developing your complex problem-solving skills, I have an online class where you can learn everything you need to know.

https://imd.widen.net/content/vvdwwyjjex/webp/20221215-Miscellaneous-Secondary-Top-Banner-A-comp-1500x800-mobile-1500x800.webp?position=c&color=ffffffff&quality=80&u=zewja0

Designed to sharpen your critical thinking skills.

Your online learning journey

Frame your problem, think critically and engage with your stakeholders.

As a manager or aspiring leader, you know that to rise through the ranks requires strong and creative problem-solving skills. To get to the top, business leaders need to act and react to the priorities set for the organization.

They must exercise their critical thinking capabilities and decide which issues to focus on and when. So, what are the essential elements of effective problem-solving that you can apply to your particular scenario?

The program is structured into five units, covering the following topics:

Learn to recognize when a problem requires a systematic approach.
Articulate how the process works and how each part contributes.
Understand why engaging stakeholders is critical throughout the process.
Learn how to develop your Frame sequence and how to frame your problem.
Apply the Holding Hands, Dolly the Sheep and Watson rules to the Frame sequence to make it robust.
Learn how to map out all the possible problem causes using a WHY map.
Identify which causes are at the root of the problem.
Update the Frame sequence to integrate new information.
Learn how to apply MECE thinking.
Identify and organize all the potential solutions using a HOW map and determine which are feasible.
Identify a set of criteria that represents your stakeholders’ views.
Create a decision matrix to rank the attractiveness of the options.
Make aligned decisions that support the solution.
Craft a storyline to defend the line of thinking (bullet proof) which is being empathetic to stakeholders.
See the big picture and plan your next actions.

Themes you will explore

Problem framing

Thinking creatively

Thinking critically

Question mapping

Hypotheses testing

Decision making

Engaging stakeholders

You will have a dedicated learning coach, making sure you receive a highly individualized learning experience.

Your professional learning coach accompanies you through your 5-week learning journey on this Complex Problem Solving online course. They provide support and feedback as you apply your learning directly into your workplace, where it has an immediate impact.

Their input helps you translate your learning to your particular context. By spreading this feedback regularly throughout the program, you’ll be sure to embed your ongoing learning directly in your daily work.

Your professional learning coach interacts with you via video, in writing, and over the phone. You have calls, spread across the 5-weeks, at intervals that consolidate your learning.

Your learning coach helps you

Find quick answers to your online program queries in our comprehensive FAQ section.

Schedule a call with a Program Advisor to discuss your professional development and see if this is the program for you.

Join online and experience a walkthrough of the Online programs, how they work, what you can expect and how it will impact your career.

IMD Executive Certificate - IMD Business School

Reinforce and consolidate your expertise in leadership, strategy, or digital acceleration through an immersive, fully online experience incorporating executive coaching.

Solutions Integral Calculator Derivative Calculator Algebra Calculator Matrix Calculator More...
Graphing Line Graph Exponential Graph Quadratic Graph Sine Graph More...
Calculators BMI Calculator Compound Interest Calculator Percentage Calculator Acceleration Calculator More...
Geometry Pythagorean Theorem Calculator Circle Area Calculator Isosceles Triangle Calculator Triangles Calculator More...
Tools Notebook Groups Cheat Sheets Worksheets Study Guides Practice Verify Solution

x^{\msquare}

\log_{\msquare}

\sqrt{\square}

\nthroot[\msquare]{\square}

\le

\ge

\frac{\msquare}{\msquare}

\cdot

\div

x^{\circ}

\pi

\left(\square\right)^{'}

\frac{d}{dx}

\frac{\partial}{\partial x}

\int

\int_{\msquare}^{\msquare}

\lim

\sum

\infty

\theta

(f\:\circ\:g)

f(x)

▭\:\longdivision{▭}

\times \twostack{▭}{▭}

+ \twostack{▭}{▭}

- \twostack{▭}{▭}

\left(

\right)

\times

\square\frac{\square}{\square}

Pre Algebra Order of Operations Factors & Primes Fractions Long Arithmetic Decimals Exponents & Radicals Ratios & Proportions Percent Modulo Number Line Expanded Form Mean, Median & Mode
Algebra Equations Inequalities System of Equations System of Inequalities Basic Operations Algebraic Properties Partial Fractions Polynomials Rational Expressions Sequences Power Sums Interval Notation Pi (Product) Notation Induction Prove That Logical Sets Word Problems
Pre Calculus Equations Inequalities Scientific Calculator Scientific Notation Arithmetics Complex Numbers Coterminal Angle Polar/Cartesian Simultaneous Equations System of Inequalities Polynomials Rationales Functions Arithmetic & Comp. Coordinate Geometry Plane Geometry Solid Geometry Conic Sections Trigonometry
Calculus Derivatives Derivative Applications Limits Integrals Integral Applications Integral Approximation Series ODE Multivariable Calculus Laplace Transform Taylor/Maclaurin Series Fourier Series Fourier Transform
Functions Line Equations Functions Arithmetic & Comp. Conic Sections Transformation
Linear Algebra Matrices Vectors
Trigonometry Identities Proving Identities Trig Equations Trig Inequalities Evaluate Functions Simplify
Statistics Mean Geometric Mean Quadratic Mean Average Median Mode Order Minimum Maximum Probability Mid-Range Range Standard Deviation Variance Lower Quartile Upper Quartile Interquartile Range Midhinge Standard Normal Distribution
Physics Mechanics
Chemistry Chemical Reactions Chemical Properties
Finance Simple Interest Compound Interest Present Value Future Value
Economics Point of Diminishing Return
Conversions Roman Numerals Radical to Exponent Exponent to Radical To Fraction To Decimal To Mixed Number To Improper Fraction Radians to Degrees Degrees to Radians Hexadecimal Scientific Notation Distance Weight Time Volume
Pre Algebra
One-Step Addition
One-Step Subtraction
One-Step Multiplication
One-Step Division
One-Step Decimals
Two-Step Integers
Two-Step Add/Subtract
Two-Step Multiply/Divide
Two-Step Fractions
Two-Step Decimals
Multi-Step Integers
Multi-Step with Parentheses
Multi-Step Rational
Multi-Step Fractions
Multi-Step Decimals
Solve by Factoring
Completing the Square
Quadratic Formula
Biquadratic
Logarithmic
Exponential
Rational Roots
Floor/Ceiling
Equation Given Roots
Equation Given Points
Newton Raphson
Substitution
Elimination
Cramer's Rule
Gaussian Elimination
System of Inequalities
Perfect Squares
Difference of Squares
Difference of Cubes
Sum of Cubes
Polynomials
Distributive Property
FOIL method
Perfect Cubes
Binomial Expansion
Negative Rule
Product Rule
Quotient Rule
Expand Power Rule
Fraction Exponent
Exponent Rules
Exponential Form
Logarithmic Form
Absolute Value
Rational Number
Powers of i
Complex Form
Partial Fractions
Is Polynomial
Leading Coefficient
Leading Term
Standard Form
Complete the Square
Synthetic Division
Linear Factors
Rationalize Denominator
Rationalize Numerator
Identify Type
Convergence
Interval Notation
Pi (Product) Notation
Boolean Algebra
Truth Table
Mutual Exclusive
Cardinality
Caretesian Product
Age Problems
Distance Problems
Cost Problems
Investment Problems
Number Problems
Percent Problems
Addition/Subtraction
Multiplication/Division
Dice Problems
Coin Problems
Card Problems
Pre Calculus
Linear Algebra
Trigonometry
Conversions