An Enlightened Approach To Assessment

Harry Webne-Behrman Author

Understanding whether you are dealing with a simple, complicated, complex, or chaotic problem helps you determine your approach to solve it.

“These are complex times — not merely complicated — and they also have many chaotic features as we wrestle with how best to respond to the challenges of the coronavirus and its sudden impacts on the world.” This excerpt from my book, What Matters at Work, seems appropriate… I hope you find it helpful.

When addressing challenges we face, we need to distinguish among the types of problems being addressed. Michael Quinn Patton (Developmental Evaluation, 2010, and other writings) has created an excellent taxonomy of issues that serves us here:

Simple problems

Some issues can be addressed by a recipe that can be easily replicated. There is high reliability that if you follow the recipe, you will achieve the same result each time. Examples of simple problems are fast food recipes (you expect a Big Mac, for example, to be prepared in the same way with the same result at any McDonald’s franchise in the world), warehouse storage strategies, usual payroll practices, room setup processes, and many maintenance problems. In order to solve a Simple problem reliably, you train towards sameness: if we teach employees how to serve food on the lunch line, for example, they can repeat that skill in settings and deliver predictable results. Simple problems are often accompanied by “best practices,” tried and true approaches that are readily shared and available across the organization and, in many cases, across the industry (see ISO 9000 standards, for example).

Complicated problems

There are other issues that are characterized by a larger number of variables, which sometimes can be disaggregated to become a chain of simple problems. A large mathematical formula is such a complicated problem, and by following the order of operations, you can solve it. Sending a spaceship to the moon is solved this way, following the laws of math and physics, with 99.99% reliability. But there is another important element of most complicated problems: There are several ways to go about solving them, resulting in natural social and technical tensions among members of a problem solving team. Some prefer to work more independently, while others like to communicate regularly. Some are deductive, others are inductive. Some can call upon past experiences from similar problems, while others can draw upon vastly different experiences. These tensions often result in conflicts over how best to solve the problem, especially as members get entrenched in their own preferred approaches and communication breaks down. There are truly several “better practices” in most Complicated problems, and each practice has its unique social and technical benefits and costs. Much of the work in scientific laboratory research, surgical team procedures, strategic planning processes, and menu planning for catering operations all are truly Complicated problems.

Complex problems

Finally, there are complex problems (sometimes called, “Wicked problems”), and they are characterized by uncertainty, dynamism, and powers of emergence. They lack the predictability of either Simple or Complicated problems, and often exhibit disproportionate impacts of variables as a result. While large amounts of data can be helpful, it is still a quality of Complexity that there will be significant uncertainty. In life, examples abound: raising a child, experiencing romance, reacting to surprise. There is little predictability involved in assessing strategies and responses to such problems. In the workplace, many of the issues are Complex problems: relationships among work teams, strategic planning in the midst of vast change, reinventing an established organization or starting one in a totally new market space. Complex problems are not approached by just throwing up our hands and praying for insight, but are well navigated through practices that are learned through group intelligence that is cultivated over time. As such, we call these approaches “emergent practices,” in contrast to the “best practices” more appropriate to Simple problems or the “better practices” of Complicated problems.

Chaotic problems

One additional category is worth mentioning at this time: Chaotic problems are those for which we cannot discern a pattern, or for which the “rules of engagement” have yet to be defined, negotiated, and agreed upon by relevant parties. There are certain issues that naturally qualify within our corporate lives.

For example, a new crisis has emerged in the organization related to larger, unpredicted political crises in the world. Our communication and decision-making channels and resources are otherwise committed, people are already busy with routine issues that align with our mission and strategic plan, etc. Suddenly, a crisis on the other side of the world disrupts our supply chain and delivery infrastructure.

Upon further analysis, we find that we have practiced similar scenarios and have answered some of these challenges before. We can generally sort the Chaotic issues into sub-issues that fit the other categories, and then respond appropriately. This requires a nimble, flexible, and committed leadership and staff, but if we have practiced our Core Values and Intentions, we can be successful in such situations.

Assessment of Simple, Complicated, and Complex Problems in Practice

The assessment approaches we need to use should vary with the types of problems we are facing:

For Simple problems, the outcomes are readily measured and generally follow recognizable criteria – did the recipe taste good? Did it look appealing? Were ingredients used in proper proportion? Did the recipe stay on budget? These types of assessment questions make sense for Simple problems. Even in IT and Project Management, much of “requirements gathering” can use this approach and make sense: What does the customer need from the system? Is the solution addressing that need in a cost-effective way? Is the project following an understood timeline that accounts for input of all variables?

For Complicated problems, use of Logic Models can be extremely helpful in assessment, as they surface for the team’s consideration all relevant variables and how they are best utilized and sequenced to achieve preferred outcomes. If we are clear about the goal we seek to achieve, a process that uses formative and summative evaluation and assessment tools makes sense; Complicated problems fit this model beautifully.

But for Complex problems, another approach is required: We need to engage in frequent “pulse checks,” sensing where the situation is now constructed and reformulating itself. This requires ongoing assessment, not only regarding anticipated outcomes, but also with sensitivity to new outcomes that are occurring.

For example, we might implement a new peer mentoring program at our company, hoping it will result in improved performance for those who are mentored. If we are open to diverse metrics, we might discover (a) social impacts on mentors from being in a helping relationship, (b) shifts in work roles for mentors, now joining leadership of new projects, (c) higher engagement and retention rates for mentees, as their learning outcomes improve and they “catch on” and feel greater overall confidence in their work skills and connectedness to colleagues, or (d) that mentees return as mentors with future employees, seeking to “pay it forward” in gratitude.

It isn’t that these results will occur, but that they may occur and get noticed so there can be a more complete evaluation and understanding of the impacts of the program.

Complex problems benefit from “utilization-focused evaluation,” an approach championed by Michael Quinn Patton (Essentials of Utilization-Focused Evaluation, 2011). This approach expressly allows us to treat innovation as an emergent process, one that respects its complexity, dynamism, and uncertainty. As such, we can pursue those strategies that become most promising (rather than having high certainty of them before starting the process), remaining flexible to notice new factors in the ever-changing currents of Complex challenges.

In “Alice in Wonderland,” the Cheshire Cat told Alice, “If you don’t know where you are going, any road will get you there!” To succeed with any significant change, people often need to know where they are headed and whether they are achieving desired results. I get that: Otherwise, they have no ability to discern whether their change efforts are progressing or adding any value. But in some situations, the goals are less discernible, the “roads” to traverse may be dusty paths, and trails are less traveled. By understanding whether we are dealing with Simple, Complicated, or Complex issues, we will gain necessary insights regarding how best to approach such change efforts, and have a far more interesting journey along the way.