The PhD Process: Tools for Making Order out of Chaos and Complexity
Research brings order to complexity and chaos. As part of the PhD process, one learns tools, methods, and strategies for creating such order from the complexity and chaos of open-ended research projects. Still, the complexity and chaos of research makes research challenging!
The Cynefin Framework and the Five Decision-Making Domains.
This post is inspired by a set of tweets from Lea Kissner. From that Twitter thread, I learned about the Cynefin framework and the five decision-making domains: simple, complicated, complex, chaotic, and confusion. I wish I had learned about the Cynefin framework earlier! It provides a natural way to discuss research, the challenges with research, and the PhD process.
Lea wrote:
Complicated problems are ones where you need to take your whole expertise and attack a hard problem and wrestle it until you come up with the right answer. Most technical people love these problems.
Lea also wrote:
Then there are complex problems: you look at them, shrug, and have to start from first principles. They don’t have a real answer, only constantly-changing trade-offs. They are ambiguous in the extreme. This is the realm of research problems.
I love that characterization! And I agree: complex (and chaotic) problems are ones that can be addressed through research.
To quote Wikipedia, “[t]he complex domain represents the ‘unknown unknowns’.” Also to quote Wikipedia, “[i]n the chaotic domain, cause and effect are unclear.”
In both cases (complex and chaotic), research can help. In the complex domain, research can change “unknown unknowns” into “known unknowns.” In the chaotic domain, research can illuminate the relationship between cause and effect.
The Cynefin Framework and the PhD Process.
At the beginning of one’s PhD journey, a PhD student often has limited experience with open-ended research questions. Generally, much of their prior education has focused on problems in the “simple” and “complicated” domains — problems that one can solve using the application of or small extensions to known ideas and methods. (See also Bloom’s taxonomy.)
Open-ended research questions are different: when beginning or conducting a research project, we may have some ideas for how to answer our research questions, but there is so much uncertainty. For example:
- Are our questions even interesting?
- Where do we start?
- How will we know if we are tackling the project in the right way(s)?
- Should we keep pursuing answers to our initial questions or should we shift focus and answer different questions as they arise?
An important part of the PhD process is the learning of tools, methods, and strategies (henceforth, the “tools”) that can help answer questions like the above. These tools can be instrumental in providing order to the chaos/complexity of a research project and move the research project from the “chaotic” domain into the “complex” domain and from the “complex” domain into the “complicated” domain.
It is possible for people, by themselves and without any mentorship, to develop their own approaches for answering open-ended research questions. I.e., getting a PhD is not a requirement to becoming a strong researcher. Rather, the PhD process — and a role of the PhD advisor — is to smooth that journey. In one’s first few projects as a PhD student, I generally encouraged students to learn from and adopt their advisor’s tools for tackling open-ended research problems. Of course, the advisor is not always right, and it is important for everyone (including the advisor) to remember that. Still, there is often value in learning how to apply and adapt their research tools.
Then, after having cultivated those tools over the course of several projects, the now-experienced researcher will be well positioned to further adapt those tools to their own research projects and their own research style as well as develop their own approaches for research.
The Uncertainty of Research is Challenging and Where to Spend One’s Time.
Spending time on chaotic or complex problems is challenging! One of the challenges: how can one measure progress when trying to tackle an open-ended chaotic or complex problem with no clear or obvious answers?
Because of these challenges, it may be tempting to spend one’s time on problems in the simple or complicated domain rather than on the research project itself. For example, because the task has less uncertainty, it might be tempting to spend time reading research papers rather than focusing on tackling one’s research project.
Of course, reading related research is valuable! My point is not to stop doing that. Rather, I find it helpful to acknowledge explicitly that spending time in the uncertainty of open-ended research projects is challenging. Sometimes, just acknowledging that it is challenging is sufficient to become comfortable spending time in the uncertainty of research. The application of tools for research (discussed above and below) can also help make open-ended research projects feel less open-ended and uncertain.
Example Tools.
One concrete tool is to thoughtfully define how to measure progress on a research project. I discuss that “tool” in my post on how to measure research progress. As part of that post, I discuss a strategy for making progress on research when the “right” next step for the research is unclear.
Another tool is to learn to be comfortable with small “failures” during the research process. Relatedly, another tool is to understand that one should not develop a sense of what a research project looks like on the inside only by evaluating what is visible in published research papers. I discuss these tools in my post on the research iceberg analogy. As part of that post, I also emphasize my dislike of the word “failure.”
Another tool is to cultivate an understanding that there is a wide diversity for what constitutes a research project. I believe that an appreciation of the full diversity of what constitutes a research project can help people become more accepting of how their own research is progressing. I.e., the researcher doesn’t need to write a paper that meets some externally-defined “norm” for research but should rather do the research that they find the most interesting and inspiring. I discuss the diversity of research projects in my post on PhD bubble diagrams.
An additional tool is to thoughtfully scope the research project to minimize and/or manage uncertainty (a topic that I would like to write about later).
A final tool, at least for this post, is learning to periodically revisit the underlying research questions. Because of the chaos/complexity of open-ended research projects, it is not uncommon for research teams to make progress on developing a methodology that is intellectually interesting but that is slightly (or more) askew from the motivating, initial research questions. Periodically pausing and asking, “what are my underlying research questions, again, and how is what I am doing right now connected to those questions?” can help keep the research project focused and on track and help mitigate the risks with open-ended projects. (This is another topic that I would like to write about later.)
Even with the Use of Tools, What if I Don’t Like Complexity or Chaos?
Lea’s original Twitter thread deals with this type of question beautifully! Everyone has different things that they enjoy and that they do not enjoy. If someone truly does not enjoy complex or chaotic problems, that is okay! If someone does not enjoy complicated or simple problems, that is also okay! Knowing what types of problems one enjoys and working in that domain is likely far more fulfilling than trying to work in a domain with which one does not truly resonate.
As Lea wrote in the thread cited at the top of this post:
… figure out what makes you happy. You’re not going to be good at work you hate.
The most overlooked aspect is whether you like and are good at well-understood, complicated, or complex problems — this somewhat maps to the Cynefin.
The above statement really resonates with me and it aligns with one of my favorite Japanese proverbs, 好きこそものの上手なれ (sukikosomononojōzunare), which translates into, “if you like something, you will become good at it.”
Complex and chaotic PhD-style research may not be right for everyone, and that is okay! As Lea wrote and as I agree, it is more important to know what types of problems really resonate with oneself and work in that domain. For those that do like complex and chaotic problems, I hope that this post and the above set of tools are useful. And, I do believe that tools and practice can make it easier to work on complex or chaotic problems, especially if one does not have much prior experience working in such domains.
Additionally, as Lea points out in the Tweets that inspired this post, there is huge value in composing teams with people who complement each other and who enjoy working on different types of problems. I really encourage everyone to read that Twitter thread, too.
Acknowledgements.
I learned so much about the PhD process from my own advisor, Mihir Bellare. After obtaining a faculty position, I continued to develop my own philosophy on advising and the PhD process through the advising my own PhD students. Much of my thoughts on advising and the PhD process have also been shaped through the co-advising PhD students with UW Security Lab co-director Franziska Roesner. Thank you to all the students and postdocs that I have advised, past and present! And thank you to Lea Kissner for the original Twitter thread that motivated this post and for excellent insights on an earlier draft of this piece.