The scientific method of investigation is built on asking questions and testing hypotheses. Many of us have questions about how something works, how applicable or valid it is, or how it can be implemented economically. Those of us who obtained a post-graduate degree were likely asked to write a thesis or a dissertation. This assignment often began with a question. It may have been stated in colloquial terms at first, but eventually it was written in scientific language with terms that could be quantified with measurement or theory. How powerful was that question – to you and the entire world? Looking back, did it put you on a course of future investigations, make you more inquisitive, increase your thirst for better understanding – or was it simply to satisfy your advisor and your academic research committee? Were the letters you received (MS, PhD, DMA, MD, EdD) more important than the discovery?

Once your question was formulated in terms of a hypothesis, attention was likely given to the statistical power of the answer. This statistical power refers to the probability of correctly rejecting a false hypothesis. It quantifies the likelihood that your study would detect an effect if that effect truly existed. A higher statistical power meant a lower chance of missing the effect, which was obviously desirable for your research goal. This statistical power is often determined by how many trials you conduct (sample size) and how valid your instrumentation is.  

Much time and money can be wasted in research by seeking powerful answers to insignificant questions. Just because something has not yet been measured is not a justification for measuring it. Existing theory, or good reasoning, may provide a satisfactory answer. Some years ago I was asked to review a research proposal that required sacrificing over 100 animals (dogs) to obtain a statistically powerful answer to a question that could easily be answered without any animal sacrifice, i.e., with computer simulation. Unfortunately, computer simulation was not available to the investigator, so repeated attempts were made to pursue the original methodology with fewer animals. Had it been funded, the research would have provided a weaker answer to an already weak methodology used for the posed question.

Here are some guidelines for conducting research with the goal of achieving personal satisfaction and being responsible to those who support it:

1. How critical is the question to advance the current state of knowledge?
2. Can it be answered adequately with existing physical laws or data?
3. If measurement is required, is it a critical measurement that will eliminate future measurements?
4. Can the question best be answered with experimentation on (a) human subjects, (b) animals, (c) physical (bench) models, or (d) computational models?
5. Which of these methodologies am I capable of handling with my training, my collaborators, and in my work setting?

Too often we approach this process backwards. We begin by choosing a research group or person we like, using equipment made available to us. Then we pursue a question. While this is often a necessity in early apprentice years, it does not lead to long-term satisfaction in research. With scientific maturity, a more successful approach is to ask a powerful question first, using our brain as the first laboratory (Gedanken experimentation in Einstein’s language), and refining the question again and again. That requires adequate think-time, writing down ideas, walking in the woods, and connecting with universal intelligence. After that, building or joining a laboratory, or buying equipment based on steps 3 and 4 above, will yield a high reward. The process of asking a good scientific question was addressed many years ago by a well-known researcher in the area of stuttering (Johnson, 1948) and an entire issue of Scientific American was devoted to this topic post World War II when economic realities in science set in (Weaver, 1953)

Many good questions begin with the words how, what, or if. A good question may have this structure: what would happen to a phenomenon if a particular variable were to be changed, in isolation or in conjunction with other variables? In physics we are always taught to examine asymptotic conditions first, which means setting a variable to zero or infinity. This often leads to insightful cause-effect relationships between inputs (independent variables) and outputs (dependent variables). One can invent a simple formula that determines the expected relationships between cause and effect. Often the invented formula comes from dimensional analysis (Gibbings, 2011). What goes in the numerator and what goes in the denominator in an algebraic formula can be clarified by simply exploring the dimensions (mass, length, and time) of the variables.

For expanding or clarifying existing data sets, focusing on anomalies rather than the general trend can lead to powerful questions. Artificial intelligence can ask and answer questions based on average trends faster and more accurately than any human can. Powerful questions come from focusing on outliers in data sets, not on the statistical regression line. Is the outlier an error or a new phenomenon to explore? With a well-crafted question concerning a recurring anomaly, a scientific revolution may occur with human investigative intelligence (Kuhn, 1963).

As far as I know, we don’t have statistics to test the power of a question. However, as individuals we are unique, thereby entitling us to unique questions and answers. I believe we are on this earth to grow intellectually. To me, asking a profound question and struggling to get an answer is more helpful for intellectual growth than learning to give perfunctory answers to many trivial questions.

References

Gibbings, J. C. (2011). Dimensional analysis. Springer Science & Business Media.

Johnso, W. (1948). How to ask a question. A Review of General Semantics, Vol. 5, No. 2 pp. 113-118. Published by: Institute of General SemanticsStable URL: https://www.jstor.org/stable/42580799

Kuhn, T. S. (1997). The structure of scientific revolutions (Vol. 962). Chicago: University of Chicago press.

Weaver, W. (1953). Fundamental questions in science. Scientific American , Vol. 189, No. 3, pp. 47-51.

Acknowledgments

Professor Brad Story brought my attention to the classic Scientific American article by Weaver and the very pertinent article in our field by Wendell Johnson. Comments by Professor Ronald Scherer were also highly appreciated and incorporated.

HOW TO CITE