My thoughts on p-value

There are many scientific domains where the p-value has been abused. I would like to discuss my views on the statistical significance and p-value in my field of research as a neuroscientist. I was routinely expected to submit a p-value of 0.05 for any data in which I claimed to have found a significant difference between two groups during my tenure as a student, postdoctoral fellow, and principal investigator. When I submit a manuscript or grant application, no reviewer will accept any of my findings until the p-value is defined. I started doing the same with other people’s articles and proposals as a means of assimilating into this field. What’s not appropriate? I think a good number of my colleagues might want to know. Unbelievably, a sizable portion of data that had a p-value of more than 0.05 was eliminated. Our scientists will explain that the difference between p=0.05 and p=0.049 in their data has to do with statistical significance if you ask them. Is significance really the main concern? Statisticians will laugh at us, I’m afraid. Let’s discuss this issue in more detail.

We may all agree that statistical significance is necessary to assess whether an effect is meaningfully detectable or significant. But in practice, statistical significance does not always suggest that an effect is important or meaningful. In some cases, we can identify a statistically significant difference between groups if the sample size is large enough. For example, a study may find a statistically significant 0.5% change in blood pressure across individuals from different regions using the p-value that we usually use. P values have been misused and overemphasized in the scientific community when determining whether an effect is statistically relevant. Many researchers and reviewers will only accept a finding if the p-value is tiny enough; a criterion of p = 0.05 or 0.01 has been applied. Even though many people question this “rule” and we all know it might not be reasonable or even truthful, it is a quasi-official norm in several different sectors.

Tian

8/16/2024