Time to alienate myself! Lets talk a bit about confirmation bias in hi-fi: the biggest and wettest blanket hovering over our community. I know, Im a brand new community member and have no right to say anything about anything, but when the first post I read is about the transformative sonic benefits of upgraded power cables, I feel the need to lay this out. I know beyond shadow of doubt that this will spark controversy, so please keep in mind that my intention is not to offend, only to share my perspective. You are free to dismiss it, however, I vastly prefer a reasoned debate. What Im seeing in most personal testimonies about hi-fi upgrades is a distinct lack of any acknowledgement of their own confirmation bias and the lack of controls in their listening experiments. Please do not underestimate the power of confirmation bias, that is, the propensity to believe that your experiment has been successful just because you want it to be and/or are invested in it. As a psychology student, I can tell you that the level of control required for a perceptual experiment like this to be considered valid is extraordinary - experimental design is very deliberate in order to control for bias. The common faults in most amateur hi-fi experiments are as follows: 1. The experimenter is the primary test subject and is not blind: Having the experimenter know exactly what has been manipulated, having had personally carried out the manipulation, and knowing when they are hearing the experimental condition and the control. These are all bad, because the experimenters expectations can color their perception of the results. 2. The experimenter is biased and invested in a particular outcome: If an experimenter went to the time, trouble, and expense of upgrading their system, theyre going to have certain expectations as to how it will improve the sound; else, they would not have done the upgrade in the first place. Also, the last thing the experimenter wants is for their upgrade to have made no appreciable difference, so they are biased to perceive a positive change. 3. The experimenter uses their friends as subjects and biases them: When telling their friends about the awesome upgrade they just did and inviting them over specifically to listen to it, the experimenter has effectively biased their subjects as well. The subjects, being neither effectively blinded nor exposed to a control condition, and being enthusiastically told that the outcome will represent an improvement, will most likely report that it does. As well, being your friends and having listened to you excitedly talk about this upgrade, they are motivated to say whatever will please you, regardless of their actual perceptions. Thats why dual-relationships between experimenter and subject are so problematic. 4. The experimenter takes no quantitative measurements: While not having taken a comprehensive review of the audiophile literature, I think its a fair assumption that the vast majority of non-professional reviewers do not use any kind of measuring equipment when describing the benefits of a particular upgrade. It seems to me a reasonable assumption that if you are hearing a difference, that difference should be measurable: given the same input media, same room, same equipment, same settings and EQ, same everything, one would expect to be able to objectively measure the impact of changing only one variable. But, hardly anyone bothers to break out so much as a dB meter, so theres no way to empirically verify the claimed results of the experiment. As such, youre forced to rely upon the subjective experiences of the experimenter and their friends, which are very likely biased. 5. The experimenter wants to be recognized for their achievement: This is a great big problem in academia, and it most certainly translates into the hi-fi community. Most who share their results want to be recognized as the discoverer of this great new innovation; they want recognition and positive feedback because it increases their perception that their efforts and expense have been worthwhile. This adds another element of bias, reporting bias: the experimenter will write likely exaggerate the benefit of their manipulations in order to increase the likelihood of positive feedback. The same motivation for talking up their innovation to their friends will cause them to exaggerate to strangers on forums as well. Heres the take-away message: The human brain is by no means a static or precise measurement machine - it is shaped very, very strongly by emotions and biases, and varies hugely from day to day, even moment to moment, and almost unspeakably so between individuals. And, as Ive explained above, were also emotionally invested in the success of our experiments based on our investments of time, effort, and money. When youre talking about whether a change youve made has improved your system, the threshold for a false-positive without instituting any controls in your experiment is unbelievably high. This becomes very problematic for all of us, particularly when someone comes on here claiming to have found the next big thing, and is making their claims based upon a completely uncontrolled experiment. One persons potential waste of time can be transferred to many other people hoping for similar benefits. This is how audiophile myths get started and snake oil sold, resulting in massive collective expenses for little to no benefit. Im not saying uncontrolled experiments never yield benefits. Sometimes, if enough people repeat an admittedly flawed experiment, we can arrive upon a collective consensus that, while not scientifically rigorous, is good enough. In those circumstances, it is best practice to take everything you read based on an uncontrolled experiment with a massive grain of salt, and to weigh heavily the costs involved if considering trying it yourself. The point is to think critically, do research, and if you want to share your own results, acknowledge the limitations and lack of controls in your experiment and invite critique from experts. Be prepared to hear that you were wrong, that the science behind your experiments conflicts with your subjective experience, because if you cant provide the controls to back up your claims, you very well might be.
Posted on: Sun, 25 Jan 2015 14:14:55 +0000
Trending Topics
Recently Viewed Topics
© 2015