A few years ago, I recall being in one on the outskirts of an industrial park and seeing a camera above a conveyor belt identify a tiny, hair-wide crack in a ceramic tile. The operator hardly raised his head. Regardless of whether the defect it detected was a flaw or, in another context, the very thing that could make an object valuable, the machine had already moved on, scanning the next tile and the next. It’s difficult to ignore that gap. The flaw was detected by the algorithm. It didn’t comprehend it in any significant way.
As machine learning permeates more aspects of daily life, this distinction is at the core of a question that keeps coming up. Because algorithms are designed to maximize, they begin by viewing flaws as mistakes that need to be fixed rather than as qualities worth considering. A statistical anomaly, such as a pixel or data point deviating from a trained baseline, is considered a flaw in a model. These days, medical models identify the irregular cell or the peculiarly shaped mole that a weary human eye might overlook, and computer vision systems detect microscopic cracks and asymmetrical weaves on assembly lines. Without a doubt, impressive work. However, identifying deviation is not the same as understanding why someone might value a chipped teacup more than a perfect one.
I’m more intrigued by the way engineers have begun intentionally incorporating imperfection. Developers purposefully add noise and variance to generative art and audio so that brushstrokes appear less robotic and musical notes have the subtle timing errors of a real player. In order to get users out of their echo chambers, some recommendation engines are intentionally designed to be a little off. There is a perception that unchecked perfection leads to sterile sameness, or what one cultural strategist recently referred to as a “sea of sameness,” a flattening that optimization frequently produces unintentionally.
The most bizarre discovery of all is the one that completely contradicts the original hypothesis. Perfect algorithms are not what people really want. Algorithm aversion, as defined by research by Berkeley Dietvorst and his colleagues at Wharton and Chicago, is the tendency for people to give up on an algorithm after witnessing it make a mistake, frequently in favor of their own worse judgment. Even after they have clearly seen that the machine performs better than them, they continue to be reluctant. It’s quite human and somewhat illogical.
Here’s the twist, though. This aversion is significantly reduced when people have even a small degree of control over an imperfect algorithm’s forecast. Participants in the studies chose to use the algorithm much more frequently and performed better when they could influence its output, even if it was limited to small changes. Strangely, it didn’t really matter how big the adjustment was. The preference persisted even when participants could only slightly alter forecasts, indicating that people were more interested in the sensation of control than actual control. They will trust the machine if it is adjusted by 10%. They recoil when you tell them it’s perfect and untouchable.
As you watch this happen, you begin to believe that accuracy was never the main concern. It had to do with being permitted to continue participating in the process. In the same way that we would not pardon an oracle, we pardon a collaborator for their errors.
Can algorithms therefore comprehend flaws? They can categorize it, imitate it, and even convincingly create it in the strict sense. The philosophical aspect, which explains how a flaw can make something unique, beautiful, or alive, remains obstinately elusive. The crack is caught by the machine. It is still unable to explain the significance of the crack. Perhaps that’s the part we should keep to ourselves. Whether we would prefer it any other way is still up for debate.
