In medical education we have a concept called longitudinal care – interns and residents need to follow a case through to its conclusion, or at least long term, so that can learn the entire process. This is why on-call work hours used to be so horrifically long (up to 36 hours), to allow doctors in training to follow an acute case to see the effects of the decisions they make. Training also includes continuity clinics where residents can follow the same patients for months or even years.
I feel that skeptical science communicators serve this role as well – we are the continuity clinics of skepticism. We follow cases through to their conclusion to see what critical thinking lessons we can derive from them. Without this follow up people tend to jump from one extraordinary claim to another, and therefore they don’t close the feedback loop and never update or improve their cognitive methods. In short, they don’t learn the lessons of their past failures, or benefit from the reinforcement of their successes.
Which is why I think it is important to follow up on the claims made earlier this year by Donut Lab. They claimed to have developed a solid state battery with a specific energy of 400 Wh/kg, with over 100,000 charge cycles, able to recharge in 5 minutes, is stable without a fire risk, operates from -30 to 100 degrees Celsius, is cheaper than current Li-ion batteries, and does not require any geopolitically sensitive materials. They presented these claims at CES in January, but notably did not have an actual battery to demonstrate. Their claims were met with pretty universal skepticism. I wrote at the time: “I’m beginning the smell the “fake it til you make it” syndrome that tanked Theranos.”
There was a strong consensus that the claims need to be backed up by independent testing of actual product, and I figured this would happen over the ensuing months. Well, here we are.
“The investigation consulted over 20 independent battery experts, including Julian Zahnow from the Fraunhofer Research Institute, Dr. Joachim Sann from Justus-Liebig University, Tom Bötticher from Litona, and Dr. Juho Heiska from Seinäjoki University of Applied Sciences. Every single one confirmed the tested cell is lithium-ion.
There are two key pieces of evidence. First, the voltage curves from VTT testing match high-nickel lithium-ion cells (NCM chemistry). The cell sits at 3.7-3.8 volts at 50% state of charge — right where lithium-ion cells operate. Sodium-ion cells don’t go significantly past 3.5 volts at 50% SOC.
The second piece of evidence is even more damning: VTT’s cell expansion data. When a battery charges, ions squeeze into the anode material, causing it to expand in a predictable pattern. A graphite anode produces a distinctive “kink” in the expansion curve around 50-70% state of charge, caused by how ions reorder themselves in graphite’s layered structure. The Donut Lab cell shows exactly that kink.”
The batteries submitted for testing turn out to be regular Li-ion batteries. This is added to demonstrable lies, such as that they were shipping production batteries for their sister company which was producing electric motorcycles. Turns out these were only for internal use, not to customers, and were using Li-ion batteries. All of this is enough to make the investigators characterize this behavior as “fraud”.
What are the lessons here? First, I do think this reinforces the skeptical approach. I have been doing this for over 30 years, and I have always taken an academic, and even medical, approach to the claims I address. What is most important is the predictive value of the arguments used for and against any specific claim. How predictive is it that we cannot fully identify a blurry photo in terms of it representing true alien technology? So far, the predictive value appears to be zero, at least for confirmed cases, but it approaches 100% in predicting that it is a misidentified mundane object.
For the Donut Labs claims there were many red flags. First, the company was claiming many advances simultaneously. This is very rarely the case. Most advances in battery tech in particular are in a single feature, and often at the expense of others. Slowly, cumulatively, the industry advances incrementally. So when a company claims they have suddenly made a suite of advances all at once, that inherently smells more like fraud than a genuine breakthrough. Further, the claims were being made by a small company, while beating out multiple big competitors with billions of dollars to invest in R&D. And then having only an empty box to show and the convention was highly suspicious.
So – we can examine how predictive each of these features are that the claims are false. We can also take them together as a pattern, and state that this pattern of extraordinary claims of extreme advance by an independent small company without anything solid to back it up is extremely highly predictive of fraud. None of this proves, by itself, that the claims were false. In some cases claims literally violate the laws of physics, and they can be rejected on that basis alone. In this case the claims were not impossible, just implausible, but also fit the highly suspicious pattern.
This is exactly what Theranos did, which is why I and others also called it early as a likely fraud. It fit the pattern. This means it is useful to understand this pattern, recognize it, and understand that it reliably predicts that the claims in question are likely not true. For many people this has immediate practical implications – don’t invest in such things, don’t buy them, don’t pin your hopes on them. Wait for actual evidence – you are very likely to be happy you did.
The other potential pattern at work here is the “fake it til you make it” behavior, which extends well beyond such cases of actual fraud. Sometimes this is “innocent” in that it is not meant to hide fraud, but appears to serve the function of timing marketing hype to actual product release. Steve Jobs famously presented any iPhone prototype in 2007 that was (charitably) not fully functional. The first Tesla-bot was a guy in a suit, and it is common for companies to present robots without disclosing that they are being teleoperated or just following motion-capture preprogramming. So sure – if you are the CEO or the head of marketing and the tech guys say the product will be ready for launch in 6 months, but you need to generate hype now, it is tempting to mock up a fake demo of what the product will be like.
My hypothesis, however, is that this behavior became so standardized and accepted in the tech industry that is slowly crept into more and more dubious behavior. Do you show a fake prototype to potential investors? Is that fraud? Do you market a product that is just in the idea stage, before the engineers have figured out if it’s even possible? So we have a continuum from Jobs to Theranos, with the fraudsters justifying their behavior by saying, “Well, Steve Jobs did it.” No – he actually made an iPhone.
Then, of course, there is deliberate premeditated fraud. We see this frequently with many pseudoscientific products, like free energy machines. But here as well there can be a complex mix of true-belief, self-deception, and outright fraud. But even here there is a line. It may be a fuzzy line at times, but if you cross it, you are committing fraud.
