According to Forbes, Consumer Report has confirmed that the low battery life was due to a bug in Safari’s code.
Following its own testing, Consumer Report reported low battery life on Apple’s new MacBook Pro machines and as a result could not recommend the laptops.
APPLE QUESTIONED THE RESULTS AND THE TWO COMPANIES HAVE BEEN EXCHANGING…
Apple questioned the results and the two companies have been exchanging diagnostics, reports and details. The results are in, and it feels like a score draw.
Consumer Report has confirmed that the low battery life was due to a bug in Safari’s code. This bug is only present when Safari is in developer mode, which Consumer Report has to use to disable the local cache:
We also turn off the local caching of web pages. In our tests, we want the computer to load each web page as if it were new content from the internet, rather than resurrecting the data from its local drive. This allows us to collect consistent results across the testing of many laptops, and it also puts batteries through a tougher workout.
According to Apple, this last part of our testing is what triggered a bug in the company’s Safari browser. Indeed, when we turned the caching function back on as part of the research we did after publishing our initial findings, the three MacBooks we’d originally tested had consistently high battery life results.
Now, here’s the point where you need to make your own call on this furore.
Consumer Report’s approach is to create a standard test across multiple laptops, from multiple manufacturers, with different operating systems. This demands that conditions should be as equal as possible. On testing the new MacBook Pros under this approach, Consumer Report triggered a bug in the software that had a material impact on the battery life, and felt it could not recommend Apple’s new laptops in good faith.
APPLE SET OUT ITS ARGUMENT IN A STATEMENT TO CONSUMER REPORT
Apple set out its argument In a statement to Consumer Report: “This is not a setting used by customers and does not reflect real-world usage.” Which implies that Apple feels the testing was not appropriate and Consumer Report should be testing its flagship laptop using mythical ‘real world’ parameters that would vary between laptops and manufacturers. When Consumer Report was asked to test the MacBook Pro machines using Apple’s normal user settings, the laptops delivered Apple’s expected battery life.
In my opinion, independent tests should be just that… independent. Manufacturers can suggest settings and use cases, but for a test to be valid everyone needs to be tested to the same standard in the same environment. Consumer Report has decided that means the local cache is switched off.
It’s important to remember that even though Apple disagreed with the testing environment, the low battery life was caused not by poor testing, but by a bug in the browser. While it is a bug that would not normally be triggered, this was the root cause of the battery drain. Apple’s code was broken, not Consumer Report’s testing.
The bug fix is currently in testing in the latest beta versions of MacOS and will be available to the public in the near future. The question now is if this will address all the battery concerns reported by countless users, or if something else is causing a real-world battery drain.