Since fitness trackers and smart watches have become popular, several bloggers have attempted to test their accuracy. Most have noticed the same thing: these suckers just aren't very accurate.

When the bloggers wore several of the gadgets at once, the devices all gave them different readings when it came to heart rate, steps taken, calories burned and other metrics. Not good.

These were just regular folks doing the testing, but now science has stepped up to really put these devices through their paces when it comes to two important metrics: heart rate and energy expenditure (calories burned.)

The Study

In a Stanford University Medical Center study, researchers recruited 60 volunteers to test out several devices in a lab setting. The subjects wore up to four devices at the same time and did various activities such as walking, running, cycling and even just sitting.

They were also hooked up to "gold standard" lab instruments that measure heart rate and energy expenditure. The idea was to compare the mass market gadgets to the super-accurate lab instruments.

The Results

For heart rate, most of the fitness trackers did okay. The Apple Watch had the lowest error rate (2%) while the Samsung Gear S2 had the highest error rate (6.8%). But that's close enough for non-medical purposes.

The problem was in the "calories burned" or energy expenditure readings. They all sucked:

• No device achieved an error rate in energy expenditure below 20%.
• The most "accurate" device was off by an average of 27%.
• The least accurate was off by 93%.
• The error rate was worse for males and those with darker skin tones.

The Apple Watch had the most favorable overall error profile while the PulseOn had the least favorable overall error profile. Here's an overview:

• Apple Watch: Not as shitty
• Basis Peak: Still kinda shitty
• Fitbit Surge: Pretty darn shitty
• Microsoft Band: Super shitty
• PulseOn: The shittiest

How to Use This Info

Whatever you do, don't adjust your training or diet based solely on what a wearable fitness gadget is telling you. When it comes to "calories burned" measurements, they're grossly inaccurate. The algorithms stink and what you're basically getting is a broad, educated guess.

But if you're absolutely determined to buy one, the Apple Watch looks like the best bet based on this study. Or you could use that inexpensive, highly-accurate fitness assessment tool you already own: a mirror.

Reference

1. Anna Shcherbina, C. Mattsson, Daryl Waggott, Heidi Salisbury, Jeffrey Christle, Trevor Hastie, Matthew Wheeler, Euan Ashley. Accuracy in Wrist-Worn, Sensor-Based Measurements of Heart Rate and Energy Expenditure in a Diverse Cohort. Journal of Personalized Medicine, 2017